DL0017 Reproducibility

Reproducibility in deep learning is achieved by controlling randomness via fixed seeds and deterministic operations, maintaining strict code and dependency versioning, managing datasets carefully, and keeping comprehensive logs of all experiment settings. These practices ensure that experiments can be reliably repeated and validated, regardless of external factors.

(1) Seed Control and Deterministic Operations:
Set random seeds for all libraries (Python, NumPy, TensorFlow/PyTorch).
Enable deterministic settings in your deep learning framework to reduce nondeterminism.
(2) Code Versioning and Configuration Management:
Use version control systems like Git.
Maintain detailed configuration files (using YAML or JSON) that log hyperparameters and settings for each experiment.
(3) Environment and Dependency Control:
Use virtual environments (e.g., Conda) or containerize your projects with Docker.
Freeze library versions to ensure consistency in the software environment.
(4) Dataset Management:
Fix train-test splits and document data preprocessing steps.
Use versioned or static datasets to prevent unintentional changes over time.
(5) Logging and Documentation:
Log hardware details, random seeds, and experiment configurations.
Utilize experiment tracking tools (like MLflow or Weights & Biases) to archive training runs and parameters.

Below is one example that illustrates the experiments are not reproducible.