dvc

Where Git tracks code line by line, DVC uses lightweight pointer files committed to Git while storing the actual data in remote storage backends (S3, GCS, Azure Blob, SSH, or local paths).

This approach lets teams version datasets and models with the same branching, merging, and tagging workflows they use for code, without bloating the repository or paying for Git LFS.

DVC's experiment tracking features allow data scientists to run systematic experiments with different hyperparameters, datasets, or model architectures, automatically logging metrics and parameters in a structured format.

The dvc exp run command executes pipeline stages defined in dvc.yaml, caching intermediate outputs so unchanged stages don't re-run. Results can be compared with dvc metrics diff and dvc plots, generating visual comparisons of training curves, confusion matrices, and custom metrics across experiment branches.

Beyond individual experiments, DVC supports complex multi-stage ML pipelines with dependency graphs, enabling reproducibility guarantees: given the same code and data versions, a pipeline always produces the same outputs.

This reproducibility is critical for regulatory compliance in industries like healthcare and finance, and for debugging model regressions when production performance degrades.

DVC integrates seamlessly with CI/CD systems (GitHub Actions, GitLab CI, Jenkins), enabling automated model retraining pipelines that trigger when either code or data changes are detected.