This repository uses DVC (Data Version Control) to manage datasets. The source code is stored in GitHub, while the datasets are stored separately in a DVC remote on Google Drive.
Before proceeding, ensure that the following are installed:
- Git
- Python 3.9+ (recommended)
- DVC with Google Drive support
Install DVC using:
pip install "dvc[gdrive]"Clone the repository and navigate to the project directory:
git clone <repository_url>
cd <repository_name>The repository contains DVC metadata files that track the dataset version associated with the current Git commit.
To download the required data:
dvc pullThe first time you run this command, DVC may prompt you to authenticate with a Google account.
Important: Use a Google account that has been granted access to the shared DVC storage.
After successful authentication, DVC will automatically download all required datasets to their correct locations.
To verify that all tracked data files have been downloaded successfully:
dvc statusNo missing files should be reported.
Once the data has been downloaded, install the project dependencies and run the project according to the instructions provided in the remaining sections of this README.
If you encounter a Google Drive authentication or permission error:
- Ensure that the Google account used during authentication has access to the shared DVC storage.
- Re-run the command with verbose logging:
dvc pull -v- If the issue persists, contact the repository owner and provide the Google account email you are using for authentication.
If some files are missing after cloning:
dvc pullto download any missing artifacts.
To verify the configured DVC remote:
dvc remote listThe default remote should already be configured within the repository.
- Do not commit downloaded datasets directly to Git.
- Use DVC commands (
dvc add,dvc push,dvc pull) to manage dataset versions. - Dataset versions are linked to Git commits, ensuring reproducibility of experiments and results.