You can integrate Parea with experiment tracking in DVC to iterate on your LLM app without polluting your git history
with a commit for every experiment and still have the ability to revert your code to the state of any experiment.
Once integrated, every time you run an experiment, Parea will automatically capture the
state of the workspace as a DVC experiment together with the associated metrics. This will enable you to compare experiments
via the DVC CLI (
dvc exp show) and to revert to the state of them (
dvc exp apply).
We assume that you have:
Setting-up the integration with DVC
You can integrate Parea with DVC by running the following command:
This command will check if DVC is installed as well as create a
.parea directory with a
dvc.yaml and a
file if they don’t exist.
If the files haven’t been committed to git before, it will ask you to commit them.
dvc.yaml file will point to the
metrics.json file and the
metrics.json file will contain the metrics of the experiments.
Both files are necessary for DVC. You can always re-run the command to check if the integration is set up properly.
Running an experiment
After updating your code, you can run an experiment as usual:
parea experiment <path/to/experiment_file.py>
This will capture the state of the workspace as a DVC experiment together with the associated metrics. To compare all ran experiments since the last commit you can run the following:
dvc exp show
To revert the code to the state of an experiment run the following:
dvc exp apply <experiment-name>
Learn more about DVC
You can learn more about