AI Research Experiment

0. POC

• Using Jupyter notebooks is also a good way to reproduce results for some people (not recommended)

• snapshot core code and scripts

• json config and output (potentially yaml)

1. Configuration

• Don't save models, only save config and result json, images, code snapshot

2. Experiment Implementation

• Git commit frequently like taking snapshots

3. Result Saving

• Copy only core code to the folder when executing is a good approach

3. Experiment batching

• Process Composer is important

• Create different folders based on run time to keep them independent (no overwriting)

• use experiment script to share a single env