Awesome

'NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search'

To run e.g. darts on search space 1 execute from the root of the repository:

PYTHONPATH=$PWD python optimizers/darts/train_search.py --seed=0 --save=baseline --search_space=1 --epochs=50

To evaluate the one-shot architectures on NAS-Bench-101 download NAS-Bench-101 and insert the path to it in nasbench_analysis/eval_darts_one_shot_model_in_nasbench.py

Then run the following for evaluation: PYTHONPATH=$PWD python nasbench_analysis/eval_darts_one_shot_model_in_nasbench.py

The NAS-Bench-101 test error and validation error for the searched architectures are written to the directory of the run and can then be analyzed using the experiment database as demonstrated in: experiments/analysis/plot_results.py