Home

Awesome

nasbot

A Python implementation of NASBOT (Neural Architecture Search with Bayesian Optimisation and Optimal Transport). This repo also provides OTMANN (Optimal Transport Metric for Architectures of Neural Networks), which is an optimal transport based distance for neural network architectures. For more details, please see our paper below.

For questions and bug reports please email kandasamy@cs.cmu.edu.

Installation

$ git clone https://github.com/kirthevasank/nasbot.git
$ pip install cython POT graphviz pygraphviz

In addition to the above, you will need numpy and scipy which can also be pip installed.

$ source set_up
$ pip install tensorflow-gpu

Testing the Installation: To test the installation, run bash run_all_tests.sh. Some of the tests are probabilistic and could fail at times. If this happens, run the same test several times and make sure it is not consistently failing. Running all tests will take a while. You can run each unit test individually simpy via python unittest_xxx.py.

Getting started

To help get started, we have some demos in the demos directory. They demonstrate how to specify a) a search space, b) the function to be optimised, and c) the number of parallel workers for NASBOT - these are the bare minimum that need to specified. Other parameters can be tuned via the APIs available in opt/nasbot.py.

demos/demo_synthetic.py demonstrates NASBOT on a synthetic function while demos/demo_mlp.py and demos/demo_cnn.py demonstrate NASBOT on MLP and CNN hyper-parameter tuning tasks respectively. The datasets can be downloaded from the author's homepage. There are some instructions in the demo files on preparing/saving the data files. To run the MLP/CNN demos, you will need access to one or more GPUs and install tensorflow, e.g. pip install tensorflow-gpu.

Using NASBOT in your architecture search task

To use NASBOT in your architecture search task, you need to write a FunctionCaller class which inherits the NNFunctionCaller class in opt/nn_function_caller.py. You should implement the method _eval_validation_score(nn, qinfo) in this class which evaluates a network nn and returns the validation score. NASBOT will maximise this score - hence, (for example) you can return the accuracy for classification problems and the negative MSE for regression problems. The qinfo argument is used to pass other ancillary information that may be required to conduct the evaluation (e.g. GPU ID).

You can follow the examples in demos/mlp_function_caller.py and demos/cnn_function_caller.py

OTMANN

Some Details

Citation

If you use any part of this code in your work, please cite our Arxiv paper:

@article{kandasamy2018neural,
  title={Neural Architecture Search with Bayesian Optimisation and Optimal Transport},
  author={Kandasamy, Kirthevasan and Neiswanger, Willie and Schneider, Jeff and Poczos,
Barnabas and Xing, Eric},
  journal={arXiv preprint arXiv:1802.07191},
  year={2018}
}

License

This software is released under the MIT license. For more details, please refer LICENSE.txt.

"Copyright 2018 Kirthevasan Kandasamy"