Awesome
CortexNet
This repo contains the PyTorch implementation of CortexNet.
Check the project website for further information.
Project structure
The project consists of the following folders and files:
data/
: contains Bash scripts and a Python class definition inherent video data loading;image-pretraining/
: hosts the code for pre-training TempoNet's discriminative branch;model/
: stores several network architectures, including PredNet, an additive feedback Model01, and a modulatory feedback Model02 (CortexNet);notebook/
: collection of Jupyter Notebooks for data exploration and results visualisation;utils/
: scripts for- (current or former) training error plotting,
- experiments
diff
, - multi-node synchronisation,
- generative predictions visualisation,
- network architecture graphing;
results@
: link to the location where experimental results will be saved within 3-digit folders;new_experiment.sh*
: creates a new experiment folder, updateslast@
, prints a memo about last used settings;last@
: symbolic link pointing to a new results sub-directory created bynew_experiment.sh
;main.py
: training script for CortexNet in MatchNet or TempoNet configuration;
Dependencies
- scikit-video: accessing images / videos
pip install sk-video
- tqdm: progress bar
conda config --add channels conda-forge
conda update --all
conda install tqdm
IDE
This project has been realised with PyCharm by JetBrains and the Vim editor. Grip has been also fundamental for crafting decent documtation locally.
Initialise environment
Once you've determined where you'd like to save your experimental results — let's call this directory <my saving location>
— run the following commands from the project's root directory:
ln -s <my saving location> results # replace <my saving location>
mkdir results/000 && touch results/000/train.log # init. placeholder
ln -s results/000 last # create pointer to the most recent result
Setup new experiment
Ready to run your first experiment? Type the following:
./new_experiment.sh
GPU selection
Let's say your machine has N
GPUs.
You can choose to use any of these, by specifying the index n = 0, ..., N-1
.
Therefore, type CUDA_VISIBLE_DEVICES=n
just before python ...
in the following sections.
Train MatchNet
- Download e-VDS35 (e.g.
e-VDS35-May17.tar
) from here. - Use
data/resize_and_split.sh
to prepare your (video) data for training. It resizes videos present in folders of folders (i.e. directory of classes) and may split them into training and validation set. May also skip short videos and trim longer ones. Checkdata/README.md
for more details. - Run the
main.py
script to start training. Use-h
to print the command line interface (CLI) arguments help.
python -u main.py --mode MatchNet <CLI arguments> | tee last/train.log
Train TempoNet
- Download e-VDS35 (e.g.
e-VDS35-May17.tar
) from here. - Pre-train the forward branch (see
image-pretraining/
) on an image data set (e.g.33-image-set.tar
from here); - Use
data/resize_and_sample.sh
to prepare your (video) data for training. It resizes videos present in folders of folders (i.e. directory of classes) and samples them. Videos are then distributed across training and validation set. May also skip short videos and trim longer ones. Checkdata/README.md
for more details. - Run the
main.py
script to start training. Use-h
to print the CLI arguments help.
python -u main.py --mode TempoNet --pre-trained <path> <CLI args> | tee last/train.log
GPU selection
To run on a specific GPU, say n
, type CUDA_VISIBLE_DEVICES=n
just before python ...
.