Awesome

Argoverse Motion Forecasting Baselines

Official GitHub repository for Argoverse Motion Forecasting Baselines. This repository is released under BSD-3-Clause-Clear License.

Table of Contents

If you have any questions, feel free to open a GitHub issue describing the problem.

• Installation
• Usage
• Contributing
• License

Installation

Requires Linux, git, and Python 3.6+

1) Setup Argoverse API

Follow instructions on Argoverse repository to setup Argoverse API. Make sure the map files are downloaded to the root directory of argoverse repo.

2) Download Argoverse-Forecasting

Argoverse provides both the full dataset and the sample version of the dataset for testing purposes. Head to their website to see the download option.

3) Install dependencies

Install the packages mentioned in requirements.txt

pip install -r requirements.txt

Usage

Running Motion Forecasting baselines has the below 3 components. The runtimes observed on a p2.8xlarge instance (8 NVIDIA K80 GPUs, 32 vCPUs and 488 GiB RAM) are also provided for each part:

1) Feature computation (compute_features.py)

To begin with, we need to compute social and map features for train, val and test set. This is the most computationally expensive part.

Run the following script to compute features for each of train/val/test.

$ python compute_features.py --data_dir <path/to/data> --feature_dir <directory/where/features/to/be/saved> --mode <train/val/test> --obs_len 20 --pred_len 30

Note: If you are not changing anything in the feature computation step, you can also download the precomputed features from this link

2) Run forecasting baselines (nn_train_test.py, lstm_train_test.py)

Once the features have been computed, we can run any of the below baselines.

Constant Velocity:

$ python const_vel_train_test.py --test_features <path/to/test/features> --obs_len 20 --pred_len 30 --traj_save_path <pkl/file/for/forecasted/trajectories>

K-Nearest Neighbors:

Using Map prior:

$ python nn_train_test.py --train_features <path/to/train/features> --val_features <path/to/val/features> --test_features <path/to/test/features> --use_map --use_delta --obs_len 20 --pred_len 30 --n_neigh 3 --model_path <pkl/file/path/for/model> --traj_save_path <pkl/file/for/forecasted/trajectories>

Neither map nor social:

$ python nn_train_test.py --train_features <path/to/train/features> --val_features <path/to/val/features> --test_features <path/to/test/features> --normalize --use_delta --obs_len 20 --pred_len 30 --n_neigh 9 --model_path <pkl/file/path/for/model> --traj_save_path <pkl/file/for/forecasted/trajectories>

LSTM:

Using Map prior:

$ python lstm_train_test.py --train_features <path/to/train/features> --val_features <path/to/val/features> --test_features <path/to/test/features> --model_path <path/to/saved/checkpoint> --use_map --use_delta --obs_len 20 --pred_len 30 --traj_save_path <pkl/file/for/forecasted/trajectories>

Using Social features:

$ python lstm_train_test.py --train_features <path/to/train/features> --val_features <path/to/val/features> --test_features <path/to/test/features> --model_path <path/to/saved/checkpoint> --use_social --use_delta --normalize --obs_len 20 --pred_len 30  --traj_save_path <pkl/file/for/forecasted/trajectories>

Neither map nor social:

$ python lstm_train_test.py --train_features <path/to/train/features> --val_features <path/to/val/features> --test_features <path/to/test/features> --model_path <path/to/saved/checkpoint> --use_delta --normalize --obs_len 20 --pred_len 30 --model_path <pkl/file/path/for/model> --traj_save_path <pkl/file/for/forecasted/trajectories>

3) Metrics and visualization

Evaluation metrics

Here we compute the metric values for the given trajectories. Since ground truth trajectories for the test set have not been released, you can run the evaluation on the val set. If doing so, make sure you don't train any of the above baselines on val set.

Some examples:

Evaluating a baseline that didn't use map and allowing 6 guesses

python eval_forecasting_helper.py --metrics --gt <path/to/ground/truth/pkl/file> --forecast <path/to/forecasted/trajectories/pkl/file> --horizon 30 --obs_len 20 --miss_threshold 2 --features <path/to/test/features> --max_n_guesses 6

Evaluating a baseline that used map prior and allowing 1 guesses along each of the 9 centerlines

python eval_forecasting_helper.py --metrics --gt <path/to/ground/truth/pkl/file> --forecast <path/to/forecasted/trajectories/pkl/file> --horizon 30 --obs_len 20 --miss_threshold 2 --features <path/to/test/features> --n_guesses_cl 1 --n_cl 9 --max_neighbors_cl 3

Evaluating a K-NN baseline that can use map for pruning and allowing 6 guesses

python eval_forecasting_helper.py --metrics --gt <path/to/ground/truth/pkl/file> --forecast <path/to/forecasted/trajectories/pkl/file> --horizon 30 --obs_len 20 --miss_threshold 2 --features <path/to/test/features> --prune_n_guesses 6

It will print out something like

------------------------------------------------
Prediction Horizon : 30, Max #guesses (K): 1
------------------------------------------------
minADE: 3.533317191869932
minFDE: 7.887520305278937
DAC: 0.8857479236783845
Miss Rate: 0.9290787402582446
------------------------------------------------

Visualization

Here we visualize the forecasted trajectories

python eval_forecasting_helper.py --viz --gt <path/to/ground/truth/pkl/file> --forecast <path/to/forecasted/trajectories/pkl/file> --horizon 30 --obs_len 20 --features <path/to/test/features>

Some sample results are shown below

Contributing

Contributions are always welcome! Please be aware of our contribution guidelines for this project.

License

The repository is released under BSD-3-Clause-Clear License. Please be aware of the constraints. See LICENSE