Awesome
TrafficWeavingCVAE
Code accompanying "Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction" (Edward Schmerling, Karen Leung, Wolf Vollprecht, Marco Pavone).
Video: http://stanford.edu/~schmrlng/Schmerling.Leung.Vollprecht.Pavone.ICRA18.mp4
Raw dataset: https://github.com/StanfordASL/TrafficWeavingCVAE/releases/download/ICRA18/human_human_traffic_weaving_rosbags.zip
Usage
Run setup.py
located in the base directory of this repository before running anything else and follow the prompts (default choices in parentheses) in order to download/unzip the dataset linked above, as well as symlink the ROS package traffic_weaving_prediction
and Julia package TrafficWeavingPlanner
into their correct locations.
- 0. Bag Extraction
- Data preprocessing: runs immediately with the ROS and Python dependencies listed below.
- 1. CVAE Training.ipynb
- Model training: runs immediately with the Python dependencies listed below.
- 2. Exploring Predictions.ipynb
- Visualizing predicted human action sequence distributions: requires an exported model from step 1 and the ROS, Python, and Julia dependencies listed below. Before using this notebook
roslaunch traffic_weaving_prediction predict.launch model:={exported_model_dir}
(wrapscvae_node_predict.py
). Pretrained models (the results of running steps 0 and 1) may be found in thepretrained_models
subdirectory of this repository, e.g., you may use{exported_model_dir} = .../TrafficWeavingCVAE/pretrained_models/slim_features_default_hps
.
- Visualizing predicted human action sequence distributions: requires an exported model from step 1 and the ROS, Python, and Julia dependencies listed below. Before using this notebook
- 3. Human-in-the-Loop Simulation.ipynb
- Testing prediction model and policy construction: requires an exported model from step 1 and the ROS, Python, Julia, and VTD dependencies listed below. Before using this notebook
roslaunch vtd_interface human_in_the_loop.launch
(handles VTD simulator input/output) and
roslaunch traffic_weaving_prediction policy.launch model:={exported_model_dir}
(wrapscvae_node_scoring.py
).
- Testing prediction model and policy construction: requires an exported model from step 1 and the ROS, Python, Julia, and VTD dependencies listed below. Before using this notebook
- 4. ICRA18 Videos.ipynb
- Visualizing predictions over the duration of a traffic weaving interaction: requires ROS, Python, and Julia dependencies below, and
roslaunch traffic_weaving_prediction predict.launch
.
- Visualizing predictions over the duration of a traffic weaving interaction: requires ROS, Python, and Julia dependencies below, and
Dependencies
- ROS Kinetic
- Python 2.7 (required for learning code, notebooks 0 and 1)
- TensorFlow 1.3
- pandas
- numpy
- matplotlib
- sympy
- json
- h5py
- Julia 0.6 (required for controls/visualization code, notebooks 2-4)
- At the Julia prompt,
Pkg.clone("https://github.com/schmrlng/DifferentialDynamicsModels.jl") Pkg.clone("https://github.com/schmrlng/LinearDynamicsModels.jl") Pkg.update()
git clone https://github.com/StanfordASL/vtd_interface.git
into thesrc
directory of your ROS catkin workspace. Even if you are not using VTD to conduct human-in-the-loop trials, theTrafficWeavingPlanner
depends on some message types defined invtd_interface
.
- VIRES Virtual Test Drive (VTD, required for human-in-the-loop simulation, notebook 3)
- Contact VIRES for a driving simulator license.
- The VTD project defining the highway on-ramp/off-ramp scenario used in this work may be accessed here: http://stanford.edu/~schmrlng/TrafficWeavingVTDProject.zip; we are not sure how portable these projects are so please file an issue if you are having difficulties loading up
TrafficWeavingVTDProject.vpj
contained within that zip file.