Awesome
MVDNet
Robust Multimodal Vehicle Detection in Foggy Weather Using Complementary Lidar and Radar Signals, CVPR 2021.
Prerequisites
- Python 3.7
- Pytorch 1.5.0
- Detectron2
- Pycocotools
MVDNet uses an old version of Detectron2 (i.e., 0.1.1) with minor modifications. To download and install the compatible version:
git clone https://github.com/qiank10/detectron2.git
git checkout alt-0.1.1
cd detectron2 && pip install -e .
Install MVDNet
git clone https://github.com/qiank10/MVDNet.git
cd MVDNet && pip install -e .
Prepare Data
Download the Oxford Radar RobotCar Dataset. Currently, only the vehicles in the first data record (Date: 10/01/2019, Time: 11:46:21 GMT) are labeled. After unzipping the files, the directory should look like this:
# Oxford Radar RobotCar Data Record
|-- DATA_PATH
|-- gt
|-- radar
|-- velodyne_left
|-- velodyne_right
|-- vo
|-- radar.timestamps
|-- velodyne_left.timestamps
|-- velodyne_right.timestamps
|-- ...
Prepare the radar data:
python data/sdk/prepare_radar_data.py --data_path DATA_PATH --image_size 320 --resolution 0.2
Prepare the lidar data:
python data/sdk/prepare_lidar_data.py --data_path DATA_PATH
Prepare the foggy lidar test set with specified fog density, e.g., 0.05:
python data/sdk/prepare_fog_data.py --data_path DATA_PATH --beta 0.05
The processed data is organized as follows:
# Oxford Radar RobotCar Data Record
|-- DATA_PATH
|-- processed
|-- radar
|-- 1547120789640420.jpg
|-- ...
|-- radar_history
|-- 1547120789640420_k.jpg # The k-th radar frame preceding the frame at the timestamp 1547120789640420, k=1,2,3,4.
|-- ...
|-- lidar
|-- 1547120789640420.bin
|-- ...
|-- lidar_history
|-- 1547120789640420_k.bin # Link to the k-th lidar frame preceding the frame at the timestamp 1547120789640420, k=1,2,3,4.
|-- 1547120789640420_k_T.bin # Transform matrix between the k-th preceding lidar frame and the current frame.
|-- ...
|-- lidar_fog_0.05 # Foggy lidar data with fog density as 0.05
|-- 1547120789640420.bin
|-- ...
|-- lidar_history_fog_0.05
|-- 1547120789640420_k.bin
|-- 1547120789640420_k_T.bin
|-- ...
Both 2D and 3D labels are in
./data/RobotCar/object/
Train MVDNet
python ./tools/train.py --config ./configs/train_config.yaml
Evaluate MVDNet
python ./tools/eval.py --config ./configs/eval_config.yaml