Awesome
Code for extract bboxes with RoI features for VidVRD and VidOR dataset
we use VinVL to extract traj bbox and RoI features.
we do this separately. The whole pipeline is 1)detect bbox --> 2)object tracking --> 3)extract RoI features. This is because at the tracking stage we will filter out many boxes, and extract RoI features based on the obtained object trajectory saves more computational cost.
We use vinvl_vg_x152c4
as the pre-trained detector to do stages 1) & 3), and for stage 2) we use Seq-NMS (which is parameter-free)
-
install VinVL and its Scene Graph Benchmark refer to https://github.com/pzzhang/VinVL
-
detect bbox:
- For VidVRD, refer to
tools/extract_bboxes/extract_video_bboxes_dataloader.py
- For VidOR, refer to
tools/extract_bboxes/extract_video_bboxes_VideoReader.py
- For VidOR, beacuse the number of videos is very large (i.e., 7000), we use
VideoReader
from decord to load videos. It saves the memory and improves speed than using open-cv.
- For VidVRD, refer to
-
object tracking:
- We use Seq-NMS following VidVRD-II.
- We modified the seq_nms script a bit. TODO: we will release the seq_nms scripts
-
extract RoI features:
- For VidVRD, refer to
tools/extract_features/extract_traj_features.py
- For VidOR, refer to
tools/extract_features/extract_traj_features_VideoReader.py
- For VidVRD, refer to