Awesome

EAO-SLAM

Related Paper:

• Wu Y, Zhang Y, Zhu D, et al. EAO-SLAM: Monocular Semi-Dense Object SLAM Based on Ensemble Data Association[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020: 4966-4973. [Paper] [Arxiv] [YouTube] [bilibili] [Project page].
• Extended Work
  • Wu Y, Zhang Y, Zhu D, et al. Object SLAM-Based Active Mapping and Robotic Grasping[C]//2021 International Conference on 3D Vision (3DV). IEEE, 2021: 1372-1381. [Paper] [Arxiv] [Project page].
  • Robotic Grasping demo: YouTube | bilibili
  • Augmented Reality demo: YouTube | bilibili
• If you use the code in your academic work, please cite the above paper.

1. Prerequisites

• Prerequisites are the same as semidense-lines. If compiling problems met, please refer to semidense-lines and ORB_SLAM2.
• The code is tested in Ubuntu 16.04, opencv 3.2.0/3.3.1, Eigen 3.2.1.

2. Building

chmod +x build.sh       
./build.sh

3. Examples

• 3.0 We provide a demo that uses the TUM rgbd_dataset_freiburg3_long_office_household sequence; please download the dataset beforehand. The offline object bounding boxes are in data/yolo_txts folder.

• 3.1 Object size and orientation estimation.

  • use iForest and line alignment:

    ./Examples/Monocular/mono_tum LineAndiForest [path of tum fr3_long_office]
    

  • only use iForest:

    ./Examples/Monocular/mono_tum iForest [path of tum fr3_long_office]
    

  • without iForest and line alignment:

    ./Examples/Monocular/mono_tum None [path of tum fr3_long_office]
    

  <figure> <p align="center" > <img src='./figures/scale_and_orientation.png' width=1000 alt="Figure 1"/> </p> </figure>

• 3.2 Data association

  • without data association:

    ./Examples/Monocular/mono_tum NA [path of tum fr3_long_office]
    

  • data association by IoU only:

    ./Examples/Monocular/mono_tum IoU [path of tum fr3_long_office]
    

  • data association by Non-Parametric-test only:

    ./Examples/Monocular/mono_tum NP [path of tum fr3_long_office]
    

  • data association by our ensemble method:

    ./Examples/Monocular/mono_tum EAO [path of tum fr3_long_office]
    

  <figure> <p align="center" > <img src='./figures/data_association.jpg' width=1000 alt="Figure 1"/> </p> </figure>

• 3.3 The full demo on TUM fr3_long_office sequence:

  ./Examples/Monocular/mono_tum Full [path of tum fr3_long_office]
  

  • If you want to see the semi-dense map, you may have to wait a while after the sequence ends.
  • Since YOLO (which was not trained in this scenario) made a lot of false detections at the start of the sequence, so we adopted a stricter elimination mechanism, which resulted in the deletion of many objects at the start.

4. Videos

• More experimental results can be found on our project page.
  • Video: YouTube | bilibili
• Extended work: project page
  • Robotic Grasping demo: YouTube | bilibili
  • Augmented Reality demo: YouTube | bilibili

5. Note

• This is an incomplete version of our paper. If you want to use it in your work or with other datasets, you should prepare the offline semantic detection/segmentation results or switch to online mode. Besides, you may need to adjust the data association strategy and abnormal object elimination mechanism (We found the misdetection from YOLO has a great impact on the results).

6. Acknowledgement

Thanks for the great work: ORB-SLAM2, Cube SLAM, and Semidense-Lines.

• Mur-Artal R, Tardós J D. Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras[J]. IEEE Transactions on Robotics, 2017, 33(5): 1255-1262. PDF, Code
• Yang S, Scherer S. Cubeslam: Monocular 3-d object slam[J]. IEEE Transactions on Robotics, 2019, 35(4): 925-938. PDF, Code
• He S, Qin X, Zhang Z, et al. Incremental 3d line segment extraction from semi-dense slam[C]//2018 24th International Conference on Pattern Recognition (ICPR). IEEE, 2018: 1658-1663. PDF, Code

7. Contact

• Yanmin Wu, Email: wuyanminmax@gmail.com
• Corresponding author: Yunzhou Zhang *, Email: zhangyunzhou@mail.neu.edu.cn

@inproceedings{wu2020eao,
  title={EAO-SLAM: Monocular semi-dense object SLAM based on ensemble data association},
  author={Wu, Yanmin and Zhang, Yunzhou and Zhu, Delong and Feng, Yonghui and Coleman, Sonya and Kerr, Dermot},
  booktitle={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  pages={4966--4973},
  year={2020},
  organization={IEEE}
}

@inproceedings{wu2021object,
  title={Object SLAM-Based Active Mapping and Robotic Grasping},
  author={Wu, Yanmin and Zhang, Yunzhou and Zhu, Delong and Chen, Xin and Coleman, Sonya and Sun, Wenkai and Hu, Xinggang and Deng, Zhiqiang},
  booktitle={2021 International Conference on 3D Vision (3DV)},
  pages={1372-1381},
  year={2021},
  organization={IEEE}
}