Home

Awesome

EndoSLAM Dataset and an Unsupervised Monocular Visual Odometry and Depth Estimation Approach for Endoscopic Videos

<p align="center"> <img align="center" src='imgs/EndoSLAM_Logo.jpeg' width=456/> </p>

EndoSLAM Dataset Overview

<p align="center"> <img src="imgs/ply2.gif" alt="ply" /> </p>

We introduce an endoscopic SLAM dataset which consists of both ex-vivo and synthetically generated data. The ex-vivo part of the dataset includes standard as well as capsule endoscopy recordings. The dataset is divided into 35 sub-datasets. Specifically, 18, 5 and 12 sub-datasets exist for colon, small intestine and stomach respectively.

Sample trajectories from each organ is publicly available in Mendeley.

1. Dataset Shooting

The experimental procedure of ex-vivo part of the dataset is demonstrated at YouTube. To get information about generation of synthetic data, please visit Virtaul Capsule Endoscopy repository.

2. Collection of frames taken on endoscope trajectories

Illustration of recorded frames are as following:

<p align="center"> <img src='imgs/camsamples.png' width=800/> </p>

The ex-vivo and synthetic parts of dataset consist of a total of 42,700 and 21,887 frames respectively. The specifications of dataset parts recorded from each camera are as follows:

<p align="center">
Parts# of FramesFPSResolution
HighCam21,428201280 x 720
LowCam17,97820640 x 480
Pillcam2394 - 35256 x 256
MiroCam3,0553320 x 320
UnityCam21,88730320 x 320
OlympusCam12,250301350 x 1080
</p>

3. Dataset Organization

<p align="left"> <img src='imgs/datatree.png' width=520/> </p> <p align="center"> <img src='imgs/EndoSfMLogo.jpeg' width=356/> </p>

Endo-SfMLearner Overview

We introduce Endo-SfMLearner framework as self-supervised spatial attantion-based monocular depth and pose estimation method.

Our main contributions are as follows:

<p align="center"> <img src='imgs/architecture.png' width=356/> </p>

Getting Started

1. Installation

Clone Endo-SfMLearner Repository

cd ~
git clone https://github.com/CapsuleEndoscope/EndoSLAM
cd EndoSLAM

Prerequisities

You will need to build the Endo-SFMLearner environment by following commend:

pip3 install -r requirements.txt

For the details of EndoSfM-Learner, please check the "EndoSfMLearner" folder.

2. Use-Cases of Endo-SfMLearner with EndoSLAM Dataset

2.1 Depth Estimation

<p align="center"> <img src='imgs/real_stom_depth_fig.png' width=800/> </p> <p align="center"> <img src='imgs/unity_stom_depth_fig.png' width=800/> </p> <p align="center">
UnityEndo-SfMLearnerEndo-SfM w/o brightnessEndo-SfM w/o attentionSC-SfMLearnerMonodepth2Monodepth2 pretrainedSfMLearnerSfMLearner pretrained
RMSE(mean,stdev)0.2966 , 0.06220.3288, 0.06080.3273, 0.10860.3692 , 0.07790.3322 , 0.08150.4531 , 0.10110.3888 , 0.07110.4911 , 0.0831
</p>

2.2 Pose Estimation

<p align="center"> <img src='imgs/real_pose.png' width=800/> </p> <p align="center"> <img src="imgs/unity_rgb_pose_new.gif" alt="ply" width=320/> </p>

2.3 3D Map Reconstruction

In this work, we propose and evaluate a hybrid 3D reconstruction technique. To exemplify the effectiveness of Endo-SfMLearner, we compare the results of reconstructions on Endo-SfMLearner, SC-SfMLearner and shape from shading method in terms of root mean square error(RMSE).

<p align="center"> <img align="center" src='imgs/3drecons.png' width=800/> </p>
AlgorithmRMSE [cm]
EndoSfMLearner0.51
SC-SfMLearner0.86
Shape from Shading0.65

Reference

If you find our work useful in your research or if you use parts of this code please consider citing our paper:

@misc{ozyoruk2020endoslam,
      title={EndoSLAM Dataset and An Unsupervised Monocular Visual Odometry and Depth Estimation Approach for Endoscopic Videos: Endo-SfMLearner}, 
      author={Kutsev Bengisu Ozyoruk and Guliz Irem Gokceler and Gulfize Coskun and Kagan Incetan and Yasin Almalioglu and Faisal Mahmood and Eva Curto and Luis Perdigoto and Marina Oliveira and Hasan Sahin and Helder Araujo and Henrique Alexandrino and Nicholas J. Durr and Hunter B. Gilbert and Mehmet Turan},
      year={2020},
      eprint={2006.16670},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
<!-- ## Help - [Frequently Asked Questions](#frequently-asked-questions) - [Limitations](#limitations) -->