Home

Awesome

3D-Reconstruction-with-Deep-Learning-Methods

The focus of this list is on open-source projects hosted on Github.

Projects released on Github

TITLEKEYWORDSURLLICENSEAwesomeness
High Quality Monocular Depth Estimation via Transfer LearningTensorFlow, PyTorchhttps://github.com/ialhashim/DenseDepth https://arxiv.org/abs/1812.11941GPL-3.0
Multi-view stereo image-based 3D reconstructionhttps://github.com/adahbingee/pais-mvsnn
Hybrid Ensemble Approach For 3D Object Reconstruction from Multi-View Monocular RGB imageshttps://github.com/Ajithbalakrishnan/3D-Object-Reconstruction-from-Multi-View-Monocular-RGB-imagesnn
Deep 3D Semantic Scene Extrapolationhybrid CNN, GAN, TensorFlowhttps://github.com/AliAbbasi/Deep-3D-Semantic-Scene-Extrapolation http://user.ceng.metu.edu.tr/~ys/pubs/extrap-tvcj18.pdfnn
ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D ScansTensorFlowhttps://github.com/angeladai/ScanCompleteApache-2.0
AtLoc: Attention Guided Camera LocalizationPyTorchhttps://github.com/BingCS/AtLoc https://arxiv.org/abs/1909.03557BY-NC-SA 4.0
PointNet: Deep Learning on Point Sets for 3D Classification and SegmentationTensorFlow, cuDNNhttps://github.com/charlesq34/pointnetMIT License
PyTorch Implementation of DeepVOPyTorch, CNNhttps://github.com/ChiWeiHsiao/DeepVO-pytorchnn
Fully Convolutional Geometric Features: Fast and accurate 3D features for registration and correspondence.PyTorchhttps://github.com/chrischoy/FCGFMIT License
Morphing and Sampling Network for Dense Point Cloud Completion (AAAI2020)PyTorchhttps://github.com/Colin97/MSN-Point-Cloud-CompletionApache-2.0
Real-Time Self-Adaptive Deep StereoTensorFlowhttps://github.com/CVLAB-Unibo/Real-time-self-adaptive-deep-stereoApache-2.0
Geometry meets semantics for semi-supervised monocular depth estimation - ACCV 2018TensorFlowhttps://github.com/CVLAB-Unibo/Semantic-Mono-DepthMIT License
BlenderProc: A procedural blender pipeline to generate images for deep learningBlenderhttps://github.com/DLR-RM/BlenderProcGPL-3.0
SingleViewReconstruction: 3D Scene Reconstruction from a Single ViewportTensorFlowhttps://github.com/DLR-RM/SingleViewReconstructionMIT License
NNCAP — Neural Network Complex Approach to Photogrammetryhttps://github.com/Dok11/nn-dldmnn
Pytorch Implementation of Deeper Depth Prediction with Fully Convolutional Residual NetworksPyTorchhttps://github.com/dontLoveBugs/FCRN_pytorchnn
Improved Adversarial Systems for 3D Object Generation and ReconstructionGANhttps://github.com/EdwardSmith1884/3D-IWGANMIT License
Deep Learning for Visual-Inertial OdometryPyTorch, CNNhttps://github.com/ElliotHYLee/Deep_Visual_Inertial_OdometryMIT License
Machine VisionListhttps://github.com/Ewenwan/MVisionnn
Mesh R-CNN, an academic publication, presented at ICCV 2019PyTorch, R-CNNhttps://github.com/facebookresearch/meshrcnnBSD-3-Clause License
PyTorch3d is FAIR's library of reusable components for deep learning with 3D data.PyTorchhttps://github.com/facebookresearch/pytorch3dBSD-3-Clause License
Self-supervised Sparse-to-Dense: Self-supervised Depth Completion from LiDAR and Monocular CameraPyTorchhttps://github.com/fangchangma/self-supervised-depth-completionMIT License
Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single ImagePyTorchhttps://github.com/fangchangma/sparse-to-denseBSD License
Sparse-to-Dense: Depth Prediction from Sparse Depth Samples and a Single ImagePyTorchhttps://github.com/fangchangma/sparse-to-dense.pytorchnn
PackNet-SfM: 3D Packing for Self-Supervised Monocular Depth EstimationPyTorchhttps://github.com/FangGet/PackNet-SFM-PyTorchGPL-3.0
InvSFM: Revealing Scenes by Inverting Structure from Motion Reconstructions [CVPR 2019]TensorFlowhttps://github.com/francescopittaluga/invsfmMIT License
Deep Monocular Visual Odometry using PyTorch (Experimental)PyTorchhttps://github.com/fshamshirdar/DeepVOnn
PointNet: Deep Learning on Point Sets for 3D Classification and SegmentationPyTorchhttps://github.com/fxia22/pointnet.pytorchMIT License
Pix2Depth - Depth Map Estimation from Monocular ImageKerashttps://github.com/gautam678/Pix2DepthGPL-3.0
3DRegNet: A Deep Neural Network for 3D Point RegistrationTensorFlowhttps://github.com/goncalo120/3DRegNetMIT License
Neural 3D Mesh Renderer – Single-Image 3D Reconstruction using Neural Rendererhttps://github.com/hiroharu-kato/mesh_reconstructionMIT License
Real-time Scalable Dense Surfel Mappinghttps://github.com/HKUST-Aerial-Robotics/DenseSurfelMappingnn
MVDepthNet: real-time multiview depth estimation neural networkPyTorchhttps://github.com/HKUST-Aerial-Robotics/MVDepthNetnn
DeepMatchVO: Beyond Photometric Loss for Self-Supervised Ego-Motion Estimationhttps://github.com/hlzz/DeepMatchVOMIT License
MIRorR: Matchable Image Retrieval by Learning from Surface ReconstructionTensorFlow, CNNhttps://github.com/hlzz/mirrorMIT License
Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature ReconstructionCaffehttps://github.com/Huangying-Zhan/Depth-VO-Featnon-commercial
Deep Learning 3D vision paperspapers, list, CNhttps://github.com/huayong/dl-vision-papersnn
Open3D PointNet implementation with PyTorchPyTorch, jupyter, Open3Dhttps://github.com/intel-isl/Open3D-PointNetMIT License
Semantic-TSDF for Self-driving Static Scene ReconstructionPyTorchhttps://github.com/irsisyphus/semantic-tsdfMIT License
Weakly supervised 3D Reconstruction with Adversarial Constrainthttps://github.com/jgwak/McReconMIT License
Using Deep learning Technique for Stereo vision and 3D reconstructionTensorFlow, CNhttps://github.com/jiafeng5513/EvisionNetnn
Unsupervised Scale-consistent Depth and Ego-motion Learning from Monocular VideoPyTorchhttps://github.com/JiawangBian/SC-SfMLearner-ReleaseGPL-3.0
Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps with Accurate Object Boundaries (official implementation)PyTorchhttps://github.com/JunjH/Revisiting_Single_Depth_Estimationnn
Visualization of Convolutional Neural Networks for Monocular Depth Estimation (official implementation)CNN, PyTorchhttps://github.com/JunjH/Visualizing-CNNs-for-monocular-depth-estimationMIT License
DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural NetworksPyTorchhttps://github.com/krrish94/DeepVOnn
DISN: Deep Implicit Surface Network for High-quality Single-view 3D ReconstructionTensorflowhttps://github.com/laughtervv/DISNnn
DeepTAM: Deep Tracking and Mappinghttps://github.com/lmb-freiburg/deeptamGPL-3.0
DeMoN: Depth and Motion NetworkTensorflowhttps://github.com/lmb-freiburg/demonGPL-3.0
PyTorch implementation of CloudWalk's recent work DenseBodyPyTorchhttps://github.com/Lotayou/densebody_pytorchGPL-3.0
Self-supervised learning for dense depth estimation in monocular endoscopyTensorflow, Torchhttps://github.com/lppllppl920/EndoscopyDepthEstimation-Pytorchnon-commercial
ContextDesc: Local Descriptor Augmentation with Cross-Modality ContextTensorflowhttps://github.com/lzx551402/contextdescnn
GL3D (Geometric Learning with 3D Reconstruction): a large-scale database created for 3D reconstruction and geometry-related learning problemshttps://github.com/lzx551402/GL3DMIT License
Deeper Depth Prediction with Fully Convolutional Residual Networks (official implementation)Tensorflowhttps://github.com/MahmoudSelmy/DeeperDepthEstimationnn
Fine-Tuning Vgg16 For Depth EstimationTensorflowhttps://github.com/MahmoudSelmy/DepthEstimationVGGnn
3D reconstruction with neural networks using Tensorflow. See link for Videohttps://github.com/micmelesse/3D-reconstruction-with-Neural-Networksnn
Learning Depth from Monocular Videos using Direct MethodsPyTorchhttps://github.com/MightyChaos/LKVOLearnerBSD-3-Clause
PointNetVLAD: Deep Point Cloud Based Retrieval for Large-Scale Place RecognitionTensorflowhttps://github.com/mikacuy/pointnetvladMIT License
Attempting to estimate topography of a region from image datahttps://github.com/nbelakovski/topography_neural_netnn
DDRNet: Depth Map Denoising and Refinement for Consumer Depth Cameras Using Cascaded CNNsTensorflowhttps://github.com/neycyanshi/DDRNetMIT License
Monocular depth estimation from a single imagePyTorchhttps://github.com/nianticlabs/monodepth2Copyright © Niantic, Inc. 2018. Patent Pending - non-commercial use only
3D-RelNet: Joint Object and Relation Network for 3D predictionTorch, jupyterhttps://github.com/nileshkulkarni/relative3dnn
PlaneRCNN detects and reconstructs piece-wise planar surfaces from a single RGB imageTorch, RCNNhttps://github.com/NVlabs/planercnnCopyright (c) 2018 NVIDIA Corp. All Rights Reserved. This work is licensed under the Creative Commons Attribution NonCommercial ShareAlike 4.0 License.
OctoMap - An Efficient Probabilistic 3D Mapping Framework Based on Octrees.https://github.com/OctoMap/octomapUniversity of Freiburg, Copyright (C) 2009-2014, octomap: New BSD License, octovis and related libraries: GPL
Unsupervised Monocular Depth Estimation neural network MonoDepth in PyTorch (Unofficial implementation)PyTorchhttps://github.com/OniroAI/MonoDepth-PyTorchnn
Learning to Sample: A learned sampling approach for point cloudshttps://github.com/orendv/learning_to_sampleMIT License
DeepMVS: Learning Multi-View StereopsisCNN, PyTorchhttps://github.com/phuang17/DeepMVSBSD 2-clause
DeepV2D: Video to Depth with Differentiable Structure from MotionTensorflowhttps://github.com/princeton-vl/DeepV2Dnn
High Quality Monocular Depth Estimation via Transfer LearningTensorflowhttps://github.com/priya-dwivedi/Deep-Learning/tree/master/depth_estimationnn (GPL-3.0 ?)
Deep Single-View 3D Object Reconstruction with Visual Hull EmbeddingCNN, Tensorflowhttps://github.com/qweas120/PSVH-3d-reconstructionMIT License
ScanNet is an RGB-D video dataset containing 2.5 million views in more than 1500 scans, annotated with 3D camera poses,...https://github.com/ScanNet/ScanNetCan be used with the restriction to give credit and include original Copyright
Visual inspection of bridges is customarily used to identify and evaluate faultsCNNhttps://github.com/Shaggyshak/CS543_project_Image-based-Localization-of-Bridge-Defects-with-AR-Visualizationnn
Semantic 3D Occupancy Mapping through Efficient High Order CRFsCNNhttps://github.com/shichaoy/semantic_3d_mappingBSD-3-Clause
Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scenehttps://github.com/shubhtuls/factored3dnn
Motion R-CNN codebase (old)RCNNhttps://github.com/simonmeister/old-motion-rcnnMIT License
Geometry-Aware Symmetric Domain Adaptation for Monocular Depth EstimationPyTorchhttps://github.com/sshan-zhao/GASDAnn
3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camerahttps://github.com/StanfordVL/3DSceneGraphMIT License
Minkowski Engine is an auto-diff convolutional neural network library for high-dimensional sparse tensorsPyTorchhttps://github.com/stanfordvl/MinkowskiEngineMIT License
Learning Single-View 3D Reconstruction with Limited Pose Supervision (Official implementation)Tensorflowhttps://github.com/stevenygd/3d-reconMIT License
VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera (Tensorflow version)Tensorflowhttps://github.com/timctho/VNect-tensorflowApache-2.0
3D-LMNet: Latent Embedding Matching for Accurate and Diverse 3D Point Cloud Reconstruction from a Single Imagehttps://github.com/val-iisc/3d-lmnetMIT License
Learning to Find Good Correspondenceshttps://github.com/vcg-uvic/learned-correspondence-releaseFor reserch and evaluation only. Commercial usage requires written approval
A Framework for the Volumetric Integration of Depth Imageshttps://github.com/victorprad/InfiniTAMnon-commercial
Pixel2Mesh++: Multi-View 3D Mesh Generation via DeformationTensorflowhttps://github.com/walsvid/Pixel2MeshPlusPlusBSD-3-Clause
Adversarial Semantic Scene Completion from a Single Depth Image (Official implementation)Tensorflowhttps://github.com/wangyida/gan-depth-semantic3dnn
SurfelWarp: Efficient Non-Volumetric Dynamic Reconstructionhttps://github.com/weigao95/surfelwarpBSD-3-Clause
PCN: Point Completion NetworkTensorflowhttps://github.com/wentaoyuan/pcnMIT License
DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructionhttps://github.com/Xharlie/DISNnn
Real-time motion from structureCNNhttps://github.com/yan99033/CNN-SVOnn
Dense 3D Object Reconstruction from a Single Depth ViewTensorflowhttps://github.com/Yang7879/3D-RecGAN-extendedMIT License
Semi-supervised monocular depth map predictionTensorflowhttps://github.com/Yevkuzn/semodepthGPL-3.0
3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud RegistrationTensorflowhttps://github.com/yewzijian/3DFeatNetMIT License
Estimated Depth Map Helps Image Classification: Depth estimation with neural network, and learning on RGBD imageshttps://github.com/yihui-he/Estimated-Depth-Map-Helps-Image-ClassificationMIT License
Fit 3DMM to front and side face images simultaneously.https://github.com/Yinghao-Li/3DMM-fittingnn
The Perfect Match: 3D Point Cloud Matching with Smoothed DensitiesCNN, Tensorflowhttps://github.com/zgojcic/3DSmoothNetBSD-3-Clause
NeurVPS: Neural Vanishing Point Scanning via Conic ConvolutionTenosorflowhttps://github.com/zhou13/neurvpsMIT License
LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image (Torch implementation)Torchhttps://github.com/zouchuhang/LayoutNetMIT License
NeRF: Neural Radiance Fieldshttps://github.com/bmild/nerfMIT License10
Local Light Field Fusion at SIGGRAPH 2019https://github.com/fyusion/llffGPL-3.010
neural-volumes-learning-dynamic-renderable-volumes-from-imageshttps://research.fb.com/publications/neural-volumes-learning-dynamic-renderable-volumes-from-images/<br/>https://github.com/facebookresearch/neuralvolumesBY-NC 4.0
Learning Less is More - 6D Camera Localization via 3D Surface Regressionhttps://github.com/vislearn/LessMoreBSD-3-Clause
Local featureshttps://github.com/vcg-uvic/lf-net-release
Pix2Voxhttps://github.com/hzxie/Pix2Vox
PlanarReconstruction: Single-Image Piece-wise Planar 3D Reconstruction via Associative Embeddingpytorchhttps://github.com/svip-lab/PlanarReconstruction
Depth estimation with deep Neural networkshttps://medium.com/@omarbarakat1995/depth-estimation-with-deep-neural-networks-part-1-5fa6d2237d0d<br/>https://medium.com/datadriveninvestor/depth-estimation-with-deep-neural-networks-part-2-81ee374888eb<br/>https://github.com/MahmoudSelmy/DeeperDepthEstimation<br/>https://github.com/MahmoudSelmy/DepthEstimationVGG/blob/master/README.md
High Quality Monocular Depth Estimation via Transfer Learninghttps://github.com/ialhashim/DenseDepth<br />https://arxiv.org/abs/1812.11941
D3Feathttps://github.com/XuyangBai/D3Feat
Hierarchical Deep Stereo Matching on High Resolution Imagespytorchhttps://github.com/gengshan-y/high-res-stereoMIT
Structure-Aware Residual Pyramid Network for Monocular Depth Estimationpytorchhttps://github.com/Xt-Chen/SARPNnn
Pytorch code to construct a 3D point cloud model from single RGB image.pytorchhttps://github.com/lkhphuc/pytorch-3d-point-cloud-generationnn
Depth estimation from RGB images using fully convolutional neural networkspytorchhttps://github.com/karoly-hars/DE_resnet_unet_hybBSD-3-Clause
Single-Image Piece-wise Planar 3D Reconstruction via Associative Embeddingtorch, tensorflowhttps://github.com/svip-lab/PlanarReconstructionMIT
TriDepth: Triangular Patch-based Deep Depth PredictionPyTorchhttps://github.com/syinari0123/tridepthMIT
Depth Map Prediction from a Single Image using a Multi-Scale Deep Networktorchhttps://github.com/imran3180/depth-map-predictionnn
Hybrid CNN for Single Image Depth Estimationtorchhttps://github.com/karoly-hars/DE_resnet_unet_hybBSD-3-Clause
MarrNet: 3D Shape Reconstruction via 2.5D Sketchestorchhttps://github.com/jiajunwu/marrnetnn
Consistent Video Depth Estimationhttps://roxanneluo.github.io/Consistent-Video-Depth-Estimation/nn
HF-Net: Robust Hierarchical Localization at Large Scaletorch, tensorflowhttps://github.com/ethz-asl/hfnetMIT
Hierarchical Deep Stereo Matching on High Resolution Imagespytorchhttps://github.com/gengshan-y/high-res-stereoMIT
Structure-Aware Residual Pyramid Network for Monocular Depth Estimationpytorchhttps://github.com/Xt-Chen/SARPNnn
Pytorch code to construct a 3D point cloud model from single RGB image.pytorchhttps://github.com/lkhphuc/pytorch-3d-point-cloud-generationnn
Depth estimation from RGB images using fully convolutional neural networkspytorchhttps://github.com/karoly-hars/DE_resnet_unet_hybBSD-3-Clause
Single-Image Piece-wise Planar 3D Reconstruction via Associative Embeddingtorch, tensorflowhttps://github.com/svip-lab/PlanarReconstructionMIT
TriDepth: Triangular Patch-based Deep Depth PredictionPyTorchhttps://github.com/syinari0123/tridepthMIT
Depth Map Prediction from a Single Image using a Multi-Scale Deep Networktorchhttps://github.com/imran3180/depth-map-predictionnn
Hybrid CNN for Single Image Depth Estimationtorchhttps://github.com/karoly-hars/DE_resnet_unet_hybBSD-3-Clause
MarrNet: 3D Shape Reconstruction via 2.5D Sketchestorchhttps://github.com/jiajunwu/marrnetnn
Consistent Video Depth Estimationhttps://roxanneluo.github.io/Consistent-Video-Depth-Estimation/nn
HF-Net: Robust Hierarchical Localization at Large Scaletorch, tensorflowhttps://github.com/ethz-asl/hfnetMIT

Other Projects

TITLEKEYWORDSURLLICENSE
3D-Scene-GAN: Three-dimensional Scene Reconstruction with Generative Adversarial Networkspaperhttps://openreview.net/forum?id=SkNEsmJwf
Google: Deep Learning Depth Predictionmagazine article, GERhttps://www.digitalproduction.com/2019/05/27/google-deep-learning-depth-prediction/
SLAM and Deep Leraning for 3D Indoor Scene UnderstandingPhD thesishttps://www.doc.ic.ac.uk/~ajd/Publications/McCormac-J-2019-PhD-Thesis.pdf
Dense 3D Object Reconstruction from a Single Depth View3D-RecGAN++https://arxiv.org/abs/1802.00411
Moving Camera, Moving People: A Deep Learning Approach to Depth Predictionhttps://ai.googleblog.com/2019/05/moving-camera-moving-people-deep.html
Depth Estimation from a Single RGB Imagehttp://campar.in.tum.de/Chair/ProjectDepthPrediction
Deep Fundamental Matrix Estimationhttp://vladlen.info/papers/deep-fundamental.pdf
depth_estimationhttps://towardsdatascience.com/depth-estimation-on-camera-images-using-densenets-ac454caa893
3D-Machine-Learning Listhttps://github.com/timzhang642/3D-Machine-Learning
DEEP LEARNING-BASED 3D OBJECT RECONSTRUCTION - A SURVEY - Image-based 3D Object Reconstruction:State-of-the-Art and Trends in the DeepLearning Erahttps://arxiv.org/pdf/1906.06543.pdf

I2-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFs (CVPR 2023) https://github.com/jingsenzhu/i2-sdf MIT

https://github.com/lioryariv/idr

https://github.com/autonomousvision/differentiable_volumetric_rendering

https://github.com/Dok11/surface-match-dataset

Image-based 3D Object Reconstruction:State-of-the-Art and Trends in the DeepLearning Era https://arxiv.org/pdf/1906.06543v3.pdf

Dense 3D Object Reconstructionfrom a Single Depth View https://arxiv.org/pdf/1802.00411v2.pdf

https://dagshub.com/OperationSavta/SavtaDepth https://colab.research.google.com/drive/1XU4DgQ217_hUMU1dllppeQNw3pTRlHy1?usp=sharing https://huggingface.co/spaces/kingabzpro/savtadepth MIT License

https://github.com/gradslam/gradslam pyTorch

https://github.com/ventusff/neurecon

https://github.com/theICTlab/3DUNDERWORLD-SLS-GPU_CPU