Home

Awesome

Kaolin: A Pytorch Library for Accelerating 3D Deep Learning Research

<p align="center"> <img src="assets/kaolin.png"> </p>

Overview

NVIDIA Kaolin library provides a PyTorch API for working with a variety of 3D representations and includes a growing collection of GPU-optimized operations such as modular differentiable rendering, fast conversions between representations, data loading, 3D checkpoints, differentiable camera API, differentiable lighting with spherical harmonics and spherical gaussians, powerful quadtree acceleration structure called Structured Point Clouds, interactive 3D visualizer for jupyter notebooks, convenient batched mesh container and more. Visit the Kaolin Library Documentation to get started!

Note that Kaolin library is part of the larger NVIDIA Kaolin effort for 3D deep learning.

Installation and Getting Started

Starting with v0.12.0, Kaolin supports installation with wheels:

# Replace TORCH_VERSION and CUDA_VERSION with your torch / cuda versions
pip install kaolin==0.16.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-{TORCH_VERSION}_cu{CUDA_VERSION}.html

For example, to install kaolin 0.16.0 over torch 2.0.0 and cuda 11.8:

pip install kaolin==0.16.0 -f https://nvidia-kaolin.s3.us-east-2.amazonaws.com/torch-2.0.0_cu118.html

About the Latest Release (0.16.0)

In this version we added a Physics section.

In this new section we implemented Simplicits a geometry-agnostic method for elastic simulation. See more information in the project page.

We've also added a math section with quaternions math features.

Finally we've improved workflow, with an easy rendering API, for quickly visualizing a model.

Check our new tutorials: Use Simplicit's Easy API to Simulate a Mesh In this notebook we show how to use the high-level Simplicit API to simulate an imported Mesh. Interact with a Physics Simulation In this notebook we show to interact with the simulation by moving objects. Use Simplicit's Full-Feature API In this notebook we show how to use the low-level API to simulate a pointcloud.

Easy rendering of Meshes Use our new high-level API to load a mesh, create a camera and lighting parameters and render interactively.

<a href="https://github.com/NVIDIAGameWorks/kaolin/tree/master/examples/tutorial/physics"><img src="./assets/physics_bulldozer.gif" alt="physics documentation" height="250" /></a>

See change logs for details.

Contributing

Please review our contribution guidelines.

External Projects using Kaolin

Licenses

Most of Kaolin's repository is under Apache v2.0 license, except under kaolin/non_commercial which is under NSCL license restricted to non commercial usage for research and evaluation purposes. For example, FlexiCubes method is included under non_commercial.

Default kaolin import includes Apache-licensed components:

import kaolin

The non-commercial components need to be explicitly imported as:

import kaolin.non_commercial

Citation

If you are using Kaolin library for your research, please cite:

@misc{KaolinLibrary,
      author = {Fuji Tsang, Clement and Shugrina, Maria and Lafleche, Jean Francois and Takikawa, Towaki and Wang, Jiehan and Loop, Charles and Chen, Wenzheng and Jatavallabhula, Krishna Murthy and Smith, Edward and Rozantsev, Artem and Perel, Or and Shen, Tianchang and Gao, Jun and Fidler, Sanja and State, Gavriel and Gorski, Jason and Xiang, Tommy and Li, Jianing and Li, Michael and Lebaredian, Rev},
      title = {Kaolin: A Pytorch Library for Accelerating 3D Deep Learning Research},
      year = {2022},
      howpublished={\url{https://github.com/NVIDIAGameWorks/kaolin}}
}

Contributors

Current Team:

Other Majors Contributors: