Home

Awesome

Mini-Omnidata

Minimal repo for generating monocular geometric priors using omnidata. It simplifies the original repo for the inference purposes.

According to the original repo's LICENSE, the usage of the omnidata dataset and the weights should only be of research purposes.

Example

Our cats!

<div float="left"> <img src="assets/image.png" width="240" /> <img src="assets/depth.png" width="240" /> <img src="assets/normal.png" width="240"/> </div>

Differences to original repo

As the backbone is a vision transformer, image resolution should not be a hard constraint as long as they are multiples of 32. In contrast to only receiving 384x384 images in the original repo and MonoSDF, we take arbitrary resolution images.

If the image is too large and GPU memory explodes, you may use down_factor to down sample the input.

Configuration

conda create --name omni python=3.8
conda activate omni
pip install -r requirements.txt

./download_weights.sh

Inference

It takes both single images and folders of images. For the latter, png/jpg images will be globbed and processed.

python run.py --task depth --image_path /path/to/image.png
python run.py --task normal --image_path /path/to/images/

You may also visualize and save the output. --plt_vis prompts plt windows. --output_npy stores outputs as they are for future usages. --output_vis saves colorized images only for the visualization purposes.

python run.py --task depth --image_path /path/to/image.png --plt_vis --output_npy depth_npy --output_vis depth_vis

Postprocessing

For further tasks, you should only postprocess the stored npy file.

normal = (normal - 0.5) * 2

followed by normalization.