Awesome
<!-- PROJECT LOGO --> <p align="center"> <h1 align="center">CR-NeRF: Cross-Ray Neural Radiance Fields for Novel-view Synthesis from Unconstrained Image Collections</h1> <p align="center"> <strong>Yifan Yang</strong></a> · <strong>Shuhai Zhang</strong></a> · <strong>Zixiong Huang</strong></a> · <strong>Yubing Zhang</strong></a> . <strong>Mingkui Tan</strong></a> </p> <h2 align="center">ICCV 2023 Oral</h2> <p align="center"> <br> <a href='https://arxiv.org/abs/2307.08093'> <img src='https://img.shields.io/badge/Paper-PDF-green?style=for-the-badge&logo=arXiv&logoColor=green' alt='Paper PDF'> </a> </p> </p> <!-- TABLE OF CONTENTS --> <details open="open" style='padding: 10px; border-radius:5px 30px 30px 5px; border-style: solid; border-width: 1px;'> <summary>Table of Contents</summary> <ol> <li> <a href="#introduction-to-cr-nerf">Introduction to CR-NeRF</a> </li> <li> <a href="#video-demo">Video Demo</a> </li> <li> <a href="#instructions">Instructions</a> </li> <li> <a href="#running-demo">Running Demo</a> </li> <li> <a href="#training-and-testing">Training and testing</a> </li> <li> <a href="#citation">Citation</a> </li> </ol> </details> <br /> <br /> <br>Introduction-to-CR-NeRF
 |
|---|
| Pipeline of CR-NeRF |
-
If you want to Train & Evaluate, please check dataset.md to prepare dataset, see <a href="#training-and-testing">Training and testing</a> to train and benchmark CR-NeRF using Brandenburg Gate tainingset
-
During evaluation, given:
- A RGB image of a desired image style
- Camera position
with our CR-NeRF You will get:
- image:
- with the same camera position as the given one
- with the same image style as the given image
For more details of our CR-NeRF, see architecture visualization in our encoder, transformation net, and decoder
<br/> <br>Video-Demo
Appearance Hallucination
Trevi Fountain
Brandenburg Gate
Cross-Appearance Hallucination
From Trevi Fountain to Brandenburg Gate
From Brandenburg Gate to Trevi Fountain
Appearance Hallucination
Comparison with NeRF-W
Instructions
- See docs/installation.md to install all the required packages and setup the models
- See docs/dataset.md to prepare the in-the-wild datasets
- See <a href="#training-and-testing">Training and testing</a> to train and benchmark CR-NeRF using Brandenburg Gate tainingset
Running Demo
Download trained checkpoints from: google drive or Baidu drive password: z6wd
If you want video demo
#Set $scene_name and $save_dir1 and cuda devices in command/get_video_demo.sh
bash command/get_video_demo.sh
The rendered video (in .gif format) will be in path "{$save_dir1}/appearance_modification/{$scene_name}"
If you want images for evaluating metrics
bash command/get_rendered_images.sh
The rendered images will be in path "{$save_dir1}/{$exp_name1}"
Training and testing
#Set experiment name and cuda devices in train.sh
bash command/train.sh
#Set the experiment name to match the training name, and set cuda devices in test.sh
bash command/test.sh
Citation
@inproceedings{yang2023cross,
title={Cross-Ray Neural Radiance Fields for Novel-view Synthesis from Unconstrained Image Collections},
author={Yang, Yifan and Zhang, Shuhai and Huang, Zixiong and Zhang, Yubing and Tan, Mingkui},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
pages={15901--15911},
year={2023}
}
Acknowledgments
We thank Dong Liu's help in making the video demo
Here are some great resources we benefit from:
<br>License
By downloading and using the code and model you agree to the terms in the LICENSE.