Lighting Every Darkness with 3DGS:
Fast Training and Real-Time Rendering for HDR View Synthesis

NeurIPS 2024

Xin Jin^1,2,*,#, Pengyi Jiao^1,*, Zheng-Peng Duan¹, Xingchao Yang², Chongyi Li¹, Chun-Le Guo^1,†, Bo Ren^1,†

¹VCIP, CS, Nankai University, ²MEGVII Technology, ^*Equal Contribution, ^†Corresponding Authors
^#This project is done during Xin Jin's Internship @ MEGVII Technology

Paper Supp (Details on Postprocessing) Code & Interactive Viewer BibTex

Click to learn more

🧑‍🎨 Create Your Own Video Storyboard! 🧑‍🎨

You could try to create your own video storyboard by tuning the exposure, color temperature, and camera pose. Try it out! ⬇

Tunable Image & Poses

Exposure Color Temperature Camera Pose

Due to LE3D's capability for real-time rendering, once the reconstruction results are obtained, we can perform real-time rendering on them. Additionally, since the reconstruction results from LE3D are in the HDR color space, it allows for exposure variation as well as local and global tone-mapping operations. Our interactive viewer enables users to edit the scene in real-time, and it will provide functionality for creating video storyboards using keyframes in the future.

🆚 Comparison Between LE3D, RawNeRF and RawGS^*

RawGS^* denotes a 3DGS trained with RawNeRF's weighted loss and multi-exposure strategy.

Comparison with RawNeRF

Comparison with RawGS

Compared to RawNeRF (left), LE3D can perform real-time rendering at 2K resolution, and as a result, it can also conduct nearly real-time postprocessing (our interactive viewer). In comparison with RawGS, LE3D possesses superior capabilities in scene structure optimization, and additionally, it has fewer residual floaters in the air.

🎑 Denoising Ability 🌃

LE3D has a natural noise resistance similar to RawNeRF, which is attributed to the noise2noise assumption and the fact that the noise is entirely located on the camera plane, rather than being distributed in space.

BibTex


@inproceedings{jin2024le3d,
      title={Lighting Every Darkness with 3DGS: Fast Training and Real-Time Rendering for HDR View Synthesis},
      author={Jin, Xin and Jiao, Pengyi and Duan, Zheng-Peng and Yang, Xingchao and Guo, Chun-Le and Ren, Bo and Li, Chong-Yi},
      booktitle={NeurIPS},
      year={2024}
}

Contact

Feel free to contact us at xjin[AT]mail.nankai.edu.cn!