Generative Occupancy Fields for 3D Surface-Aware Image Synthesis
Dec 6, 2021
Speakers
About
The advent of generative radiance fields has significantly promoted the development of 3D-aware image synthesis. The cumulative rendering process in radiance fields makes training these generative models much easier since gradients are distributed over the entire volume, but leads to diffused object surfaces. In the meantime, compared to radiance fields occupancy representations could inherently ensure deterministic surfaces. However, if we directly apply occupancy representations to generative models, during training they will only receive sparse gradients located on object surfaces and eventually suffer from the convergence problem. In this paper, we propose Generative Occupancy Fields (GOF), a novel model based on generative radiance fields that can learn compact object surfaces without impeding its training convergence. The key insight of GOF is a dedicated transition from the cumulative rendering in radiance fields to rendering with only the surface points as the learned surface gets more and more accurate. In this way, GOF combines the merits of two representations in a unified framework. In practice, the training-time transition of start from radiance fields and march to occupancy representations is achieved in GOF by gradually shrinking the sampling region in its rendering process from the entire volume to a minimal neighboring region around the surface. Through comprehensive experiments on multiple datasets, we demonstrate that GON can synthesize high-fidelity images with 3D consistency and simultaneously learn compact and smooth object surfaces.The advent of generative radiance fields has significantly promoted the development of 3D-aware image synthesis. The cumulative rendering process in radiance fields makes training these generative models much easier since gradients are distributed over the entire volume, but leads to diffused object surfaces. In the meantime, compared to radiance fields occupancy representations could inherently ensure deterministic surfaces. However, if we directly apply occupancy representations to generative…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Sign-to-Speech Model for Sign Language Understanding: A Case Study of Nigerian Sign Language
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Post-Training Sparsity-Aware Quantization
Watch later
Gil Shomron, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Wasserstein distance maximizing Intrinsic Control
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Q&A
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Nested Graph Neural Networks
Watch later
Muhan Zhang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Data-Centric Al Strategies
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%