Text-driven PhotoreAlistic 3D Stylization via LiGthing DecOmposition
Nov 28, 2022
Speakers
Yongwei Chen
Speaker · 0 followers
Jiabao Lei
Speaker · 0 followers
Yabin Zhang
Speaker · 0 followers
About
Creation of 3D content by stylization is a promising yet challenging problem in computer vision and graphics research. In this work, we focus on stylizing photorealistic appearance renderings of a given surface mesh of arbitrary topology. Motivated by the recent surge of cross-modal supervision of the Contrastive Language-Image Pre-training (CLIP) model, we propose to transfer the appearance style of a given 3D shape according to a text prompt in a photorealistic manner. Technically, we propose to disentangle the appearance style as the spatially varying bidirectional reflectance distribution function, the local geometric variation, and the lighting condition, which are jointly optimized, via supervision of the CLIP loss, by a spherical Gaussians based differentiable renderer. As such, our method enables photorealistic 3D style transfer by automatically predicting reflectance effects even for bare, low-quality meshes, without training on a task-specific dataset. Extensive experiments show that our method outperforms existing methods of text-driven 3D style transfer in terms of photorealistic quality, consistency of 3D geometry, and robustness when stylizing low-quality meshes.Creation of 3D content by stylization is a promising yet challenging problem in computer vision and graphics research. In this work, we focus on stylizing photorealistic appearance renderings of a given surface mesh of arbitrary topology. Motivated by the recent surge of cross-modal supervision of the Contrastive Language-Image Pre-training (CLIP) model, we propose to transfer the appearance style of a given 3D shape according to a text prompt in a photorealistic manner. Technically, we propose…
Organizer
NeurIPS 2022
Account · 961 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Training self-supervised peptide sequence models on artificially chopped proteins
Watch later
Gil Sadeh, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Making Sense of Dependence: Efficient Black-box Explanations Using Dependence Measure
Watch later
Paul Novello, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Bayesian State-Space SCM for Deforestation Baseline Estimation for Forest Carbon Credit
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Neural Set Function Extensions: Learning with Discrete Functions in High Dimensions
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Dynamic Sparse Network for Time Series Classification: Learning What to “See”
Watch later
Qiao Xiao, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Neural Transmitted Radiance Fields
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%