ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction
Dec 6, 2021
Speakers
Gengshan Yang
Speaker · 0 followers
Deqing Sun
Speaker · 0 followers
Varun Jampani
Speaker · 0 followers
Daniel Vlasic
Speaker · 0 followers
Forrester Cole
Speaker · 0 followers
Deva Ramanan
Speaker · 0 followers
About
We introduce ViSER, a method for recovering articulated 3D shapes and dense 3D part trajectories from monocular videos. Previous work on high-quality reconstruction of dynamic 3D shapes typically relies on multiple camera views, strong category-specific priors, or 2D keypoint supervision. We show that none of these are required with reliable estimation of long-range 2D point correspondences, making use of only 2D object masks and two-frame optical flow as inputs. ViSER infers correspondences by matching 2D pixels to a canonical, deformable 3D mesh via video-specific surface embeddings that capture the pixel appearance of each surface point. These surface embeddings behave as a continuous set of keypoint descriptors defined over the mesh surface, which can be used to establish dense long-range correspondences across pixels. These surface embeddings are implemented via surface-coordinate-based MLPs that are fit to each video via contrastive reconstruction losses. Experimental results show that ViSER compares favorably against prior work on challenging videos of humans with loose clothing and unusual poses as well as animals videos from DAVIS and YTVOS.We introduce ViSER, a method for recovering articulated 3D shapes and dense 3D part trajectories from monocular videos. Previous work on high-quality reconstruction of dynamic 3D shapes typically relies on multiple camera views, strong category-specific priors, or 2D keypoint supervision. We show that none of these are required with reliable estimation of long-range 2D point correspondences, making use of only 2D object masks and two-frame optical flow as inputs. ViSER infers correspondences by…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Grounding Aleatoric Uncertainty in Unsupervised Environment Design
Watch later
Minqi Jiang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Memory to Map: Improving Radar Flood Maps With Temporal Context and Semantic Segmentation
Watch later
Veda Sunkara, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Influence of human-expert labels on a neonatal seizure detector based on a convolutional neural network
Watch later
Ana Borovac, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Unifying Gradient Estimators for Meta-Reinforcement Learning via Off-Policy Evaluation
Watch later
Yunhao Tang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement
Watch later
Heyang Qin, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Non Equilibrium Sampling on the Orbits of a Deterministic Transform: NEO estimator
Watch later
Achille Thin, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%