ViSER: Video-Specific Surface Embeddings for Articulated 3D Shape Reconstruction
Dez 6, 2021
Sprecher:innen
Gengshan Yang
Speaker · 0 followers
Deqing Sun
Speaker · 0 followers
Varun Jampani
Speaker · 0 followers
Daniel Vlasic
Speaker · 0 followers
Forrester Cole
Speaker · 0 followers
Deva Ramanan
Speaker · 0 followers
Über
We introduce ViSER, a method for recovering articulated 3D shapes and dense 3D part trajectories from monocular videos. Previous work on high-quality reconstruction of dynamic 3D shapes typically relies on multiple camera views, strong category-specific priors, or 2D keypoint supervision. We show that none of these are required with reliable estimation of long-range 2D point correspondences, making use of only 2D object masks and two-frame optical flow as inputs. ViSER infers correspondences by matching 2D pixels to a canonical, deformable 3D mesh via video-specific surface embeddings that capture the pixel appearance of each surface point. These surface embeddings behave as a continuous set of keypoint descriptors defined over the mesh surface, which can be used to establish dense long-range correspondences across pixels. These surface embeddings are implemented via surface-coordinate-based MLPs that are fit to each video via contrastive reconstruction losses. Experimental results show that ViSER compares favorably against prior work on challenging videos of humans with loose clothing and unusual poses as well as animals videos from DAVIS and YTVOS.We introduce ViSER, a method for recovering articulated 3D shapes and dense 3D part trajectories from monocular videos. Previous work on high-quality reconstruction of dynamic 3D shapes typically relies on multiple camera views, strong category-specific priors, or 2D keypoint supervision. We show that none of these are required with reliable estimation of long-range 2D point correspondences, making use of only 2D object masks and two-frame optical flow as inputs. ViSER infers correspondences by…
Organisator
NeurIPS 2021
Account · 1.9k followers
Über NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
Domain Invariant Representation Learning with Domain Density Transformations
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
HyAR: Addressing Discrete-Continuous Action Reinforcement Learning via Hybrid Action Representation
Watch later
Boyan Li, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Bandit Phase Retrieval
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Heuristic-Guided Reinforcement Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Grounding Spatio-Temporal Language With Transformers
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning
Watch later
Ji Lin, …
Total of 1 viewers voted for saving the presentation to eternal vault which is 0.1%