CASA: Category-agnostic Skeletal Animal Reconstruction
Nov 28, 2022
Speakers
Zeyuan Chen
Speaker · 0 followers
Shaowei Liu
Speaker · 0 followers
Zhongzheng Ren
Speaker · 0 followers
Shenlong Wang
Speaker · 0 followers
About
Recovering a skeletal shape from a monocular video is a longstanding challenge. Prevailing nonrigid animal reconstruction methods often adopt a control-point driven animation model and optimize bone transforms individually without considering skeletal topology, yielding unsatisfactory shape and articulation. In contrast, humans can easily infer the articulation structure of an unknown character by associating it with a seen articulated object in their memory. Inspired by this fact, we present CASA, a novel category-agnostic articulated animal reconstruction method. Our method consists of two components, a video-to-shape retrieval process and a neural inverse graphics framework. During inference, CASA first finds a matched articulated shape from a 3D character assets bank so that the input video scores highly with the rendered image, according to a pretrained image-language model. It then integrates the retrieved character into an inverse graphics framework and jointly infers the shape deformation, skeleton structure, and skinning weights through optimization. Experiments validate the efficacy of our method in shape reconstruction and articulation. We further show that we can use the resulting skeletal-animated character for re-animation.Recovering a skeletal shape from a monocular video is a longstanding challenge. Prevailing nonrigid animal reconstruction methods often adopt a control-point driven animation model and optimize bone transforms individually without considering skeletal topology, yielding unsatisfactory shape and articulation. In contrast, humans can easily infer the articulation structure of an unknown character by associating it with a seen articulated object in their memory. Inspired by this fact, we present CA…
Organizer
NeurIPS 2022
Account · 956 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Multivariate Time-Series Forecasting with Temporal Polynomial Graph Neural Networks
Watch later
Yijing Liu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Relational Proxies: Emergent Relationships as Fine-Grained Discriminators
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
SWAMPNN: End-to-end protein structures alignment
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
On using pre-trained language models for RL: An NLP perspective
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Multi-agent Dynamic Algorithm Configuration
Watch later
Ke Xue, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
On the Representation Collapse of Sparse Mixture of Experts
Watch later
Zewen Chi, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%