Coarse-to-fine Animal Pose and Shape Estimation
Dez 6, 2021
Sprecher:innen
Gim Hee Lee
Speaker · 0 followers
Über
Most existing animal pose and shape estimation approaches reconstruct animal meshes with a parametric SMAL model. This is because the low-dimensional pose and shape parameters of the SMAL model makes it easier for deep networks to learn the high-dimensional animal meshes. However, the SMAL model is obtained from scans of toy animals with limited pose and shape variations, and thus may not be able to represent highly varying real animals well. This can result in ill-fittings of the estimated mesh to the 2D evidences, e.g. 2D keypoints or silhouettes. To mitigate this problem, we propose a coarse-to-fine approach to reconstruct 3D animal mesh from a single image. The coarse estimation stage first estimates the pose, shape and translation parameters of the SMAL model. The estimated meshes are then used as a starting point by a graph convolutional network (GCN) to predict a per-vertex deformation in the refinement stage. This combination of SMAL-based and vertex-based representations benefits from both parametric and non-parametric representations. We design our mesh refinement GCN (MRGCN) as an encoder-decoder structure with hierarchical feature representation to overcome the limited receptive field of traditional GCNs.Moreover, we observe that the global image feature used by all existing animal mesh reconstruction works is unable to capture detailed shape information for mesh refinement. We thus introduce a local feature extractor to retrieve a vertex-level feature and use it together with the global feature as the input of the MRGCN. We test our approach on the StanfordExtra dataset and achieve state-of-the-art results. Furthermore, we test the generalization capacity of our approach on the Animal Pose and BADJA datasets. Our code will be made available for research purpose upon acceptance of this paper.Most existing animal pose and shape estimation approaches reconstruct animal meshes with a parametric SMAL model. This is because the low-dimensional pose and shape parameters of the SMAL model makes it easier for deep networks to learn the high-dimensional animal meshes. However, the SMAL model is obtained from scans of toy animals with limited pose and shape variations, and thus may not be able to represent highly varying real animals well. This can result in ill-fittings of the estimated mesh…
Organisator
NeurIPS 2021
Account · 1.9k followers
Über NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
Bounded Fairness Transferability: Subject to Distribution Shift
Watch later
Yatong Chen, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Towards Stable and Robust AdderNets
Watch later
Minjing Dong, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Discussion
Watch later
Dorsa Sadigh, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
VisDA21: Visual Domain Adaptation
Watch later
Kate Saenko, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Non-asymptotic Error Bounds for Bidirectional GANs
Watch later
Shiao Liu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Using Deep Learning to Annotate the Protein Universe
Watch later
Total of 1 viewers voted for saving the presentation to eternal vault which is 0.1%