MBW: Multiview bootstrapping in the wild
Nov 28, 2022
Speakers
Mosam Dabhi
Speaker · 0 followers
Chaoyang Wang
Speaker · 0 followers
Tim Clifford
Speaker · 0 followers
László A. Jeni
Speaker · 0 followers
Simon Lucey
Speaker · 0 followers
About
Labeling articulated objects in unconstrained settings have a wide variety of applications including entertainment, neuroscience, psychology, ethology, and many fields of medicine. Large offline labeled datasets do not exist for all but the most common articulated object categories (e.g., humans). Hand labeling these landmarks within a video sequence is a laborious task. Learned landmark detectors can help, but can be error-prone when trained from only a few examples. Multi-camera systems that train fine-grained detectors have shown significant promise in detecting such errors, allowing for self-supervised solutions that only need a small percentage of the video sequence to be hand-labeled. The approach, however, is based on calibrated cameras and rigid geometry, making it expensive, difficult to manage, and impractical in real-world scenarios. In this paper, we address these bottlenecks by combining a non-rigid 3D neural prior with deep flow to obtain high-fidelity landmark estimates from videos with only two or three uncalibrated, handheld cameras. With just a few annotations (representing 1-2% of the frames), we are able to produce 2D results comparable to state-of-the-art fully supervised methods, and obtain 3D reconstructions that are impossible with other existing approaches. Our Multi-view Bootstrapping in the Wild (MBW) approach demonstrates impressive results on standard human datasets, as well as tigers, cheetahs, fish, colobus monkeys, chimpanzees, and flamingos from videos captured casually in a zoo. We release this challenging dataset consisting more than a thousand image frames of tail-end distribution categories with their corresponding 2D, 3D labels generated from minimal human intervention.Labeling articulated objects in unconstrained settings have a wide variety of applications including entertainment, neuroscience, psychology, ethology, and many fields of medicine. Large offline labeled datasets do not exist for all but the most common articulated object categories (e.g., humans). Hand labeling these landmarks within a video sequence is a laborious task. Learned landmark detectors can help, but can be error-prone when trained from only a few examples. Multi-camera systems that t…
Organizer
NeurIPS 2022
Account · 952 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Evaluating Robustness to Dataset Shift via Parametric Robustness Sets
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Asynchronous Actor-Critic for Multi-Agent Reinforcement Learning
Watch later
Yuchen Xiao, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Protein Sequence Design in a Latent Space via Model-based Reinforcement Learning
Watch later
Minji Lee, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Universally Expressive Communication in Multi-Agent Reinforcement Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Differentiable hierarchical and surrogate gradient search for spiking neural networks
Watch later
Luziwei Leng, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Emergent collective intelligence from massive-agent cooperation and competition
Watch later
Hanmo Chen, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%