Effective Adaptation in Multi-Task Co-Training for Unified Autonomous Driving
Nov 28, 2022
Speakers
Xiwen Liang
Speaker · 0 followers
Yangxin Wu
Speaker · 0 followers
Jianhua Han
Speaker · 0 followers
Chunjing Xu
Speaker · 0 followers
Xiaodan Liang
Speaker · 0 followers
About
Aiming towards a holistic understanding of multiple downstream tasks simultaneously, there is a need for extracting features with better transferability. Though many latest self-supervised pre-training methods have achieved impressive performance on various vision tasks under the prevailing pretrain-finetune paradigm, their generalization capacity to multi-task learning scenarios is yet to be explored. In this paper, we extensively investigate the transfer performance of various types of self-supervised methods, e.g., MoCo and SimCLR, on three downstream tasks, including semantic segmentation, drivable area segmentation, and traffic object detection, on the large-scale driving dataset BDD100K. We surprisingly find that their performances are sub-optimal or even lag far behind the single-task baseline, which may be due to the distinctions of training objectives and architectural design lied in the pretrain-finetune paradigm. To overcome this dilemma as well as avoid redesigning the resource-intensive pre-training stage, we propose a simple yet effective pretrain-adapt-finetune paradigm for general multi-task training, where the off-the-shelf pretrained models can be effectively adapted without increasing the training overhead. During the adapt stage, we utilize learnable multi-scale adapters to dynamically adjust the pretrained model weights supervised by multi-task objectives while leaving the pretrained knowledge untouched. Furthermore, we regard the vision-language pre-training model CLIP as a strong complement to the pretrain-adapt-finetune paradigm and propose a novel adapter named LV-Adapter, which incorporates language priors in the multi-task model via task-specific prompting and alignment between visual and textual features. Our experiments demonstrate that the adapt stage significantly improves the overall performance of those off-the-shelf pretrained models and the contextual features generated by LV-Adapter are of general benefits for downstream tasks.Aiming towards a holistic understanding of multiple downstream tasks simultaneously, there is a need for extracting features with better transferability. Though many latest self-supervised pre-training methods have achieved impressive performance on various vision tasks under the prevailing pretrain-finetune paradigm, their generalization capacity to multi-task learning scenarios is yet to be explored. In this paper, we extensively investigate the transfer performance of various types of self-su…
Organizer
NeurIPS 2022
Account · 961 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
MissDAG: Causal Discovery in the Presence of Missing Data with Continuous Additive Noise Models
Watch later
Erdun Gao, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Towards Versatile Embodied Navigation
Watch later
Hanqing Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Evaluating the Practicality of Counterfactual Explanation
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Learnable Polyphase Sampling for Shift Invariant and Equivariant Convolutional Networks
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learning
Watch later
Annie Xie, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Improving Domain Generalization with Interpolation Robustness
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%