Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning
Dec 6, 2021
Speakers
Yifan Zhang
Speaker · 2 followers
Bryan Hooi
Speaker · 0 followers
Jian Liang
Speaker · 0 followers
Jiashi Feng
Speaker · 0 followers
About
Contrastive self-supervised learning (CSL) has attracted increasing attention for model pre-training via unlabeled data. The resulted CSL models provide instance-discriminative visual features that are uniformly scattered in the feature space. During deployment, the common practice is to directly fine-tune CSL models with cross-entropy, which however may not be the best strategy in practice. Although cross-entropy tends to separate inter-class features, the resulting models still have limited capability for reducing intra-class feature scattering that exists in CSL models. In this paper, we investigate whether applying contrastive learning to fine-tuning would bring further benefits, and analytically find that optimizing the contrastive loss benefits both discriminative representation learning and model optimization during fine-tuning. Inspired by these findings, we propose Contrast-regularized tuning (Core-tuning), a new approach for fine-tuning CSL models. Instead of simply adding the contrastive loss to the objective of fine-tuning, Core-tuning further applies a novel hard pair mining strategy for more effective contrastive fine-tuning, as well as smoothing the decision boundary to better exploit the learned discriminative feature space. Extensive experiments on image classification and semantic segmentation verify the effectiveness of Core-tuning.Contrastive self-supervised learning (CSL) has attracted increasing attention for model pre-training via unlabeled data. The resulted CSL models provide instance-discriminative visual features that are uniformly scattered in the feature space. During deployment, the common practice is to directly fine-tune CSL models with cross-entropy, which however may not be the best strategy in practice. Although cross-entropy tends to separate inter-class features, the resulting models still have limited ca…
Organizer
NeurIPS 2021
Account · 1.9k followers
About NeurIPS 2021
Neural Information Processing Systems (NeurIPS) is a multi-track machine learning and computational neuroscience conference that includes invited talks, demonstrations, symposia and oral and poster presentations of refereed papers. Following the conference, there are workshops which provide a less formal setting.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Adversarial Robustness with Semi-Infinite Constrained Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Fitting summary statistics of neural data with a differentiable spiking network simulator
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
FIgLib & SmokeyNet: Dataset and Deep Learning Model for Real-Time Wildland Fire Smoke Detection
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Optimizing Functionals on the Space of Probabilities with Input Convex Neural Network
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Self-Supervised Multi-Object Tracking with Cross-input Consistency
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Identification of Enzymatic Active Sites with Unsupervised Language Modelling
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%