Jul 24, 2023
Speaker · 0 followers
Speaker · 0 followers
Speaker · 0 followers
Speaker · 0 followers
Out-of-distribution (OOD) generalization remains a major issue in modern deep learning. One of the main causes of poor OOD generalization is the reliance on spurious features — patterns that are predictive of the class label in the training data distribution, but that are not causally related to the target. Standard methods for reducing the reliance on spurious features typically assume that we know what the spurious feature is, and methods that attempt to alleviate this limitation are complex, hard to tune, and lead to a significant computational overhead compared to standard training. In this paper we propose Automatic Feature Reweighting (AFR), a simple and fast method for updating the model to reduce the reliance on spurious features. Inspired by the Deep Feature Reweighting (DFR), in AFR we retrain the last layer of the base model with a weighted loss, where we upweight the datapoints on which the base model provides poor predictions. With this simple procedure, we improve upon the best reported results among methods trained without access to the spurious attributes on several vision and natural language classification benchmarks, using only a fraction of the compute required by the competing methods.Out-of-distribution (OOD) generalization remains a major issue in modern deep learning. One of the main causes of poor OOD generalization is the reliance on spurious features — patterns that are predictive of the class label in the training data distribution, but that are not causally related to the target. Standard methods for reducing the reliance on spurious features typically assume that we know what the spurious feature is, and methods that attempt to alleviate this limitation are complex,…
Professional recording and live streaming, delivered globally.
Presentations on similar topic, category or speaker
Nicholas Gao, …
Uriel Singer, …