Automatic Data Augmentation via Invariance-Constrained Learning
Jul 24, 2023
Speakers
Ignacio Hounie
Speaker · 0 followers
Luiz F. O. Chamon
Speaker · 0 followers
Alejandro Ribeiro
Speaker · 0 followers
About
Underlying data structures, such as symmetries or invariance to transformations, are often exploited to improve the solution of learning tasks. However, embedding these properties in models or learning algorithms can be challenging and computationally intensive. Data augmentation, on the other hand, induces these symmetries during training by applying multiple transformations to the input data. Despite its ubiquity, its effectiveness depends on the choices of which transformations to apply, when to do so, and how often. In fact, there is both empirical and theoretical evidence that the indiscriminate use of data augmentation can introduce biases that outweigh its benefits. This work tackles these issues by automatically adapting the data augmentation while solving the learning task. To do so, it formulates data augmentation as an invariance constrained learning problem and leverages Monte Carlo Markov Chain (MCMC) sampling to solve it. The result is a practical algorithm that not only does away with a priori searches for augmentation distributions, but also dynamically controls if and when data augmentation is applied. Our experiments illustrate the performance of this method, which achieves state-of-the-art results in automatic data augmentation benchmarks for CIFAR datasets. Furthermore, this approach can be used to gather insights on the actual symmetries underlying a learning task.Underlying data structures, such as symmetries or invariance to transformations, are often exploited to improve the solution of learning tasks. However, embedding these properties in models or learning algorithms can be challenging and computationally intensive. Data augmentation, on the other hand, induces these symmetries during training by applying multiple transformations to the input data. Despite its ubiquity, its effectiveness depends on the choices of which transformations to apply, when…
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Moderately Distributional Exploration for Domain Generalization
Watch later
Rui Dai, …
Aligning Language Models with Preferences through f-divergence Minimization
Watch later
Dongyoung Go, …
Polarity is all you need to learn and transfer faster
Watch later
Alice, …
A General Representation Learning Framework with Generalization Performance Guarantees
Watch later
Junbiao Cui, …
Bit Allocation using Optimization
Watch later
Tongda Xu, …