ModelDiff: A Framework for Comparing Learning Algorithms
Jul 24, 2023
Speakers
Harshay Shah
Speaker · 0 followers
Andrew Ilyas
Speaker · 1 follower
Aleksander Madry
Speaker · 0 followers
About
We study the problem of (learning) algorithm comparison, where the goal is to find differences between models trained with two different learning algorithms. We begin by formalizing this goal as one of finding distinguishing feature transformations, i.e., input transformations that change the predictions of models trained with one learning algorithm but not the other. We then present ModelDiff, a method that leverages the datamodels framework (Ilyas et al., 2022) to compare learning algorithms based on how they use their training data. We demonstrate ModelDiff through three case studies, comparing models trained with/without data augmentation, with/without pre-training, and with different SGD hyperparameters.We study the problem of (learning) algorithm comparison, where the goal is to find differences between models trained with two different learning algorithms. We begin by formalizing this goal as one of finding distinguishing feature transformations, i.e., input transformations that change the predictions of models trained with one learning algorithm but not the other. We then present ModelDiff, a method that leverages the datamodels framework (Ilyas et al., 2022) to compare learning algorithms b…
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Panel on Reasoning Capabilities of LLMs
Watch later
Efficient RL via Disentangled Environment and Agent Representations
Watch later
Kevin Gmelin, …
Regression with Sensor Data Containing Incomplete Observations
Watch later
Efficient Personalized Federated Learning via Sparse Model-Adaptation
Watch later
Daoyuan Chen, …