The goal of data attribution is to trace model predictions back to the training data. Prior approaches to this task exhibit a strict tradeoff between computational demand and efficacy—e.g., methods that are effective for deep neural networks require training thousands of models, making them impractical for large models or datasets.In this work, we introduce TRAK (Tracing by Rewinding the After Kernel), a data attribution method for overparameterized models that brings us closer to the best of bo…