TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second
Dec 2, 2022
Speakers
Noah Hollmann
Speaker · 0 followers
Samuel Müller
Speaker · 0 followers
Katharina Eggensperger
Speaker · 0 followers
Frank Hutter
Speaker · 2 followers
About
We present TabPFN, a trained Transformer model that can do tabular supervised classification for small datasets in less than a second, needs no hyperparameter tuning and is competitive with state-of-the-art classification methods.TabPFN is entailed in the weights of our network, which accepts training and test samples as a set-valued input and yields predictions for the entire test set in a single forward pass. TabPFN is a Prior-Data Fitted Network (PFN) and is trained offline once, to approximate Bayesian inference on synthetic datasets drawn from our prior. Our prior incorporates ideas from causal learning: It entails a large space of structural causal models with a preference for simple structures. Afterwards, the trained TabPFN approximates Bayesian prediction on any unseen tabular dataset, without any hyperparameter tuning or gradient-based learning.On 30 datasets from the OpenML-CC18 suite, we show that our method outperforms boosted trees and performs on par with complex state-of-the-art AutoML systems with a 70× speedup. This increases to a 3 200× speedup when a GPU is available.We provide all our code and the trained TabPFN at https://anonymous.4open.science/r/TabPFN-2AEE. We also provide an online demo at https://huggingface.co/spaces/TabPFN/TabPFNPrediction.We present TabPFN, a trained Transformer model that can do tabular supervised classification for small datasets in less than a second, needs no hyperparameter tuning and is competitive with state-of-the-art classification methods.TabPFN is entailed in the weights of our network, which accepts training and test samples as a set-valued input and yields predictions for the entire test set in a single forward pass. TabPFN is a Prior-Data Fitted Network (PFN) and is trained offline once, to approxim…
Organizer
NeurIPS 2022
Account · 963 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Efficient coding, channel capacity, and the emergence of retinal mosaics
Watch later
Na Young Jun, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Plan To Predict: Learning an Uncertainty-Foreseeing Model For Model-Based Reinforcement Learning
Watch later
Zifan Wu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Decomposable Non-Smooth Convex Optimization with Nearly-Linear Gradient Oracle Complexity
Watch later
Sally Dong, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Pitfalls of Epistemic Uncertainty Quantification through Loss Minimisation
Watch later
Viktor Bengs, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Comparing radiologists' gaze and saliency maps generated by interpretability methods for chest x-rays
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%