Efficient Queries Transformer Neural Processes
Dec 2, 2022
Speakers
Hossein Hajimirsadeghi
Speaker · 0 followers
Yoshua Bengio
Speaker · 266 followers
Mohamed Osama Ahmed
Speaker · 0 followers
About
Neural Processes (NPs) are popular methods in meta-learning that can estimate predictive uncertainty on target datapoints by conditioning on a context dataset. Previous state-of-the-art method Transformer Neural Processes (TNPs) achieve strong performance but require quadratic computation with respect to the number of context datapoints per query, limiting its applications. Conversely, existing sub-quadratic NP variants perform significantly worse than that of TNPs. Tackling this issue, we propose Efficient Queries Transformer Neural Processes (EQTNPs), a more computationally efficient NP variant. The model encodes the context dataset into a set of vectors that is linear in the number of context datapoints. When making predictions, the model retrieves higher-order information from the context dataset via multiple cross-attention mechanisms on the context vectors. We empirically show that EQTNPs achieve results competitive with the state-of-the-art.Neural Processes (NPs) are popular methods in meta-learning that can estimate predictive uncertainty on target datapoints by conditioning on a context dataset. Previous state-of-the-art method Transformer Neural Processes (TNPs) achieve strong performance but require quadratic computation with respect to the number of context datapoints per query, limiting its applications. Conversely, existing sub-quadratic NP variants perform significantly worse than that of TNPs. Tackling this issue, we propo…
Organizer
NeurIPS 2022
Account · 953 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Estimating the Arc Length of the Optimal ROC Curve and Lower Bounding the Maximal AUC
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Wasserstein Logistic Regression with Mixed Features
Watch later
Aras Selvi, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Moment Distributionally Robust Tree Structured Prediction
Watch later
Yeshu Li, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
MAgNet: Mesh Agnostic Neural PDE Solvers
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Connections between Large Sequence Modeling and Reinforcement Learning
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
The promises and pitfalls of CVAR
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%