Couplings for Multinomial Hamiltonian Monte Carlo
Apr 14, 2021
Speakers
Tor Erlend Fjelde
Speaker · 0 followers
Charles Sutton
Speaker · 1 follower
About
Hamiltonian Monte Carlo (HMC) is a popular sampling method in Bayesian inference. Recently, Heng & Jacob (2019) studied Metropolis HMC with couplings for unbiased Monte Carlo estimation, establishing a generic parallelizable sampling scheme. However, in practice a different HMC method, multinomial HMC, is considered as the go-to method, e.g. as part of the no-U-turn sampler (NUTS). In multinomial HMC, proposed states are not limited to the end-points as in Metropolis HMC; instead points along the entire trajectory can be proposed. In this paper, we establish couplings for multinomial HMC, based on optimal transport for the multinomial sampling in the transition. We prove an upper bound for the meeting time -- the time it takes for the coupled chains to meet -- using analysis based on local contractivity. We evaluate our methods using three targets: a high-dimensional Gaussian, Bayesian logistic regression and log-Gaussian Cox point processes. Compared to Heng & Jacob (2019), we find that coupled multinomial HMC is more robust to the choice of step sizes and trajectory lengths and generally attains a smaller meeting time with a better variance-bias trade-off. The robustness allows re-use of existing HMC adaptation methods and, together with other improvements, paves the way for a wider and more practical use of coupled HMC methods.Hamiltonian Monte Carlo (HMC) is a popular sampling method in Bayesian inference. Recently, Heng & Jacob (2019) studied Metropolis HMC with couplings for unbiased Monte Carlo estimation, establishing a generic parallelizable sampling scheme. However, in practice a different HMC method, multinomial HMC, is considered as the go-to method, e.g. as part of the no-U-turn sampler (NUTS). In multinomial HMC, proposed states are not limited to the end-points as in Metropolis HMC; instead points alo…
Organizer
AISTATS 2021
Account · 63 followers
Categories
Mathematics
Category · 2.4k presentations
AI & Data Science
Category · 10.8k presentations
About AISTATS 2021
The 24th International Conference on Artificial Intelligence and Statistics was held virtually from Tuesday, 13 April 2021 to Thursday, 15 April 2021.
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Sparse Algorithms for Markovian Gaussian Processes
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Spectral Tensor Train Parameterization of Deep Learning Layers
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Stability and Differential Privacy of SGD for Pairwise Learning with NonSmooth Loss
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Dynamic Cutset Networks
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Last-Iterate Convergence in No-regret Learning: Constrained Min-Max Optimization for Convex-Concave Landscapes
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Deep Fourier Kernel for Self-Attentive Point Processes
Watch later
Shixiang Zhu, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%