ContraBAR: Contrastive Bayes-Adaptive Deep RL
Jul 24, 2023
Speakers
Era Choshen
Speaker · 0 followers
Aviv Tamar
Speaker · 2 followers
About
In meta reinforcement learning (meta RL), an agent seeks a Bayes-optimal policy – the optimal policy when facing an unknown task that is sampled from some known task distribution. Previous approaches tackled this problem by inferring a belief over task parameters, using variational inference methods. Motivated by recent successes of contrastive learning approaches in RL, such as contrastive predictive coding (CPC), we investigate whether contrastive methods can be used for learning Bayes-optimal behavior. We begin by proving that representations learned by CPC are indeed sufficient for Bayes optimality. Based on this observation, we propose a simple meta RL algorithm that uses CPC in lieu of variational belief inference. Our method, ContraBAR, achieves comparable performance to state-of-the-art in domains with state-based observation and circumvents the computational toll of future observation reconstruction, enabling learning in domains with image-based observations. It can also be combined with image augmentations for domain randomization and used seamlessly in both online and offline meta RL settings.In meta reinforcement learning (meta RL), an agent seeks a Bayes-optimal policy – the optimal policy when facing an unknown task that is sampled from some known task distribution. Previous approaches tackled this problem by inferring a belief over task parameters, using variational inference methods. Motivated by recent successes of contrastive learning approaches in RL, such as contrastive predictive coding (CPC), we investigate whether contrastive methods can be used for learning Bayes-optimal…
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Provably Efficient Offline Reinforcement Learning with Perturbed Data Sources
Watch later
Deep Anomaly Detection under Labeling Budget Constraints
Watch later
Aodong Li, …
Provably Efficient Representation Learning with Tractable Planning in Low-Rank POMDP
Watch later
Jiacheng Guo, …
Are Equivariant Equilibrium Approximators Beneficial?
Watch later
Zhijian Duan, …
