Next
Livestream will start soon!
Livestream has already ended.
Presentation has not been recorded yet!
  • title: Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning
      0:00 / 0:00
      • Report Issue
      • Settings
      • Playlists
      • Bookmarks
      • Subtitles Off
      • Playback rate
      • Quality
      • Settings
      • Debug information
      • Server sl-yoda-v2-stream-010-alpha.b-cdn.net
      • Subtitles size Medium
      • Bookmarks
      • Server
      • sl-yoda-v2-stream-010-alpha.b-cdn.net
      • sl-yoda-v2-stream-010-beta.b-cdn.net
      • 1759419103.rsc.cdn77.org
      • 1016618226.rsc.cdn77.org
      • Subtitles
      • Off
      • English
      • Playback rate
      • Quality
      • Subtitles size
      • Large
      • Medium
      • Small
      • Mode
      • Video Slideshow
      • Audio Slideshow
      • Slideshow
      • Video
      My playlists
        Bookmarks
          00:00:00
            Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning
            • Settings
            • Sync diff
            • Quality
            • Settings
            • Server
            • Quality
            • Server

            Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning

            Nov 28, 2022

            Speakers

            WL

            Weixin Liang

            Speaker · 1 follower

            YZ

            Yuhui Zhang

            Speaker · 0 followers

            YK

            Yongchan Kwon

            Speaker · 0 followers

            About

            We present modality gap, an intriguing geometric phenomenon of the representation space of multi-modal models. Specifically, we show that different data modalities (e.g. images and text) are embedded at arm's length in their shared representation in multi-modal models such as CLIP. Our systematic analysis demonstrates that this gap is caused by a combination of model initialization and contrastive learning optimization. In model initialization, we show empirically and theoretically that the repr…

            Organizer

            N2
            N2

            NeurIPS 2022

            Account · 952 followers

            Like the format? Trust SlidesLive to capture your next event!

            Professional recording and live streaming, delivered globally.

            Sharing

            Recommended Videos

            Presentations on similar topic, category or speaker

            Toward Causal-Aware RL: State-Wise Action-Refined Temporal Difference
            02:04

            Toward Causal-Aware RL: State-Wise Action-Refined Temporal Difference

            Hao Sun, …

            N2
            N2
            NeurIPS 2022 2 years ago

            Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%

            Explaining Graph Neural Networks with Structure-Aware Cooperative Games
            05:04

            Explaining Graph Neural Networks with Structure-Aware Cooperative Games

            Shichang Zhang, …

            N2
            N2
            NeurIPS 2022 2 years ago

            Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%

            Beyond the Best:  Distribution Functional Estimation in Infinite-Armed Bandits
            05:08

            Beyond the Best: Distribution Functional Estimation in Infinite-Armed Bandits

            Yifei Wang, …

            N2
            N2
            NeurIPS 2022 2 years ago

            Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%

            HAPI: A Large-scale Longitudinal Dataset of Commercial ML API Predictions
            05:10

            HAPI: A Large-scale Longitudinal Dataset of Commercial ML API Predictions

            Lingjiao Chen, …

            N2
            N2
            NeurIPS 2022 2 years ago

            Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%

            Fair Rank Aggregation
            04:59

            Fair Rank Aggregation

            Diptarka Chakraborty, …

            N2
            N2
            NeurIPS 2022 2 years ago

            Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%

            AV2.0: Learning to Drive at a Global Scale
            31:34

            AV2.0: Learning to Drive at a Global Scale

            Alex Kendall

            N2
            N2
            NeurIPS 2022 2 years ago

            Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%

            Interested in talks like this? Follow NeurIPS 2022