Next
Livestream will start soon!
Livestream has already ended.
Presentation has not been recorded yet!
  • title: Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning
      0:00 / 0:00
      • Report Issue
      • Settings
      • Playlists
      • Bookmarks
      • Subtitles Off
      • Playback rate
      • Quality
      • Settings
      • Debug information
      • Server sl-yoda-v2-stream-010-alpha.b-cdn.net
      • Subtitles size Medium
      • Bookmarks
      • Server
      • sl-yoda-v2-stream-010-alpha.b-cdn.net
      • sl-yoda-v2-stream-010-beta.b-cdn.net
      • 1759419103.rsc.cdn77.org
      • 1016618226.rsc.cdn77.org
      • Subtitles
      • Off
      • English
      • Playback rate
      • Quality
      • Subtitles size
      • Large
      • Medium
      • Small
      • Mode
      • Video Slideshow
      • Audio Slideshow
      • Slideshow
      • Video
      My playlists
        Bookmarks
          00:00:00
            Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning
            • Settings
            • Sync diff
            • Quality
            • Settings
            • Server
            • Quality
            • Server

            Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning

            Nov 28, 2022

            Speakers

            WL

            Weixin Liang

            Speaker · 1 follower

            YZ

            Yuhui Zhang

            Speaker · 0 followers

            YK

            Yongchan Kwon

            Speaker · 0 followers

            About

            We present modality gap, an intriguing geometric phenomenon of the representation space of multi-modal models. Specifically, we show that different data modalities (e.g. images and text) are embedded at arm's length in their shared representation in multi-modal models such as CLIP. Our systematic analysis demonstrates that this gap is caused by a combination of model initialization and contrastive learning optimization. In model initialization, we show empirically and theoretically that the repr…

            Organizer

            N2
            N2

            NeurIPS 2022

            Account · 952 followers

            Like the format? Trust SlidesLive to capture your next event!

            Professional recording and live streaming, delivered globally.

            Sharing

            Recommended Videos

            Presentations on similar topic, category or speaker

            GNM: A General Navigation Model to Drive Any Robot
            05:04

            GNM: A General Navigation Model to Drive Any Robot

            Dhruv Shah, …

            N2
            N2
            NeurIPS 2022 2 years ago

            Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%

            First is Better Than Last for Language Data Influence
            04:58

            First is Better Than Last for Language Data Influence

            Chih-Kuan Yeh, …

            N2
            N2
            NeurIPS 2022 2 years ago

            Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%

            Social-Inverse: Inverse Decision-making of Social Contagion Management with Task Migrations
            05:02

            Social-Inverse: Inverse Decision-making of Social Contagion Management with Task Migrations

            Guangmo Tong

            N2
            N2
            NeurIPS 2022 2 years ago

            Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%

            Active Learning with Neural Networks: Insights from Nonparametric Statistics
            04:51

            Active Learning with Neural Networks: Insights from Nonparametric Statistics

            Yinglun Zhu, …

            N2
            N2
            NeurIPS 2022 2 years ago

            Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%

            Class-Aware Generative AdverSarial Transformers for Medical Image Segmentation
            05:12

            Class-Aware Generative AdverSarial Transformers for Medical Image Segmentation

            Chenyu You

            N2
            N2
            NeurIPS 2022 2 years ago

            Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%

            Learning Structure from the Ground up—Hierarchical Representation Learning by Chunking
            00:58

            Learning Structure from the Ground up—Hierarchical Representation Learning by Chunking

            Shuchen Wu, …

            N2
            N2
            NeurIPS 2022 2 years ago

            Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%

            Interested in talks like this? Follow NeurIPS 2022