Distilling Knowledge from Reader to Retriever for Question Answering
Mai 3, 2021
Sprecher:innen
Gautier Izacard
Sprecher:in · 0 Follower:innen
Edouard Grave
Sprecher:in · 0 Follower:innen
Über
The task of information retrieval is an important component of many natural language processing systems, such as open domain question answering. While traditional methods were based on hand-crafted features, continuous representations based on neural networks recently obtained competitive results. A challenge of using such methods is to obtain supervised data to train the retriever model, corresponding to pairs of query and support documents. In this paper, we propose a technique to learn retriever models for downstream tasks, inspired by knowledge distillation, and which does not require annotated pairs of query and documents. Our approach leverages attention scores of a reader model, used to solve the task based on retrieved documents, to obtain synthetic labels for the retriever. We evaluate our method on question answering, obtaining state-of-the-art results.The task of information retrieval is an important component of many natural language processing systems, such as open domain question answering. While traditional methods were based on hand-crafted features, continuous representations based on neural networks recently obtained competitive results. A challenge of using such methods is to obtain supervised data to train the retriever model, corresponding to pairs of query and support documents. In this paper, we propose a technique to learn retrie…
Kategorien
KI und Datenwissenschaft
Kategorie · 10,8k Präsentationen
Über ICLR 2021
The International Conference on Learning Representations (ICLR) is the premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence called representation learning, but generally referred to as deep learning. ICLR is globally renowned for presenting and publishing cutting-edge research on all aspects of deep learning used in the fields of artificial intelligence, statistics and data science, as well as important application areas such as machine vision, computational biology, speech recognition, text understanding, gaming, and robotics.
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
Panel #5: Hyo Gweon & Matt Botvinick
Später ansehen
Hyo Gweon, …
Training GANs with Stronger Augmentations via Contrastive Discriminator
Später ansehen
Risk-Averse Offline Reinforcement Learning
Später ansehen
Conjugate Energy-Based Models
Später ansehen
Hao Wu, …
Announcement of the different award winners and closing remarks
Später ansehen
