Diplomat: A Dialogue Dataset for Situated PragMATic Reasoning
Dez 10, 2023
Sprecher:innen
Song-Chun Zhu
Sprecher:in · 0 Follower:innen
Zilong Zheng
Sprecher:in · 0 Follower:innen
Über
Pragmatic reasoning aims at resolving implicit meanings that commonly occur in real-life and is crucial for building communicative social agents.We introduce a new benchmark, Diplomat, aiming at a unified paradigm for pragmatic reasoning and situated conversational understanding. Compared with previous works that treat different figurative expressions (eg, metaphor, sarcasm) as individual tasks, Diplomat provides a unified understanding towards general pragmatic understanding. Our dataset is created using Amazon Mechanical Turk (AMT), resulting in 4,177 multi-turn dialogues. In company with the dataset, we propose two tasks: Pragmatic Identification and Reasoning and Conversational Question Answering. Experimental results with state-of-the-art (SOTA) neural architectures demonstrate that: 1) large language models show poor performances in this subjective topic. 2) Context understanding is a crucial factor in building benign human-machine interaction. 3) Current models defect in the application of pragmatic reasoning. As a result, we call on more attention to improve the ability of context understanding, reasoning and implied meaning modeling.Pragmatic reasoning aims at resolving implicit meanings that commonly occur in real-life and is crucial for building communicative social agents.We introduce a new benchmark, Diplomat, aiming at a unified paradigm for pragmatic reasoning and situated conversational understanding. Compared with previous works that treat different figurative expressions (eg, metaphor, sarcasm) as individual tasks, Diplomat provides a unified understanding towards general pragmatic understanding. Our dataset is cre…
Organisator
NeurIPS 2023
Konto · 648 Follower:innen
Gefällt euch das Format? Vertraut auf SlidesLive, um euer nächstes Event festzuhalten!
Professionelle Aufzeichnung und Livestreaming – weltweit.
Freigeben
Empfohlene Videos
Präsentationen, deren Thema, Kategorie oder Sprecher:in ähnlich sind
Data-Driven Network Neuroscience: On Data Collection and Benchmark
Später ansehen
Jiaxing Xu, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Posterior Contraction Rates for Matérn Gaussian Processes on Riemannian Manifolds
Später ansehen
Paul Rosa, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Voyager: An Open-Ended Embodied Agent with Large Language Models
Später ansehen
Guanzhi Wang, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Accelerating Monte Carlo Tree Search with Probability Tree State Abstraction
Später ansehen
Yangqing Fu, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%
NAVI: Category-Agnostic Image Collections with High-Quality 3D Shape and Pose Annotations
Später ansehen
Ewigspeicher-Fortschrittswert: 0 = 0.0%
Sample Efficient Reinforcement Learning in Mixed Systems through Augmented Samples and Its Applications to Queueing Networks
Später ansehen
Honghao Wei, …
Ewigspeicher-Fortschrittswert: 0 = 0.0%