Diplomat: A Dialogue Dataset for Situated PragMATic Reasoning
Dec 10, 2023
Speakers
Song-Chun Zhu
Speaker · 0 followers
Zilong Zheng
Speaker · 0 followers
About
Pragmatic reasoning aims at resolving implicit meanings that commonly occur in real-life and is crucial for building communicative social agents.We introduce a new benchmark, Diplomat, aiming at a unified paradigm for pragmatic reasoning and situated conversational understanding. Compared with previous works that treat different figurative expressions (eg, metaphor, sarcasm) as individual tasks, Diplomat provides a unified understanding towards general pragmatic understanding. Our dataset is created using Amazon Mechanical Turk (AMT), resulting in 4,177 multi-turn dialogues. In company with the dataset, we propose two tasks: Pragmatic Identification and Reasoning and Conversational Question Answering. Experimental results with state-of-the-art (SOTA) neural architectures demonstrate that: 1) large language models show poor performances in this subjective topic. 2) Context understanding is a crucial factor in building benign human-machine interaction. 3) Current models defect in the application of pragmatic reasoning. As a result, we call on more attention to improve the ability of context understanding, reasoning and implied meaning modeling.Pragmatic reasoning aims at resolving implicit meanings that commonly occur in real-life and is crucial for building communicative social agents.We introduce a new benchmark, Diplomat, aiming at a unified paradigm for pragmatic reasoning and situated conversational understanding. Compared with previous works that treat different figurative expressions (eg, metaphor, sarcasm) as individual tasks, Diplomat provides a unified understanding towards general pragmatic understanding. Our dataset is cre…
Organizer
NeurIPS 2023
Account · 645 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Enhancing Knowledge Transfer for Task Incremental Learning with Data-free Subnetwork
Watch later
Qiang Gao, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Identifying Causal Mechanism Shifts among Nonlinear Additive Noise Models
Watch later
Tianyu Chen, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Training Neural Networks is NP-Hard in Fixed Dimension
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Quantification of Uncertainty with Adversarial Models
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
On Slicing Optimality for Mutual Information
Watch later
Ammar Fayad, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Active Learning for Semantic Segmentation with Multi-class Label Query
Watch later
Sehyun Hwang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%