Can LLMs Accurately Assess Human Expert Confidence in Climate Statements?
Dec 15, 2023
Speakers
Romain Lacombe
Speaker · 0 followers
Eddie Dilworth
Speaker · 0 followers
About
The potential for public misinformation fueled by “confidently wrong” Large Language Models (LLMs) is especially salient in the climate science and policy domain. We introduce the ICCS dataset, a novel, curated, expert-labeled NLP dataset consisting of 8094 climate science statements and their associated confidence levels collected from the latest IPCC AR6 reports. Using this dataset, we show that recent LLMs can classify human expert confidence in climate-related statements with reasonable—if limited—accuracy, especially in a few-shot learning setting. Overall, models exhibit consistent and significant overconfidence on low and medium confidence statements. We highlight important implications from our results for climate policy and the use of LLMs in information retrieval systems.The potential for public misinformation fueled by “confidently wrong” Large Language Models (LLMs) is especially salient in the climate science and policy domain. We introduce the ICCS dataset, a novel, curated, expert-labeled NLP dataset consisting of 8094 climate science statements and their associated confidence levels collected from the latest IPCC AR6 reports. Using this dataset, we show that recent LLMs can classify human expert confidence in climate-related statements with reasonable—if l…
Organizer
NeurIPS 2023
Account · 644 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Do Temporal Knowledge Graph Embedding Models Learn or Memorize
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Hierarchical Randomized Smoothing
Watch later
Yan Scholten, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Opening Speech
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
SutraNets: Sub-series Autoregressive Networks for Long-Sequence, Probabilistic Forecasting
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Robust Knowledge Transfer in Tiered Reinforcement Learning
Watch later
Jiawei Huang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
TensorNet: Cartesian Tensor Representations for Efficient Learning of Molecular Potentials
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%