Can LLMs Accurately Assess Human Expert Confidence in Climate Statements?

Dec 15, 2023



The potential for public misinformation fueled by “confidently wrong” Large Language Models (LLMs) is especially salient in the climate science and policy domain. We introduce the ICCS dataset, a novel, curated, expert-labeled NLP dataset consisting of 8094 climate science statements and their associated confidence levels collected from the latest IPCC AR6 reports. Using this dataset, we show that recent LLMs can classify human expert confidence in climate-related statements with reasonable—if limited—accuracy, especially in a few-shot learning setting. Overall, models exhibit consistent and significant overconfidence on low and medium confidence statements. We highlight important implications from our results for climate policy and the use of LLMs in information retrieval systems.


Store presentation

Should this presentation be stored for 1000 years?

How do we store presentations

Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%


Recommended Videos

Presentations on similar topic, category or speaker

Interested in talks like this? Follow NeurIPS 2023