What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods
Nov 28, 2022
Speakers
Julien Colin
Speaker · 0 followers
Thomas Fel
Speaker · 0 followers
Remi Cadene
Speaker · 0 followers
Thomas Serre
Speaker · 0 followers
About
A multitude of explainability methods and theoretical evaluation scores have been proposed. However, it is not yet known: (1) how useful these methods are in real-world scenarios and (2) how well theoretical measures predict the usefulness of these methods for practical use by a human. To fill this gap, we conducted human psychophysics experiments at scale to evaluate the ability of human participants (n=1,150) to leverage representative attribution methods to predicting the decision of different image classifiers. We carried out this analysis on 3 datasets, each dedicated to one of the end-goal of explainability: bias detection, identification of new strategies and the understanding of failure cases. Our results demonstrate that the degree to which individual attribution methods helped human participants better understand a model varied widely across categorization tasks and datasets. We test several hypotheses to understand the reasons for these failures by investigating: (1) the relationship between explanation fidelity and usefulness, (2) the effects of explanation complexity, and (3) the prediction of usefulness using a human visual perceptual similarity proxy.Overall, our results highlight fundamental challenges for the field – suggesting a critical need to refocus the development of explainability tools that go beyond attribution methods. We will make the code of our framework and results available to ease the systematic evaluation of novel explainability methods and to support the development of theoretical measures more aligned with human.A multitude of explainability methods and theoretical evaluation scores have been proposed. However, it is not yet known: (1) how useful these methods are in real-world scenarios and (2) how well theoretical measures predict the usefulness of these methods for practical use by a human. To fill this gap, we conducted human psychophysics experiments at scale to evaluate the ability of human participants (n=1,150) to leverage representative attribution methods to predicting the decision of differen…
Organizer
NeurIPS 2022
Account · 952 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
Neuron with Steady Response Leads to Better Generalization
Watch later
Haitao Mao, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
VeriDark: A Large-Scale Benchmark for Authorship Verification on the Dark Web
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
CLEVRER-Humans: Describing Physical and Causal Events the Human Way
Watch later
Xuelin Yang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Deep Active Learning by Leveraging Training Dynamics
Watch later
Haonan Wang, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge
Watch later
Linxi Fan, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Estimating Corporate Scope 1 Emissions Using Tree-Based Machine Learning Methods
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%