What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods
Nov 28, 2022
Speakers
Julien Colin
Speaker · 0 followers
Thomas Fel
Speaker · 0 followers
Remi Cadene
Speaker · 0 followers
Thomas Serre
Speaker · 0 followers
About
A multitude of explainability methods and theoretical evaluation scores have been proposed. However, it is not yet known: (1) how useful these methods are in real-world scenarios and (2) how well theoretical measures predict the usefulness of these methods for practical use by a human. To fill this gap, we conducted human psychophysics experiments at scale to evaluate the ability of human participants (n=1,150) to leverage representative attribution methods to predicting the decision of different image classifiers. We carried out this analysis on 3 datasets, each dedicated to one of the end-goal of explainability: bias detection, identification of new strategies and the understanding of failure cases. Our results demonstrate that the degree to which individual attribution methods helped human participants better understand a model varied widely across categorization tasks and datasets. We test several hypotheses to understand the reasons for these failures by investigating: (1) the relationship between explanation fidelity and usefulness, (2) the effects of explanation complexity, and (3) the prediction of usefulness using a human visual perceptual similarity proxy.Overall, our results highlight fundamental challenges for the field – suggesting a critical need to refocus the development of explainability tools that go beyond attribution methods. We will make the code of our framework and results available to ease the systematic evaluation of novel explainability methods and to support the development of theoretical measures more aligned with human.A multitude of explainability methods and theoretical evaluation scores have been proposed. However, it is not yet known: (1) how useful these methods are in real-world scenarios and (2) how well theoretical measures predict the usefulness of these methods for practical use by a human. To fill this gap, we conducted human psychophysics experiments at scale to evaluate the ability of human participants (n=1,150) to leverage representative attribution methods to predicting the decision of differen…
Organizer
NeurIPS 2022
Account · 952 followers
Like the format? Trust SlidesLive to capture your next event!
Professional recording and live streaming, delivered globally.
Sharing
Recommended Videos
Presentations on similar topic, category or speaker
When does return-conditioned supervised learning work for offline RL?
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Fast Bayesian Inference with Batch Bayesian Quadrature via Kernel Recombination
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Inductive Logical Query Answering in Knowledge Graphs
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
TGEA 2.0: A Large-Scale Diagnostically Annotated Dataset with Benchmark Tasks for Text Generation of Pretrained Language Models
Watch later
Huibin Ge, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Featured Papers Panels 4B
Watch later
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Representation of missense variants for predicting modes of action
Watch later
Guojie Zhong, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%