Nov 28, 2022
Speaker · 0 followers
Speaker · 0 followers
Speaker · 0 followers
Speaker · 0 followers
A multitude of explainability methods and theoretical evaluation scores have been proposed. However, it is not yet known: (1) how useful these methods are in real-world scenarios and (2) how well theoretical measures predict the usefulness of these methods for practical use by a human. To fill this gap, we conducted human psychophysics experiments at scale to evaluate the ability of human participants (n=1,150) to leverage representative attribution methods to predicting the decision of different image classifiers. We carried out this analysis on 3 datasets, each dedicated to one of the end-goal of explainability: bias detection, identification of new strategies and the understanding of failure cases. Our results demonstrate that the degree to which individual attribution methods helped human participants better understand a model varied widely across categorization tasks and datasets. We test several hypotheses to understand the reasons for these failures by investigating: (1) the relationship between explanation fidelity and usefulness, (2) the effects of explanation complexity, and (3) the prediction of usefulness using a human visual perceptual similarity proxy.Overall, our results highlight fundamental challenges for the field – suggesting a critical need to refocus the development of explainability tools that go beyond attribution methods. We will make the code of our framework and results available to ease the systematic evaluation of novel explainability methods and to support the development of theoretical measures more aligned with human.A multitude of explainability methods and theoretical evaluation scores have been proposed. However, it is not yet known: (1) how useful these methods are in real-world scenarios and (2) how well theoretical measures predict the usefulness of these methods for practical use by a human. To fill this gap, we conducted human psychophysics experiments at scale to evaluate the ability of human participants (n=1,150) to leverage representative attribution methods to predicting the decision of differen…
Account · 952 followers
Professional recording and live streaming, delivered globally.
Presentations on similar topic, category or speaker
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Huibin Ge, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%
Guojie Zhong, …
Total of 0 viewers voted for saving the presentation to eternal vault which is 0.0%