From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidence

David Alvarez-Melis, Harmanpreet Kaur, Hal Daume, Hanna Wallach, Jennifer Wortman Vaughan

Research output: Chapter in Book/Report/Conference proceedingConference contribution

12 Scopus citations

Abstract

We take inspiration from the study of human explanation to inform the design and evaluation of interpretability methods in machine learning. First, we survey the literature on human explanation in philosophy, cognitive science, and the social sciences, and propose a list of design principles for machinegenerated explanations that are meaningful to humans. Using the concept of weight of evidence from information theory, we develop a method for generating explanations that adhere to these principles. We show that this method can be adapted to handle high-dimensional, multi-class settings, yielding a flexible framework for generating explanations. We demonstrate that these explanations can be estimated accurately from finite samples and are robust to small perturbations of the inputs. We also evaluate our method through a qualitative user study with machine learning practitioners, where we observe that the resulting explanations are usable despite some participants struggling with background concepts like prior class probabilities. Finally, we conclude by surfacing design implications for interpretability tools in general.

Original languageEnglish (US)
Title of host publicationHCOMP 2021 - Proceedings of the 9th AAAI Conference on Human Computation and Crowdsourcing
EditorsEce Kamar, Kurt Luther
PublisherAssociation for the Advancement of Artificial Intelligence
Pages35-47
Number of pages13
ISBN (Print)9781577358725
DOIs
StatePublished - 2021
Externally publishedYes
Event9th AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2021 - Virtual, Online
Duration: Nov 14 2021Nov 18 2021

Publication series

NameProceedings of the AAAI Conference on Human Computation and Crowdsourcing
Volume9
ISSN (Print)2769-1330
ISSN (Electronic)2769-1349

Conference

Conference9th AAAI Conference on Human Computation and Crowdsourcing, HCOMP 2021
CityVirtual, Online
Period11/14/2111/18/21

Bibliographical note

Publisher Copyright:
© 2021, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.

Fingerprint

Dive into the research topics of 'From Human Explanation to Model Interpretability: A Framework Based on Weight of Evidence'. Together they form a unique fingerprint.

Cite this