Predictive and Causal Implications of using Shapley Value for Model Interpretation

Research output: Contribution to journalConference articlepeer-review

26 Scopus citations

Abstract

Shapley value is a concept from game theory. Recently, it has been used for explaining complex models produced by machine learning techniques. Although the mathematical definition of Shapley value is straight-forward, the implication of using it as a model interpretation tool is yet to be described. In the current paper, we analyzed Shapley value in the Bayesian network framework. We established the relationship between Shapley value and conditional independence, a key concept in both predictive and causal modeling. Our results indicate that, eliminating a variable with high Shapley value from a model do not necessarily impair predictive performance, whereas eliminating a variable with low Shapley value from a model could impair performance. Therefore, using Shapley value for feature selection do not result in the most parsimonious and predictively optimal model in the general case. More importantly, Shapley value of a variable do not reflect their causal relationship with the target of interest.

Original languageEnglish (US)
Pages (from-to)23-38
Number of pages16
JournalProceedings of Machine Learning Research
Volume127
StatePublished - 2020
Event2020 ACM SIGKDD workshop on Causal Discovery, CD 2020 - Virtual, Online, United States
Duration: Aug 24 2020 → …

Bibliographical note

Publisher Copyright:
© 2020 Sisi Ma and Roshan Tourani.

Keywords

  • Causal Bayesian Networks
  • Model Explanation
  • Model Interpretability
  • Predictive Models
  • Shapley Value

Fingerprint

Dive into the research topics of 'Predictive and Causal Implications of using Shapley Value for Model Interpretation'. Together they form a unique fingerprint.

Cite this