AtmSeer: Increasing transparency and controllability in automated machine learning

Qianwen Wang; Yao Ming; Zhihua Jin; Qiaomu Shen; Dongyu Liu; Micah J. Smith; Kalyan Veeramachaneni; Huamin Qu

doi:10.1145/3290605.3300911

AtmSeer: Increasing transparency and controllability in automated machine learning

Qianwen Wang, Yao Ming, Zhihua Jin, Qiaomu Shen, Dongyu Liu, Micah J. Smith, Kalyan Veeramachaneni, Huamin Qu

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

75 Scopus citations

Abstract

To relieve the pain of manually selecting machine learning algorithms and tuning hyperparameters, automated machine learning (AutoML) methods have been developed to automatically search for good models. Due to the huge model search space, it is impossible to try all models. Users tend to distrust automatic results and increase the search budget as much as they can, thereby undermining the efficiency of AutoML. To address these issues, we design and implement ATMSeer, an interactive visualization tool that supports users in refining the search space of AutoML and analyzing the results. To guide the design of ATMSeer, we derive a workflow of using AutoML based on interviews with machine learning experts. A multi-granularity visualization is proposed to enable users to monitor the AutoML process, analyze the searched models, and refine the search space in real time. We demonstrate the utility and usability of ATMSeer through two case studies, expert interviews, and a user study with 13 end users.

Original language	English (US)
Title of host publication	CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
Publisher	Association for Computing Machinery
ISBN (Electronic)	9781450359702
DOIs	https://doi.org/10.1145/3290605.3300911
State	Published - May 2 2019
Externally published	Yes
Event	2019 CHI Conference on Human Factors in Computing Systems, CHI 2019 - Glasgow, United Kingdom Duration: May 4 2019 → May 9 2019

Publication series

Name	Conference on Human Factors in Computing Systems - Proceedings

Conference

Conference	2019 CHI Conference on Human Factors in Computing Systems, CHI 2019
Country/Territory	United Kingdom
City	Glasgow
Period	5/4/19 → 5/9/19

Bibliographical note

Publisher Copyright:
© 2019 Copyright held by the owner/author(s).

Keywords

Automated machine learning
Data visualization

Publisher link

10.1145/3290605.3300911

Find It

Find It at U of M Libraries

Cite this

Wang, Q., Ming, Y., Jin, Z., Shen, Q., Liu, D., Smith, M. J., Veeramachaneni, K., & Qu, H. (2019). AtmSeer: Increasing transparency and controllability in automated machine learning. In CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Conference on Human Factors in Computing Systems - Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3290605.3300911

AtmSeer: Increasing transparency and controllability in automated machine learning. / Wang, Qianwen; Ming, Yao; Jin, Zhihua et al.
CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, 2019. (Conference on Human Factors in Computing Systems - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Wang, Q, Ming, Y, Jin, Z, Shen, Q, Liu, D, Smith, MJ, Veeramachaneni, K & Qu, H 2019, AtmSeer: Increasing transparency and controllability in automated machine learning. in CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Conference on Human Factors in Computing Systems - Proceedings, Association for Computing Machinery, 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019, Glasgow, United Kingdom, 5/4/19. https://doi.org/10.1145/3290605.3300911

@inproceedings{fdc80e3f623e4ffba03ddecfc3dc995e,

title = "AtmSeer: Increasing transparency and controllability in automated machine learning",

abstract = "To relieve the pain of manually selecting machine learning algorithms and tuning hyperparameters, automated machine learning (AutoML) methods have been developed to automatically search for good models. Due to the huge model search space, it is impossible to try all models. Users tend to distrust automatic results and increase the search budget as much as they can, thereby undermining the efficiency of AutoML. To address these issues, we design and implement ATMSeer, an interactive visualization tool that supports users in refining the search space of AutoML and analyzing the results. To guide the design of ATMSeer, we derive a workflow of using AutoML based on interviews with machine learning experts. A multi-granularity visualization is proposed to enable users to monitor the AutoML process, analyze the searched models, and refine the search space in real time. We demonstrate the utility and usability of ATMSeer through two case studies, expert interviews, and a user study with 13 end users.",

keywords = "Automated machine learning, Data visualization",

author = "Qianwen Wang and Yao Ming and Zhihua Jin and Qiaomu Shen and Dongyu Liu and Smith, {Micah J.} and Kalyan Veeramachaneni and Huamin Qu",

note = "Publisher Copyright: {\textcopyright} 2019 Copyright held by the owner/author(s).; 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019 ; Conference date: 04-05-2019 Through 09-05-2019",

year = "2019",

month = may,

day = "2",

doi = "10.1145/3290605.3300911",

language = "English (US)",

series = "Conference on Human Factors in Computing Systems - Proceedings",

publisher = "Association for Computing Machinery",

booktitle = "CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems",

}

TY - GEN

T1 - AtmSeer

T2 - 2019 CHI Conference on Human Factors in Computing Systems, CHI 2019

AU - Wang, Qianwen

AU - Ming, Yao

AU - Jin, Zhihua

AU - Shen, Qiaomu

AU - Liu, Dongyu

AU - Smith, Micah J.

AU - Veeramachaneni, Kalyan

AU - Qu, Huamin

PY - 2019/5/2

Y1 - 2019/5/2

N2 - To relieve the pain of manually selecting machine learning algorithms and tuning hyperparameters, automated machine learning (AutoML) methods have been developed to automatically search for good models. Due to the huge model search space, it is impossible to try all models. Users tend to distrust automatic results and increase the search budget as much as they can, thereby undermining the efficiency of AutoML. To address these issues, we design and implement ATMSeer, an interactive visualization tool that supports users in refining the search space of AutoML and analyzing the results. To guide the design of ATMSeer, we derive a workflow of using AutoML based on interviews with machine learning experts. A multi-granularity visualization is proposed to enable users to monitor the AutoML process, analyze the searched models, and refine the search space in real time. We demonstrate the utility and usability of ATMSeer through two case studies, expert interviews, and a user study with 13 end users.

AB - To relieve the pain of manually selecting machine learning algorithms and tuning hyperparameters, automated machine learning (AutoML) methods have been developed to automatically search for good models. Due to the huge model search space, it is impossible to try all models. Users tend to distrust automatic results and increase the search budget as much as they can, thereby undermining the efficiency of AutoML. To address these issues, we design and implement ATMSeer, an interactive visualization tool that supports users in refining the search space of AutoML and analyzing the results. To guide the design of ATMSeer, we derive a workflow of using AutoML based on interviews with machine learning experts. A multi-granularity visualization is proposed to enable users to monitor the AutoML process, analyze the searched models, and refine the search space in real time. We demonstrate the utility and usability of ATMSeer through two case studies, expert interviews, and a user study with 13 end users.

KW - Automated machine learning

KW - Data visualization

UR - http://www.scopus.com/inward/record.url?scp=85067625599&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85067625599&partnerID=8YFLogxK

U2 - 10.1145/3290605.3300911

DO - 10.1145/3290605.3300911

M3 - Conference contribution

AN - SCOPUS:85067625599

T3 - Conference on Human Factors in Computing Systems - Proceedings

BT - CHI 2019 - Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

Y2 - 4 May 2019 through 9 May 2019

ER -

AtmSeer: Increasing transparency and controllability in automated machine learning

Abstract

Publication series

Conference

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this