Gesture and Voice Commands to Interact With AR Windshield Display in Automated Vehicle: A Remote Elicitation Study

Nabil Al Nahin Ch; Diana Tosca; Tyanna Crump; Alberta Ansah; Andrew Kun; Orit Shaer

doi:10.1145/3543174.3545257

Gesture and Voice Commands to Interact With AR Windshield Display in Automated Vehicle: A Remote Elicitation Study

Nabil Al Nahin Ch
, Diana Tosca
, Tyanna Crump
, Alberta Ansah
, Andrew Kun
, Orit Shaer

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

19 Scopus citations

Abstract

Augmented reality (AR) windshield display (WSD) offers promising ways to engage in non-driving tasks in automated vehicles. Previous studies explored different ways WSD can be used to present driving and other tasks-related information and how that can affect driving performance, user experience, and performance in secondary tasks. Our goal for this study was to examine how drivers expect to use gesture and voice commands for interacting with WSD for performing complex, multi-step personal and work-related tasks in an automated vehicle. In this remote unmoderated online elicitation study, 31 participants proposed 373 gestures and 373 voice commands for performing 24 tasks. We analyzed the elicited interactions, their preferred modality of interaction, and the reasons behind this preference. Lastly, we discuss our results and their implications for designing AR WSD in automated vehicles.

Original language	English (US)
Title of host publication	Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022
Publisher	Association for Computing Machinery, Inc
Pages	171-182
Number of pages	12
ISBN (Electronic)	9781450394154
DOIs	https://doi.org/10.1145/3543174.3545257
State	Published - Sep 17 2022
Externally published	Yes
Event	14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022 - Seoul, Korea, Republic of Duration: Sep 17 2022 → Sep 20 2022

Publication series

Name	Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022

Conference

Conference	14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022
Country/Territory	Korea, Republic of
City	Seoul
Period	9/17/22 → 9/20/22

Bibliographical note

Publisher Copyright:
© 2022 ACM.

Keywords

automated driving
gesture
head-up display
voice commands
Windshield display

Publisher link

10.1145/3543174.3545257

Find It

Find It at U of M Libraries

Cite this

Ch, N. A. N., Tosca, D., Crump, T., Ansah, A., Kun, A., & Shaer, O. (2022). Gesture and Voice Commands to Interact With AR Windshield Display in Automated Vehicle: A Remote Elicitation Study. In Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022 (pp. 171-182). (Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022). Association for Computing Machinery, Inc. https://doi.org/10.1145/3543174.3545257

Gesture and Voice Commands to Interact With AR Windshield Display in Automated Vehicle: A Remote Elicitation Study. / Ch, Nabil Al Nahin; Tosca, Diana; Crump, Tyanna et al.
Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022. Association for Computing Machinery, Inc, 2022. p. 171-182 (Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Ch, NAN, Tosca, D, Crump, T, Ansah, A, Kun, A & Shaer, O 2022, Gesture and Voice Commands to Interact With AR Windshield Display in Automated Vehicle: A Remote Elicitation Study. in Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022. Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022, Association for Computing Machinery, Inc, pp. 171-182, 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022, Seoul, Korea, Republic of, 9/17/22. https://doi.org/10.1145/3543174.3545257

Ch NAN, Tosca D, Crump T, Ansah A, Kun A, Shaer O. Gesture and Voice Commands to Interact With AR Windshield Display in Automated Vehicle: A Remote Elicitation Study. In Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022. Association for Computing Machinery, Inc. 2022. p. 171-182. (Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022). doi: 10.1145/3543174.3545257

Ch, Nabil Al Nahin ; Tosca, Diana ; Crump, Tyanna et al. / Gesture and Voice Commands to Interact With AR Windshield Display in Automated Vehicle : A Remote Elicitation Study. Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022. Association for Computing Machinery, Inc, 2022. pp. 171-182 (Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022).

@inproceedings{b93c3873a5014020abc76957f1a06b57,

title = "Gesture and Voice Commands to Interact With AR Windshield Display in Automated Vehicle: A Remote Elicitation Study",

abstract = "Augmented reality (AR) windshield display (WSD) offers promising ways to engage in non-driving tasks in automated vehicles. Previous studies explored different ways WSD can be used to present driving and other tasks-related information and how that can affect driving performance, user experience, and performance in secondary tasks. Our goal for this study was to examine how drivers expect to use gesture and voice commands for interacting with WSD for performing complex, multi-step personal and work-related tasks in an automated vehicle. In this remote unmoderated online elicitation study, 31 participants proposed 373 gestures and 373 voice commands for performing 24 tasks. We analyzed the elicited interactions, their preferred modality of interaction, and the reasons behind this preference. Lastly, we discuss our results and their implications for designing AR WSD in automated vehicles.",

keywords = "automated driving, gesture, head-up display, voice commands, Windshield display",

author = "Ch, \{Nabil Al Nahin\} and Diana Tosca and Tyanna Crump and Alberta Ansah and Andrew Kun and Orit Shaer",

note = "Publisher Copyright: {\textcopyright} 2022 ACM.; 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022 ; Conference date: 17-09-2022 Through 20-09-2022",

year = "2022",

month = sep,

day = "17",

doi = "10.1145/3543174.3545257",

language = "English (US)",

series = "Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022",

publisher = "Association for Computing Machinery, Inc",

pages = "171--182",

booktitle = "Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022",

}

TY - GEN

T1 - Gesture and Voice Commands to Interact With AR Windshield Display in Automated Vehicle

T2 - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022

AU - Ch, Nabil Al Nahin

AU - Tosca, Diana

AU - Crump, Tyanna

AU - Ansah, Alberta

AU - Kun, Andrew

AU - Shaer, Orit

PY - 2022/9/17

Y1 - 2022/9/17

N2 - Augmented reality (AR) windshield display (WSD) offers promising ways to engage in non-driving tasks in automated vehicles. Previous studies explored different ways WSD can be used to present driving and other tasks-related information and how that can affect driving performance, user experience, and performance in secondary tasks. Our goal for this study was to examine how drivers expect to use gesture and voice commands for interacting with WSD for performing complex, multi-step personal and work-related tasks in an automated vehicle. In this remote unmoderated online elicitation study, 31 participants proposed 373 gestures and 373 voice commands for performing 24 tasks. We analyzed the elicited interactions, their preferred modality of interaction, and the reasons behind this preference. Lastly, we discuss our results and their implications for designing AR WSD in automated vehicles.

AB - Augmented reality (AR) windshield display (WSD) offers promising ways to engage in non-driving tasks in automated vehicles. Previous studies explored different ways WSD can be used to present driving and other tasks-related information and how that can affect driving performance, user experience, and performance in secondary tasks. Our goal for this study was to examine how drivers expect to use gesture and voice commands for interacting with WSD for performing complex, multi-step personal and work-related tasks in an automated vehicle. In this remote unmoderated online elicitation study, 31 participants proposed 373 gestures and 373 voice commands for performing 24 tasks. We analyzed the elicited interactions, their preferred modality of interaction, and the reasons behind this preference. Lastly, we discuss our results and their implications for designing AR WSD in automated vehicles.

KW - automated driving

KW - gesture

KW - head-up display

KW - voice commands

KW - Windshield display

UR - https://www.scopus.com/pages/publications/85139495586

UR - https://www.scopus.com/pages/publications/85139495586#tab=citedBy

U2 - 10.1145/3543174.3545257

DO - 10.1145/3543174.3545257

M3 - Conference contribution

AN - SCOPUS:85139495586

T3 - Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022

SP - 171

EP - 182

BT - Main Proceedings - 14th International ACM Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI 2022

PB - Association for Computing Machinery, Inc

Y2 - 17 September 2022 through 20 September 2022

ER -

Gesture and Voice Commands to Interact With AR Windshield Display in Automated Vehicle: A Remote Elicitation Study

Abstract

Publication series

Conference

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this