EasyCaption: Investigating the Impact of Prolonged Exposure to Captioning on VR HMD on General Population

Sunday D. Ubur; Naome A. Etori; Shiva Ghasemi; Kenneth King; Denis Gračanin; Maria Gini

doi:10.1007/978-3-031-60881-0_24

EasyCaption: Investigating the Impact of Prolonged Exposure to Captioning on VR HMD on General Population

Sunday D. Ubur, Naome A. Etori, Shiva Ghasemi, Kenneth King, Denis Gračanin, Maria Gini

Computer Science and Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Scopus citations

Abstract

Recent research explores the potential of captioning within virtual environments. However, the effects of prolonged exposure to captioning in these environments have received scant attention. This study delves into the impacts of extended exposure to captions in a virtual reality (VR) setting, utilizing Head-Mounted Displays (HMDs). Its objective is to augment communication accessibility for individuals who are Deaf or Hard of Hearing (DHH), as well as the broader populace, by evaluating aspects such as user experience, cognitive load, and comprehension. To achieve this, the study adopts a mixed-methods framework, integrating quantitative data gathered via user surveys with qualitative insights obtained from semi-structured interviews. The findings suggest that prolonged eye gaze negatively impacts engagement levels, with participants without avatars exhibiting higher attention and engagement. Additionally, participants with avatars tend to experience slightly higher stress levels. The study concludes by recommending further exploration of robust personalization features and conducting user experience studies to refine captioning environments for improved readability and user satisfaction. This research contributes to the understanding of accessibility and VR technology for both Deaf and Hard of Hearing (DHH) and non DHH populations, offering valuable insights for designers, developers, and educators working towards creating more inclusive VR experiences.

Original language	English (US)
Title of host publication	Universal Access in Human-Computer Interaction - 18th International Conference, UAHCI 2024, Held as Part of the 26th HCI International Conference, HCII 2024, Proceedings
Editors	Margherita Antona, Constantine Stephanidis
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	382-403
Number of pages	22
ISBN (Print)	9783031608803
DOIs	https://doi.org/10.1007/978-3-031-60881-0_24
State	Published - 2024
Event	18th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2024, Held as Part of the 26th HCI International Conference, HCII 2024 - Washington, United States Duration: Jun 29 2024 → Jul 4 2024

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14697 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	18th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2024, Held as Part of the 26th HCI International Conference, HCII 2024
Country/Territory	United States
City	Washington
Period	6/29/24 → 7/4/24

Bibliographical note

Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

Keywords

Captions
Cognitive Load
Eye Tracking
User Experience
Virtual Reality

Publisher link

10.1007/978-3-031-60881-0_24

Find It

Find It at U of M Libraries

Cite this

Ubur, S. D., Etori, N. A., Ghasemi, S., King, K., Gračanin, D., & Gini, M. (2024). EasyCaption: Investigating the Impact of Prolonged Exposure to Captioning on VR HMD on General Population. In M. Antona, & C. Stephanidis (Eds.), Universal Access in Human-Computer Interaction - 18th International Conference, UAHCI 2024, Held as Part of the 26th HCI International Conference, HCII 2024, Proceedings (pp. 382-403). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14697 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-60881-0_24

EasyCaption: Investigating the Impact of Prolonged Exposure to Captioning on VR HMD on General Population. / Ubur, Sunday D.; Etori, Naome A.; Ghasemi, Shiva et al.
Universal Access in Human-Computer Interaction - 18th International Conference, UAHCI 2024, Held as Part of the 26th HCI International Conference, HCII 2024, Proceedings. ed. / Margherita Antona; Constantine Stephanidis. Springer Science and Business Media Deutschland GmbH, 2024. p. 382-403 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14697 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Ubur, SD, Etori, NA, Ghasemi, S, King, K, Gračanin, D & Gini, M 2024, EasyCaption: Investigating the Impact of Prolonged Exposure to Captioning on VR HMD on General Population. in M Antona & C Stephanidis (eds), Universal Access in Human-Computer Interaction - 18th International Conference, UAHCI 2024, Held as Part of the 26th HCI International Conference, HCII 2024, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14697 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 382-403, 18th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2024, Held as Part of the 26th HCI International Conference, HCII 2024, Washington, United States, 6/29/24. https://doi.org/10.1007/978-3-031-60881-0_24

Ubur SD, Etori NA, Ghasemi S, King K, Gračanin D, Gini M. EasyCaption: Investigating the Impact of Prolonged Exposure to Captioning on VR HMD on General Population. In Antona M, Stephanidis C, editors, Universal Access in Human-Computer Interaction - 18th International Conference, UAHCI 2024, Held as Part of the 26th HCI International Conference, HCII 2024, Proceedings. Springer Science and Business Media Deutschland GmbH. 2024. p. 382-403. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-60881-0_24

Ubur, Sunday D. ; Etori, Naome A. ; Ghasemi, Shiva et al. / EasyCaption : Investigating the Impact of Prolonged Exposure to Captioning on VR HMD on General Population. Universal Access in Human-Computer Interaction - 18th International Conference, UAHCI 2024, Held as Part of the 26th HCI International Conference, HCII 2024, Proceedings. editor / Margherita Antona ; Constantine Stephanidis. Springer Science and Business Media Deutschland GmbH, 2024. pp. 382-403 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{ce2f5a1836e7464998c8726081969178,

title = "EasyCaption: Investigating the Impact of Prolonged Exposure to Captioning on VR HMD on General Population",

abstract = "Recent research explores the potential of captioning within virtual environments. However, the effects of prolonged exposure to captioning in these environments have received scant attention. This study delves into the impacts of extended exposure to captions in a virtual reality (VR) setting, utilizing Head-Mounted Displays (HMDs). Its objective is to augment communication accessibility for individuals who are Deaf or Hard of Hearing (DHH), as well as the broader populace, by evaluating aspects such as user experience, cognitive load, and comprehension. To achieve this, the study adopts a mixed-methods framework, integrating quantitative data gathered via user surveys with qualitative insights obtained from semi-structured interviews. The findings suggest that prolonged eye gaze negatively impacts engagement levels, with participants without avatars exhibiting higher attention and engagement. Additionally, participants with avatars tend to experience slightly higher stress levels. The study concludes by recommending further exploration of robust personalization features and conducting user experience studies to refine captioning environments for improved readability and user satisfaction. This research contributes to the understanding of accessibility and VR technology for both Deaf and Hard of Hearing (DHH) and non DHH populations, offering valuable insights for designers, developers, and educators working towards creating more inclusive VR experiences.",

keywords = "Captions, Cognitive Load, Eye Tracking, User Experience, Virtual Reality",

author = "Ubur, \{Sunday D.\} and Etori, \{Naome A.\} and Shiva Ghasemi and Kenneth King and Denis Gra{\v c}anin and Maria Gini",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.; 18th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2024, Held as Part of the 26th HCI International Conference, HCII 2024 ; Conference date: 29-06-2024 Through 04-07-2024",

year = "2024",

doi = "10.1007/978-3-031-60881-0\_24",

language = "English (US)",

isbn = "9783031608803",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "382--403",

editor = "Margherita Antona and Constantine Stephanidis",

booktitle = "Universal Access in Human-Computer Interaction - 18th International Conference, UAHCI 2024, Held as Part of the 26th HCI International Conference, HCII 2024, Proceedings",

address = "Germany",

}

TY - GEN

T1 - EasyCaption

T2 - 18th International Conference on Universal Access in Human-Computer Interaction, UAHCI 2024, Held as Part of the 26th HCI International Conference, HCII 2024

AU - Ubur, Sunday D.

AU - Etori, Naome A.

AU - Ghasemi, Shiva

AU - King, Kenneth

AU - Gračanin, Denis

AU - Gini, Maria

PY - 2024

Y1 - 2024

N2 - Recent research explores the potential of captioning within virtual environments. However, the effects of prolonged exposure to captioning in these environments have received scant attention. This study delves into the impacts of extended exposure to captions in a virtual reality (VR) setting, utilizing Head-Mounted Displays (HMDs). Its objective is to augment communication accessibility for individuals who are Deaf or Hard of Hearing (DHH), as well as the broader populace, by evaluating aspects such as user experience, cognitive load, and comprehension. To achieve this, the study adopts a mixed-methods framework, integrating quantitative data gathered via user surveys with qualitative insights obtained from semi-structured interviews. The findings suggest that prolonged eye gaze negatively impacts engagement levels, with participants without avatars exhibiting higher attention and engagement. Additionally, participants with avatars tend to experience slightly higher stress levels. The study concludes by recommending further exploration of robust personalization features and conducting user experience studies to refine captioning environments for improved readability and user satisfaction. This research contributes to the understanding of accessibility and VR technology for both Deaf and Hard of Hearing (DHH) and non DHH populations, offering valuable insights for designers, developers, and educators working towards creating more inclusive VR experiences.

AB - Recent research explores the potential of captioning within virtual environments. However, the effects of prolonged exposure to captioning in these environments have received scant attention. This study delves into the impacts of extended exposure to captions in a virtual reality (VR) setting, utilizing Head-Mounted Displays (HMDs). Its objective is to augment communication accessibility for individuals who are Deaf or Hard of Hearing (DHH), as well as the broader populace, by evaluating aspects such as user experience, cognitive load, and comprehension. To achieve this, the study adopts a mixed-methods framework, integrating quantitative data gathered via user surveys with qualitative insights obtained from semi-structured interviews. The findings suggest that prolonged eye gaze negatively impacts engagement levels, with participants without avatars exhibiting higher attention and engagement. Additionally, participants with avatars tend to experience slightly higher stress levels. The study concludes by recommending further exploration of robust personalization features and conducting user experience studies to refine captioning environments for improved readability and user satisfaction. This research contributes to the understanding of accessibility and VR technology for both Deaf and Hard of Hearing (DHH) and non DHH populations, offering valuable insights for designers, developers, and educators working towards creating more inclusive VR experiences.

KW - Captions

KW - Cognitive Load

KW - Eye Tracking

KW - User Experience

KW - Virtual Reality

UR - https://www.scopus.com/pages/publications/85196100597

UR - https://www.scopus.com/inward/citedby.url?scp=85196100597&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-60881-0_24

DO - 10.1007/978-3-031-60881-0_24

M3 - Conference contribution

AN - SCOPUS:85196100597

SN - 9783031608803

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 382

EP - 403

BT - Universal Access in Human-Computer Interaction - 18th International Conference, UAHCI 2024, Held as Part of the 26th HCI International Conference, HCII 2024, Proceedings

A2 - Antona, Margherita

A2 - Stephanidis, Constantine

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 29 June 2024 through 4 July 2024

ER -