Assessing enactment of content regulation policies: A post hoc crowd-sourced audit of election misinformation on YouTube

Prerna Juneja; Md Momen Bhuiyan; Tanushree Mitra

doi:10.1145/3544548.3580846

Assessing enactment of content regulation policies: A post hoc crowd-sourced audit of election misinformation on YouTube

Prerna Juneja
, Md Momen Bhuiyan
, Tanushree Mitra

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

29 Scopus citations

Abstract

With the 2022 US midterm elections approaching, conspiratorial claims about the 2020 presidential elections continue to threaten users' trust in the electoral process. To regulate election misinformation, YouTube introduced policies to remove such content from its searches and recommendations. In this paper, we conduct a 9-day crowd-sourced audit on YouTube to assess the extent of enactment of such policies. We recruited 99 users who installed a browser extension that enabled us to collect up-next recommendation trails and search results for 45 videos and 88 search queries about the 2020 elections. We find that YouTube's search results, irrespective of search query bias, contain more videos that oppose rather than support election misinformation. However, watching misinformative election videos still lead users to a small number of misinformative videos in the up-next trails. Our results imply that while YouTube largely seems successful in regulating election misinformation, there is still room for improvement.

Original language	English (US)
Title of host publication	CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
Publisher	Association for Computing Machinery
ISBN (Electronic)	9781450394215
DOIs	https://doi.org/10.1145/3544548.3580846
State	Published - Apr 19 2023
Externally published	Yes
Event	2023 CHI Conference on Human Factors in Computing Systems, CHI 2023 - Hamburg, Germany Duration: Apr 23 2023 → Apr 28 2023

Publication series

Name	Conference on Human Factors in Computing Systems - Proceedings

Conference

Conference	2023 CHI Conference on Human Factors in Computing Systems, CHI 2023
Country/Territory	Germany
City	Hamburg
Period	4/23/23 → 4/28/23

Bibliographical note

Publisher Copyright:
© 2023 Owner/Author.

Keywords

algorithm audit
elections
fairness
misinformation
recommendations
voter fraud

Publisher link

10.1145/3544548.3580846

Find It

Find It at U of M Libraries

Cite this

Juneja, P., Bhuiyan, M. M., & Mitra, T. (2023). Assessing enactment of content regulation policies: A post hoc crowd-sourced audit of election misinformation on YouTube. In CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems Article 545 (Conference on Human Factors in Computing Systems - Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3544548.3580846

Assessing enactment of content regulation policies: A post hoc crowd-sourced audit of election misinformation on YouTube. / Juneja, Prerna; Bhuiyan, Md Momen; Mitra, Tanushree.
CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, 2023. 545 (Conference on Human Factors in Computing Systems - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Juneja, P, Bhuiyan, MM & Mitra, T 2023, Assessing enactment of content regulation policies: A post hoc crowd-sourced audit of election misinformation on YouTube. in CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems., 545, Conference on Human Factors in Computing Systems - Proceedings, Association for Computing Machinery, 2023 CHI Conference on Human Factors in Computing Systems, CHI 2023, Hamburg, Germany, 4/23/23. https://doi.org/10.1145/3544548.3580846

Juneja P, Bhuiyan MM, Mitra T. Assessing enactment of content regulation policies: A post hoc crowd-sourced audit of election misinformation on YouTube. In CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery. 2023. 545. (Conference on Human Factors in Computing Systems - Proceedings). doi: 10.1145/3544548.3580846

Juneja, Prerna ; Bhuiyan, Md Momen ; Mitra, Tanushree. / Assessing enactment of content regulation policies : A post hoc crowd-sourced audit of election misinformation on YouTube. CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, 2023. (Conference on Human Factors in Computing Systems - Proceedings).

@inproceedings{a929ed26fe52458591af0174d8ac1ae9,

title = "Assessing enactment of content regulation policies: A post hoc crowd-sourced audit of election misinformation on YouTube",

abstract = "With the 2022 US midterm elections approaching, conspiratorial claims about the 2020 presidential elections continue to threaten users' trust in the electoral process. To regulate election misinformation, YouTube introduced policies to remove such content from its searches and recommendations. In this paper, we conduct a 9-day crowd-sourced audit on YouTube to assess the extent of enactment of such policies. We recruited 99 users who installed a browser extension that enabled us to collect up-next recommendation trails and search results for 45 videos and 88 search queries about the 2020 elections. We find that YouTube's search results, irrespective of search query bias, contain more videos that oppose rather than support election misinformation. However, watching misinformative election videos still lead users to a small number of misinformative videos in the up-next trails. Our results imply that while YouTube largely seems successful in regulating election misinformation, there is still room for improvement.",

keywords = "algorithm audit, elections, fairness, misinformation, recommendations, voter fraud",

author = "Prerna Juneja and Bhuiyan, \{Md Momen\} and Tanushree Mitra",

note = "Publisher Copyright: {\textcopyright} 2023 Owner/Author.; 2023 CHI Conference on Human Factors in Computing Systems, CHI 2023 ; Conference date: 23-04-2023 Through 28-04-2023",

year = "2023",

month = apr,

day = "19",

doi = "10.1145/3544548.3580846",

language = "English (US)",

series = "Conference on Human Factors in Computing Systems - Proceedings",

publisher = "Association for Computing Machinery",

booktitle = "CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems",

}

TY - GEN

T1 - Assessing enactment of content regulation policies

T2 - 2023 CHI Conference on Human Factors in Computing Systems, CHI 2023

AU - Juneja, Prerna

AU - Bhuiyan, Md Momen

AU - Mitra, Tanushree

PY - 2023/4/19

Y1 - 2023/4/19

N2 - With the 2022 US midterm elections approaching, conspiratorial claims about the 2020 presidential elections continue to threaten users' trust in the electoral process. To regulate election misinformation, YouTube introduced policies to remove such content from its searches and recommendations. In this paper, we conduct a 9-day crowd-sourced audit on YouTube to assess the extent of enactment of such policies. We recruited 99 users who installed a browser extension that enabled us to collect up-next recommendation trails and search results for 45 videos and 88 search queries about the 2020 elections. We find that YouTube's search results, irrespective of search query bias, contain more videos that oppose rather than support election misinformation. However, watching misinformative election videos still lead users to a small number of misinformative videos in the up-next trails. Our results imply that while YouTube largely seems successful in regulating election misinformation, there is still room for improvement.

AB - With the 2022 US midterm elections approaching, conspiratorial claims about the 2020 presidential elections continue to threaten users' trust in the electoral process. To regulate election misinformation, YouTube introduced policies to remove such content from its searches and recommendations. In this paper, we conduct a 9-day crowd-sourced audit on YouTube to assess the extent of enactment of such policies. We recruited 99 users who installed a browser extension that enabled us to collect up-next recommendation trails and search results for 45 videos and 88 search queries about the 2020 elections. We find that YouTube's search results, irrespective of search query bias, contain more videos that oppose rather than support election misinformation. However, watching misinformative election videos still lead users to a small number of misinformative videos in the up-next trails. Our results imply that while YouTube largely seems successful in regulating election misinformation, there is still room for improvement.

KW - algorithm audit

KW - elections

KW - fairness

KW - misinformation

KW - recommendations

KW - voter fraud

UR - https://www.scopus.com/pages/publications/85160009474

UR - https://www.scopus.com/pages/publications/85160009474#tab=citedBy

U2 - 10.1145/3544548.3580846

DO - 10.1145/3544548.3580846

M3 - Conference contribution

AN - SCOPUS:85160009474

T3 - Conference on Human Factors in Computing Systems - Proceedings

BT - CHI 2023 - Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

PB - Association for Computing Machinery

Y2 - 23 April 2023 through 28 April 2023

ER -

Assessing enactment of content regulation policies: A post hoc crowd-sourced audit of election misinformation on YouTube

Abstract

Publication series

Conference

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this