TY - JOUR
T1 - The Future of Coding
T2 - A Comparison of Hand-Coding and Three Types of Computer-Assisted Text Analysis Methods
AU - Nelson, Laura K.
AU - Burk, Derek
AU - Knudsen, Marcel
AU - McCall, Leslie
N1 - Publisher Copyright:
© 2018, The Author(s) 2018.
Copyright:
Copyright 2018 Elsevier B.V., All rights reserved.
PY - 2018
Y1 - 2018
N2 - Advances in computer science and computational linguistics have yielded new, and faster, computational approaches to structuring and analyzing textual data. These approaches perform well on tasks like information extraction, but their ability to identify complex, socially constructed, and unsettled theoretical concepts—a central goal of sociological content analysis—has not been tested. To fill this gap, we compare the results produced by three common computer-assisted approaches—dictionary, supervised machine learning (SML), and unsupervised machine learning—to those produced through a rigorous hand-coding analysis of inequality in the news (N = 1,253 articles). Although we find that SML methods perform best in replicating hand-coded results, we document and clarify the strengths and weaknesses of each approach, including how they can complement one another. We argue that content analysts in the social sciences would do well to keep all these approaches in their toolkit, deploying them purposefully according to the task at hand.
AB - Advances in computer science and computational linguistics have yielded new, and faster, computational approaches to structuring and analyzing textual data. These approaches perform well on tasks like information extraction, but their ability to identify complex, socially constructed, and unsettled theoretical concepts—a central goal of sociological content analysis—has not been tested. To fill this gap, we compare the results produced by three common computer-assisted approaches—dictionary, supervised machine learning (SML), and unsupervised machine learning—to those produced through a rigorous hand-coding analysis of inequality in the news (N = 1,253 articles). Although we find that SML methods perform best in replicating hand-coded results, we document and clarify the strengths and weaknesses of each approach, including how they can complement one another. We argue that content analysts in the social sciences would do well to keep all these approaches in their toolkit, deploying them purposefully according to the task at hand.
KW - content/text analysis
KW - dictionary methods
KW - hand-coding methods
KW - inequality
KW - supervised machine learning
KW - unsupervised machine learning
UR - http://www.scopus.com/inward/record.url?scp=85047665832&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85047665832&partnerID=8YFLogxK
U2 - 10.1177/0049124118769114
DO - 10.1177/0049124118769114
M3 - Article
AN - SCOPUS:85047665832
JO - Sociological Methods and Research
JF - Sociological Methods and Research
SN - 0049-1241
ER -