Lab experiment and field experiment in the digital age

Friend or foe?

Ravi Bapna, Gordon Burtch, Yili Hong, Tianshu Sun, Jason Thatcher

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The emergence of digital platforms offered new venues for quasi-controlled experimentation, e.g., MTurk, or as hosts or collaborators for field experimentation, resulting in renewed interest in experimentation as a method for understanding causal relationships in the real-world, with delicate control and precision, at unprecedented scales. While digitization has significantly expanded the scope and scale of field experimentation, it also benefits laboratory experiment researchers, as it has extended understanding of the utility of experiments for understanding many different phenomena in online environments. With the rapid growth of digital capabilities to conduct both field experiments and lab experiments, it behooves us, as researchers and experimentalists to take a fresh look at both flavors of the method, and the interplay between them. In this digital age, are field experiments and lab experiments friends or foes? While the academic field of information systems (IS), which often has taken the intellectual leadership role in matters related to digital, we have largely lagged in our discussion of this important methodological issue. Despite the rapid evolution of the experiment landscape and the strong interest from the research communities in both lab and field experimentation, no systematic discussion has been conducted to arrive at clarity about the scope and future directions for IS research on this frontier, at our conferences or at our journals. Thus, in this panel, we open a discussion of how digital capabilities have enabled large-scale, randomized field experiments, how randomized field experiments may substitute or complement traditional laboratory experiments, and how these methods may be creatively combined and advance our understanding of the digital phenomenon in the future. Below we highlight several major issues, which we provide a detailed discussion.

Original languageEnglish (US)
Title of host publicationInternational Conference on Information Systems 2018, ICIS 2018
PublisherAssociation for Information Systems
ISBN (Electronic)9780996683173
StatePublished - Jan 1 2018
Event39th International Conference on Information Systems, ICIS 2018 - San Francisco, United States
Duration: Dec 13 2018Dec 16 2018

Publication series

NameInternational Conference on Information Systems 2018, ICIS 2018

Conference

Conference39th International Conference on Information Systems, ICIS 2018
CountryUnited States
CitySan Francisco
Period12/13/1812/16/18

Fingerprint

Field Experiment
Experimentation
experiment
Randomized Experiments
Experiment
Experiments
Information Systems
laboratory experiment
information system
Digitization
Leadership
Substitute
Information systems
Field experiment
Lab experiment
systems research
Complement
Flavors
Analog to digital conversion
leadership

Keywords

  • Digital experimentation
  • Field experiment
  • Lab experiment

Cite this

Bapna, R., Burtch, G., Hong, Y., Sun, T., & Thatcher, J. (2018). Lab experiment and field experiment in the digital age: Friend or foe? In International Conference on Information Systems 2018, ICIS 2018 (International Conference on Information Systems 2018, ICIS 2018). Association for Information Systems.

Lab experiment and field experiment in the digital age : Friend or foe? / Bapna, Ravi; Burtch, Gordon; Hong, Yili; Sun, Tianshu; Thatcher, Jason.

International Conference on Information Systems 2018, ICIS 2018. Association for Information Systems, 2018. (International Conference on Information Systems 2018, ICIS 2018).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Bapna, R, Burtch, G, Hong, Y, Sun, T & Thatcher, J 2018, Lab experiment and field experiment in the digital age: Friend or foe? in International Conference on Information Systems 2018, ICIS 2018. International Conference on Information Systems 2018, ICIS 2018, Association for Information Systems, 39th International Conference on Information Systems, ICIS 2018, San Francisco, United States, 12/13/18.
Bapna R, Burtch G, Hong Y, Sun T, Thatcher J. Lab experiment and field experiment in the digital age: Friend or foe? In International Conference on Information Systems 2018, ICIS 2018. Association for Information Systems. 2018. (International Conference on Information Systems 2018, ICIS 2018).
Bapna, Ravi ; Burtch, Gordon ; Hong, Yili ; Sun, Tianshu ; Thatcher, Jason. / Lab experiment and field experiment in the digital age : Friend or foe?. International Conference on Information Systems 2018, ICIS 2018. Association for Information Systems, 2018. (International Conference on Information Systems 2018, ICIS 2018).
@inproceedings{fb9e182e23984bf2a94c089d808287e5,
title = "Lab experiment and field experiment in the digital age: Friend or foe?",
abstract = "The emergence of digital platforms offered new venues for quasi-controlled experimentation, e.g., MTurk, or as hosts or collaborators for field experimentation, resulting in renewed interest in experimentation as a method for understanding causal relationships in the real-world, with delicate control and precision, at unprecedented scales. While digitization has significantly expanded the scope and scale of field experimentation, it also benefits laboratory experiment researchers, as it has extended understanding of the utility of experiments for understanding many different phenomena in online environments. With the rapid growth of digital capabilities to conduct both field experiments and lab experiments, it behooves us, as researchers and experimentalists to take a fresh look at both flavors of the method, and the interplay between them. In this digital age, are field experiments and lab experiments friends or foes? While the academic field of information systems (IS), which often has taken the intellectual leadership role in matters related to digital, we have largely lagged in our discussion of this important methodological issue. Despite the rapid evolution of the experiment landscape and the strong interest from the research communities in both lab and field experimentation, no systematic discussion has been conducted to arrive at clarity about the scope and future directions for IS research on this frontier, at our conferences or at our journals. Thus, in this panel, we open a discussion of how digital capabilities have enabled large-scale, randomized field experiments, how randomized field experiments may substitute or complement traditional laboratory experiments, and how these methods may be creatively combined and advance our understanding of the digital phenomenon in the future. Below we highlight several major issues, which we provide a detailed discussion.",
keywords = "Digital experimentation, Field experiment, Lab experiment",
author = "Ravi Bapna and Gordon Burtch and Yili Hong and Tianshu Sun and Jason Thatcher",
year = "2018",
month = "1",
day = "1",
language = "English (US)",
series = "International Conference on Information Systems 2018, ICIS 2018",
publisher = "Association for Information Systems",
booktitle = "International Conference on Information Systems 2018, ICIS 2018",

}

TY - GEN

T1 - Lab experiment and field experiment in the digital age

T2 - Friend or foe?

AU - Bapna, Ravi

AU - Burtch, Gordon

AU - Hong, Yili

AU - Sun, Tianshu

AU - Thatcher, Jason

PY - 2018/1/1

Y1 - 2018/1/1

N2 - The emergence of digital platforms offered new venues for quasi-controlled experimentation, e.g., MTurk, or as hosts or collaborators for field experimentation, resulting in renewed interest in experimentation as a method for understanding causal relationships in the real-world, with delicate control and precision, at unprecedented scales. While digitization has significantly expanded the scope and scale of field experimentation, it also benefits laboratory experiment researchers, as it has extended understanding of the utility of experiments for understanding many different phenomena in online environments. With the rapid growth of digital capabilities to conduct both field experiments and lab experiments, it behooves us, as researchers and experimentalists to take a fresh look at both flavors of the method, and the interplay between them. In this digital age, are field experiments and lab experiments friends or foes? While the academic field of information systems (IS), which often has taken the intellectual leadership role in matters related to digital, we have largely lagged in our discussion of this important methodological issue. Despite the rapid evolution of the experiment landscape and the strong interest from the research communities in both lab and field experimentation, no systematic discussion has been conducted to arrive at clarity about the scope and future directions for IS research on this frontier, at our conferences or at our journals. Thus, in this panel, we open a discussion of how digital capabilities have enabled large-scale, randomized field experiments, how randomized field experiments may substitute or complement traditional laboratory experiments, and how these methods may be creatively combined and advance our understanding of the digital phenomenon in the future. Below we highlight several major issues, which we provide a detailed discussion.

AB - The emergence of digital platforms offered new venues for quasi-controlled experimentation, e.g., MTurk, or as hosts or collaborators for field experimentation, resulting in renewed interest in experimentation as a method for understanding causal relationships in the real-world, with delicate control and precision, at unprecedented scales. While digitization has significantly expanded the scope and scale of field experimentation, it also benefits laboratory experiment researchers, as it has extended understanding of the utility of experiments for understanding many different phenomena in online environments. With the rapid growth of digital capabilities to conduct both field experiments and lab experiments, it behooves us, as researchers and experimentalists to take a fresh look at both flavors of the method, and the interplay between them. In this digital age, are field experiments and lab experiments friends or foes? While the academic field of information systems (IS), which often has taken the intellectual leadership role in matters related to digital, we have largely lagged in our discussion of this important methodological issue. Despite the rapid evolution of the experiment landscape and the strong interest from the research communities in both lab and field experimentation, no systematic discussion has been conducted to arrive at clarity about the scope and future directions for IS research on this frontier, at our conferences or at our journals. Thus, in this panel, we open a discussion of how digital capabilities have enabled large-scale, randomized field experiments, how randomized field experiments may substitute or complement traditional laboratory experiments, and how these methods may be creatively combined and advance our understanding of the digital phenomenon in the future. Below we highlight several major issues, which we provide a detailed discussion.

KW - Digital experimentation

KW - Field experiment

KW - Lab experiment

UR - http://www.scopus.com/inward/record.url?scp=85062508979&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85062508979&partnerID=8YFLogxK

M3 - Conference contribution

T3 - International Conference on Information Systems 2018, ICIS 2018

BT - International Conference on Information Systems 2018, ICIS 2018

PB - Association for Information Systems

ER -