Predicting Individual Treatment Effects of Large-scale Team Competitions in a Ride-sharing Economy

Teng Ye; Wei Ai; Lingyu Zhang; Ning Luo; Lulu Zhang; Jieping Ye; Qiaozhu Mei

doi:10.1145/3394486.3403286

Predicting Individual Treatment Effects of Large-scale Team Competitions in a Ride-sharing Economy

Teng Ye, Wei Ai, Lingyu Zhang, Ning Luo, Lulu Zhang, Jieping Ye, Qiaozhu Mei

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

8 Scopus citations

Abstract

Millions of drivers worldwide have enjoyed financial benefits and work schedule flexibility through a ride-sharing economy, but meanwhile they have suffered from the lack of a sense of identity and career achievement. Equipped with social identity and contest theories, financially incentivized team competitions have been an effective instrument to increase drivers' productivity, job satisfaction, and retention, and to improve revenue over cost for ride-sharing platforms. While these competitions are overall effective, the decisive factors behind the treatment effects and how they affect the outcomes of individual drivers have been largely mysterious. In this study, we analyze data collected from more than 500 large-scale team competitions organized by a leading ride-sharing platform, building machine learning models to predict individual treatment effects. Through a careful investigation of features and predictors, we are able to reduce out-sample prediction error by more than 24%. Through interpreting the best-performing models, we discover many novel and actionable insights regarding how to optimize the design and the execution of team competitions on ride-sharing platforms. A simulated analysis demonstrates that by simply changing a few contest design options, the average treatment effect of a real competition is expected to increase by as much as 26%. Our procedure and findings shed light on how to analyze and optimize large-scale online field experiments in general.

Original language	English (US)
Title of host publication	KDD 2020 - Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
Publisher	Association for Computing Machinery
Pages	2368-2377
Number of pages	10
ISBN (Electronic)	9781450379984
DOIs	https://doi.org/10.1145/3394486.3403286
State	Published - Aug 23 2020
Externally published	Yes
Event	26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2020 - Virtual, Online, United States Duration: Aug 23 2020 → Aug 27 2020

Publication series

Name	Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Conference

Conference	26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2020
Country/Territory	United States
City	Virtual, Online
Period	8/23/20 → 8/27/20

Bibliographical note

Funding Information:
We thank Yan Chen for designing the team competition experiments and Hongtu Zhu for helpful discussions. This work is funded in part by the DiDi Research Partnership with the University of Michigan. Jieping Ye and Lingyu Zhang’s work is in part funded by the National Key Research and Development Program of China under grant 2018AAA0101100. Qiaozhu Mei’s work is in part supported by the National Science Foundation under grant no. 1633370.

Publisher Copyright:
© 2020 ACM.

Keywords

field experiment
individual treatment effect
machine learning
sharing economy
team competition

Publisher link

10.1145/3394486.3403286

Find It

Find It at U of M Libraries

Cite this

Ye, T., Ai, W., Zhang, L., Luo, N., Zhang, L., Ye, J., & Mei, Q. (2020). Predicting Individual Treatment Effects of Large-scale Team Competitions in a Ride-sharing Economy. In KDD 2020 - Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 2368-2377). (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining). Association for Computing Machinery. https://doi.org/10.1145/3394486.3403286

Predicting Individual Treatment Effects of Large-scale Team Competitions in a Ride-sharing Economy. / Ye, Teng; Ai, Wei; Zhang, Lingyu et al.
KDD 2020 - Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, 2020. p. 2368-2377 (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Ye, T, Ai, W, Zhang, L, Luo, N, Zhang, L, Ye, J & Mei, Q 2020, Predicting Individual Treatment Effects of Large-scale Team Competitions in a Ride-sharing Economy. in KDD 2020 - Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Association for Computing Machinery, pp. 2368-2377, 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2020, Virtual, Online, United States, 8/23/20. https://doi.org/10.1145/3394486.3403286

Ye T, Ai W, Zhang L, Luo N, Zhang L, Ye J et al. Predicting Individual Treatment Effects of Large-scale Team Competitions in a Ride-sharing Economy. In KDD 2020 - Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery. 2020. p. 2368-2377. (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining). doi: 10.1145/3394486.3403286

Ye, Teng ; Ai, Wei ; Zhang, Lingyu et al. / Predicting Individual Treatment Effects of Large-scale Team Competitions in a Ride-sharing Economy. KDD 2020 - Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, 2020. pp. 2368-2377 (Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining).

@inproceedings{a399e569c0f9470badd0adb73f5d6b38,

title = "Predicting Individual Treatment Effects of Large-scale Team Competitions in a Ride-sharing Economy",

abstract = "Millions of drivers worldwide have enjoyed financial benefits and work schedule flexibility through a ride-sharing economy, but meanwhile they have suffered from the lack of a sense of identity and career achievement. Equipped with social identity and contest theories, financially incentivized team competitions have been an effective instrument to increase drivers' productivity, job satisfaction, and retention, and to improve revenue over cost for ride-sharing platforms. While these competitions are overall effective, the decisive factors behind the treatment effects and how they affect the outcomes of individual drivers have been largely mysterious. In this study, we analyze data collected from more than 500 large-scale team competitions organized by a leading ride-sharing platform, building machine learning models to predict individual treatment effects. Through a careful investigation of features and predictors, we are able to reduce out-sample prediction error by more than 24%. Through interpreting the best-performing models, we discover many novel and actionable insights regarding how to optimize the design and the execution of team competitions on ride-sharing platforms. A simulated analysis demonstrates that by simply changing a few contest design options, the average treatment effect of a real competition is expected to increase by as much as 26%. Our procedure and findings shed light on how to analyze and optimize large-scale online field experiments in general.",

keywords = "field experiment, individual treatment effect, machine learning, sharing economy, team competition",

author = "Teng Ye and Wei Ai and Lingyu Zhang and Ning Luo and Lulu Zhang and Jieping Ye and Qiaozhu Mei",

note = "Funding Information: We thank Yan Chen for designing the team competition experiments and Hongtu Zhu for helpful discussions. This work is funded in part by the DiDi Research Partnership with the University of Michigan. Jieping Ye and Lingyu Zhang{\textquoteright}s work is in part funded by the National Key Research and Development Program of China under grant 2018AAA0101100. Qiaozhu Mei{\textquoteright}s work is in part supported by the National Science Foundation under grant no. 1633370. Publisher Copyright: {\textcopyright} 2020 ACM.; 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2020 ; Conference date: 23-08-2020 Through 27-08-2020",

year = "2020",

month = aug,

day = "23",

doi = "10.1145/3394486.3403286",

language = "English (US)",

series = "Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining",

publisher = "Association for Computing Machinery",

pages = "2368--2377",

booktitle = "KDD 2020 - Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining",

}

TY - GEN

T1 - Predicting Individual Treatment Effects of Large-scale Team Competitions in a Ride-sharing Economy

AU - Ye, Teng

AU - Ai, Wei

AU - Zhang, Lingyu

AU - Luo, Ning

AU - Zhang, Lulu

AU - Ye, Jieping

AU - Mei, Qiaozhu

N1 - Funding Information: We thank Yan Chen for designing the team competition experiments and Hongtu Zhu for helpful discussions. This work is funded in part by the DiDi Research Partnership with the University of Michigan. Jieping Ye and Lingyu Zhang’s work is in part funded by the National Key Research and Development Program of China under grant 2018AAA0101100. Qiaozhu Mei’s work is in part supported by the National Science Foundation under grant no. 1633370. Publisher Copyright: © 2020 ACM.

PY - 2020/8/23

Y1 - 2020/8/23

N2 - Millions of drivers worldwide have enjoyed financial benefits and work schedule flexibility through a ride-sharing economy, but meanwhile they have suffered from the lack of a sense of identity and career achievement. Equipped with social identity and contest theories, financially incentivized team competitions have been an effective instrument to increase drivers' productivity, job satisfaction, and retention, and to improve revenue over cost for ride-sharing platforms. While these competitions are overall effective, the decisive factors behind the treatment effects and how they affect the outcomes of individual drivers have been largely mysterious. In this study, we analyze data collected from more than 500 large-scale team competitions organized by a leading ride-sharing platform, building machine learning models to predict individual treatment effects. Through a careful investigation of features and predictors, we are able to reduce out-sample prediction error by more than 24%. Through interpreting the best-performing models, we discover many novel and actionable insights regarding how to optimize the design and the execution of team competitions on ride-sharing platforms. A simulated analysis demonstrates that by simply changing a few contest design options, the average treatment effect of a real competition is expected to increase by as much as 26%. Our procedure and findings shed light on how to analyze and optimize large-scale online field experiments in general.

AB - Millions of drivers worldwide have enjoyed financial benefits and work schedule flexibility through a ride-sharing economy, but meanwhile they have suffered from the lack of a sense of identity and career achievement. Equipped with social identity and contest theories, financially incentivized team competitions have been an effective instrument to increase drivers' productivity, job satisfaction, and retention, and to improve revenue over cost for ride-sharing platforms. While these competitions are overall effective, the decisive factors behind the treatment effects and how they affect the outcomes of individual drivers have been largely mysterious. In this study, we analyze data collected from more than 500 large-scale team competitions organized by a leading ride-sharing platform, building machine learning models to predict individual treatment effects. Through a careful investigation of features and predictors, we are able to reduce out-sample prediction error by more than 24%. Through interpreting the best-performing models, we discover many novel and actionable insights regarding how to optimize the design and the execution of team competitions on ride-sharing platforms. A simulated analysis demonstrates that by simply changing a few contest design options, the average treatment effect of a real competition is expected to increase by as much as 26%. Our procedure and findings shed light on how to analyze and optimize large-scale online field experiments in general.

KW - field experiment

KW - individual treatment effect

KW - machine learning

KW - sharing economy

KW - team competition

UR - http://www.scopus.com/inward/record.url?scp=85090415934&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85090415934&partnerID=8YFLogxK

U2 - 10.1145/3394486.3403286

DO - 10.1145/3394486.3403286

M3 - Conference contribution

AN - SCOPUS:85090415934

T3 - Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

SP - 2368

EP - 2377

BT - KDD 2020 - Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

PB - Association for Computing Machinery

T2 - 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD 2020

Y2 - 23 August 2020 through 27 August 2020

ER -

Predicting Individual Treatment Effects of Large-scale Team Competitions in a Ride-sharing Economy

Abstract

Publication series

Conference

Bibliographical note

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this