As the gig economy continues to grow and freelance work moves online, five-star reputation systems are becoming more and more common. At the same time, there are increasing accounts of race and gender bias in evaluations of gig workers, with negative impacts for those workers. We report on a series of four Mechanical Turk-based studies in which participants who rated simulated gig work did not show race- or gender bias, while manipulation checks showed they reliably distinguished between low- and high-quality work. Given prior research, this was a striking result. To explore further, we used a Bayesian approach to verify absence of ratings bias (as opposed to merely not detecting bias). This Bayesian test let us identify an upperbound: if any bias did exist in our studies, it was below an average of 0.2 stars on a five-star scale. We discuss possible interpretations of our results and outline future work to better understand the results.
|Original language||English (US)|
|Journal||Proceedings of the ACM on Human-Computer Interaction|
|State||Published - Nov 2017|
Bibliographical notePublisher Copyright:
© 2017 Association for Computing Machinery.
- Bayesian statistics
- Gender bias
- Gig economy
- Racial bias
- Reputation bias
- Reputation systems