Design challenge levels in e-learning? Insights from a large-scale field experiment

The existing adaptive e-learning literature focuses on detecting and mitigating knowledge gaps but has paid scant attention to the issue of regulating the challenge levels. The latter issue is especially relevant when it comes to sequence the set of practice problems. Insights from flow theory suggest that learners would be more engaged if their perceived challenges match their abilities. Despite these theoretical predictions, there are gaps in terms of how to administrate the desirable level of challenge in dynamic learning environments and whether it pays to do so despite the fact that one may not perfectly control the level of challenge. Field experiments at seven middle schools reveal that steady challenge benefits both weak and strong learners compared with fluctuating challenges. Moreover, the optimal challenge is heterogeneous on learner's preparations, and weak learners benefit more from low challenges, while strong learners are relatively insensitive to the challenge level.