Categorical Perception of p-Values

Traditional statistics instruction emphasizes a.05 significance level for hypothesis tests. Here, we investigate the consequences of this training for researchers’ mental representations of probabilities — whether.05 becomes a boundary, that is, a discontinuity of the mental number line, and alters their reasoning about p-values. Graduate students with statistical training (n = 25) viewed pairs of p-values and judged whether they were “similar” or “different.” After controlling for several covariates, participants were more likely and faster to judge p-values as “different” when they crossed the.05 boundary (e.g.,.046 vs.052) compared to when they did not (e.g.,.026 vs.032). This result suggests a categorical perception-like effect for the processing of p-values. It may be a consequence of traditional statistical instruction creating a psychologically real divide between so-called statistical “significance” and “nonsignificance.” Such a distortion is undesirable given modern approaches to statistical reasoning that de-emphasize dichotomizing the p-value continuum.