Information utility: Quantifying the total psychometric information provided by a measure

Although advances have improved our ability to describe the measurement precision of a test, it often remains challenging to summarize how well a test is performing overall. Reliability, for example, provides an overall summary of measurement precision, but it is sample-specific and might not reflect the potential usefulness of a test if the sample is poorly suited for the test's purposes. The test information function, conversely, provides detailed sample-independent information about measurement precision, but it does not provide an overall summary of test performance. Here, the concept of information utility is introduced. Information utility provides an index of how much psychometric information a measure (e.g., item, test) provides about a trait overall. Information utility has a number of important applied implications, including test selection, trait estimation, computerized adaptive testing, and hypothesis testing. Information utility may have particular utility in situations where the accuracy of prior information about trait level is vague or unclear.