New semiparametric method for predicting high-cost patients

Adam Maidman, Lan Wang

Research output: Contribution to journalArticlepeer-review

6 Scopus citations


Motivated by the Medical Expenditure Panel Survey containing data from individuals’ medical providers and employers across the United States, we propose a new semiparametric procedure for predicting whether a patient will incur high medical expenditure. Problems of the same nature arise in many other important applications where one would like to predict if a future response occurs at the upper (or lower) tail of the response distribution. The common practice is to artificially dichotomize the response variable and then apply an existing classification method such as binomial regression or a classification tree. We propose a new semiparametric prediction rule to classify whether a future response occurs at the upper tail of the response distribution. The new method can be considered a semiparametric estimator of the Bayes rule for classification and enjoys some nice features. It does not require an artificially dichotomized response and better uses the information contained in the data. It does not require any parametric distributional assumptions and tends to be more robust. It incorporates nonlinear covariate effects and can be adapted to construct a prediction interval and hence provides more information about the future response. We provide an R package plaqr to implement the proposed procedure and demonstrate its performance in Monte Carlo simulations. We illustrate the application of the new method on a subset of the Medical Expenditure Panel Survey data.

Original languageEnglish (US)
Pages (from-to)1104-1111
Number of pages8
Issue number3
StatePublished - Sep 2018

Bibliographical note

Funding Information:
We thank the co-editor, the associate editor, and the anonymous referees for their helpful comments which helped us improve the article significantly. The research was partially supported by DMS-1712706 and a grant from the U.S. Department of Veterans Affairs.

Publisher Copyright:
© 2017, The International Biometric Society


  • Conditional quantile
  • Expenditure prediction
  • High-cost patient
  • Partially linear additive model
  • Semiparametric regression
  • Upper tail


Dive into the research topics of 'New semiparametric method for predicting high-cost patients'. Together they form a unique fingerprint.

Cite this