Supervised pretraining through contrastive categorical positive samplings to improve COVID-19 mortality prediction

Tingyi Wanyan; Mingquan Lin; Eyal Klang; Kartikeya M. Menon; Faris F. Gulamali; Ariful Azad; Yiye Zhang; Ying Ding; Zhangyang Wang; Fei Wang; Benjamin Glicksberg; Yifan Peng

doi:10.1145/3535508.3545541

Supervised pretraining through contrastive categorical positive samplings to improve COVID-19 mortality prediction

Tingyi Wanyan
, Mingquan Lin
, Eyal Klang
, Kartikeya M. Menon
, Faris F. Gulamali
, Ariful Azad
, Yiye Zhang
, Ying Ding
, Zhangyang Wang
, Fei Wang
, Benjamin Glicksberg
, Yifan Peng

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

7 Scopus citations

Abstract

Clinical EHR data is naturally heterogeneous, where it contains abundant sub-phenotype. Such diversity creates challenges for outcome prediction using a machine learning model since it leads to high intra-class variance. To address this issue, we propose a supervised pre-Training model with a unique embedded k-nearest-neighbor positive sampling strategy. We demonstrate the enhanced performance value of this framework theoretically and show that it yields highly competitive experimental results in predicting patient mortality in real-world COVID-19 EHR data with a total of over 7,000 patients admitted to a large, urban health system. Our method achieves a better AUROC prediction score of 0.872, which outperforms the alternative pre-Training models and traditional machine learning methods. Additionally, our method performs much better when the training data size is small (345 training instances).

Original language	English (US)
Title of host publication	Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022
Publisher	Association for Computing Machinery, Inc
ISBN (Electronic)	9781450393867
DOIs	https://doi.org/10.1145/3535508.3545541
State	Published - Aug 7 2022
Externally published	Yes
Event	13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022 - Chicago, United States Duration: Aug 7 2022 → Aug 8 2022

Publication series

Name	Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022

Conference

Conference	13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022
Country/Territory	United States
City	Chicago
Period	8/7/22 → 8/8/22

Bibliographical note

Publisher Copyright:
© 2022 ACM.

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being

Keywords

Intra-class variance
Mortality prediction
Pre-Training
Self-supervised learning
Sub-phenotype
Supervised contrastive learning

Publisher link

10.1145/3535508.3545541

Find It

Find It at U of M Libraries

Cite this

Wanyan, T., Lin, M., Klang, E., Menon, K. M., Gulamali, F. F., Azad, A., Zhang, Y., Ding, Y., Wang, Z., Wang, F., Glicksberg, B., & Peng, Y. (2022). Supervised pretraining through contrastive categorical positive samplings to improve COVID-19 mortality prediction. In Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022 Article 9 (Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022). Association for Computing Machinery, Inc. https://doi.org/10.1145/3535508.3545541

Supervised pretraining through contrastive categorical positive samplings to improve COVID-19 mortality prediction. / Wanyan, Tingyi; Lin, Mingquan; Klang, Eyal et al.
Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022. Association for Computing Machinery, Inc, 2022. 9 (Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Wanyan, T, Lin, M, Klang, E, Menon, KM, Gulamali, FF, Azad, A, Zhang, Y, Ding, Y, Wang, Z, Wang, F, Glicksberg, B & Peng, Y 2022, Supervised pretraining through contrastive categorical positive samplings to improve COVID-19 mortality prediction. in Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022., 9, Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022, Association for Computing Machinery, Inc, 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022, Chicago, United States, 8/7/22. https://doi.org/10.1145/3535508.3545541

Wanyan T, Lin M, Klang E, Menon KM, Gulamali FF, Azad A et al. Supervised pretraining through contrastive categorical positive samplings to improve COVID-19 mortality prediction. In Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022. Association for Computing Machinery, Inc. 2022. 9. (Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022). doi: 10.1145/3535508.3545541

Wanyan, Tingyi ; Lin, Mingquan ; Klang, Eyal et al. / Supervised pretraining through contrastive categorical positive samplings to improve COVID-19 mortality prediction. Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022. Association for Computing Machinery, Inc, 2022. (Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022).

@inproceedings{2dfffdcb251b4e5c8ad2ae45481dcd95,

title = "Supervised pretraining through contrastive categorical positive samplings to improve COVID-19 mortality prediction",

abstract = "Clinical EHR data is naturally heterogeneous, where it contains abundant sub-phenotype. Such diversity creates challenges for outcome prediction using a machine learning model since it leads to high intra-class variance. To address this issue, we propose a supervised pre-Training model with a unique embedded k-nearest-neighbor positive sampling strategy. We demonstrate the enhanced performance value of this framework theoretically and show that it yields highly competitive experimental results in predicting patient mortality in real-world COVID-19 EHR data with a total of over 7,000 patients admitted to a large, urban health system. Our method achieves a better AUROC prediction score of 0.872, which outperforms the alternative pre-Training models and traditional machine learning methods. Additionally, our method performs much better when the training data size is small (345 training instances).",

keywords = "Intra-class variance, Mortality prediction, Pre-Training, Self-supervised learning, Sub-phenotype, Supervised contrastive learning",

author = "Tingyi Wanyan and Mingquan Lin and Eyal Klang and Menon, \{Kartikeya M.\} and Gulamali, \{Faris F.\} and Ariful Azad and Yiye Zhang and Ying Ding and Zhangyang Wang and Fei Wang and Benjamin Glicksberg and Yifan Peng",

note = "Publisher Copyright: {\textcopyright} 2022 ACM.; 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022 ; Conference date: 07-08-2022 Through 08-08-2022",

year = "2022",

month = aug,

day = "7",

doi = "10.1145/3535508.3545541",

language = "English (US)",

series = "Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022",

publisher = "Association for Computing Machinery, Inc",

booktitle = "Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022",

}

TY - GEN

T1 - Supervised pretraining through contrastive categorical positive samplings to improve COVID-19 mortality prediction

AU - Wanyan, Tingyi

AU - Lin, Mingquan

AU - Klang, Eyal

AU - Menon, Kartikeya M.

AU - Gulamali, Faris F.

AU - Azad, Ariful

AU - Zhang, Yiye

AU - Ding, Ying

AU - Wang, Zhangyang

AU - Wang, Fei

AU - Glicksberg, Benjamin

AU - Peng, Yifan

PY - 2022/8/7

Y1 - 2022/8/7

N2 - Clinical EHR data is naturally heterogeneous, where it contains abundant sub-phenotype. Such diversity creates challenges for outcome prediction using a machine learning model since it leads to high intra-class variance. To address this issue, we propose a supervised pre-Training model with a unique embedded k-nearest-neighbor positive sampling strategy. We demonstrate the enhanced performance value of this framework theoretically and show that it yields highly competitive experimental results in predicting patient mortality in real-world COVID-19 EHR data with a total of over 7,000 patients admitted to a large, urban health system. Our method achieves a better AUROC prediction score of 0.872, which outperforms the alternative pre-Training models and traditional machine learning methods. Additionally, our method performs much better when the training data size is small (345 training instances).

AB - Clinical EHR data is naturally heterogeneous, where it contains abundant sub-phenotype. Such diversity creates challenges for outcome prediction using a machine learning model since it leads to high intra-class variance. To address this issue, we propose a supervised pre-Training model with a unique embedded k-nearest-neighbor positive sampling strategy. We demonstrate the enhanced performance value of this framework theoretically and show that it yields highly competitive experimental results in predicting patient mortality in real-world COVID-19 EHR data with a total of over 7,000 patients admitted to a large, urban health system. Our method achieves a better AUROC prediction score of 0.872, which outperforms the alternative pre-Training models and traditional machine learning methods. Additionally, our method performs much better when the training data size is small (345 training instances).

KW - Intra-class variance

KW - Mortality prediction

KW - Pre-Training

KW - Self-supervised learning

KW - Sub-phenotype

KW - Supervised contrastive learning

UR - https://www.scopus.com/pages/publications/85136490570

UR - https://www.scopus.com/pages/publications/85136490570#tab=citedBy

U2 - 10.1145/3535508.3545541

DO - 10.1145/3535508.3545541

M3 - Conference contribution

AN - SCOPUS:85136490570

T3 - Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022

BT - Proceedings of the 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022

PB - Association for Computing Machinery, Inc

T2 - 13th ACM International Conference on Bioinformatics, Computational Biology and Health Informatics, BCB 2022

Y2 - 7 August 2022 through 8 August 2022

ER -

Supervised pretraining through contrastive categorical positive samplings to improve COVID-19 mortality prediction

Abstract

Publication series

Conference

Bibliographical note

UN SDGs

Keywords

Publisher link

Find It

Other files and links

Fingerprint

Cite this