Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype

Heather D. Couture, Lindsay A. Williams, Joseph Geradts, Sarah J. Nyante, Ebonee N. Butler, J. S. Marron, Charles M. Perou, Melissa A. Troester, Marc Niethammer

Research output: Contribution to journalArticlepeer-review

75 Scopus citations


RNA-based, multi-gene molecular assays are available and widely used for patients with ER-positive/HER2-negative breast cancers. However, RNA-based genomic tests can be costly and are not available in many countries. Methods for inferring molecular subtype from histologic images may identify patients most likely to benefit from further genomic testing. To identify patients who could benefit from molecular testing based on H&E stained histologic images, we developed an image analysis approach using deep learning. A training set of 571 breast tumors was used to create image-based classifiers for tumor grade, ER status, PAM50 intrinsic subtype, histologic subtype, and risk of recurrence score (ROR-PT). The resulting classifiers were applied to an independent test set (n = 288), and accuracy, sensitivity, and specificity of each was assessed on the test set. Histologic image analysis with deep learning distinguished low-intermediate vs. high tumor grade (82% accuracy), ER status (84% accuracy), Basal-like vs. non-Basal-like (77% accuracy), Ductal vs. Lobular (94% accuracy), and high vs. low-medium ROR-PT score (75% accuracy). Sampling considerations in the training set minimized bias in the test set. Incorrect classification of ER status was significantly more common for Luminal B tumors. These data provide proof of principle that molecular marker status, including a critical clinical biomarker (i.e., ER status), can be predicted with accuracy >75% based on H&E features. Image-based methods could be promising for identifying patients with a greater need for further genomic testing, or in place of classically scored variables typically accomplished using human-based scoring.

Original languageEnglish (US)
Article number30
Journalnpj Breast Cancer
Issue number1
StatePublished - Dec 1 2018
Externally publishedYes

Bibliographical note

Funding Information:
This study was supported by funds from the following sources: a grant from the UNC Lineberger Comprehensive Cancer Center funded by the University Cancer Research Fund (LCCC2017T204), NCI P50-CA58223 Breast SPORE program, R01-CA148761 (CMP), the Breast Cancer Research Foundation, Susan G. Komen, and U01-CA179715. The Tesla K40 used for this research was donated by the NVIDIA Corporation.

Publisher Copyright:
© 2018, The Author(s).


Dive into the research topics of 'Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype'. Together they form a unique fingerprint.

Cite this