DeepSeg: A transfer-learning segmentation tool for limited sample training of nonhuman primate MRI

Tissue segmentation of individual magnetic resonance imaging (MRI) is a fundamental step in building accurate head models for brain stimulation. In nonhuman primates (NHPs), due to limited sample size, site variability, and sub-optimal image quality, it is challenging to automate the tissue segmentation process. To overcome these challenges, we leveraged a recent transfer-learning framework for brain extraction and developed an automatic segmentation tool, DeepSeg, a U-Net model for NHP MRI data. We trained two DeepSeg models - a brain tissue model and a full head model - in a relatively large human dataset and then transferred them to limited macaque samples. We demonstrated that both full head and brain tissue models achieved good segmentation performance on the macaque test samples from the same training sites and also showed promising results on multi-site data (Dice coefficient mean±standard deviation for full-head within-site test sample: 0.88 ± 0.08; full-head out-of-site test sample: 0.72 ± 0.17). We further showed that the transferred brain tissue model outperformed a traditional template-driven approach, the prior-based ANTs segmentation (Dice coefficient mean±standard deviation for white matter: 0.90 ± 0.04 vs. 0.85 ± 0.03; gray matter: 0.82±0.07 vs. 0.81±0.04). We then discussed possible solutions to improve model generalizability. Overall, despite limited training samples, our preliminary results demonstrate that DeepSeg is a promising segmentation tool for NHP MRI data.