Deep Residual Network Predicts Cortical Representation and Organization of Visual Features for Rapid Categorization

Haiguang Wen, Junxing Shi, Wei Chen, Zhongming Liu

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

The brain represents visual objects with topographic cortical patterns. To address how distributed visual representations enable object categorization, we established predictive encoding models based on a deep residual network, and trained them to predict cortical responses to natural movies. Using this predictive model, we mapped human cortical representations to 64,000 visual objects from 80 categories with high throughput and accuracy. Such representations covered both the ventral and dorsal pathways, reflected multiple levels of object features, and preserved semantic relationships between categories. In the entire visual cortex, object representations were organized into three clusters of categories: Biological objects, non-biological objects, and background scenes. In a finer scale specific to each cluster, object representations revealed sub-clusters for further categorization. Such hierarchical clustering of category representations was mostly contributed by cortical representations of object features from middle to high levels. In summary, this study demonstrates a useful computational strategy to characterize the cortical organization and representations of visual features for rapid categorization.

Original languageEnglish (US)
Article number3752
JournalScientific reports
Volume8
Issue number1
DOIs
StatePublished - Dec 1 2018

Fingerprint

Brain
Semantics
Throughput

PubMed: MeSH publication types

  • Journal Article
  • Research Support, N.I.H., Extramural
  • Research Support, Non-U.S. Gov't

Cite this

Deep Residual Network Predicts Cortical Representation and Organization of Visual Features for Rapid Categorization. / Wen, Haiguang; Shi, Junxing; Chen, Wei; Liu, Zhongming.

In: Scientific reports, Vol. 8, No. 1, 3752, 01.12.2018.

Research output: Contribution to journalArticle

@article{1a528dc384384aa2b37d1102198ddd04,
title = "Deep Residual Network Predicts Cortical Representation and Organization of Visual Features for Rapid Categorization",
abstract = "The brain represents visual objects with topographic cortical patterns. To address how distributed visual representations enable object categorization, we established predictive encoding models based on a deep residual network, and trained them to predict cortical responses to natural movies. Using this predictive model, we mapped human cortical representations to 64,000 visual objects from 80 categories with high throughput and accuracy. Such representations covered both the ventral and dorsal pathways, reflected multiple levels of object features, and preserved semantic relationships between categories. In the entire visual cortex, object representations were organized into three clusters of categories: Biological objects, non-biological objects, and background scenes. In a finer scale specific to each cluster, object representations revealed sub-clusters for further categorization. Such hierarchical clustering of category representations was mostly contributed by cortical representations of object features from middle to high levels. In summary, this study demonstrates a useful computational strategy to characterize the cortical organization and representations of visual features for rapid categorization.",
author = "Haiguang Wen and Junxing Shi and Wei Chen and Zhongming Liu",
year = "2018",
month = "12",
day = "1",
doi = "10.1038/s41598-018-22160-9",
language = "English (US)",
volume = "8",
journal = "Scientific Reports",
issn = "2045-2322",
publisher = "Nature Publishing Group",
number = "1",

}

TY - JOUR

T1 - Deep Residual Network Predicts Cortical Representation and Organization of Visual Features for Rapid Categorization

AU - Wen, Haiguang

AU - Shi, Junxing

AU - Chen, Wei

AU - Liu, Zhongming

PY - 2018/12/1

Y1 - 2018/12/1

N2 - The brain represents visual objects with topographic cortical patterns. To address how distributed visual representations enable object categorization, we established predictive encoding models based on a deep residual network, and trained them to predict cortical responses to natural movies. Using this predictive model, we mapped human cortical representations to 64,000 visual objects from 80 categories with high throughput and accuracy. Such representations covered both the ventral and dorsal pathways, reflected multiple levels of object features, and preserved semantic relationships between categories. In the entire visual cortex, object representations were organized into three clusters of categories: Biological objects, non-biological objects, and background scenes. In a finer scale specific to each cluster, object representations revealed sub-clusters for further categorization. Such hierarchical clustering of category representations was mostly contributed by cortical representations of object features from middle to high levels. In summary, this study demonstrates a useful computational strategy to characterize the cortical organization and representations of visual features for rapid categorization.

AB - The brain represents visual objects with topographic cortical patterns. To address how distributed visual representations enable object categorization, we established predictive encoding models based on a deep residual network, and trained them to predict cortical responses to natural movies. Using this predictive model, we mapped human cortical representations to 64,000 visual objects from 80 categories with high throughput and accuracy. Such representations covered both the ventral and dorsal pathways, reflected multiple levels of object features, and preserved semantic relationships between categories. In the entire visual cortex, object representations were organized into three clusters of categories: Biological objects, non-biological objects, and background scenes. In a finer scale specific to each cluster, object representations revealed sub-clusters for further categorization. Such hierarchical clustering of category representations was mostly contributed by cortical representations of object features from middle to high levels. In summary, this study demonstrates a useful computational strategy to characterize the cortical organization and representations of visual features for rapid categorization.

UR - http://www.scopus.com/inward/record.url?scp=85043235657&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85043235657&partnerID=8YFLogxK

U2 - 10.1038/s41598-018-22160-9

DO - 10.1038/s41598-018-22160-9

M3 - Article

C2 - 29491405

AN - SCOPUS:85043235657

VL - 8

JO - Scientific Reports

JF - Scientific Reports

SN - 2045-2322

IS - 1

M1 - 3752

ER -