Image Visual Realism: From Human Perception to Machine Computation

Shaojing Fan, Tian Tsong Ng, Bryan L. Koenig, Jonathan S. Herberg, Ming Jiang, Zhiqi Shen, Qi Zhao

Research output: Contribution to journalArticlepeer-review

9 Scopus citations

Abstract

Visual realism is defined as the extent to which an image appears to people as a photo rather than computer generated. Assessing visual realism is important in applications like computer graphics rendering and photo retouching. However, current realism evaluation approaches use either labor-intensive human judgments or automated algorithms largely dependent on comparing renderings to reference images. We develop a reference-free computational framework for visual realism prediction to overcome these constraints. First, we construct a benchmark dataset of 2,520 images with comprehensive human annotated attributes. From statistical modeling on this data, we identify image attributes most relevant for visual realism. We propose both empirically-based (guided by our statistical modeling of human data) and deep convolutional neural network models to predict visual realism of images. Our framework has the following advantages: (1) it creates an interpretable and concise empirical model that characterizes human perception of visual realism; (2) it links computational features to latent factors of human image perception.

Original languageEnglish (US)
Article number8022957
Pages (from-to)2180-2193
Number of pages14
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume40
Issue number9
DOIs
StatePublished - Sep 1 2018

Bibliographical note

Funding Information:
We thank Qi Shan and his colleagues from University of Washington for sharing their dataset. We thank following people for their contribution to this work in one way or another: Dr. Cheston Y.-C Tan, Mr. Karianto Leman, Mr. Zhang Fan, Dr. Chu Xinqi, Dr. Wang Hee Lin, Prof. Liu Zhen, and all the reviewers for our previous papers on this topic. This research is supported by the National Research Foundation, Prime Minister’s Office, Singapore under its International Research Centre in Singapore Funding Initiative, and a University of Minnesota Department of Computer Science and Engineering Start-up Fund (QZ).

Publisher Copyright:
© 1979-2012 IEEE.

Keywords

  • Visual realism
  • convolutional neural network
  • human psychophysics
  • statistical modeling

Fingerprint

Dive into the research topics of 'Image Visual Realism: From Human Perception to Machine Computation'. Together they form a unique fingerprint.

Cite this