In many sub-fields, researchers collect datasets of human ground truth that are used to create a new algorithm. For example, in research on image perception, datasets have been collected for topics such as what makes an image aesthetic or memorable. Despite high costs for human data collection, datasets are infrequently reused beyond their own fields of interest. Moreover, the algorithms built from them are domain-specific (predict a small set of attributes) and usually unconnected to one another. In this paper, we present a paradigm for building generalized and expandable models of human image perception. First, we fuse multiple fragmented and partially-overlapping datasets through data imputation. We then create a theoretically-structured statistical model of human image perception that is fit to the fused datasets. The resulting model has many advantages. (1) It is generalized, going beyond the content of the constituent datasets, and can be easily expanded by fusing additional datasets. (2) It provides a new ontology usable as a network to expand human data in a cost-effective way. (3) It can guide the design of a generalized computational algorithm for multi-dimensional visual perception. Indeed, experimental results show that a model-based algorithm outperforms state-of-the-art methods on predicting visual sentiment, visual realism and interestingness. Our paradigm can be used in various visual tasks (e.g., video summarization).
|Original language||English (US)|
|Title of host publication||Proceedings - 29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016|
|Publisher||IEEE Computer Society|
|Number of pages||10|
|State||Published - Dec 9 2016|
|Event||29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016 - Las Vegas, United States|
Duration: Jun 26 2016 → Jul 1 2016
|Name||Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition|
|Conference||29th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016|
|Period||6/26/16 → 7/1/16|
Bibliographical noteFunding Information:
We would like to thank Robert Kirkpatrick and Michael Neale for helpful discussions on statistical modeling. This research is supported by the National Research Foundation, Prime Ministers Office, Singapore under its International Research Centre in Singapore Funding Initiative
© 2016 IEEE.
Copyright 2017 Elsevier B.V., All rights reserved.