An investigation into three visual characteristics of complex scenes that evoke human emotion

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Prior computational studies have examined hundreds of visual characteristics related to color, texture, and composition in an attempt to predict human emotional responses. Beyond those myriad features examined in computer science, roundness, angularity, and visual complexity have also been found to evoke emotions in human perceivers, as demonstrated in psychological studies of facial expressions, dance poses, and even simple synthetic visual patterns. Capturing these characteristics algorithmically to incorporate in computational studies, however, has proven difficult. Here we expand the scope of previous computer vision work by examining these three visual characteristics in computer analysis of complex scenes, and compare the results to the hundreds of visual qualities previously examined. A large collection of ecologically valid stimuli (i.e., photos that humans regularly encounter on the web), named the EmoSet and containing more than 40,000 images crawled from web albums, was generated using crowd-sourcing and subjected to human subject emotion ratings. We developed computational methods to the separate indices of roundness, angularity, and complexity, thereby establishing three new computational constructs. Critically, these three new physically interpretable visual constructs achieve comparable classification accuracy to the hundreds of shape, texture, composition, and facial feature characteristics previously examined. In addition, our experimental results show that color features related most strongly with the positivity of perceived emotions, the texture features related more to calmness or excitement, and roundness, angularity, and simplicity related similarly with both of these emotions dimensions.

Original languageEnglish (US)
Title of host publication2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages440-447
Number of pages8
ISBN (Electronic)9781538605639
DOIs
StatePublished - Jul 2 2017
Event7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017 - San Antonio, United States
Duration: Oct 23 2017Oct 26 2017

Publication series

Name2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017
Volume2018-January

Other

Other7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017
CountryUnited States
CitySan Antonio
Period10/23/1710/26/17

Fingerprint

Emotions
Textures
Color
Crowdsourcing
Dancing
Computational methods
Chemical analysis
Computer science
Computer vision
Facial Expression
Psychology

All Science Journal Classification (ASJC) codes

  • Human-Computer Interaction
  • Behavioral Neuroscience
  • Social Psychology
  • Artificial Intelligence

Cite this

Lu, X., Adams, R. B., Li, J., Newman, M. G., & Wang, J. Z. (2017). An investigation into three visual characteristics of complex scenes that evoke human emotion. In 2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017 (pp. 440-447). (2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017; Vol. 2018-January). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ACII.2017.8273637
Lu, Xin ; Adams, Reginald B. ; Li, Jia ; Newman, Michelle G. ; Wang, James Z. / An investigation into three visual characteristics of complex scenes that evoke human emotion. 2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017. Institute of Electrical and Electronics Engineers Inc., 2017. pp. 440-447 (2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017).
@inproceedings{c65f54e72de94b3f9d8e4fbd2f0632b2,
title = "An investigation into three visual characteristics of complex scenes that evoke human emotion",
abstract = "Prior computational studies have examined hundreds of visual characteristics related to color, texture, and composition in an attempt to predict human emotional responses. Beyond those myriad features examined in computer science, roundness, angularity, and visual complexity have also been found to evoke emotions in human perceivers, as demonstrated in psychological studies of facial expressions, dance poses, and even simple synthetic visual patterns. Capturing these characteristics algorithmically to incorporate in computational studies, however, has proven difficult. Here we expand the scope of previous computer vision work by examining these three visual characteristics in computer analysis of complex scenes, and compare the results to the hundreds of visual qualities previously examined. A large collection of ecologically valid stimuli (i.e., photos that humans regularly encounter on the web), named the EmoSet and containing more than 40,000 images crawled from web albums, was generated using crowd-sourcing and subjected to human subject emotion ratings. We developed computational methods to the separate indices of roundness, angularity, and complexity, thereby establishing three new computational constructs. Critically, these three new physically interpretable visual constructs achieve comparable classification accuracy to the hundreds of shape, texture, composition, and facial feature characteristics previously examined. In addition, our experimental results show that color features related most strongly with the positivity of perceived emotions, the texture features related more to calmness or excitement, and roundness, angularity, and simplicity related similarly with both of these emotions dimensions.",
author = "Xin Lu and Adams, {Reginald B.} and Jia Li and Newman, {Michelle G.} and Wang, {James Z.}",
year = "2017",
month = "7",
day = "2",
doi = "10.1109/ACII.2017.8273637",
language = "English (US)",
series = "2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "440--447",
booktitle = "2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017",
address = "United States",

}

Lu, X, Adams, RB, Li, J, Newman, MG & Wang, JZ 2017, An investigation into three visual characteristics of complex scenes that evoke human emotion. in 2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017. 2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017, vol. 2018-January, Institute of Electrical and Electronics Engineers Inc., pp. 440-447, 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017, San Antonio, United States, 10/23/17. https://doi.org/10.1109/ACII.2017.8273637

An investigation into three visual characteristics of complex scenes that evoke human emotion. / Lu, Xin; Adams, Reginald B.; Li, Jia; Newman, Michelle G.; Wang, James Z.

2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017. Institute of Electrical and Electronics Engineers Inc., 2017. p. 440-447 (2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017; Vol. 2018-January).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - An investigation into three visual characteristics of complex scenes that evoke human emotion

AU - Lu, Xin

AU - Adams, Reginald B.

AU - Li, Jia

AU - Newman, Michelle G.

AU - Wang, James Z.

PY - 2017/7/2

Y1 - 2017/7/2

N2 - Prior computational studies have examined hundreds of visual characteristics related to color, texture, and composition in an attempt to predict human emotional responses. Beyond those myriad features examined in computer science, roundness, angularity, and visual complexity have also been found to evoke emotions in human perceivers, as demonstrated in psychological studies of facial expressions, dance poses, and even simple synthetic visual patterns. Capturing these characteristics algorithmically to incorporate in computational studies, however, has proven difficult. Here we expand the scope of previous computer vision work by examining these three visual characteristics in computer analysis of complex scenes, and compare the results to the hundreds of visual qualities previously examined. A large collection of ecologically valid stimuli (i.e., photos that humans regularly encounter on the web), named the EmoSet and containing more than 40,000 images crawled from web albums, was generated using crowd-sourcing and subjected to human subject emotion ratings. We developed computational methods to the separate indices of roundness, angularity, and complexity, thereby establishing three new computational constructs. Critically, these three new physically interpretable visual constructs achieve comparable classification accuracy to the hundreds of shape, texture, composition, and facial feature characteristics previously examined. In addition, our experimental results show that color features related most strongly with the positivity of perceived emotions, the texture features related more to calmness or excitement, and roundness, angularity, and simplicity related similarly with both of these emotions dimensions.

AB - Prior computational studies have examined hundreds of visual characteristics related to color, texture, and composition in an attempt to predict human emotional responses. Beyond those myriad features examined in computer science, roundness, angularity, and visual complexity have also been found to evoke emotions in human perceivers, as demonstrated in psychological studies of facial expressions, dance poses, and even simple synthetic visual patterns. Capturing these characteristics algorithmically to incorporate in computational studies, however, has proven difficult. Here we expand the scope of previous computer vision work by examining these three visual characteristics in computer analysis of complex scenes, and compare the results to the hundreds of visual qualities previously examined. A large collection of ecologically valid stimuli (i.e., photos that humans regularly encounter on the web), named the EmoSet and containing more than 40,000 images crawled from web albums, was generated using crowd-sourcing and subjected to human subject emotion ratings. We developed computational methods to the separate indices of roundness, angularity, and complexity, thereby establishing three new computational constructs. Critically, these three new physically interpretable visual constructs achieve comparable classification accuracy to the hundreds of shape, texture, composition, and facial feature characteristics previously examined. In addition, our experimental results show that color features related most strongly with the positivity of perceived emotions, the texture features related more to calmness or excitement, and roundness, angularity, and simplicity related similarly with both of these emotions dimensions.

UR - http://www.scopus.com/inward/record.url?scp=85047316093&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85047316093&partnerID=8YFLogxK

U2 - 10.1109/ACII.2017.8273637

DO - 10.1109/ACII.2017.8273637

M3 - Conference contribution

AN - SCOPUS:85047316093

T3 - 2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017

SP - 440

EP - 447

BT - 2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Lu X, Adams RB, Li J, Newman MG, Wang JZ. An investigation into three visual characteristics of complex scenes that evoke human emotion. In 2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017. Institute of Electrical and Electronics Engineers Inc. 2017. p. 440-447. (2017 7th International Conference on Affective Computing and Intelligent Interaction, ACII 2017). https://doi.org/10.1109/ACII.2017.8273637