Beyond saliency: Assessing visual balance with high-level cues

Baris Kandemir, Zihan Zhou, Jia Li, James Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Automatic composition optimization is a vital technique for computational photography systems. Balance in composition is one of the agreed-upon principles of aesthetics and is commonly employed as a visual feature in many computational aesthetics studies. It refers to an equilibrium of visual weights within composition. Existing composition optimization and aesthetic quality assessment systems utilize the saliency map to represent balance. However, saliency map methods fail to account for high-level visual features that are important for compositional balance. Our work establishes a framework for the purpose of evaluating the relationship between visual features and compositional balance. This provides a better understanding of compositional balance and help improve composition optimization performance. A dataset based on a human subject study was created with photos representing main balance concepts such as symmetric, dynamic balance, and imbalance. We take the visual center given by human subjects as the dependent variable and the center-of-mass for each type of visual features as the predictor variable. Based on a linear regression model, we can assess how much each type of visual features contributes to the prediction of the visual center. Our findings show that highlevel visual elements can help increase prediction accuracy with significance on top of saliency maps. Specifically, extra information provided through human and dominant vanishing point detection is statistically significant for assessing balance in the composition.

Original languageEnglish (US)
Title of host publicationThematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017
PublisherAssociation for Computing Machinery, Inc
Pages26-34
Number of pages9
ISBN (Electronic)9781450354165
DOIs
StatePublished - Oct 23 2017
Event1st International ACM Thematic Workshops, Thematic Workshops 2017 - Mountain View, United States
Duration: Oct 23 2017Oct 27 2017

Other

Other1st International ACM Thematic Workshops, Thematic Workshops 2017
CountryUnited States
CityMountain View
Period10/23/1710/27/17

Fingerprint

Saliency
Chemical analysis
Saliency Map
Photography
Linear regression
Vision
Optimization
Performance Optimization
Prediction
Quality Assessment
Barycentre
Linear Regression Model
Predictors
Dependent

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Computational Theory and Mathematics
  • Theoretical Computer Science

Cite this

Kandemir, B., Zhou, Z., Li, J., & Wang, J. (2017). Beyond saliency: Assessing visual balance with high-level cues. In Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017 (pp. 26-34). Association for Computing Machinery, Inc. https://doi.org/10.1145/3126686.3126712
Kandemir, Baris ; Zhou, Zihan ; Li, Jia ; Wang, James. / Beyond saliency : Assessing visual balance with high-level cues. Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017. Association for Computing Machinery, Inc, 2017. pp. 26-34
@inproceedings{2bf0d4672bef4ab584383a47efe06631,
title = "Beyond saliency: Assessing visual balance with high-level cues",
abstract = "Automatic composition optimization is a vital technique for computational photography systems. Balance in composition is one of the agreed-upon principles of aesthetics and is commonly employed as a visual feature in many computational aesthetics studies. It refers to an equilibrium of visual weights within composition. Existing composition optimization and aesthetic quality assessment systems utilize the saliency map to represent balance. However, saliency map methods fail to account for high-level visual features that are important for compositional balance. Our work establishes a framework for the purpose of evaluating the relationship between visual features and compositional balance. This provides a better understanding of compositional balance and help improve composition optimization performance. A dataset based on a human subject study was created with photos representing main balance concepts such as symmetric, dynamic balance, and imbalance. We take the visual center given by human subjects as the dependent variable and the center-of-mass for each type of visual features as the predictor variable. Based on a linear regression model, we can assess how much each type of visual features contributes to the prediction of the visual center. Our findings show that highlevel visual elements can help increase prediction accuracy with significance on top of saliency maps. Specifically, extra information provided through human and dominant vanishing point detection is statistically significant for assessing balance in the composition.",
author = "Baris Kandemir and Zihan Zhou and Jia Li and James Wang",
year = "2017",
month = "10",
day = "23",
doi = "10.1145/3126686.3126712",
language = "English (US)",
pages = "26--34",
booktitle = "Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017",
publisher = "Association for Computing Machinery, Inc",

}

Kandemir, B, Zhou, Z, Li, J & Wang, J 2017, Beyond saliency: Assessing visual balance with high-level cues. in Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017. Association for Computing Machinery, Inc, pp. 26-34, 1st International ACM Thematic Workshops, Thematic Workshops 2017, Mountain View, United States, 10/23/17. https://doi.org/10.1145/3126686.3126712

Beyond saliency : Assessing visual balance with high-level cues. / Kandemir, Baris; Zhou, Zihan; Li, Jia; Wang, James.

Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017. Association for Computing Machinery, Inc, 2017. p. 26-34.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Beyond saliency

T2 - Assessing visual balance with high-level cues

AU - Kandemir, Baris

AU - Zhou, Zihan

AU - Li, Jia

AU - Wang, James

PY - 2017/10/23

Y1 - 2017/10/23

N2 - Automatic composition optimization is a vital technique for computational photography systems. Balance in composition is one of the agreed-upon principles of aesthetics and is commonly employed as a visual feature in many computational aesthetics studies. It refers to an equilibrium of visual weights within composition. Existing composition optimization and aesthetic quality assessment systems utilize the saliency map to represent balance. However, saliency map methods fail to account for high-level visual features that are important for compositional balance. Our work establishes a framework for the purpose of evaluating the relationship between visual features and compositional balance. This provides a better understanding of compositional balance and help improve composition optimization performance. A dataset based on a human subject study was created with photos representing main balance concepts such as symmetric, dynamic balance, and imbalance. We take the visual center given by human subjects as the dependent variable and the center-of-mass for each type of visual features as the predictor variable. Based on a linear regression model, we can assess how much each type of visual features contributes to the prediction of the visual center. Our findings show that highlevel visual elements can help increase prediction accuracy with significance on top of saliency maps. Specifically, extra information provided through human and dominant vanishing point detection is statistically significant for assessing balance in the composition.

AB - Automatic composition optimization is a vital technique for computational photography systems. Balance in composition is one of the agreed-upon principles of aesthetics and is commonly employed as a visual feature in many computational aesthetics studies. It refers to an equilibrium of visual weights within composition. Existing composition optimization and aesthetic quality assessment systems utilize the saliency map to represent balance. However, saliency map methods fail to account for high-level visual features that are important for compositional balance. Our work establishes a framework for the purpose of evaluating the relationship between visual features and compositional balance. This provides a better understanding of compositional balance and help improve composition optimization performance. A dataset based on a human subject study was created with photos representing main balance concepts such as symmetric, dynamic balance, and imbalance. We take the visual center given by human subjects as the dependent variable and the center-of-mass for each type of visual features as the predictor variable. Based on a linear regression model, we can assess how much each type of visual features contributes to the prediction of the visual center. Our findings show that highlevel visual elements can help increase prediction accuracy with significance on top of saliency maps. Specifically, extra information provided through human and dominant vanishing point detection is statistically significant for assessing balance in the composition.

UR - http://www.scopus.com/inward/record.url?scp=85034836058&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85034836058&partnerID=8YFLogxK

U2 - 10.1145/3126686.3126712

DO - 10.1145/3126686.3126712

M3 - Conference contribution

AN - SCOPUS:85034836058

SP - 26

EP - 34

BT - Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017

PB - Association for Computing Machinery, Inc

ER -

Kandemir B, Zhou Z, Li J, Wang J. Beyond saliency: Assessing visual balance with high-level cues. In Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017. Association for Computing Machinery, Inc. 2017. p. 26-34 https://doi.org/10.1145/3126686.3126712