Intelligent portrait composition assistance: Integrating deep-learned models and photography idea retrieval

Farshid Farhat, Mohammad Mahdi Kamani, Sahil Mishra, James Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

Retrieving photography ideas corresponding to a given location facilitates the usage of smart cameras, where there is a high interest among amateurs and enthusiasts to take astonishing photos at anytime and in any location. Existing research captures some aesthetic techniques and retrieves useful feedbacks based on one technique. However, they are restricted to a particular technique and the retrieved results have room to improve as they are confined to the quality of the query. There is a lack of a holistic framework to capture important aspects of a given scene and help a novice photographer by informative feedback to take a be.er shot in his/her photography adventure. This work proposes an intelligent framework of portrait composition using our deep-learned models and image retrieval methods. A highly-rated web-crawled portrait dataset is exploited for retrieval purposes. Our framework detects and extracts ingredients of a given scene representing as a correlated semantic model. It then matches extracted semantics with the dataset of aesthetically composed photos to investigate a ranked list of photography ideas, and gradually optimizes the human pose and other artistic aspects of the composed scene supposed to be captured. The conducted user study demonstrates that our approach is more helpful than other feedback retrieval systems.

Original languageEnglish (US)
Title of host publicationThematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017
PublisherAssociation for Computing Machinery, Inc
Pages17-25
Number of pages9
ISBN (Electronic)9781450354165
DOIs
StatePublished - Oct 23 2017
Event1st International ACM Thematic Workshops, Thematic Workshops 2017 - Mountain View, United States
Duration: Oct 23 2017Oct 27 2017

Publication series

NameThematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017

Other

Other1st International ACM Thematic Workshops, Thematic Workshops 2017
CountryUnited States
CityMountain View
Period10/23/1710/27/17

Fingerprint

Photography
Retrieval
Feedback
Chemical analysis
Semantics
User Studies
Image retrieval
Image Retrieval
Camera
Cameras
Optimise
Model
Query
Demonstrate
Framework

All Science Journal Classification (ASJC) codes

  • Computer Science Applications
  • Computational Theory and Mathematics
  • Theoretical Computer Science

Cite this

Farhat, F., Kamani, M. M., Mishra, S., & Wang, J. (2017). Intelligent portrait composition assistance: Integrating deep-learned models and photography idea retrieval. In Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017 (pp. 17-25). (Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017). Association for Computing Machinery, Inc. https://doi.org/10.1145/3126686.3126710
Farhat, Farshid ; Kamani, Mohammad Mahdi ; Mishra, Sahil ; Wang, James. / Intelligent portrait composition assistance : Integrating deep-learned models and photography idea retrieval. Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017. Association for Computing Machinery, Inc, 2017. pp. 17-25 (Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017).
@inproceedings{d347b40e4c9f4813ab1848a8b655edb0,
title = "Intelligent portrait composition assistance: Integrating deep-learned models and photography idea retrieval",
abstract = "Retrieving photography ideas corresponding to a given location facilitates the usage of smart cameras, where there is a high interest among amateurs and enthusiasts to take astonishing photos at anytime and in any location. Existing research captures some aesthetic techniques and retrieves useful feedbacks based on one technique. However, they are restricted to a particular technique and the retrieved results have room to improve as they are confined to the quality of the query. There is a lack of a holistic framework to capture important aspects of a given scene and help a novice photographer by informative feedback to take a be.er shot in his/her photography adventure. This work proposes an intelligent framework of portrait composition using our deep-learned models and image retrieval methods. A highly-rated web-crawled portrait dataset is exploited for retrieval purposes. Our framework detects and extracts ingredients of a given scene representing as a correlated semantic model. It then matches extracted semantics with the dataset of aesthetically composed photos to investigate a ranked list of photography ideas, and gradually optimizes the human pose and other artistic aspects of the composed scene supposed to be captured. The conducted user study demonstrates that our approach is more helpful than other feedback retrieval systems.",
author = "Farshid Farhat and Kamani, {Mohammad Mahdi} and Sahil Mishra and James Wang",
year = "2017",
month = "10",
day = "23",
doi = "10.1145/3126686.3126710",
language = "English (US)",
series = "Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017",
publisher = "Association for Computing Machinery, Inc",
pages = "17--25",
booktitle = "Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017",

}

Farhat, F, Kamani, MM, Mishra, S & Wang, J 2017, Intelligent portrait composition assistance: Integrating deep-learned models and photography idea retrieval. in Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017. Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017, Association for Computing Machinery, Inc, pp. 17-25, 1st International ACM Thematic Workshops, Thematic Workshops 2017, Mountain View, United States, 10/23/17. https://doi.org/10.1145/3126686.3126710

Intelligent portrait composition assistance : Integrating deep-learned models and photography idea retrieval. / Farhat, Farshid; Kamani, Mohammad Mahdi; Mishra, Sahil; Wang, James.

Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017. Association for Computing Machinery, Inc, 2017. p. 17-25 (Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Intelligent portrait composition assistance

T2 - Integrating deep-learned models and photography idea retrieval

AU - Farhat, Farshid

AU - Kamani, Mohammad Mahdi

AU - Mishra, Sahil

AU - Wang, James

PY - 2017/10/23

Y1 - 2017/10/23

N2 - Retrieving photography ideas corresponding to a given location facilitates the usage of smart cameras, where there is a high interest among amateurs and enthusiasts to take astonishing photos at anytime and in any location. Existing research captures some aesthetic techniques and retrieves useful feedbacks based on one technique. However, they are restricted to a particular technique and the retrieved results have room to improve as they are confined to the quality of the query. There is a lack of a holistic framework to capture important aspects of a given scene and help a novice photographer by informative feedback to take a be.er shot in his/her photography adventure. This work proposes an intelligent framework of portrait composition using our deep-learned models and image retrieval methods. A highly-rated web-crawled portrait dataset is exploited for retrieval purposes. Our framework detects and extracts ingredients of a given scene representing as a correlated semantic model. It then matches extracted semantics with the dataset of aesthetically composed photos to investigate a ranked list of photography ideas, and gradually optimizes the human pose and other artistic aspects of the composed scene supposed to be captured. The conducted user study demonstrates that our approach is more helpful than other feedback retrieval systems.

AB - Retrieving photography ideas corresponding to a given location facilitates the usage of smart cameras, where there is a high interest among amateurs and enthusiasts to take astonishing photos at anytime and in any location. Existing research captures some aesthetic techniques and retrieves useful feedbacks based on one technique. However, they are restricted to a particular technique and the retrieved results have room to improve as they are confined to the quality of the query. There is a lack of a holistic framework to capture important aspects of a given scene and help a novice photographer by informative feedback to take a be.er shot in his/her photography adventure. This work proposes an intelligent framework of portrait composition using our deep-learned models and image retrieval methods. A highly-rated web-crawled portrait dataset is exploited for retrieval purposes. Our framework detects and extracts ingredients of a given scene representing as a correlated semantic model. It then matches extracted semantics with the dataset of aesthetically composed photos to investigate a ranked list of photography ideas, and gradually optimizes the human pose and other artistic aspects of the composed scene supposed to be captured. The conducted user study demonstrates that our approach is more helpful than other feedback retrieval systems.

UR - http://www.scopus.com/inward/record.url?scp=85034851160&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85034851160&partnerID=8YFLogxK

U2 - 10.1145/3126686.3126710

DO - 10.1145/3126686.3126710

M3 - Conference contribution

AN - SCOPUS:85034851160

T3 - Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017

SP - 17

EP - 25

BT - Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017

PB - Association for Computing Machinery, Inc

ER -

Farhat F, Kamani MM, Mishra S, Wang J. Intelligent portrait composition assistance: Integrating deep-learned models and photography idea retrieval. In Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017. Association for Computing Machinery, Inc. 2017. p. 17-25. (Thematic Workshops 2017 - Proceedings of the Thematic Workshops of ACM Multimedia 2017, co-located with MM 2017). https://doi.org/10.1145/3126686.3126710