Repfinder: Finding approximately repeated scene elements for image editing

Ming Ming Cheng, Fang Lue Zhang, Niloy J. Mitra, Sharon Xiaolei Huang, Shi Min Hu

Research output: Contribution to journalArticle

91 Citations (Scopus)

Abstract

Repeated elements are ubiquitous and abundant in both manmade and natural scenes. Editing such images while preserving the repetitions and their relations is nontrivial due to overlap, missing parts, deformation across instances, illumination variation, etc. Manually enforcing such relations is laborious and error-prone. We propose a novel framework where user scribbles are used to guide detection and extraction of such repeated elements. Our detection process, which is based on a novel boundary band method, robustly extracts the repetitions along with their deformations. The algorithm only considers the shape of the elements, and ignores similarity based on color, texture, etc. We then use topological sorting to establish a partial depth ordering of overlapping repeated instances. Missing parts on occluded instances are completed using information from other instances. The extracted repeated instances can then be seamlessly edited and manipulated for a variety of high level tasks that are otherwise difficult to perform. We demonstrate the versatility of our framework on a large set of inputs of varying complexity, showing applications to image rearrangement, edit transfer, deformation propagation, and instance replacement.

Original languageEnglish (US)
Article number83
JournalACM Transactions on Graphics
Volume29
Issue number4
DOIs
StatePublished - Sep 14 2010

Fingerprint

Sorting
Textures
Lighting
Color

All Science Journal Classification (ASJC) codes

  • Computer Graphics and Computer-Aided Design

Cite this

Cheng, Ming Ming ; Zhang, Fang Lue ; Mitra, Niloy J. ; Huang, Sharon Xiaolei ; Hu, Shi Min. / Repfinder : Finding approximately repeated scene elements for image editing. In: ACM Transactions on Graphics. 2010 ; Vol. 29, No. 4.
@article{88cac2a7e6564ffdb2152d80e47b6da9,
title = "Repfinder: Finding approximately repeated scene elements for image editing",
abstract = "Repeated elements are ubiquitous and abundant in both manmade and natural scenes. Editing such images while preserving the repetitions and their relations is nontrivial due to overlap, missing parts, deformation across instances, illumination variation, etc. Manually enforcing such relations is laborious and error-prone. We propose a novel framework where user scribbles are used to guide detection and extraction of such repeated elements. Our detection process, which is based on a novel boundary band method, robustly extracts the repetitions along with their deformations. The algorithm only considers the shape of the elements, and ignores similarity based on color, texture, etc. We then use topological sorting to establish a partial depth ordering of overlapping repeated instances. Missing parts on occluded instances are completed using information from other instances. The extracted repeated instances can then be seamlessly edited and manipulated for a variety of high level tasks that are otherwise difficult to perform. We demonstrate the versatility of our framework on a large set of inputs of varying complexity, showing applications to image rearrangement, edit transfer, deformation propagation, and instance replacement.",
author = "Cheng, {Ming Ming} and Zhang, {Fang Lue} and Mitra, {Niloy J.} and Huang, {Sharon Xiaolei} and Hu, {Shi Min}",
year = "2010",
month = "9",
day = "14",
doi = "10.1145/1778765.1778820",
language = "English (US)",
volume = "29",
journal = "ACM Transactions on Graphics",
issn = "0730-0301",
publisher = "Association for Computing Machinery (ACM)",
number = "4",

}

Repfinder : Finding approximately repeated scene elements for image editing. / Cheng, Ming Ming; Zhang, Fang Lue; Mitra, Niloy J.; Huang, Sharon Xiaolei; Hu, Shi Min.

In: ACM Transactions on Graphics, Vol. 29, No. 4, 83, 14.09.2010.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Repfinder

T2 - Finding approximately repeated scene elements for image editing

AU - Cheng, Ming Ming

AU - Zhang, Fang Lue

AU - Mitra, Niloy J.

AU - Huang, Sharon Xiaolei

AU - Hu, Shi Min

PY - 2010/9/14

Y1 - 2010/9/14

N2 - Repeated elements are ubiquitous and abundant in both manmade and natural scenes. Editing such images while preserving the repetitions and their relations is nontrivial due to overlap, missing parts, deformation across instances, illumination variation, etc. Manually enforcing such relations is laborious and error-prone. We propose a novel framework where user scribbles are used to guide detection and extraction of such repeated elements. Our detection process, which is based on a novel boundary band method, robustly extracts the repetitions along with their deformations. The algorithm only considers the shape of the elements, and ignores similarity based on color, texture, etc. We then use topological sorting to establish a partial depth ordering of overlapping repeated instances. Missing parts on occluded instances are completed using information from other instances. The extracted repeated instances can then be seamlessly edited and manipulated for a variety of high level tasks that are otherwise difficult to perform. We demonstrate the versatility of our framework on a large set of inputs of varying complexity, showing applications to image rearrangement, edit transfer, deformation propagation, and instance replacement.

AB - Repeated elements are ubiquitous and abundant in both manmade and natural scenes. Editing such images while preserving the repetitions and their relations is nontrivial due to overlap, missing parts, deformation across instances, illumination variation, etc. Manually enforcing such relations is laborious and error-prone. We propose a novel framework where user scribbles are used to guide detection and extraction of such repeated elements. Our detection process, which is based on a novel boundary band method, robustly extracts the repetitions along with their deformations. The algorithm only considers the shape of the elements, and ignores similarity based on color, texture, etc. We then use topological sorting to establish a partial depth ordering of overlapping repeated instances. Missing parts on occluded instances are completed using information from other instances. The extracted repeated instances can then be seamlessly edited and manipulated for a variety of high level tasks that are otherwise difficult to perform. We demonstrate the versatility of our framework on a large set of inputs of varying complexity, showing applications to image rearrangement, edit transfer, deformation propagation, and instance replacement.

UR - http://www.scopus.com/inward/record.url?scp=77956410897&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77956410897&partnerID=8YFLogxK

U2 - 10.1145/1778765.1778820

DO - 10.1145/1778765.1778820

M3 - Article

AN - SCOPUS:77956410897

VL - 29

JO - ACM Transactions on Graphics

JF - ACM Transactions on Graphics

SN - 0730-0301

IS - 4

M1 - 83

ER -