Interpreting idea maps

Pairwise comparisons reveal what makes ideas novel

Faez Ahmed, Sharath Kumar Ramachandran, Mark Fuge, Samuel Todd Hunter, Scarlett Rae Miller

Research output: Contribution to journalArticle

Abstract

Assessing similarity between design ideas is an inherent part of many design evaluations to measure novelty. In such evaluation tasks, humans excel at making mental connections among diverse knowledge sets to score ideas on their uniqueness. However, their decisions about novelty are often subjective and difficult to explain. In this paper, we demonstrate a way to uncover human judgment of design idea similarity using two-dimensional (2D) idea maps. We derive these maps by asking participants for simple similarity comparisons of the form "Is idea A more similar to idea B or to idea C?" We show that these maps give insight into the relationships between ideas and help understand the design domain. We also propose that novel ideas can be identified by finding outliers on these idea maps. To demonstrate our method, we conduct experimental evaluations on two datasets - colored polygons (known answer) and milk frother sketches (unknown answer). We show that idea maps shed light on factors considered by participants in judging idea similarity and the maps are robust to noisy ratings. We also compare physical maps made by participants on a white-board to their computationally generated idea maps to compare how people think about spatial arrangement of design items. This method provides a new direction of research into deriving ground truth novelty metrics by combining human judgments and computational methods.

Original languageEnglish (US)
Article number021102
JournalJournal of Mechanical Design, Transactions Of the ASME
Volume141
Issue number2
DOIs
StatePublished - Feb 1 2019

Fingerprint

Computational methods
Milk

All Science Journal Classification (ASJC) codes

  • Mechanics of Materials
  • Mechanical Engineering
  • Computer Science Applications
  • Computer Graphics and Computer-Aided Design

Cite this

@article{7481b9e5b510443380c775a0dce63a50,
title = "Interpreting idea maps: Pairwise comparisons reveal what makes ideas novel",
abstract = "Assessing similarity between design ideas is an inherent part of many design evaluations to measure novelty. In such evaluation tasks, humans excel at making mental connections among diverse knowledge sets to score ideas on their uniqueness. However, their decisions about novelty are often subjective and difficult to explain. In this paper, we demonstrate a way to uncover human judgment of design idea similarity using two-dimensional (2D) idea maps. We derive these maps by asking participants for simple similarity comparisons of the form {"}Is idea A more similar to idea B or to idea C?{"} We show that these maps give insight into the relationships between ideas and help understand the design domain. We also propose that novel ideas can be identified by finding outliers on these idea maps. To demonstrate our method, we conduct experimental evaluations on two datasets - colored polygons (known answer) and milk frother sketches (unknown answer). We show that idea maps shed light on factors considered by participants in judging idea similarity and the maps are robust to noisy ratings. We also compare physical maps made by participants on a white-board to their computationally generated idea maps to compare how people think about spatial arrangement of design items. This method provides a new direction of research into deriving ground truth novelty metrics by combining human judgments and computational methods.",
author = "Faez Ahmed and Ramachandran, {Sharath Kumar} and Mark Fuge and Hunter, {Samuel Todd} and Miller, {Scarlett Rae}",
year = "2019",
month = "2",
day = "1",
doi = "10.1115/1.4041856",
language = "English (US)",
volume = "141",
journal = "Journal of Mechanical Design - Transactions of the ASME",
issn = "1050-0472",
publisher = "ASME",
number = "2",

}

Interpreting idea maps : Pairwise comparisons reveal what makes ideas novel. / Ahmed, Faez; Ramachandran, Sharath Kumar; Fuge, Mark; Hunter, Samuel Todd; Miller, Scarlett Rae.

In: Journal of Mechanical Design, Transactions Of the ASME, Vol. 141, No. 2, 021102, 01.02.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Interpreting idea maps

T2 - Pairwise comparisons reveal what makes ideas novel

AU - Ahmed, Faez

AU - Ramachandran, Sharath Kumar

AU - Fuge, Mark

AU - Hunter, Samuel Todd

AU - Miller, Scarlett Rae

PY - 2019/2/1

Y1 - 2019/2/1

N2 - Assessing similarity between design ideas is an inherent part of many design evaluations to measure novelty. In such evaluation tasks, humans excel at making mental connections among diverse knowledge sets to score ideas on their uniqueness. However, their decisions about novelty are often subjective and difficult to explain. In this paper, we demonstrate a way to uncover human judgment of design idea similarity using two-dimensional (2D) idea maps. We derive these maps by asking participants for simple similarity comparisons of the form "Is idea A more similar to idea B or to idea C?" We show that these maps give insight into the relationships between ideas and help understand the design domain. We also propose that novel ideas can be identified by finding outliers on these idea maps. To demonstrate our method, we conduct experimental evaluations on two datasets - colored polygons (known answer) and milk frother sketches (unknown answer). We show that idea maps shed light on factors considered by participants in judging idea similarity and the maps are robust to noisy ratings. We also compare physical maps made by participants on a white-board to their computationally generated idea maps to compare how people think about spatial arrangement of design items. This method provides a new direction of research into deriving ground truth novelty metrics by combining human judgments and computational methods.

AB - Assessing similarity between design ideas is an inherent part of many design evaluations to measure novelty. In such evaluation tasks, humans excel at making mental connections among diverse knowledge sets to score ideas on their uniqueness. However, their decisions about novelty are often subjective and difficult to explain. In this paper, we demonstrate a way to uncover human judgment of design idea similarity using two-dimensional (2D) idea maps. We derive these maps by asking participants for simple similarity comparisons of the form "Is idea A more similar to idea B or to idea C?" We show that these maps give insight into the relationships between ideas and help understand the design domain. We also propose that novel ideas can be identified by finding outliers on these idea maps. To demonstrate our method, we conduct experimental evaluations on two datasets - colored polygons (known answer) and milk frother sketches (unknown answer). We show that idea maps shed light on factors considered by participants in judging idea similarity and the maps are robust to noisy ratings. We also compare physical maps made by participants on a white-board to their computationally generated idea maps to compare how people think about spatial arrangement of design items. This method provides a new direction of research into deriving ground truth novelty metrics by combining human judgments and computational methods.

UR - http://www.scopus.com/inward/record.url?scp=85059077795&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85059077795&partnerID=8YFLogxK

U2 - 10.1115/1.4041856

DO - 10.1115/1.4041856

M3 - Article

VL - 141

JO - Journal of Mechanical Design - Transactions of the ASME

JF - Journal of Mechanical Design - Transactions of the ASME

SN - 1050-0472

IS - 2

M1 - 021102

ER -