A novel method for providing global assessments of design concepts using single-word adjectives and semantic similarity

Christopher A. Gosnell, Scarlett Rae Miller

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Engineering design idea-generation sessions often result in dozens, if not hundreds, of ideas. These ideas must be quickly evaluated and filtered in order to select a few candidate concepts to move forward in the design process. While creativity is often stressed in the conceptual phases of design, it receives little attention in these later phases - particularly during concept selection. This is largely because there are no methods for quickly rating or identifying worthwhile creative concepts during this process. Therefore, the purpose of this study was to develop and test a novel method for evaluating the creativity and feasibility of design concepts and compare this method to gold standards in our field. The SCAT method employed in this paper uses word selections and semantic similarity to quickly and effectively evaluate candidate concepts for their creativity and feasibility. This method requires little knowledge of the rating process by the evaluator. We tested this method with 10 engineering designers and three different design tasks. Our results revealed that SCAT ratings can be used as a proxy for measuring design concepts but there are modifications that could enhance its utility. This work contributes to our understanding of how to evaluate creativity after idea generation and provides a framework for further research in this field.

Original languageEnglish (US)
Title of host publication2nd Biennial International Conference on Dynamics for Design; 26th International Conference on Design Theory and Methodology
PublisherAmerican Society of Mechanical Engineers (ASME)
ISBN (Electronic)9780791846407
DOIs
StatePublished - Jan 1 2014
EventASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC/CIE 2014 - Buffalo, United States
Duration: Aug 17 2014Aug 20 2014

Publication series

NameProceedings of the ASME Design Engineering Technical Conference
Volume7

Other

OtherASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC/CIE 2014
CountryUnited States
CityBuffalo
Period8/17/148/20/14

Fingerprint

Semantic Similarity
Semantics
Evaluate
Engineering Design
Gold
Design Process
Design
Concepts
Engineering
Creativity

All Science Journal Classification (ASJC) codes

  • Modeling and Simulation
  • Mechanical Engineering
  • Computer Science Applications
  • Computer Graphics and Computer-Aided Design

Cite this

Gosnell, C. A., & Miller, S. R. (2014). A novel method for providing global assessments of design concepts using single-word adjectives and semantic similarity. In 2nd Biennial International Conference on Dynamics for Design; 26th International Conference on Design Theory and Methodology (Proceedings of the ASME Design Engineering Technical Conference; Vol. 7). American Society of Mechanical Engineers (ASME). https://doi.org/10.1115/DETC201435380
Gosnell, Christopher A. ; Miller, Scarlett Rae. / A novel method for providing global assessments of design concepts using single-word adjectives and semantic similarity. 2nd Biennial International Conference on Dynamics for Design; 26th International Conference on Design Theory and Methodology. American Society of Mechanical Engineers (ASME), 2014. (Proceedings of the ASME Design Engineering Technical Conference).
@inproceedings{d629c5c3a794478ba51a233919cc34a4,
title = "A novel method for providing global assessments of design concepts using single-word adjectives and semantic similarity",
abstract = "Engineering design idea-generation sessions often result in dozens, if not hundreds, of ideas. These ideas must be quickly evaluated and filtered in order to select a few candidate concepts to move forward in the design process. While creativity is often stressed in the conceptual phases of design, it receives little attention in these later phases - particularly during concept selection. This is largely because there are no methods for quickly rating or identifying worthwhile creative concepts during this process. Therefore, the purpose of this study was to develop and test a novel method for evaluating the creativity and feasibility of design concepts and compare this method to gold standards in our field. The SCAT method employed in this paper uses word selections and semantic similarity to quickly and effectively evaluate candidate concepts for their creativity and feasibility. This method requires little knowledge of the rating process by the evaluator. We tested this method with 10 engineering designers and three different design tasks. Our results revealed that SCAT ratings can be used as a proxy for measuring design concepts but there are modifications that could enhance its utility. This work contributes to our understanding of how to evaluate creativity after idea generation and provides a framework for further research in this field.",
author = "Gosnell, {Christopher A.} and Miller, {Scarlett Rae}",
year = "2014",
month = "1",
day = "1",
doi = "10.1115/DETC201435380",
language = "English (US)",
series = "Proceedings of the ASME Design Engineering Technical Conference",
publisher = "American Society of Mechanical Engineers (ASME)",
booktitle = "2nd Biennial International Conference on Dynamics for Design; 26th International Conference on Design Theory and Methodology",

}

Gosnell, CA & Miller, SR 2014, A novel method for providing global assessments of design concepts using single-word adjectives and semantic similarity. in 2nd Biennial International Conference on Dynamics for Design; 26th International Conference on Design Theory and Methodology. Proceedings of the ASME Design Engineering Technical Conference, vol. 7, American Society of Mechanical Engineers (ASME), ASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, IDETC/CIE 2014, Buffalo, United States, 8/17/14. https://doi.org/10.1115/DETC201435380

A novel method for providing global assessments of design concepts using single-word adjectives and semantic similarity. / Gosnell, Christopher A.; Miller, Scarlett Rae.

2nd Biennial International Conference on Dynamics for Design; 26th International Conference on Design Theory and Methodology. American Society of Mechanical Engineers (ASME), 2014. (Proceedings of the ASME Design Engineering Technical Conference; Vol. 7).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - A novel method for providing global assessments of design concepts using single-word adjectives and semantic similarity

AU - Gosnell, Christopher A.

AU - Miller, Scarlett Rae

PY - 2014/1/1

Y1 - 2014/1/1

N2 - Engineering design idea-generation sessions often result in dozens, if not hundreds, of ideas. These ideas must be quickly evaluated and filtered in order to select a few candidate concepts to move forward in the design process. While creativity is often stressed in the conceptual phases of design, it receives little attention in these later phases - particularly during concept selection. This is largely because there are no methods for quickly rating or identifying worthwhile creative concepts during this process. Therefore, the purpose of this study was to develop and test a novel method for evaluating the creativity and feasibility of design concepts and compare this method to gold standards in our field. The SCAT method employed in this paper uses word selections and semantic similarity to quickly and effectively evaluate candidate concepts for their creativity and feasibility. This method requires little knowledge of the rating process by the evaluator. We tested this method with 10 engineering designers and three different design tasks. Our results revealed that SCAT ratings can be used as a proxy for measuring design concepts but there are modifications that could enhance its utility. This work contributes to our understanding of how to evaluate creativity after idea generation and provides a framework for further research in this field.

AB - Engineering design idea-generation sessions often result in dozens, if not hundreds, of ideas. These ideas must be quickly evaluated and filtered in order to select a few candidate concepts to move forward in the design process. While creativity is often stressed in the conceptual phases of design, it receives little attention in these later phases - particularly during concept selection. This is largely because there are no methods for quickly rating or identifying worthwhile creative concepts during this process. Therefore, the purpose of this study was to develop and test a novel method for evaluating the creativity and feasibility of design concepts and compare this method to gold standards in our field. The SCAT method employed in this paper uses word selections and semantic similarity to quickly and effectively evaluate candidate concepts for their creativity and feasibility. This method requires little knowledge of the rating process by the evaluator. We tested this method with 10 engineering designers and three different design tasks. Our results revealed that SCAT ratings can be used as a proxy for measuring design concepts but there are modifications that could enhance its utility. This work contributes to our understanding of how to evaluate creativity after idea generation and provides a framework for further research in this field.

UR - http://www.scopus.com/inward/record.url?scp=84926030401&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84926030401&partnerID=8YFLogxK

U2 - 10.1115/DETC201435380

DO - 10.1115/DETC201435380

M3 - Conference contribution

AN - SCOPUS:84926030401

T3 - Proceedings of the ASME Design Engineering Technical Conference

BT - 2nd Biennial International Conference on Dynamics for Design; 26th International Conference on Design Theory and Methodology

PB - American Society of Mechanical Engineers (ASME)

ER -

Gosnell CA, Miller SR. A novel method for providing global assessments of design concepts using single-word adjectives and semantic similarity. In 2nd Biennial International Conference on Dynamics for Design; 26th International Conference on Design Theory and Methodology. American Society of Mechanical Engineers (ASME). 2014. (Proceedings of the ASME Design Engineering Technical Conference). https://doi.org/10.1115/DETC201435380