Metamodel-driven interfaces for engineering design: Impact of delay and problem size on user performance

Timothy W. Simpson, Parameshwaran S. Iyer, Ling Rothrock, Mary Frecker, Russell R. Barton, Kimberly A. Barron, Martin Meckesheimer

Research output: Contribution to journalConference article

9 Citations (Scopus)

Abstract

In this paper, we present results from a follow-up study to our previous research involving the use of a metamodel-driven, graphical design interface for sizing the plan view layout of an aircraft wing to minimize cost subject to three constraints. This study extends our work by examining the impact of (i) varying problem size, i.e., the number of design variables (2, 4, or 6) that the user can manipulate, and (ii) a longer response delay (1.5 seconds) on user performance. Design effectiveness is measured by calculating the percent error between a submitted design and the known optimum, and design efficiency is measured by recording the completion time for solving the wing design problem. Fortyeight users participated in the study. Response delay and problem size both had a significant effect on design effectiveness: the 1.5 second delay increased average error by 150% compared to the no delay case, and the average error more than doubled each time the problem size increased. Problem size also had a significant effect on design efficiency: the average completion times for the 4 and 6 variable problems were more than double the completion time for the 2 variable problem. Average completion time also increased as the response delay increased, but this effect was not significant at the 5% level. Although the interaction between delay and problem size was not significant, the impact of the response delay on design efficiency and design effectiveness did decrease as problem size increased. Correlations between user performance and responses to p re- and post-test questionnaires are also discussed along with the implications of our findings for the development of metamodel-driven, graphical design interfaces.

Original languageEnglish (US)
Pages (from-to)3198-3208
Number of pages11
JournalCollection of Technical Papers - AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference
Volume5
StatePublished - Dec 19 2005
Event46th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference - Austin, TX, United States
Duration: Apr 18 2005Apr 21 2005

Fingerprint

Costs

All Science Journal Classification (ASJC) codes

  • Architecture
  • Materials Science(all)
  • Aerospace Engineering
  • Mechanics of Materials
  • Mechanical Engineering

Cite this

@article{8372e52aa9fa49f39b3860a48433464a,
title = "Metamodel-driven interfaces for engineering design: Impact of delay and problem size on user performance",
abstract = "In this paper, we present results from a follow-up study to our previous research involving the use of a metamodel-driven, graphical design interface for sizing the plan view layout of an aircraft wing to minimize cost subject to three constraints. This study extends our work by examining the impact of (i) varying problem size, i.e., the number of design variables (2, 4, or 6) that the user can manipulate, and (ii) a longer response delay (1.5 seconds) on user performance. Design effectiveness is measured by calculating the percent error between a submitted design and the known optimum, and design efficiency is measured by recording the completion time for solving the wing design problem. Fortyeight users participated in the study. Response delay and problem size both had a significant effect on design effectiveness: the 1.5 second delay increased average error by 150{\%} compared to the no delay case, and the average error more than doubled each time the problem size increased. Problem size also had a significant effect on design efficiency: the average completion times for the 4 and 6 variable problems were more than double the completion time for the 2 variable problem. Average completion time also increased as the response delay increased, but this effect was not significant at the 5{\%} level. Although the interaction between delay and problem size was not significant, the impact of the response delay on design efficiency and design effectiveness did decrease as problem size increased. Correlations between user performance and responses to p re- and post-test questionnaires are also discussed along with the implications of our findings for the development of metamodel-driven, graphical design interfaces.",
author = "Simpson, {Timothy W.} and Iyer, {Parameshwaran S.} and Ling Rothrock and Mary Frecker and Barton, {Russell R.} and Barron, {Kimberly A.} and Martin Meckesheimer",
year = "2005",
month = "12",
day = "19",
language = "English (US)",
volume = "5",
pages = "3198--3208",
journal = "Collection of Technical Papers - AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference",
issn = "0273-4508",
publisher = "American Institute of Aeronautics and Astronautics Inc. (AIAA)",

}

TY - JOUR

T1 - Metamodel-driven interfaces for engineering design

T2 - Impact of delay and problem size on user performance

AU - Simpson, Timothy W.

AU - Iyer, Parameshwaran S.

AU - Rothrock, Ling

AU - Frecker, Mary

AU - Barton, Russell R.

AU - Barron, Kimberly A.

AU - Meckesheimer, Martin

PY - 2005/12/19

Y1 - 2005/12/19

N2 - In this paper, we present results from a follow-up study to our previous research involving the use of a metamodel-driven, graphical design interface for sizing the plan view layout of an aircraft wing to minimize cost subject to three constraints. This study extends our work by examining the impact of (i) varying problem size, i.e., the number of design variables (2, 4, or 6) that the user can manipulate, and (ii) a longer response delay (1.5 seconds) on user performance. Design effectiveness is measured by calculating the percent error between a submitted design and the known optimum, and design efficiency is measured by recording the completion time for solving the wing design problem. Fortyeight users participated in the study. Response delay and problem size both had a significant effect on design effectiveness: the 1.5 second delay increased average error by 150% compared to the no delay case, and the average error more than doubled each time the problem size increased. Problem size also had a significant effect on design efficiency: the average completion times for the 4 and 6 variable problems were more than double the completion time for the 2 variable problem. Average completion time also increased as the response delay increased, but this effect was not significant at the 5% level. Although the interaction between delay and problem size was not significant, the impact of the response delay on design efficiency and design effectiveness did decrease as problem size increased. Correlations between user performance and responses to p re- and post-test questionnaires are also discussed along with the implications of our findings for the development of metamodel-driven, graphical design interfaces.

AB - In this paper, we present results from a follow-up study to our previous research involving the use of a metamodel-driven, graphical design interface for sizing the plan view layout of an aircraft wing to minimize cost subject to three constraints. This study extends our work by examining the impact of (i) varying problem size, i.e., the number of design variables (2, 4, or 6) that the user can manipulate, and (ii) a longer response delay (1.5 seconds) on user performance. Design effectiveness is measured by calculating the percent error between a submitted design and the known optimum, and design efficiency is measured by recording the completion time for solving the wing design problem. Fortyeight users participated in the study. Response delay and problem size both had a significant effect on design effectiveness: the 1.5 second delay increased average error by 150% compared to the no delay case, and the average error more than doubled each time the problem size increased. Problem size also had a significant effect on design efficiency: the average completion times for the 4 and 6 variable problems were more than double the completion time for the 2 variable problem. Average completion time also increased as the response delay increased, but this effect was not significant at the 5% level. Although the interaction between delay and problem size was not significant, the impact of the response delay on design efficiency and design effectiveness did decrease as problem size increased. Correlations between user performance and responses to p re- and post-test questionnaires are also discussed along with the implications of our findings for the development of metamodel-driven, graphical design interfaces.

UR - http://www.scopus.com/inward/record.url?scp=28844457664&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=28844457664&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:28844457664

VL - 5

SP - 3198

EP - 3208

JO - Collection of Technical Papers - AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference

JF - Collection of Technical Papers - AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference

SN - 0273-4508

ER -