Learning to design from humans: Imitating human designers through deep learning

Ayush Raina, Christopher McComb, Jonathan Cagan

Research output: Contribution to journalArticle

Abstract

Humans as designers have quite versatile problem-solving strategies. Computer agents on the other hand can access large-scale computational resources to solve certain design problems. Hence, if agents can learn from human behavior, a synergetic human-agent problem-solving team can be created. This paper presents an approach to extract human design strategies and implicit rules, purely from historical human data, and use that for design generation. A two-step framework that learns to imitate human design strategies from observation is proposed and implemented. This framework makes use of deep learning constructs to learn to generate designs without any explicit information about objective and performance metrics. The framework is designed to interact with the problem through a visual interface as humans did when solving the problem. It is trained to imitate a set of human designers by observing their design state sequences without inducing problem-specific modeling bias or extra information about the problem. Furthermore, an end-to-end agent is developed that uses this deep learning framework as its core in conjunction with image processing to map pixel-to-design moves as a mechanism to generate designs. Finally, the designs generated by a computational team of these agents are then compared with actual human data for teams solving a truss design problem. Results demonstrate that these agents are able to create feasible and efficient truss designs without guidance, showing that this methodology allows agents to learn effective design strategies.

Original languageEnglish (US)
Article numbere4044256
JournalJournal of Mechanical Design, Transactions of the ASME
Volume141
Issue number11
DOIs
StatePublished - Nov 1 2019

Fingerprint

Deep learning
Image processing
Pixels

All Science Journal Classification (ASJC) codes

  • Mechanics of Materials
  • Mechanical Engineering
  • Computer Science Applications
  • Computer Graphics and Computer-Aided Design

Cite this

@article{e365ff89523644b4a6b89abef5ddf704,
title = "Learning to design from humans: Imitating human designers through deep learning",
abstract = "Humans as designers have quite versatile problem-solving strategies. Computer agents on the other hand can access large-scale computational resources to solve certain design problems. Hence, if agents can learn from human behavior, a synergetic human-agent problem-solving team can be created. This paper presents an approach to extract human design strategies and implicit rules, purely from historical human data, and use that for design generation. A two-step framework that learns to imitate human design strategies from observation is proposed and implemented. This framework makes use of deep learning constructs to learn to generate designs without any explicit information about objective and performance metrics. The framework is designed to interact with the problem through a visual interface as humans did when solving the problem. It is trained to imitate a set of human designers by observing their design state sequences without inducing problem-specific modeling bias or extra information about the problem. Furthermore, an end-to-end agent is developed that uses this deep learning framework as its core in conjunction with image processing to map pixel-to-design moves as a mechanism to generate designs. Finally, the designs generated by a computational team of these agents are then compared with actual human data for teams solving a truss design problem. Results demonstrate that these agents are able to create feasible and efficient truss designs without guidance, showing that this methodology allows agents to learn effective design strategies.",
author = "Ayush Raina and Christopher McComb and Jonathan Cagan",
year = "2019",
month = "11",
day = "1",
doi = "10.1115/1.4044256",
language = "English (US)",
volume = "141",
journal = "Journal of Mechanical Design - Transactions of the ASME",
issn = "1050-0472",
publisher = "ASME",
number = "11",

}

Learning to design from humans : Imitating human designers through deep learning. / Raina, Ayush; McComb, Christopher; Cagan, Jonathan.

In: Journal of Mechanical Design, Transactions of the ASME, Vol. 141, No. 11, e4044256, 01.11.2019.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Learning to design from humans

T2 - Imitating human designers through deep learning

AU - Raina, Ayush

AU - McComb, Christopher

AU - Cagan, Jonathan

PY - 2019/11/1

Y1 - 2019/11/1

N2 - Humans as designers have quite versatile problem-solving strategies. Computer agents on the other hand can access large-scale computational resources to solve certain design problems. Hence, if agents can learn from human behavior, a synergetic human-agent problem-solving team can be created. This paper presents an approach to extract human design strategies and implicit rules, purely from historical human data, and use that for design generation. A two-step framework that learns to imitate human design strategies from observation is proposed and implemented. This framework makes use of deep learning constructs to learn to generate designs without any explicit information about objective and performance metrics. The framework is designed to interact with the problem through a visual interface as humans did when solving the problem. It is trained to imitate a set of human designers by observing their design state sequences without inducing problem-specific modeling bias or extra information about the problem. Furthermore, an end-to-end agent is developed that uses this deep learning framework as its core in conjunction with image processing to map pixel-to-design moves as a mechanism to generate designs. Finally, the designs generated by a computational team of these agents are then compared with actual human data for teams solving a truss design problem. Results demonstrate that these agents are able to create feasible and efficient truss designs without guidance, showing that this methodology allows agents to learn effective design strategies.

AB - Humans as designers have quite versatile problem-solving strategies. Computer agents on the other hand can access large-scale computational resources to solve certain design problems. Hence, if agents can learn from human behavior, a synergetic human-agent problem-solving team can be created. This paper presents an approach to extract human design strategies and implicit rules, purely from historical human data, and use that for design generation. A two-step framework that learns to imitate human design strategies from observation is proposed and implemented. This framework makes use of deep learning constructs to learn to generate designs without any explicit information about objective and performance metrics. The framework is designed to interact with the problem through a visual interface as humans did when solving the problem. It is trained to imitate a set of human designers by observing their design state sequences without inducing problem-specific modeling bias or extra information about the problem. Furthermore, an end-to-end agent is developed that uses this deep learning framework as its core in conjunction with image processing to map pixel-to-design moves as a mechanism to generate designs. Finally, the designs generated by a computational team of these agents are then compared with actual human data for teams solving a truss design problem. Results demonstrate that these agents are able to create feasible and efficient truss designs without guidance, showing that this methodology allows agents to learn effective design strategies.

UR - http://www.scopus.com/inward/record.url?scp=85072533979&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85072533979&partnerID=8YFLogxK

U2 - 10.1115/1.4044256

DO - 10.1115/1.4044256

M3 - Article

AN - SCOPUS:85072533979

VL - 141

JO - Journal of Mechanical Design - Transactions of the ASME

JF - Journal of Mechanical Design - Transactions of the ASME

SN - 1050-0472

IS - 11

M1 - e4044256

ER -