Using q-learning and genetic algorithms to improve the efficiency of weight adjustments for optimal control and design problems

Kaivan Kamali, Lijun Jiang, John Yen, K. W. Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In traditional optimal control and design problems, the control gains and design parameters are usually derived to minimize a cost function reflecting the system performance and control effort. One major challenge of such approaches is the selection of weighting matrices in the cost function, which are usually determined via trial and error and human intuition. While various techniques have been proposed to automate the weight selection process, they either can not address complex design problems or suffer from slow convergence rate and high computational costs. We propose a layered approach based on Q-learning, a reinforcement learning technique, on top of genetic algorithms (GA) to determine the best weightings for optimal control and design problems. The layered approach allows for reuse of knowledge. Knowledge obtained via Q-learning in a design problem can be used to speed up the convergence rate of a similar design problem. Moreover, the layered approach allows for solving optimizations that cannot be solved by GA alone. To test the proposed method, we perform numerical experiments on a sample active-passive hybrid vibration control problem, namely adaptive structures with active-passive hybrid piezoelectric networks (APPN). These numerical experiments show that the proposed Q-learning scheme is a promising approach for.

Original languageEnglish (US)
Title of host publicationProceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conferences - DETC2005
Subtitle of host publication31st Design Automation Conference
Pages43-50
Number of pages8
StatePublished - Dec 1 2005
EventDETC2005: ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference - Long Beach, CA, United States
Duration: Sep 24 2005Sep 28 2005

Publication series

NameProceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference - DETC2005
Volume2 A

Other

OtherDETC2005: ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
CountryUnited States
CityLong Beach, CA
Period9/24/059/28/05

Fingerprint

Learning algorithms
Genetic algorithms
Cost functions
Gain control
Reinforcement learning
Vibration control
Experiments
Costs

All Science Journal Classification (ASJC) codes

  • Engineering(all)

Cite this

Kamali, K., Jiang, L., Yen, J., & Wang, K. W. (2005). Using q-learning and genetic algorithms to improve the efficiency of weight adjustments for optimal control and design problems. In Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conferences - DETC2005: 31st Design Automation Conference (pp. 43-50). (Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference - DETC2005; Vol. 2 A).
Kamali, Kaivan ; Jiang, Lijun ; Yen, John ; Wang, K. W. / Using q-learning and genetic algorithms to improve the efficiency of weight adjustments for optimal control and design problems. Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conferences - DETC2005: 31st Design Automation Conference. 2005. pp. 43-50 (Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference - DETC2005).
@inproceedings{f4ccaf33564044e4833d46c38f546aac,
title = "Using q-learning and genetic algorithms to improve the efficiency of weight adjustments for optimal control and design problems",
abstract = "In traditional optimal control and design problems, the control gains and design parameters are usually derived to minimize a cost function reflecting the system performance and control effort. One major challenge of such approaches is the selection of weighting matrices in the cost function, which are usually determined via trial and error and human intuition. While various techniques have been proposed to automate the weight selection process, they either can not address complex design problems or suffer from slow convergence rate and high computational costs. We propose a layered approach based on Q-learning, a reinforcement learning technique, on top of genetic algorithms (GA) to determine the best weightings for optimal control and design problems. The layered approach allows for reuse of knowledge. Knowledge obtained via Q-learning in a design problem can be used to speed up the convergence rate of a similar design problem. Moreover, the layered approach allows for solving optimizations that cannot be solved by GA alone. To test the proposed method, we perform numerical experiments on a sample active-passive hybrid vibration control problem, namely adaptive structures with active-passive hybrid piezoelectric networks (APPN). These numerical experiments show that the proposed Q-learning scheme is a promising approach for.",
author = "Kaivan Kamali and Lijun Jiang and John Yen and Wang, {K. W.}",
year = "2005",
month = "12",
day = "1",
language = "English (US)",
isbn = "079184739X",
series = "Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference - DETC2005",
pages = "43--50",
booktitle = "Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conferences - DETC2005",

}

Kamali, K, Jiang, L, Yen, J & Wang, KW 2005, Using q-learning and genetic algorithms to improve the efficiency of weight adjustments for optimal control and design problems. in Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conferences - DETC2005: 31st Design Automation Conference. Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference - DETC2005, vol. 2 A, pp. 43-50, DETC2005: ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Long Beach, CA, United States, 9/24/05.

Using q-learning and genetic algorithms to improve the efficiency of weight adjustments for optimal control and design problems. / Kamali, Kaivan; Jiang, Lijun; Yen, John; Wang, K. W.

Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conferences - DETC2005: 31st Design Automation Conference. 2005. p. 43-50 (Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference - DETC2005; Vol. 2 A).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Using q-learning and genetic algorithms to improve the efficiency of weight adjustments for optimal control and design problems

AU - Kamali, Kaivan

AU - Jiang, Lijun

AU - Yen, John

AU - Wang, K. W.

PY - 2005/12/1

Y1 - 2005/12/1

N2 - In traditional optimal control and design problems, the control gains and design parameters are usually derived to minimize a cost function reflecting the system performance and control effort. One major challenge of such approaches is the selection of weighting matrices in the cost function, which are usually determined via trial and error and human intuition. While various techniques have been proposed to automate the weight selection process, they either can not address complex design problems or suffer from slow convergence rate and high computational costs. We propose a layered approach based on Q-learning, a reinforcement learning technique, on top of genetic algorithms (GA) to determine the best weightings for optimal control and design problems. The layered approach allows for reuse of knowledge. Knowledge obtained via Q-learning in a design problem can be used to speed up the convergence rate of a similar design problem. Moreover, the layered approach allows for solving optimizations that cannot be solved by GA alone. To test the proposed method, we perform numerical experiments on a sample active-passive hybrid vibration control problem, namely adaptive structures with active-passive hybrid piezoelectric networks (APPN). These numerical experiments show that the proposed Q-learning scheme is a promising approach for.

AB - In traditional optimal control and design problems, the control gains and design parameters are usually derived to minimize a cost function reflecting the system performance and control effort. One major challenge of such approaches is the selection of weighting matrices in the cost function, which are usually determined via trial and error and human intuition. While various techniques have been proposed to automate the weight selection process, they either can not address complex design problems or suffer from slow convergence rate and high computational costs. We propose a layered approach based on Q-learning, a reinforcement learning technique, on top of genetic algorithms (GA) to determine the best weightings for optimal control and design problems. The layered approach allows for reuse of knowledge. Knowledge obtained via Q-learning in a design problem can be used to speed up the convergence rate of a similar design problem. Moreover, the layered approach allows for solving optimizations that cannot be solved by GA alone. To test the proposed method, we perform numerical experiments on a sample active-passive hybrid vibration control problem, namely adaptive structures with active-passive hybrid piezoelectric networks (APPN). These numerical experiments show that the proposed Q-learning scheme is a promising approach for.

UR - http://www.scopus.com/inward/record.url?scp=33144463894&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33144463894&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:33144463894

SN - 079184739X

SN - 9780791847398

T3 - Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference - DETC2005

SP - 43

EP - 50

BT - Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conferences - DETC2005

ER -

Kamali K, Jiang L, Yen J, Wang KW. Using q-learning and genetic algorithms to improve the efficiency of weight adjustments for optimal control and design problems. In Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conferences - DETC2005: 31st Design Automation Conference. 2005. p. 43-50. (Proceedings of the ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference - DETC2005).