Using learning curves to assess resident surgical skill acquisition: A case study of ultrasound-guided catheter insertion

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

While there has been a wealth of performance criteria proposed and validated for objective assessment of surgical skills, only less than one third of the metrics are adopted in clinical practice, and even fewer are used for medical examination. This is due in part to the fact that the evaluation criteria proposed are focused on the task-centered outcomes which are not generalizable to broader sets of emerging surgical procedures. With increasing pressure from the media, regulatory bodies, and patients, to scrutinize physician performance, generalized standard criteria for assessing surgical performance are needed. This can be realized only if Multivariate Performance Criteria (MPC) are defined with respect to inherent human capabilities, rather than to transient task conditions, and if these criteria become related and prioritized consistently. In light of this, the current study proposes a method for relating and prioritizing the multivariate criteria through the use of Learning Curves. To this end, ten desired capabilities were identified from a Task Analysis of the Central Venous Catheter (CVC) placement procedure. In the preliminary study, motion tracking of video footage for six medical residents, taken at the beginning, middle, and end of a three-months-long training session, was used to confirm learning of the desired capabilities. Specifically, the economy of motion, eye-hand coordination, motion stability, and visual detection were tracked and quantified according to the defined metrics. Among them, both economy of motion and eye-hand coordination had a statistically significant learning effect (p<0.05) with the different learning rates-eye-hand coordination needed more training to reach a plateau. Our results imply that eye-hand coordination is relatively more comprehensive capability and requires to set higher priority in training. The current method can contribute to establishing hierarchical structures in MPC based on learning curves, ultimately building a standard system of performance. The results of this proceeding have implications for using learning curves to assess medical skill performance in a systematic manner.

Original languageEnglish (US)
Title of host publication2014 International Annual Meeting of the Human Factors and Ergonomics Society, HFES 2014
PublisherHuman Factors an Ergonomics Society Inc.
Pages2350-2354
Number of pages5
ISBN (Electronic)9780945289456
DOIs
StatePublished - Jan 1 2014
Event58th International Annual Meeting of the Human Factors and Ergonomics Society, HFES 2014 - Chicago, United States
Duration: Oct 27 2014Oct 31 2014

Publication series

NameProceedings of the Human Factors and Ergonomics Society
Volume2014-January
ISSN (Print)1071-1813

Other

Other58th International Annual Meeting of the Human Factors and Ergonomics Society, HFES 2014
CountryUnited States
CityChicago
Period10/27/1410/31/14

Fingerprint

Catheters
Ultrasonics
resident
learning
performance
medical examination
performance standard
economy
learning success
video
physician
evaluation

All Science Journal Classification (ASJC) codes

  • Human Factors and Ergonomics

Cite this

Kim, I., Miller, S. R., & Freivalds, A. (2014). Using learning curves to assess resident surgical skill acquisition: A case study of ultrasound-guided catheter insertion. In 2014 International Annual Meeting of the Human Factors and Ergonomics Society, HFES 2014 (pp. 2350-2354). (Proceedings of the Human Factors and Ergonomics Society; Vol. 2014-January). Human Factors an Ergonomics Society Inc.. https://doi.org/10.1177/1541931214581489
Kim, Inki ; Miller, Scarlett Rae ; Freivalds, Andris. / Using learning curves to assess resident surgical skill acquisition : A case study of ultrasound-guided catheter insertion. 2014 International Annual Meeting of the Human Factors and Ergonomics Society, HFES 2014. Human Factors an Ergonomics Society Inc., 2014. pp. 2350-2354 (Proceedings of the Human Factors and Ergonomics Society).
@inproceedings{52b3a55590ad43cbb27cf48ad54b2902,
title = "Using learning curves to assess resident surgical skill acquisition: A case study of ultrasound-guided catheter insertion",
abstract = "While there has been a wealth of performance criteria proposed and validated for objective assessment of surgical skills, only less than one third of the metrics are adopted in clinical practice, and even fewer are used for medical examination. This is due in part to the fact that the evaluation criteria proposed are focused on the task-centered outcomes which are not generalizable to broader sets of emerging surgical procedures. With increasing pressure from the media, regulatory bodies, and patients, to scrutinize physician performance, generalized standard criteria for assessing surgical performance are needed. This can be realized only if Multivariate Performance Criteria (MPC) are defined with respect to inherent human capabilities, rather than to transient task conditions, and if these criteria become related and prioritized consistently. In light of this, the current study proposes a method for relating and prioritizing the multivariate criteria through the use of Learning Curves. To this end, ten desired capabilities were identified from a Task Analysis of the Central Venous Catheter (CVC) placement procedure. In the preliminary study, motion tracking of video footage for six medical residents, taken at the beginning, middle, and end of a three-months-long training session, was used to confirm learning of the desired capabilities. Specifically, the economy of motion, eye-hand coordination, motion stability, and visual detection were tracked and quantified according to the defined metrics. Among them, both economy of motion and eye-hand coordination had a statistically significant learning effect (p<0.05) with the different learning rates-eye-hand coordination needed more training to reach a plateau. Our results imply that eye-hand coordination is relatively more comprehensive capability and requires to set higher priority in training. The current method can contribute to establishing hierarchical structures in MPC based on learning curves, ultimately building a standard system of performance. The results of this proceeding have implications for using learning curves to assess medical skill performance in a systematic manner.",
author = "Inki Kim and Miller, {Scarlett Rae} and Andris Freivalds",
year = "2014",
month = "1",
day = "1",
doi = "10.1177/1541931214581489",
language = "English (US)",
series = "Proceedings of the Human Factors and Ergonomics Society",
publisher = "Human Factors an Ergonomics Society Inc.",
pages = "2350--2354",
booktitle = "2014 International Annual Meeting of the Human Factors and Ergonomics Society, HFES 2014",
address = "United States",

}

Kim, I, Miller, SR & Freivalds, A 2014, Using learning curves to assess resident surgical skill acquisition: A case study of ultrasound-guided catheter insertion. in 2014 International Annual Meeting of the Human Factors and Ergonomics Society, HFES 2014. Proceedings of the Human Factors and Ergonomics Society, vol. 2014-January, Human Factors an Ergonomics Society Inc., pp. 2350-2354, 58th International Annual Meeting of the Human Factors and Ergonomics Society, HFES 2014, Chicago, United States, 10/27/14. https://doi.org/10.1177/1541931214581489

Using learning curves to assess resident surgical skill acquisition : A case study of ultrasound-guided catheter insertion. / Kim, Inki; Miller, Scarlett Rae; Freivalds, Andris.

2014 International Annual Meeting of the Human Factors and Ergonomics Society, HFES 2014. Human Factors an Ergonomics Society Inc., 2014. p. 2350-2354 (Proceedings of the Human Factors and Ergonomics Society; Vol. 2014-January).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Using learning curves to assess resident surgical skill acquisition

T2 - A case study of ultrasound-guided catheter insertion

AU - Kim, Inki

AU - Miller, Scarlett Rae

AU - Freivalds, Andris

PY - 2014/1/1

Y1 - 2014/1/1

N2 - While there has been a wealth of performance criteria proposed and validated for objective assessment of surgical skills, only less than one third of the metrics are adopted in clinical practice, and even fewer are used for medical examination. This is due in part to the fact that the evaluation criteria proposed are focused on the task-centered outcomes which are not generalizable to broader sets of emerging surgical procedures. With increasing pressure from the media, regulatory bodies, and patients, to scrutinize physician performance, generalized standard criteria for assessing surgical performance are needed. This can be realized only if Multivariate Performance Criteria (MPC) are defined with respect to inherent human capabilities, rather than to transient task conditions, and if these criteria become related and prioritized consistently. In light of this, the current study proposes a method for relating and prioritizing the multivariate criteria through the use of Learning Curves. To this end, ten desired capabilities were identified from a Task Analysis of the Central Venous Catheter (CVC) placement procedure. In the preliminary study, motion tracking of video footage for six medical residents, taken at the beginning, middle, and end of a three-months-long training session, was used to confirm learning of the desired capabilities. Specifically, the economy of motion, eye-hand coordination, motion stability, and visual detection were tracked and quantified according to the defined metrics. Among them, both economy of motion and eye-hand coordination had a statistically significant learning effect (p<0.05) with the different learning rates-eye-hand coordination needed more training to reach a plateau. Our results imply that eye-hand coordination is relatively more comprehensive capability and requires to set higher priority in training. The current method can contribute to establishing hierarchical structures in MPC based on learning curves, ultimately building a standard system of performance. The results of this proceeding have implications for using learning curves to assess medical skill performance in a systematic manner.

AB - While there has been a wealth of performance criteria proposed and validated for objective assessment of surgical skills, only less than one third of the metrics are adopted in clinical practice, and even fewer are used for medical examination. This is due in part to the fact that the evaluation criteria proposed are focused on the task-centered outcomes which are not generalizable to broader sets of emerging surgical procedures. With increasing pressure from the media, regulatory bodies, and patients, to scrutinize physician performance, generalized standard criteria for assessing surgical performance are needed. This can be realized only if Multivariate Performance Criteria (MPC) are defined with respect to inherent human capabilities, rather than to transient task conditions, and if these criteria become related and prioritized consistently. In light of this, the current study proposes a method for relating and prioritizing the multivariate criteria through the use of Learning Curves. To this end, ten desired capabilities were identified from a Task Analysis of the Central Venous Catheter (CVC) placement procedure. In the preliminary study, motion tracking of video footage for six medical residents, taken at the beginning, middle, and end of a three-months-long training session, was used to confirm learning of the desired capabilities. Specifically, the economy of motion, eye-hand coordination, motion stability, and visual detection were tracked and quantified according to the defined metrics. Among them, both economy of motion and eye-hand coordination had a statistically significant learning effect (p<0.05) with the different learning rates-eye-hand coordination needed more training to reach a plateau. Our results imply that eye-hand coordination is relatively more comprehensive capability and requires to set higher priority in training. The current method can contribute to establishing hierarchical structures in MPC based on learning curves, ultimately building a standard system of performance. The results of this proceeding have implications for using learning curves to assess medical skill performance in a systematic manner.

UR - http://www.scopus.com/inward/record.url?scp=84957714802&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84957714802&partnerID=8YFLogxK

U2 - 10.1177/1541931214581489

DO - 10.1177/1541931214581489

M3 - Conference contribution

AN - SCOPUS:84957714802

T3 - Proceedings of the Human Factors and Ergonomics Society

SP - 2350

EP - 2354

BT - 2014 International Annual Meeting of the Human Factors and Ergonomics Society, HFES 2014

PB - Human Factors an Ergonomics Society Inc.

ER -

Kim I, Miller SR, Freivalds A. Using learning curves to assess resident surgical skill acquisition: A case study of ultrasound-guided catheter insertion. In 2014 International Annual Meeting of the Human Factors and Ergonomics Society, HFES 2014. Human Factors an Ergonomics Society Inc. 2014. p. 2350-2354. (Proceedings of the Human Factors and Ergonomics Society). https://doi.org/10.1177/1541931214581489