From Image to Stability: Learning Dynamics from Human Pose

Jesse Scott, Bharadwaj Ravichandran, Christopher Funk, Robert T. Collins, Yanxi Liu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We propose and validate two end-to-end deep learning architectures to learn foot pressure distribution maps (dynamics) from 2D or 3D human pose (kinematics). The networks are trained using 1.36 million synchronized pose+pressure data pairs from 10 subjects performing multiple takes of a 5-min long choreographed Taiji sequence. Using leave-one-subject-out cross validation, we demonstrate reliable and repeatable foot pressure prediction, setting the first baseline for solving a non-obvious pose to pressure cross-modality mapping problem in computer vision. Furthermore, we compute and quantitatively validate Center of Pressure (CoP) and Base of Support (BoS), two key components for stability analysis, from the predicted foot pressure distributions.

Original languageEnglish (US)
Title of host publicationComputer Vision – ECCV 2020 - 16th European Conference, Glasgow, 2020, Proceedings
EditorsAndrea Vedaldi, Horst Bischof, Thomas Brox, Jan-Michael Frahm
PublisherSpringer Science and Business Media Deutschland GmbH
Pages536-554
Number of pages19
ISBN (Print)9783030585914
DOIs
StatePublished - 2020
Event16th European Conference on Computer Vision, ECCV 2020 - Glasgow, United Kingdom
Duration: Aug 23 2020Aug 28 2020

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume12368 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference16th European Conference on Computer Vision, ECCV 2020
Country/TerritoryUnited Kingdom
CityGlasgow
Period8/23/208/28/20

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint

Dive into the research topics of 'From Image to Stability: Learning Dynamics from Human Pose'. Together they form a unique fingerprint.

Cite this