TY - JOUR
T1 - Efficient Bronchoscopic Video Summarization
AU - Byrnes, Patrick D.
AU - Higgins, William Evan
N1 - Funding Information:
Manuscript received March 12, 2018; revised June 13, 2018; accepted July 12, 2018. Date of publication July 24, 2018; date of current version February 18, 2019. This work was supported by NIH Grant R01-CA151433 from the National Cancer Institute. (Corresponding author: William Evan Higgins.) P. D. Byrnes is with the School of Engineering, Penn State Erie, The Behrend College.
Publisher Copyright:
© 1964-2012 IEEE.
PY - 2019/3
Y1 - 2019/3
N2 - Bronchoscopy enables many minimally invasive chest procedures for diseases such as lung cancer and asthma. Guided by the bronchoscope's video stream, a physician can navigate the complex three-dimensional (3-D) airway tree to collect tissue samples or administer a disease treatment. Unfortunately, physicians currently discard procedural video because of the overwhelming amount of data generated. Hence, they must rely on memory and anecdotal snapshots to document a procedure. We propose a robust automatic method for summarizing an endobronchial video stream. Inspired by the multimedia concept of the video summary and by research in other endoscopy domains, our method consists of three main steps: 1) shot segmentation, 2) motion analysis, and 3) keyframe selection. Overall, the method derives a true hierarchical decomposition, consisting of a shot set and constituent keyframe set, for a given procedural video. No other method to our knowledge gives such a structured summary for the raw, unscripted, unedited videos arising in endoscopy. Results show that our method more efficiently covers the observed endobronchial regions than other keyframe-selection approaches and is robust to parameter variations. Over a wide range of video sequences, our method required on average only 6.5% of available video frames to achieve a video coverage = 92.7%. We also demonstrate how the derived video summary facilitates direct fusion with a patient's 3-D chest computed-tomography scan in a system under development, thereby enabling efficient video browsing and retrieval through the complex airway tree.
AB - Bronchoscopy enables many minimally invasive chest procedures for diseases such as lung cancer and asthma. Guided by the bronchoscope's video stream, a physician can navigate the complex three-dimensional (3-D) airway tree to collect tissue samples or administer a disease treatment. Unfortunately, physicians currently discard procedural video because of the overwhelming amount of data generated. Hence, they must rely on memory and anecdotal snapshots to document a procedure. We propose a robust automatic method for summarizing an endobronchial video stream. Inspired by the multimedia concept of the video summary and by research in other endoscopy domains, our method consists of three main steps: 1) shot segmentation, 2) motion analysis, and 3) keyframe selection. Overall, the method derives a true hierarchical decomposition, consisting of a shot set and constituent keyframe set, for a given procedural video. No other method to our knowledge gives such a structured summary for the raw, unscripted, unedited videos arising in endoscopy. Results show that our method more efficiently covers the observed endobronchial regions than other keyframe-selection approaches and is robust to parameter variations. Over a wide range of video sequences, our method required on average only 6.5% of available video frames to achieve a video coverage = 92.7%. We also demonstrate how the derived video summary facilitates direct fusion with a patient's 3-D chest computed-tomography scan in a system under development, thereby enabling efficient video browsing and retrieval through the complex airway tree.
UR - http://www.scopus.com/inward/record.url?scp=85050607012&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85050607012&partnerID=8YFLogxK
U2 - 10.1109/TBME.2018.2859322
DO - 10.1109/TBME.2018.2859322
M3 - Article
C2 - 30047870
AN - SCOPUS:85050607012
VL - 66
SP - 848
EP - 863
JO - IRE transactions on medical electronics
JF - IRE transactions on medical electronics
SN - 0018-9294
IS - 3
M1 - 8418753
ER -