Team activity analysis and recognition based on kinect depth map and optical imagery techniques

Vinayak Elangovan, Vinod K. Bandaru, Amir Shirkhodaie

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Citations (Scopus)

Abstract

Kinect cameras produce low-cost depth map video streams applicable for conventional surveillance systems. However, commonly applied image processing techniques are not directly applicable for depth map video processing. Kinect depth map images contain range measurement of objects at expense of having spatial features of objects suppressed. For example, typical objects' attributes such as textures, color tones, intensity, and other characteristic attributes cannot be fully realized by processing depth map imagery. In this paper, we demonstrate application of Kinect depth map and optical imagery for characterization of indoor and outdoor group activities. A Casual-Events State Inference (CESI) technique is proposed for spatiotemporal recognition and reasoning of group activities. CESI uses an ontological scheme for representation of casual distinctiveness of a priori known group activities. By tracking and serializing distinctive atomic group activities, CESI allows discovery of more complex group activities. A Modified Sequential Hidden Markov Model (MS-HMM) is implemented for trail analysis of atomic events representing correlated group activities. CESI reasons about five levels of group activities including: Merging, Planning, Cooperation, Coordination, and Dispersion. In this paper, we present results of capability of CESI approach for characterization of group activities taking place both in indoor and outdoor. Based on spatiotemporal pattern matching of atomic activities representing a known group activities, the CESI is able to discriminate suspicious group activity from normal activities. This paper also presents technical details of imagery techniques implemented for detection, tracking, and characterization of atomic events based on Kinect depth map and optical imagery data sets. Various experimental scenarios in indoors and outdoors (e.g. loading and unloading of objects, human-vehicle interactions etc.,) are carried to demonstrate effectiveness and efficiency of the proposed model for characterization of distinctive group activities.

Original languageEnglish (US)
Title of host publicationSignal Processing, Sensor Fusion, and Target Recognition XXI
DOIs
StatePublished - Jul 2 2012
EventSignal Processing, Sensor Fusion, and Target Recognition XXI - Baltimore, MD, United States
Duration: Apr 23 2012Apr 25 2012

Publication series

NameProceedings of SPIE - The International Society for Optical Engineering
Volume8392
ISSN (Print)0277-786X

Other

OtherSignal Processing, Sensor Fusion, and Target Recognition XXI
CountryUnited States
CityBaltimore, MD
Period4/23/124/25/12

Fingerprint

Depth Map
imagery
inference
Pattern matching
Hidden Markov models
Processing
Unloading
Merging
Imagery
Image processing
Textures
Cameras
Attribute
Color
Planning
Video Processing
Range Image
Spatio-temporal Patterns
unloading
rangefinding

All Science Journal Classification (ASJC) codes

  • Electronic, Optical and Magnetic Materials
  • Condensed Matter Physics
  • Computer Science Applications
  • Applied Mathematics
  • Electrical and Electronic Engineering

Cite this

Elangovan, V., Bandaru, V. K., & Shirkhodaie, A. (2012). Team activity analysis and recognition based on kinect depth map and optical imagery techniques. In Signal Processing, Sensor Fusion, and Target Recognition XXI [83920W] (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 8392). https://doi.org/10.1117/12.919946
Elangovan, Vinayak ; Bandaru, Vinod K. ; Shirkhodaie, Amir. / Team activity analysis and recognition based on kinect depth map and optical imagery techniques. Signal Processing, Sensor Fusion, and Target Recognition XXI. 2012. (Proceedings of SPIE - The International Society for Optical Engineering).
@inproceedings{bafde9f396f74dc5bd6febaea5160dee,
title = "Team activity analysis and recognition based on kinect depth map and optical imagery techniques",
abstract = "Kinect cameras produce low-cost depth map video streams applicable for conventional surveillance systems. However, commonly applied image processing techniques are not directly applicable for depth map video processing. Kinect depth map images contain range measurement of objects at expense of having spatial features of objects suppressed. For example, typical objects' attributes such as textures, color tones, intensity, and other characteristic attributes cannot be fully realized by processing depth map imagery. In this paper, we demonstrate application of Kinect depth map and optical imagery for characterization of indoor and outdoor group activities. A Casual-Events State Inference (CESI) technique is proposed for spatiotemporal recognition and reasoning of group activities. CESI uses an ontological scheme for representation of casual distinctiveness of a priori known group activities. By tracking and serializing distinctive atomic group activities, CESI allows discovery of more complex group activities. A Modified Sequential Hidden Markov Model (MS-HMM) is implemented for trail analysis of atomic events representing correlated group activities. CESI reasons about five levels of group activities including: Merging, Planning, Cooperation, Coordination, and Dispersion. In this paper, we present results of capability of CESI approach for characterization of group activities taking place both in indoor and outdoor. Based on spatiotemporal pattern matching of atomic activities representing a known group activities, the CESI is able to discriminate suspicious group activity from normal activities. This paper also presents technical details of imagery techniques implemented for detection, tracking, and characterization of atomic events based on Kinect depth map and optical imagery data sets. Various experimental scenarios in indoors and outdoors (e.g. loading and unloading of objects, human-vehicle interactions etc.,) are carried to demonstrate effectiveness and efficiency of the proposed model for characterization of distinctive group activities.",
author = "Vinayak Elangovan and Bandaru, {Vinod K.} and Amir Shirkhodaie",
year = "2012",
month = "7",
day = "2",
doi = "10.1117/12.919946",
language = "English (US)",
isbn = "9780819490704",
series = "Proceedings of SPIE - The International Society for Optical Engineering",
booktitle = "Signal Processing, Sensor Fusion, and Target Recognition XXI",

}

Elangovan, V, Bandaru, VK & Shirkhodaie, A 2012, Team activity analysis and recognition based on kinect depth map and optical imagery techniques. in Signal Processing, Sensor Fusion, and Target Recognition XXI., 83920W, Proceedings of SPIE - The International Society for Optical Engineering, vol. 8392, Signal Processing, Sensor Fusion, and Target Recognition XXI, Baltimore, MD, United States, 4/23/12. https://doi.org/10.1117/12.919946

Team activity analysis and recognition based on kinect depth map and optical imagery techniques. / Elangovan, Vinayak; Bandaru, Vinod K.; Shirkhodaie, Amir.

Signal Processing, Sensor Fusion, and Target Recognition XXI. 2012. 83920W (Proceedings of SPIE - The International Society for Optical Engineering; Vol. 8392).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Team activity analysis and recognition based on kinect depth map and optical imagery techniques

AU - Elangovan, Vinayak

AU - Bandaru, Vinod K.

AU - Shirkhodaie, Amir

PY - 2012/7/2

Y1 - 2012/7/2

N2 - Kinect cameras produce low-cost depth map video streams applicable for conventional surveillance systems. However, commonly applied image processing techniques are not directly applicable for depth map video processing. Kinect depth map images contain range measurement of objects at expense of having spatial features of objects suppressed. For example, typical objects' attributes such as textures, color tones, intensity, and other characteristic attributes cannot be fully realized by processing depth map imagery. In this paper, we demonstrate application of Kinect depth map and optical imagery for characterization of indoor and outdoor group activities. A Casual-Events State Inference (CESI) technique is proposed for spatiotemporal recognition and reasoning of group activities. CESI uses an ontological scheme for representation of casual distinctiveness of a priori known group activities. By tracking and serializing distinctive atomic group activities, CESI allows discovery of more complex group activities. A Modified Sequential Hidden Markov Model (MS-HMM) is implemented for trail analysis of atomic events representing correlated group activities. CESI reasons about five levels of group activities including: Merging, Planning, Cooperation, Coordination, and Dispersion. In this paper, we present results of capability of CESI approach for characterization of group activities taking place both in indoor and outdoor. Based on spatiotemporal pattern matching of atomic activities representing a known group activities, the CESI is able to discriminate suspicious group activity from normal activities. This paper also presents technical details of imagery techniques implemented for detection, tracking, and characterization of atomic events based on Kinect depth map and optical imagery data sets. Various experimental scenarios in indoors and outdoors (e.g. loading and unloading of objects, human-vehicle interactions etc.,) are carried to demonstrate effectiveness and efficiency of the proposed model for characterization of distinctive group activities.

AB - Kinect cameras produce low-cost depth map video streams applicable for conventional surveillance systems. However, commonly applied image processing techniques are not directly applicable for depth map video processing. Kinect depth map images contain range measurement of objects at expense of having spatial features of objects suppressed. For example, typical objects' attributes such as textures, color tones, intensity, and other characteristic attributes cannot be fully realized by processing depth map imagery. In this paper, we demonstrate application of Kinect depth map and optical imagery for characterization of indoor and outdoor group activities. A Casual-Events State Inference (CESI) technique is proposed for spatiotemporal recognition and reasoning of group activities. CESI uses an ontological scheme for representation of casual distinctiveness of a priori known group activities. By tracking and serializing distinctive atomic group activities, CESI allows discovery of more complex group activities. A Modified Sequential Hidden Markov Model (MS-HMM) is implemented for trail analysis of atomic events representing correlated group activities. CESI reasons about five levels of group activities including: Merging, Planning, Cooperation, Coordination, and Dispersion. In this paper, we present results of capability of CESI approach for characterization of group activities taking place both in indoor and outdoor. Based on spatiotemporal pattern matching of atomic activities representing a known group activities, the CESI is able to discriminate suspicious group activity from normal activities. This paper also presents technical details of imagery techniques implemented for detection, tracking, and characterization of atomic events based on Kinect depth map and optical imagery data sets. Various experimental scenarios in indoors and outdoors (e.g. loading and unloading of objects, human-vehicle interactions etc.,) are carried to demonstrate effectiveness and efficiency of the proposed model for characterization of distinctive group activities.

UR - http://www.scopus.com/inward/record.url?scp=84862901459&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84862901459&partnerID=8YFLogxK

U2 - 10.1117/12.919946

DO - 10.1117/12.919946

M3 - Conference contribution

AN - SCOPUS:84862901459

SN - 9780819490704

T3 - Proceedings of SPIE - The International Society for Optical Engineering

BT - Signal Processing, Sensor Fusion, and Target Recognition XXI

ER -

Elangovan V, Bandaru VK, Shirkhodaie A. Team activity analysis and recognition based on kinect depth map and optical imagery techniques. In Signal Processing, Sensor Fusion, and Target Recognition XXI. 2012. 83920W. (Proceedings of SPIE - The International Society for Optical Engineering). https://doi.org/10.1117/12.919946