Video mining is to discover interesting knowledge, patterns and events, or called semantic structures, in the video data, and its benefits range from efficient browsing and summarization of video content of interest to facilitating video access and retrieval in a large database. In this work, we study sports video mining where relatively definite semantic structures exist and that has tremendous commercial value. However, there could be multiple semantic structures concurring in a sports video, e.g., play types (what happened?) and camera views (where it happened?). Specifically, we are interested in developing powerful generative models that are able to capture the interaction between multiple semantic structures and improve the overall performance. We propose a new multi-channel segmental hidden Markov model (MCSHMM) that is inspired by recent progress on graphical model and machine learning theory. As a case study, we test the new MCSHMM for American football video analysis where we want to explore two kinds of semantic structures, i.e., play types and camera views. This work will deliver the building blocks for our future research that will focus on the high-level semantic structures.

The two semantic structures defined in the American football videos.

The proposed multi-channel segmental hidden Markov models (MCSHMM).

Related Publications

• Y. Ding and G. Fan, "Multi-channel Segmental Hidden Markov Models for Sports Video Mining", in Proc. the ACM Multimedia Conference, Vancouver, Canada, Oct. 27-Nov. 1 , 2008.

• Y. Ding and G. Fan, "Segmental Hidden Markov Models for View-based Sport Video Analysis", in Proc. of International Workshop on Semantic Learning Applications in Multimedia (SLAM07), in conjunction with CVPR07, Minneapolis, MN, June 22, 2007.

• Y. Ding and G. Fan, "Two-layer Generative Models for Video Mining", in Proc. of IEEE International Conference on Multimedia and Expo (ICME), Beijing, China, July 2007.

• Y. Ding and G. Fan, “Camera View Based American Football Video Analysis”, in IEEE Proc. International Symposium on Multimedia, San Diego, CA, Dec. 11-13.