Motion History Classifier; Classifying Video

Motion history images are motion-blurred silhouettes. Here is an example. The original is on the left, the background is removed, A cleaned-up mask produced. The resulting motion history images are the frames off the animation on the right.


Hu Moments and Classification

The Hu moments of the motion history image can be computed using OpenCV (or other libraries, or by hand). These seven moments can be used as a feature vector in classification tasks.

Hu Moments are invarient to rotation, scaling, and translation. But not affine or perspective translations (or changes in viewpoint). See Appendix for the images.

**Image** **Hu1** **Hu2** **Hu3** **Hu4** **Hu5** **Hu6** **Hu7**
None 1.157e-03 6.721e-07 1.824e-10 5.402e-11 5.194e-21 4.381e-14 1.336e-21
Horizontal Flip 1.157e-03 6.721e-07 1.824e-10 5.402e-11 5.194e-21 4.381e-14 -1.336e-21
Vertical Flip 1.157e-03 6.721e-07 1.824e-10 5.402e-11 5.194e-21 4.381e-14 -1.336e-21
Translation 1.157e-03 6.721e-07 1.824e-10 5.402e-11 5.194e-21 4.381e-14 1.336e-21
Rotation 1.157e-03 6.722e-07 1.825e-10 5.410e-11 5.207e-21 4.387e-14 1.341e-21
Scaled 1.157e-03 6.693e-07 1.801e-10 5.405e-11 5.148e-21 4.367e-14 1.398e-21
Affine 1.446e-03 1.427e-06 2.398e-10 1.089e-10 1.761e-20 1.239e-13 2.499e-23


Transformed Images

Basic Image


Flipped Image


Horizontal Image










Python Code For Generating Hu Moments Examples

import cv2
import numpy as np

frame = cv2.imread("example_mhi.png",0)
print "No Transform"
print cv2.HuMoments(cv2.moments(frame)).flatten()

print "Horz Flip"
flip_horz = cv2.flip( frame, 1 )
cv2.imwrite("example_flip_horz.png", flip_horz)
print cv2.HuMoments(cv2.moments(flip_horz)).flatten()

print "Vert Flip"
flip_vert = cv2.flip( frame, 0 )
cv2.imwrite("example_flip_vert.png", flip_vert)
print cv2.HuMoments(cv2.moments(flip_vert)).flatten()

print "Translate"
M = np.float32([[1,0,-100],[0,1,-10]])
rows,cols = frame.shape
translated = cv2.warpAffine(frame,M,(cols, rows))
cv2.imwrite("example_translated.png", translated)
print cv2.HuMoments(cv2.moments(translated)).flatten()

print "Rotate"
rows,cols = frame.shape
center = (cols/2,rows/2)
M = cv2.getRotationMatrix2D(center,45,1)
rotated = cv2.warpAffine(frame,M,(cols,rows))
cv2.imwrite("example_rotated.png", rotated)
print cv2.HuMoments(cv2.moments(rotated)).flatten()

print "Scale"
M = np.float32([[.5,0,-10],[0,.5,10]])
rows,cols = frame.shape
scaled = cv2.warpAffine(frame,M,(cols, rows))
cv2.imwrite("example_scaled.png", scaled)
print cv2.HuMoments(cv2.moments(scaled)).flatten()

print "Affine"
M = np.float32([[1,.45,-100],[.1,1.0,-15]])
rows,cols = frame.shape
affine = cv2.warpAffine(frame,M,(cols, rows))
cv2.imwrite("example_affine.png", affine)
print cv2.HuMoments(cv2.moments(affine)).flatten()


  • A. F. Bobick and J. W. Davis, “The recognition of human movement using temporal templates,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 3, pp. 257-267, Mar 2001.

doi: 10.110934.910878