Motion History Classifier; Classifying Video

Motion history images are motion-blurred silhouettes. Here is an example. The original is on the left, the background is removed, A cleaned-up mask produced. The resulting motion history images are the frames off the animation on the right.

img

Hu Moments and Classification

The Hu moments of the motion history image can be computed using OpenCV (or other libraries, or by hand). These seven moments can be used as a feature vector in classification tasks.

Hu Moments are invarient to rotation, scaling, and translation. But not affine or perspective translations (or changes in viewpoint). See Appendix for the images.

**Image** **Hu1** **Hu2** **Hu3** **Hu4** **Hu5** **Hu6** **Hu7**
None 1.157e-03 6.721e-07 1.824e-10 5.402e-11 5.194e-21 4.381e-14 1.336e-21
Horizontal Flip 1.157e-03 6.721e-07 1.824e-10 5.402e-11 5.194e-21 4.381e-14 -1.336e-21
Vertical Flip 1.157e-03 6.721e-07 1.824e-10 5.402e-11 5.194e-21 4.381e-14 -1.336e-21
Translation 1.157e-03 6.721e-07 1.824e-10 5.402e-11 5.194e-21 4.381e-14 1.336e-21
Rotation 1.157e-03 6.722e-07 1.825e-10 5.410e-11 5.207e-21 4.387e-14 1.341e-21
Scaled 1.157e-03 6.693e-07 1.801e-10 5.405e-11 5.148e-21 4.367e-14 1.398e-21
Affine 1.446e-03 1.427e-06 2.398e-10 1.089e-10 1.761e-20 1.239e-13 2.499e-23

Appendix

Transformed Images

Basic Image

img

Flipped Image

img

Horizontal Image

img

Translated

img

Rotated

img

Scaled

img

Affine

img

Python Code For Generating Hu Moments Examples

import cv2
import numpy as np

frame = cv2.imread("example_mhi.png",0)
print "No Transform"
print cv2.HuMoments(cv2.moments(frame)).flatten()

print "Horz Flip"
flip_horz = cv2.flip( frame, 1 )
cv2.imwrite("example_flip_horz.png", flip_horz)
print cv2.HuMoments(cv2.moments(flip_horz)).flatten()

print "Vert Flip"
flip_vert = cv2.flip( frame, 0 )
cv2.imwrite("example_flip_vert.png", flip_vert)
print cv2.HuMoments(cv2.moments(flip_vert)).flatten()

print "Translate"
M = np.float32([[1,0,-100],[0,1,-10]])
rows,cols = frame.shape
translated = cv2.warpAffine(frame,M,(cols, rows))
cv2.imwrite("example_translated.png", translated)
print cv2.HuMoments(cv2.moments(translated)).flatten()

print "Rotate"
rows,cols = frame.shape
center = (cols/2,rows/2)
M = cv2.getRotationMatrix2D(center,45,1)
rotated = cv2.warpAffine(frame,M,(cols,rows))
cv2.imwrite("example_rotated.png", rotated)
print cv2.HuMoments(cv2.moments(rotated)).flatten()

print "Scale"
M = np.float32([[.5,0,-10],[0,.5,10]])
rows,cols = frame.shape
scaled = cv2.warpAffine(frame,M,(cols, rows))
cv2.imwrite("example_scaled.png", scaled)
print cv2.HuMoments(cv2.moments(scaled)).flatten()

print "Affine"
M = np.float32([[1,.45,-100],[.1,1.0,-15]])
rows,cols = frame.shape
affine = cv2.warpAffine(frame,M,(cols, rows))
cv2.imwrite("example_affine.png", affine)
print cv2.HuMoments(cv2.moments(affine)).flatten()

References

  • A. F. Bobick and J. W. Davis, “The recognition of human movement using temporal templates,” in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 3, pp. 257-267, Mar 2001.

doi: 10.110934.910878