Full Body Tracking

With Nebojsa Jojic, Beckman Institute, University of Illinois at Urbana-Champaign

	Mpeg videos showing short tracking segments: 2 blob tracking (984 KB), 4 blob tracking (906 KB).
	Paper on this work

We are developing an algorithm for tracking the human body in real time using disparity data from a commercial stereo system. The tracked object is modeled as a connected set of 3D blob segments, each modeled as a Gaussian distribution. Classification of the disparity map pixels into the segments of the articulated object is based on the maximum likelihood principle with an additional mechanism for filling the missing data created by self-occlusions. The articulation constraints are enforced through an Extended Kalman Filter, which can also be used to model the dynamics of the tracked object. We are able to successfully track upper human body motion in (almost) real time and in the presence of self-occlusions.

Here is a block diagram of the tracker:

Here are a few example images from tracking sequences.  First, tracking two blobs (head and torso):

Now tracking four blobs (head, torso, upper arm, lower arm):