Aaron Bobick: Representation and Recognition of Human Behavior for Perception

We have developed several approaches to the representation and recognition of human behavior, focusing primarily computer vision. We divide behaviors into movement, activity, and action. Movements are the most atomic primitives, requiring no contextual or sophisticated sequence knowledge to be recognized; movement is often represented and recognized using either view-invariant or view specific geometric techniques. Activity refers to sequences of movements, represented primarily by statistical descriptions; much of the recent work in gesture understanding falls within this category of behavior recognition. Finally, actions are larger scale events that typically include interaction with the environment and causal relationships; action understanding straddles the division between perception and cognition, computer vision and artificial intelligence/cognitive science. Fundamental questions underlying these techniques include how is time represented, what is the relationship between structural and statistical representations of behavior, and can the recognition of high level actions - involving, for example, intentionality - be achieved by compiled visual routines. I will present examples of our work in each of these areas covering domains ranging from the recognition of aerobics movements, to recovering parametric gestures, to visual surveillance, to interpreting football plays. I will also show video of some interactive spaces that leverage the techniques described.