Learning to recognize video-based spatiotemporal events

Harini Veeraraghavan, Nikolaos P. Papanikolopoulos

Research output: Contribution to journalArticlepeer-review

19 Scopus citations


A key research issue in activity recognition in real-world applications, such as in intelligent transportation systems (ITS), is to automatically learn robust models of activities that require minimal human training. In this paper, we contribute a novel approach for learning sequenced spatiotemporal activities in outdoor traffic intersections. Concretely, by representing the activities as sequences of actions, we contribute a semisupervised learning algorithm that learns activities as complete stochastic context-free grammars (SCFGs), namely, the grammar structure and the parameters. Our approach has been implemented and tested on real-world scenes, and we present experimental results of the grammar learning and activity recognition applied to datacollection and traffic monitoring applications using video data.

Original languageEnglish (US)
Article number5166486
Pages (from-to)628-638
Number of pages11
JournalIEEE Transactions on Intelligent Transportation Systems
Issue number4
StatePublished - Dec 2009

Bibliographical note

Funding Information:
Manuscript received September 15, 2008; revised March 27, 2009. First published July 17, 2009; current version published December 3, 2009. This work was supported in part by the National Science Foundation under Grant IIS-0219863, Grant CNS-0224363, Grant CNS-0324864, Grant CNS-0420836, Grant IIP-0443945, Grant IIP-0726109, and Grant CNS-0708344, by the ITS Institute at the University of Minnesota, and by the Minnesota Department of Transportation.


  • Context-free grammars
  • Intelligent transportation system (ITS) applications
  • Machine learning
  • Vehicle tracking
  • Video analysis


Dive into the research topics of 'Learning to recognize video-based spatiotemporal events'. Together they form a unique fingerprint.

Cite this