Object Recognition by Scene Alignment

Antonio Torralba

Electrical Engineering and Computer Science

MIT

Date: Friday, May 30, 2008
Place: Harold Frank Hall, 4164
Time: 2:00 pm — 3:00 pm


Abstract:

Object detection and recognition is generally posed as a matching problem between the object representation and the image features (e.g., aligning pictorial cues, shape correspondence, constellations of parts, etc.) while rejecting the background features using an outlier process. In this work, we take a different approach: we formulate the object detection problem as a problem of aligning elements of the entire scene. The background, instead of being treated as a set of outliers, is used to guide the detection process. Our approach relies on the observation that when we have a big enough database then we can find with high probability some images in the database very close to a query image, as in similar scenes with similar objects arranged in similar spatial configurations. If the images in the retrieval set are
partially labeled, then we can transfer the knowledge of the labeling to the query image, and the problem of object recognition becomes a problem of aligning scene regions. But, can we find a dataset large enough to cover a large number of scene configurations? Given an input image, how do we find a good retrieval set, and, finally, how we do transfer the labels to the input image? We will use two datasets; 1) the LabelMe dataset, which contains more than 10,000 labeled images with over 180,000 annotated objects. 2) The tiny images dataset: A dataset of weakly labeled images with more than 79,000,000 images. We use this database to perform object and scene classification, examining performance over a range of semantic levels.

Work in collaboration with Rob Fergus, Bryan Russell, Ce Liu and William
T. Freeman.

Additional information and links to relevant papers can be found at:
URL: http://people.csail.mit.edu/torralba/tinyimages/

A review paper describing some of the background for this work is
available here: http://cvcl.mit.edu/Papers/OlivaTorralbaTICS2007.pdf

Code, and additional material can be found here: http://people.csail.mit.edu/torralba/code/spatialenvelope/

ANTONIO TORRALBA joined the Department of Electrical Engineering and Computer Science at MIT in August 2007 as an Assistant Professor and a member of the Computer Science and Artificial Intelligence Laboratory. He earned a degree in telecommunications engineering from the Technical University of Catalonia (Spain) in 1994. He received his Ph.D. in Image and Signal Processing from the Institut National Polytechnique, Grenoble, France, in 2000. Then, he moved to Boston as a postdoctoral fellow at MIT. In 2004, he became a research scientist at the Computer Science and Artificial Intelligence Laboratory.

Host: Prof. Miguel Eckstein, Dept. of Psychology