Object Recognition by Scene Alignment

Bryan C. Russell, Antonio Torralba, Ce Liu, Rob Fergus, William T. Freeman

Current object recognition systems can only recognize a limited number of object categories; scaling up to many categories is the next challenge. We seek to build a system to recognize and localize many different object categories in complex scenes. We achieve this through a simple approach: by matching the input image, in an appropriate representation, to images in a large training set of labeled images. Due to regularities in object identities across similar scenes, the retrieved matches provide hypotheses for object identities and locations. We build a probabilistic model to transfer the labels from the retrieval set to the input image. We demonstrate the effectiveness of this approach and study algorithm component contributions using held-out test sets from the LabelMe database.

Dataset

We ran experiments using a subset of the LabelMe dataset. Please use the LabelMe Matlab toolbox to interact with the dataset.

Downloads: Images.tar.gz (468 MB); Annotations.tar.gz (12 MB); train.txt (lists training set folders); test.txt (lists test set folders); demo.m; nips_rocs.zip (raw ROC curves in paper)

Publication

B. C. Russell, A. Torralba, C. Liu, R. Fergus, W. T. Freeman. Object Recognition by Scene Alignment. Advances in Neural Information Processing Systems, 2007. (Paper: PDF)