 |
 |
Bryan C. Russell, Alexei A. Efros, Josef Sivic, William T. Freeman, Andrew Zisserman
In this paper, we investigate how, given an image, similar images sharing the same global description can help with unsupervised scene segmentation. In contrast to recent work in semantic alignment of scenes, we allow an input image to be explained by partial matches of similar scenes. This allows for a better explanation of the input scenes. We perform MRF-based segmentation that optimizes over matches, while respecting boundary information. The recovered segments are then used to re-query a large database of images to retrieve better matches for the target regions. We show improved performance in detecting the principal occluding and contact boundaries for the scene over previous methods on data gathered from the LabelMe database.
Publication
B. C. Russell, A. A. Efros, J. Sivic, W. T. Freeman, and A. Zisserman, Segmenting Scenes by Matching Image Composites, Advances in Neural Information Processing Systems (NIPS), 2009. (paper.pdf, poster.pdf)
Downloads
1. Matlab source code
2. Modified GraphCuts library (note: we used version 2.2 from the original source)
3. (optional) Data structures for example image: DB.mat (2.5MB), G.mat
(177MB), images.tar.gz (574MB),
features.tar.gz (1.6GB)
Installation instructions
1. Unzip all downloaded code and data tarballs.
2. Compile the ".cpp" files in the Matlab source code via the "mex" command in Matlab.
3. Download and install the Berkeley PB library.
Running the code
The algorithm is demonstrated on an example image in "demo.m" using
pre-computed data structures on a subset of our Flickr dataset
(unfortunately, we cannot release the full Flickr dataset). Before
running, adjust the global variables at the top of the script to point
to the locations of the downloaded libraries and data structures.
We have included a script to pre-compute the necessary data structures
given a (large) dataset of images. Please see "precompute.m". Make
sure to modifiy this script according to your system setup, along with
the global variables at the top of the "demo.m" script.
Example result
The code is demonstrated on the following image. You should get the following results:
|
|
