Awesome
Superpixels Revisited
A more comprehensive comparison of superpixel algorithms, including the corresponding benchmark and implementations, can be found here: davidstutz/superpixel-benchmark.
This library combines several state-of-the-art superpixel algorithms in a single library. For each approach, a user-friendly command line tool is provided - these command line tools were used for evaluation in [1] and [2]. An overview over all superpixel approaches is provided below.
[1] D. Stutz, A. Hermans, B. Leibe.
Superpixel Segmentation using Depth Information.
Bachelor thesis, RWTH Aachen University, Aachen, Germany, 2014.
[2] D. Stutz.
Superpixel Segmentation: An Evaluation.
Pattern Recognition (J. Gall, P. Gehler, B. Leibe (Eds.)), Lecture Notes in Computer Science, vol. 9358, pages 555 - 562, 2015.
An overview over all superpixel algorithms and their evaluation results can be found online at [3]:
[3] http://davidstutz.de/projects/superpixelsseeds/
Index
Superpixel Algorithms
Provided superpixels algorithms:
- FH - Felzenswalb & Huttenlocher [4];
- SLIC - Simple Linear Iterative Clustering [5];
- CIS/CS - Constant Intensity Superpixels/Compact Superpixels [6];
- ERS - Entropy Rate Superpixels [7];
- PB - Pseudo Boolean Superpixels [8];
- CRS - Contour Relaxed Superpixels [9];
- SEEDS - Superpixels Extracted via Energy-Driven Sampling [10].
Note that the library of CIS/CS is, due to license restrictions, not directly included. See lib_cis/README.md for details.
Further, note that for SLIC, both the original implementation as well as the implementation as part of the VLFeat library [11] is provided. Similarly, for SEEDS the original implementation and SEEDS Revised - an implementation written during the bachelor thesis [2] - is provided.
[4] P. F. Felzenswalb, D. P. Huttenlocher.
Efficient graph-based image segmentation.
International Journal of Computer Vision, 59(2), 2004.
[5] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. Süsstrunk.
SLIC superpixels.
Technical report, École Polytechnique Fédérale de Lausanne, 2010.
[6] O. Veksler, Y. Boykov, P. Mehrani.
Superpixels and supervoxels in an energy optimization framework.
European Conference on Computer Vision, pages 211–224, 2010.
[7] M. Y. Lui, O. Tuzel, S. Ramalingam, R. Chellappa.
Entropy rate superpixel segmentation.
Conference on Computer Vision and Pattern Recognition, pages 2097–2104, 2011.
[8] Superpixels via pseudo-boolean optimization.
Y. Zhang, R. Hartley, J. Mashford, and S. Burn.
In International Conference on Computer Vision, 2011.
[9] C. Conrad, M. Mertz, R. Mester.
Contour-relaxed superpixels.
Energy Minimization Methods in Computer Vision and Pattern Recognition,
volume 8081 of Lecture Notes in Computer Science, pages 280–293, 2013.
[10] M. van den Bergh, X. Boix, G. Roig, B. de Capitani, L. van Gool.
SEEDS: Superpixels extracted via energy-driven sampling.
European Conference on Computer Vision, pages 13–26, 2012.
[11] A. Vedaldi, B. Fulkerson.
VLFeat: An Open and Portable Library of Computer Vision Algorithms.
\url{http://www.vlfeat.org/, 2008.
Building
Note: The library was tested primarily on Ubuntu 14.04, Ubuntu 16.04 and OpenCV 2.4.10 as well as OpenCV 2.4.13. Comments on building instructions are welcome!
The library can be built using CMake and Boost:
sudo apt-get install build-essential
sudo apt-get install cmake
sudo apt-get install libboost-all-dev
OpenCV can be installed using:
sudo apt-get install libopencv-dev
Or following this guide: http://docs.opencv.org/doc/tutorials/introduction/linux_install/linux_install.html. Then, the library can be built using:
git clone --recursive https://github.com/davidstutz/superpixels-revisited.git
cd superpixels-revisited
mkdir -p build
cd build
cmake ..
make
Note: This repository currently includes the VLFeat library [11] for simplicity. However, VLFeat can also be installed using:
sudo apt-get install libvlfeat-dev libvlfeat0
Then, the target vlfeat in vlfeat_slic_cli/CMakeLists.txt can be commented out.
Also see .travis.yml for building instructions.
The executables will be created in superpixels-revisited/bin while the libraries will be written to superpixels-revisited/lib.
For building CIS/CS [6] you need to download the corresponding library first, see lib_cis/README.md.
Per default, all superpixel algorithms are built. By adapting superpixels-revisited/CMakeLists.txt, this behavior can be adapted by commenting out the corresponding subdirectories:
# SEEDS Revised
add_subdirectory(lib_seeds_revised)
# Constant Intensity Superpixels/Compact Superpixels
# Remove comments after installing the library as described in
# lib_cli/README.md!
# add_subdirectory(lib_cis)
# add_subdirectory(cis_cli)
# Entropy Rate Superpixels
add_subdirectory(lib_ers)
add_subdirectory(ers_cli)
# Contour Relaxed Superpixels
add_subdirectory(lib_crs)
add_subdirectory(crs_cli)
# Felzenswalb & Huttenlocher
add_subdirectory(lib_fh)
add_subdirectory(fh_cli)
# Pseudo Boolean Superpixels
add_subdirectory(lib_pb)
add_subdirectory(pb_cli)
# SEEDS
add_subdirectory(lib_seeds)
add_subdirectory(seeds_cli)
# SLIC
add_subdirectory(lib_slic)
add_subdirectory(slic_cli)
# VLFeat SLIC
add_subdirectory(vlfeat_slic_cli)
Usage
Note: Usage details can also be found in the corresponding main.cpp files (e.g. crs_cli/main.cpp or lib_seeds_revised/cli/main.cpp).
In general, the following executables are provided:
bin/cli: SEEDS Revised;bin/cis_cli: CIS/CS;bin/crs_cli: CRS;bin/ers_cli: ERS;bin/fh_cli: FH;bin/pb_cli: PB;bin/seeds_cli: SEEDS;bin/slic_cli: SLIC;bin/vlfeat_slic_cli: VLFeat SLIC.
Each command line tool is provided on an input directory containing a variables number of images to be oversegmented. Further, each executable is able to write the resulting segmentations to .csv files and create boundary images using the --csv and --contour options, respectively, see the example below. Using the --help option, all available options are printed, e.g.:
$ ./bin/cli --help
Allowed options:
--help produce help message
--input arg the folder to process, may contain several
images
--bins arg (=5) number of bins used for color histograms
--neighborhood arg (=1) neighborhood size used for smoothing prior
--confidence arg (=0.100000001) minimum confidence used for block update
--iterations arg (=2) iterations at each level
--spatial-weight arg (=0.25) spatial weight
--superpixels arg (=400) desired number of supüerpixels
--verbose show additional information while processing
--csv save segmentation as CSV file
--contour save contour image of segmentation
--labels save label image of segmentation
--mean save mean colored image of segmentation
--output arg (=output) specify the output directory (default is
./output)
As example, for running SEEDS Revised on the test set of the Berkeley Segmentation Dataset [12], use:
$ cd superpixels-revisited
$ wget http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/BSR/BSR_bsds500.tgz
$ tar -xvzf BSR_bsds500.tgz
$ mkdir output
$ ./bin/cli ./BSR/BSDS500/data/images/test/ ./output --contour
200 images total ...
On average, 0.118183 seconds needed ...
For details on the Berkeley Segmentation Dataset [12], see:
[12] P. Arbeláez, M. Maire, C. Fowlkes, J. Malik.
Contour detection and hierarchical image segmentation.
Transactions on Pattern Analysis and Machine Intelligence, volume 33, number 5, pages 898–916, 2011.
In the following, each executable is described in detail.
FH
$ ./bin/fh_cli --help
Allowed options:
--help produce help message
--input arg the folder to process
--sigma arg (=1) sigma used for smoothing
--threshold arg (=20) constant for threshold function
--minimum-size arg (=10) minimum component size
--time arg time the algorithm and save results to the given
directory
--process show additional information while processing
--csv save segmentation as CSV file
--contour save contour image of segmentation
--mean save mean colored image of segmentation
--output arg (=output) specify the output directory (default is ./output)
SLIC
For SLIC, two executables are provided, the original implementation and the implementation as part of the VLFeat library:
# OriginalSLIC:
$ ./bin/slic_cli --help
Allowed options:
--help produce help message
--input arg the folder to process (can also be passed as
positional argument)
--superpixels arg (=400) number of superpixles
--compactness arg (=40) compactness
--perturb-seeds perturb seeds
--iterations arg (=10) iterations
--time arg time the algorithm and save results to the given
directory
--process show additional information while processing
--csv save segmentation as CSV file
--contour save contour image of segmentation
--mean save mean colored image of segmentation
--output arg (=output) specify the output directory (default is ./output)
# VLFeat SLIC:
$ ./bin/vlfeat_slic_cli --help
Allowed options:
--help produce help message
--input arg the folder to process (can also be passed as
positional argument)
--region-size arg (=10) region size used; defines the number of
superpixels
--minimum-region-size arg (=1) minimum region size allowed
--regularization arg (=100) regularization trades off color for spatial
closeness
--time arg time the algorithm and save results to the
given directory
--process show additional information while processing
--csv save segmentation as CSV file
--contour save contour image of segmentation
--mean save mean colored image of segmentation
--process show additional information
--output arg (=output) specify the output directory (default is
./output)
CIS
$ ./bin/cis_cli --help
Allowed options:
--help produce help message
--input arg folder containing the images to process
--region-size arg (=10) maxmimum allowed region size (that is region size x
region size patches)
--type arg (=1) 0 for compact superpixels, 1 for constant intensity
superpixels
--iterations arg (=2) number of iterations
--lambda arg (=50) lambda only influences constant intensity
superpixels; larger lambda results in smoother
boundaries
--process show additional information while processing
--time arg time the algorithm and save results to the given
directory
--csv save segmentation as CSV file
--contour save contour image of segmentation
--mean save mean colored image of segmentation
--time save timings in BSD evaluation format in the given
directory
--output arg (=output) specify the output directory (default is ./output)
ERS
$ ./bin/ers_cli --help
Allowed options:
--help produce help message
--input arg the folder to process
--lambda arg (=0.5) lambda
--sigma arg (=5) sigma
--four-connected use 4-connected
--superpixels arg (=400) number of superpixels
--time arg time the algorithm and save results to the given
directory
--process show additional information while processing
--csv save segmentation as CSV file
--contour save contour image of segmentation
--mean save mean colored image of segmentation
--output arg (=output) specify the output directory (default is ./output)
PB
$ ./bin/pb_cli --help
Allowed options:
--help produce help message
--input arg the folder to process (can also be passed as
positional argument)
--height arg (=10) height of initial vertical strips
--width arg (=10) width of initial vertical strips
--sigma arg (=20) balancing the weight between regular shape and
accurate edge
--max-flow use max flow algorithm instead of elimination
--time arg time the algorithm and save results to the given
directory
--process show additional information while processing
--csv save segmentation as CSV file
--contour save contour image of segmentation
--mean save mean colored image of segmentation
--output arg (=output) specify the output directory (default is ./output)
CRS
$ ./bin/crs_cli --help
Allowed options:
--help produce help message
--input arg the folder to process
--width arg (=20) width of blocks in initial superpixel
segmentation
--height arg (=20) height of blocks in initial superpixel
segmentation
--compactness arg (=0.045) compactness weight
--clique-cost arg (=0.3) direct clique cost
--iterations arg (=3) number of iterations to perform
--time arg time the algorithm and save results to
the given directory
--process show additional information while
processing
--csv save segmentation as CSV file
--contour save contour image of segmentation
--mean save mean colored image of segmentation
--output arg (=output) specify the output directory (default
is ./output)
SEEDS
For SEEDS, two implementations are provided. The first implementation, called SEEDS Revised, is published as result of the bachelor thesis [1]. The second implementation is the original implementation provided by van den Bergh et al.:
# SEEDS Revised:
$ ./bin/cli --help
Allowed options:
--help produce help message
--input arg the folder to process, may contain several
images
--bins arg (=5) number of bins used for color histograms
--neighborhood arg (=1) neighborhood size used for smoothing prior
--confidence arg (=0.100000001) minimum confidence used for block update
--iterations arg (=2) iterations at each level
--spatial-weight arg (=0.25) spatial weight
--superpixels arg (=400) desired number of supüerpixels
--verbose show additional information while processing
--csv save segmentation as CSV file
--contour save contour image of segmentation
--labels save label image of segmentation
--mean save mean colored image of segmentation
--output arg (=output) specify the output directory (default is
./output)
# Original SEEDS:
$ ./bin/seeds_cli --help
Allowed options:
--help produce help message
--input arg the folder to process (can also be passed as
positional argument)
--bins arg (=5) number of bins
--iterations arg (=2) iterations at each level
--bsd arg number of superpixels for BSDS500
--nyucropped arg number of superpixels for the cropped NYU Depth V2
--nyuhalf arg number of superpixel for NYU Depth V2 halfed
--nyuhalfcropped arg number of superpixels for the cropped NYU Depth V2
halfed
--time arg time the algorithm and save results to the given
directory
--process show additional information while processing
--csv save segmentation as CSV file
--contour save contour image of segmentation
--mean save mean colored image of segmentation
--output arg (=output) specify the output directory (default is ./output)
License
Licenses for source code corresponding to:
D. Stutz. Superpixel Segmentation using Depth Information. Bachelor Thesis, RWTH Aachen University, 2014.
D. Stutz. Superpixel Segmentation: An Evaluation. Pattern Recognition (J. Gall, P. Gehler, B. Leibe (Eds.)), Lecture Notes in Computer Science, vol. 9358, pages 555 - 562, 2015.
Note that for individual superpixel algorithms, separate linceses apply; see the README.md in the corresponding folder (e.g. lib_crs/README.md).
Copyright (c) 2014-2018 David Stutz, RWTH Aachen University
Please read carefully the following terms and conditions and any accompanying documentation before you download and/or use this software and associated documentation files (the "Software").
The authors hereby grant you a non-exclusive, non-transferable, free of charge right to copy, modify, merge, publish, distribute, and sublicense the Software for the sole purpose of performing non-commercial scientific research, non-commercial education, or non-commercial artistic projects.
Any other use, in particular any use for commercial purposes, is prohibited. This includes, without limitation, incorporation in a commercial product, use in a commercial service, or production of other artefacts for commercial purposes.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
You understand and agree that the authors are under no obligation to provide either maintenance services, update services, notices of latent defects, or corrections of defects with regard to the Software. The authors nevertheless reserve the right to update, modify, or discontinue the Software at any time.
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. You agree to cite the corresponding papers (see above) in documents and papers that report on research using this Software.