Image segmentation is an essential tool to enhance the ability of computer systems to efficiently perform elementary cognitive tasks such as detection, recognition and tracking. In this thesis we concentrate on the investigation of two fundamental topics in the context of image segmentation: spectral clustering and seeded image segmentation. We introduce two new algorithms for those topics that, in summary, rely on Laplacian-based operators, spectral graph theory, and minimization of energy functionals. The effectiveness of both segmentation algorithms is verified by visually evaluating the resulting partitions against state-of-the-art methods as well as through a variety of quantitative measures typically employed as benchmark by the image segmentation community.
Our spectral-based segmentation algorithm combines image decomposition, similarity metrics, and spectral graph theory into a concise and powerful framework. An image decomposition is performed to split the input image into texture and cartoon components. Then, an affinity graph is generated and weights are assigned to the edges of the graph according to a gradient-based inner-product function. From the eigenstructure of the affinity graph, the image is partitioned through the spectral cut of the underlying graph. Moreover, the image partitioning can be improved by changing the graph weights by sketching interactively. Visual and numerical evaluation were conducted against representative spectral-based segmentation techniques using boundary and partition quality measures in the well-known BSDS dataset.
Unlike most existing seed-based methods that rely on complex mathematical formulations that typically do not guarantee unique solution for the segmentation problem while still being prone to be trapped in local minima, our segmentation approach is mathematically simple to formulate, easy-to-implement, and it guarantees to produce a unique solution. Moreover, the formulation holds an anisotropic behavior, that is, pixels sharing similar attributes are preserved closer to each other while big discontinuities are naturally imposed on the boundary between image regions, thus ensuring better fitting on object boundaries. We show that the proposed approach significantly outperforms competing techniques both quantitatively as well as qualitatively, using the classical “GrabCut” dataset from Microsoft as a benchmark.
While most of this research concentrates on the particular problem of segmenting an image, we also develop two new techniques to address the problem of image inpainting and photo colorization. Both methods couple the developed segmentation tools with other computer vision approaches in order to operate properly.
Luis Gustavo Nonato, Wallace Casaca
W Casaca, D Motta, G Taubin and L G Nonato. A user-friendly interactive image inpainting framework using Laplacian coordinates. In Image Processing (ICIP), 2015 IEEE International Conference on. 2015, 862-866.
George Azzopardi and Nicolai Petkov (eds.). Computer Analysis of Images and Patterns: 16th International Conference, CAIP 2015, Valletta, Malta, September 2-4, 2015, Proceedings, Part II. Chapter Interactive Image Colorization Using Laplacian Coordinates, pages 675–686, Springer International Publishing, 2015.
W Casaca, L G Nonato and G Taubin. Laplacian Coordinates for Seeded Image Segmentation. In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on. Junho 2014, 384-391.
Wallace Casaca, Maurílio Boaventura, Marcos Proença Almeida and Luis Gustavo Nonato. Combining anisotropic diffusion, transport equation and texture synthesis for inpainting textured images. Pattern Recognition Letters 36:36 – 45, 2014.
Wallace Casaca, Afonso Paiva, Erick Gomez-Nieto, Paulo Joia and Luis Gustavo Nonato. Spectral Image Segmentation Using Image Decomposition and Inner Product-Based Metric. Journal of Mathematical Imaging and Vision 45(3):227–238, 2012.