Efficient Multiscale Object-based Superpixel Framework
DOI:
https://doi.org/10.5753/jbcs.2025.4278Keywords:
Superpixel Delineation, Image Foresting Transform, Object Saliency Map, Image SegmentationAbstract
Superpixel segmentation can be used as an intermediary step in many applications, often to improve object delineation and reduce computer workload. However, classical methods do not incorporate information about the desired object. Deep-learning-based approaches consider object information, but their delineation performance depends on data annotation. Additionally, the computational time of object-based methods is usually much higher than desired. In this work, we propose a novel superpixel framework which exploits object information being able to generate a multiscale segmentation on-the-fly. Our method starts off from seed oversampling and repeats optimal connectivity-based superpixel delineation and object-based seed removal until a desired number of superpixels is reached. It generalizes recent superpixel methods, surpassing them and other state-of-the-art approaches in efficiency and effectiveness according to multiple delineation metrics.
Downloads
References
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., and Süsstrunk, S. (2012). SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11):2274-2282. DOI: 10.1109/TPAMI.2012.120.
Achanta, R. and Süsstrunk, S. (2017). Superpixels and polygons using simple non-iterative clustering. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4895-4904. DOI: 10.1109/CVPR.2017.520.
Awaisu, M., Li, L., Peng, J., and Zhang, J. (2019). Fast superpixel segmentation with deep features. In Advances in Computer Graphics, volume 11542, pages 410-416. DOI: 10.1007/978-3-030-22514-8_38.
Ban, Z., Liu, J., and Cao, L. (2018). Superpixel segmentation using gaussian mixture model. IEEE Transactions on Image Processing, 27(8):4105-4117. DOI: 10.1109/TIP.2018.2836306.
Belém, F., Borlido, I., João, L., Perret, B., Cousty, J., Guimarães, S., and Falcão, A. (2022). Fast and effective superpixel segmentation using accurate saliency estimation. In Discrete Geometry and Mathematical Morphology, pages 261-273. DOI: 10.1007/978-3-031-19897-7_21.
Belém, F., Borlido, I., João, L., Perret, B., Cousty, J., Guimarães, S., and Falcão, A. (2023). Novel arc-cost functions and seed relevance estimations for compact and accurate superpixels. Journal of Mathematical Imaging and Vision, 65(5):770-786. DOI: 10.1007/s10851-023-01156-9.
Belém, F., Cousty, J., Perret, B., Guimarães, S., and Falcão, A. (2021). Towards a simple and efficient object-based superpixel delineation framework. In 34th Conference on Graphics, Patterns and Images (SIBGRAPI), pages 346-353. DOI: 10.1109/SIBGRAPI54419.2021.00054.
Belém, F., Guimarães, S., and Falcão, A. (2019a). Superpixel segmentation by object-based iterative spanning forest. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, volume 11401, pages 334-341. DOI: 10.1007/978-3-030-13469-3_39.
Belém, F., Guimarães, S., and Falcão, A. (2020a). Superpixel generation by the iterative spanning forest using object information. In 33rd Conference on Graphics, Patterns and Images (SIBGRAPI), pages 22-28. Workshop of Thesis and Dissertations. DOI: 10.5753/sibgrapi.est.2020.12979.
Belém, F., Guimarães, S., and Falcão, A. (2020b). Superpixel segmentation using dynamic and iterative spanning forest. Signal Processing Letters, 27:1440-1444. DOI: 10.1109/LSP.2020.3015433.
Belém, F., Melo, L., Guimarães, S., and Falcão, A. (2019b). The importance of object-based seed sampling for superpixel segmentation. In 32nd Conference on Graphics, Patterns and Images (SIBGRAPI), pages 108-115. DOI: 10.1109/SIBGRAPI.2019.00023.
Bragantini, J., Martins, S., Castelo-Fernandez, C., and Falcão, A. (2018). Graph-based image segmentation using dynamic trees. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, pages 470-478. DOI: 10.1007/978-3-030-13469-3_55.
Castelo-Fernandez, C. and Falcão, A. (2019). Learning visual dictionaries from class-specific superpixel segmentation. In Computer Analysis of Images and Patterns, pages 171-182. DOI: 10.1007/978-3-030-29888-3_14.
Dhore, S. and Abin, D. (2021). Chest x-ray segmentation using watershed and super pixel segmentation technique. In International Conference on Communication information and Computing Technology (ICCICT), pages 1-4. DOI: 10.1109/ICCICT50803.2021.9510078.
Ding, J., Xue, N., Xia, G., Schiele, B., and Dai, D. (2023). Hgformer: Hierarchical grouping transformer for domain generalized semantic segmentation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15413-15423. DOI: 10.1109/CVPR52729.2023.01479.
Falcão, A. and Bergo, F. (2004). Interactive volume segmentation with differential image foresting transforms. IEEE Transactions on Medical Imaging, 23(9):1100-1108. DOI: 10.1109/TMI.2004.829335.
Falcão, A., Stolfi, J., and Lotufo, R. (2004). The image foresting transform: Theory, algorithms, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(1):19-29. DOI: 10.1109/TPAMI.2004.1261076.
Galvão, F., Falcão, A., and Chowdhury, A. (2018). RISF: recursive iterative spanning forest for superpixel segmentation. In 31st Conference on Graphics, Patterns and Images (SIBGRAPI), pages 408-415. DOI: 10.1109/SIBGRAPI.2018.00059.
Gonçalves, H., Vasconcelos, G., Rangel, P., Carvalho, M., Archilha, N., and Spina, T. (2019). cudaift: 180x faster image foresting transform for waterpixel estimation using cuda. In 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISAPP), volume 4, pages 395-404. DOI: 10.5220/0007402703950404.
Jampani, V., Sun, D., Liu, M., Yang, M., and Kautz, J. (2018). Superpixel sampling networks. In European Conference on Computer Vision (ECCV), volume 11211, pages 363-380. DOI: 10.1007/978-3-030-01234-2_22.
Li, Z. and Chen, J. (2015). Superpixel segmentation using linear spectral clustering. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1356-1363. DOI: 10.1109/CVPR.2015.7298741.
Liu, C., Zhao, R., and Pang, M. (2019). A fully automatic segmentation algorithm for ct lung images based on random forest. Medical Physics, 47(2):518-529. DOI: 10.1002/mp.13939.
Liu, M., Tuzel, O., Ramalingam, S., and Chellappa, R. (2011). Entropy rate superpixel segmentation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2097-2104. DOI: 10.1109/CVPR.2011.5995323.
Liu, Y., Yu, M., Li, B., and He, Y. (2018). Intrinsic manifold SLIC: A simple and efficient method for computing content-sensitive superpixels. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(3):653-666. DOI: 10.1109/TPAMI.2017.2686857.
Mansilla, L. and Miranda, P. (2016). Oriented image foresting transform segmentation: Connectivity constraints with adjustable width. In 29th Conference on Graphics, Patterns and Images (SIBGRAPI), pages 289-296. DOI: 10.1109/SIBGRAPI.2016.047.
Martins, S., Ruppert, G., Reis, F., Yasuda, C., and Falcão, A. (2019). A supervoxel-based approach for unsupervised abnormal asymmetry detection in MR images of the brain. In IEEE 16th International Symposium on Biomedical Imaging (ISBI), pages 882-885. DOI: 10.1109/ISBI.2019.8759166.
Miranda, P.and Mansilla, L. (2014). Oriented image foresting transform segmentation by seed competition. IEEE Transactions on Image Processing, 23(1):389-398. DOI: 10.1109/TIP.2013.2288867.
Neubert, P. and Protzel, P. (2012). Superpixel Benchmark and Comparison, volume 6. KIT Scientific Publishing. Book.
Peng, H., Aviles-Rivero, A., and Schonlieb, C. (2022). Hers superpixels: Deep affinity learning for hierarchical entropy rate segmentation. pages 72-81. DOI: 10.1109/WACV51458.2022.00015.
Qin, X., Zhang, Z., Huang, C., Dehghan, M., Zaiane, O., and Jagersand, M. (2020). U2-net: Going deeper with nested u-structure for salient object detection. Pattern Recognition, 106:107404. DOI: 10.1016/j.patcog.2020.107404.
Shen, J., Hao, X., Liang, Z., Liu, Y., Wang, W., and Shao, L. (2016). Real-time superpixel segmentation by DBSCAN clustering algorithm. IEEE Transactions on Image Processing, 25(12):5933-5942. DOI: 10.1109/TIP.2016.2616302.
Shi, J., Yan, Q., Xu, L., and Jia, J. (2015). Hierarchical image saliency detection on extended cssd. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(4):717-729. DOI: 10.1109/TPAMI.2015.2465960.
Sousa, A., Martins, S., Falcão, A., Reis, F., Bagatin, E., and Irion, K. (2019). Altis: A fast and automatic lung and trachea ct-image segmentation method. Medical Physics, 46(11):4970-4982. DOI: 10.1002/mp.13773.
Stutz, D., Hermans, A., and Leibe, B. (2018). Superpixels: An evaluation of the state-of-the-art. Computer Vision and Image Understanding, 166:1-27. DOI: 10.1016/j.cviu.2017.03.007.
Suzuki, T. (2020). Superpixel segmentation via convolutional neural networks with regularized information maximization. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2573-2577. DOI: 10.1109/ICASSP40776.2020.9054140.
Tu, W., Liu, M., Jampani, V., Sun, D., Chien, S., Yang, M., and Kautz, J. (2018). Learning superpixels with segmentation-aware affinity loss. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 568-576. DOI: 10.1109/CVPR.2018.00066.
Vargas-Muñoz, J., Chowdhury, A., Alexandre, E., Galvão, F., Miranda, P., and Falcão, A. (2019). An iterative spanning forest framework for superpixel segmentation. IEEE Transactions on Image Processing, 28(7):3477-3489. DOI: 10.1109/TIP.2019.2897941.
Wei, X., Yang, Q., Gong, Y., Ahuja, N., and Yang, M. (2018). Superpixel hierarchy. IEEE Transactions on Image Processing, 27(10):4838-4849. DOI: 10.1109/TIP.2018.2836300.
Wu, J., Liu, C., and Li, B. (2021). Texture-aware and structure-preserving superpixel segmentation. Computers and Graphics, 94:152-163. DOI: 10.1016/j.cag.2020.12.002.
Xiao, X., Zhou, Y., and Gong, Y. (2018). Content-adaptive superpixel segmentation. IEEE Transactions on Image Processing, 27(6):2883-2896. DOI: 10.1109/TIP.2018.2810541.
Xu, L., Zeng, L., and Wang, Z. (2014). Saliency-based superpixels. Signal, Image and Video Processing, 8(1):181-190. DOI: 10.1007/s11760-013-0520-8.
Yang, F., Sun, Q., Jin, H., and Zhou, Z. (2020). Superpixel segmentation with fully convolutional networks. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13961-13970. DOI: 10.1109/CVPR42600.2020.01398.
Yi, S., Ma, H., Wang, X., Hu, T., Li, X., and Wang, Y. (2022). Weakly-supervised semantic segmentation with superpixel guided local and global consistency. Pattern Recognition, 124:108504. DOI: 10.1016/j.patcog.2021.108504.
Yu, Y., Makihara, Y., and Yagi, Y. (2019). Pedestrian segmentation based on a spatio-temporally consistent graph-cut with optimal transport. IPSJ Transactions on Computer Vision and Applications, 11:10. DOI: 10.1186/s41074-019-0062-2.
Yu, Y., Yang, Y., and Liu, K. (2021). Edge-aware superpixel segmentation with unsupervised convolutional neural networks. In IEEE International Conference on Image Processing (ICIP), pages 1504-1508. DOI: 10.1109/ICIP42928.2021.9506289.
Zhang, S., Wang, H., Huang, W., and You, Z. (2018). Plant diseased leaf segmentation and recognition by fusion of superpixel, K-means and PHOG. Optik, 157:866-872. DOI: 10.1016/j.ijleo.2017.11.190.
Zhou, J., Ruan, J., Wu, C., Ye, G., Zhu, Z., Yue, J., and Zhang, Y. (2019). Superpixel segmentation of breast cancer pathology images based on features extracted from the autoencoder. In IEEE 11th International Conference on Communication Software and Networks (ICCSN), pages 366-370. DOI: 10.1109/ICCSN.2019.8905358.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Felipe C. Belém, Benjamin Perret, Jean Cousty, Silvio J. F. Guimarães, Alexandre X. Falcão
This work is licensed under a Creative Commons Attribution 4.0 International License.
