Learning on hierarchical trees with Random Forest
DOI:
https://doi.org/10.5753/jbcs.2026.4242Keywords:
Morphological hierarchies, Random Forest, Machine learning, Graphs, Image processingAbstract
Hierarchies, as described in mathematical morphology, represent nested regions of interest and provide mechanisms to create coherent data organization. They facilitate high-level analysis and management of large amounts of data. Represented as hierarchical trees, they have formalisms intersecting with graph theory and generalizable applications. Due to the deterministic algorithms, the multiform representations, and the absence of a direct quality evaluation, it is hard to insert hierarchical information into a learning framework and benefit from the recent advances. Researchers usually tackle this problem by refining the hierarchies for a specific media and assessing their quality for a particular task. The downside of this approach is that it depends on the application, and the formulations limit the generalization to similar data. This work aims to create a learning framework that can operate with hierarchical data and is agnostic to the input and application. The idea is to transform the data into a regular representation required by most learning models while preserving the rich information in the hierarchical structure. The proposed methods use edge-weighted image graphs and hierarchical trees as input, and they evaluate different proposals on the edge detection and segmentation tasks. The learning model is the Random Forest, a fast and scalable method for working with high-dimensional data. Results demonstrate that it is possible to create a learning framework dependent only on the hierarchical data that presents a state-of-the-art performance in multiple tasks.
Downloads
References
Adão, M. M., Guimarães, S. J. F., and Patrocínio Jr, Z. K. G. (2020). Learning to realign hierarchy for image segmentation. Pattern Recognition Letters, 133:287-294. DOI: 10.1016/j.patrec.2020.03.010.
Almeida, R., Kijak, E., Malinowski, S., Patrocínio Jr, Z. K., Araújo, A. A., and Guimarães, S. J. (2022). Graph-based image gradients aggregated with random forests. Pattern Recognition Letters. DOI: 10.1016/j.patrec.2022.08.015.
Almeida, R., Patrocínio Jr., Z. K. G., Araújo, A. d. A., Kijak, E., Malinowski, S., and Guimarães, S. J. F. (2021). Descriptive image gradient from edge-weighted image graph and random forests. In 34th Conference on Graphics, Patterns and Images, pages 338-345. DOI: 10.1109/SIBGRAPI54419.2021.00053.
Aptoula, E., Weber, J., and Lefèvre, S. (2013). Vectorial quasi-flat zones for color image simplification. In Mathematical Morphology and Its Applications to Signal and Image Processing., pages 231-242. Springer Berlin Heidelberg. DOI: 10.1007/978-3-642-38294-9_20.
Arbelaez, P., Maire, M., Fowlkes, C., and Malik, J. (2009). From contours to regions: An empirical evaluation. In IEEE Conference on Computer Vision and Pattern Recognition, pages 2294-2301. IEEE. DOI: 10.1109/CVPR.2009.5206707.
Arbelaez, P., Pont-Tuset, J., Barron, J., Marques, F., and Malik, J. (2014). Multiscale combinatorial grouping. In IEEE Conference on Computer Vision and Pattern Recognition, pages 328-335. IEEE. DOI: 10.1109/CVPR.2014.49.
Beucher, S. (1979). Use of watersheds in contour detection. In International Workshop on Image Processing, pages 2.1-2.12. CCETT/IRISA. Available at:[link].
Beucher, S. (1994). Watershed, hierarchical segmentation and waterfall algorithm. In Computational Imaging and Vision, volume 2, pages 69-76. Springer. DOI: 10.1007/978-94-011-1040-2_10.
Bondy, J. A., Murty, U. S. R., et al. (1976). Graph theory with applications, volume 290. Macmillan London. DOI: 10.1007/978-1-349-03521-2.
Bosilj, P., Kijak, E., and Lefèvre, S. (2018). Partition and inclusion hierarchies of images: a comprehensive survey. Journal of Imaging, 4:33. DOI: 10.3390/jimaging4020033.
Breiman, L. (2001). Random forests. Machine Learning, 45:5-32. DOI: 10.1023/A:1010933404324.
Chen, C., Wu, Y., Dai, Q., Zhou, H.-Y., Xu, M., Yang, S., Han, X., and Yu, Y. (2024). A survey on graph neural networks and graph transformers in computer vision: A task-oriented perspective. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12):10297-10318. DOI: 10.1109/TPAMI.2024.3445463.
Chen, D., Wu, X., Dong, J., He, Y., Xue, H., and Mao, F. (2020). Hierarchical sequence representation with graph network. In IEEE International Conference on Acoustics, Speech and Signal Processing, pages 2288-2292. IEEE. DOI: 10.1109/ICASSP40776.2020.9054195.
Chierchia, G. and Perret, B. (2020). Ultrametric fitting by gradient descent. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, page 124004, Red Hook, NY, USA. Curran Associates Inc.. DOI: 10.1088/1742-5468/abc62d.
Chuang, C.-Y., Li, J., Torralba, A., and Fidler, S. (2018). Learning to act properly: predicting and explaining affordances from images. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 975-983. IEEE. DOI: 10.1109/CVPR.2018.00108.
Clément, M., Kurtz, C., and Wendling, L. (2018). Learning spatial relations and shapes for structural object description and scene recognition. Pattern Recognition, 84:197-210. DOI: 10.1016/j.patcog.2018.06.017.
Cousty, J., Bertrand, G., Najman, L., and Couprie, M. (2009). Watershed cuts: minimum spanning forests and the drop of water principle. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31:1362-1374. DOI: 10.1109/TPAMI.2008.173.
Cousty, J., Najman, L., Kenmochi, Y., and Guimarães, S. (2018). Hierarchical segmentations with graphs: quasi-flat zones, minimum spanning trees, and saliency maps. Journal of Mathematical Imaging and Vision, 60:479-502. DOI: 10.1007/s10851-017-0768-7.
Cousty, J., Najman, L., and Perret, B. (2013). Constructive links between some morphological hierarchies on edge-weighted graphs. In Mathematical Morphology and Its Applications to Signal and Image Processing, pages 86-97. Springer Berlin Heidelberg. DOI: 10.1007/978-3-642-38294-9_8.
Dollar, P., Belongie, S., and Perona, P. (2010). The fastest pedestrian detector in the west. In Procedings of the British Machine Vision Conference, pages 68.1-68.11. British Machine Vision Association. DOI: 10.5244/C.24.68.
Dollar, P. and Zitnick, C. L. (2015). Fast edge detection using structured forests. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37:1558-1570. DOI: 10.1109/TPAMI.2014.2377715.
dos Santos Belo, L., Caetano Jr, C. A., do Patrocínio Jr, Z. K. G., and Guimaraes, S. J. F. (2016). Summarizing video sequence using a graph-based hierarchical approach. Neurocomputing, 173:1001-1016. DOI: 10.1016/j.neucom.2015.08.057.
Díaz, G., González, F. A., and Romero, E. (2009). A semi-automatic method for quantification and classification of erythrocytes infected with malaria parasites in microscopic images. Journal of Biomedical Informatics, 42:296-307. DOI: 10.1016/j.jbi.2008.11.005.
Fan, J., Zhao, T., Kuang, Z., Zheng, Y., Zhang, J., Yu, J., and Peng, J. (2017). Hd-mtl: hierarchical deep multi-task learning for large-scale visual recognition. IEEE Transactions on Image Processing, 26:1923-1938. DOI: 10.1109/TIP.2017.2667405.
Grossiord, E., Talbot, H., Passat, N., Meignan, M., and Najman, L. (2017). Automated 3d lymphoma lesion segmentation from pet/ct characteristics. In International Symposium on Biomedical Imaging, pages 174-178. IEEE. DOI: 10.1109/ISBI.2017.7950495.
Grover, A. and Leskovec, J. (2016). Node2vec: scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 855-864. ACM. DOI: 10.1145/2939672.2939754.
Guigues, L., Cocquerez, J. P., and Men, H. L. (2006). Scale-sets image analysis. International Journal of Computer Vision, 68(3):289-317. DOI: 10.1007/s11263-005-6299-0.
Hu, Z., Shi, T., Wang, C., Li, Q., and Wu, G. (2021). Scale-sets image classification with hierarchical sample enriching and automatic scale selection. International Journal of Applied Earth Observation and Geoinformation, 105:102605. DOI: 10.1016/j.jag.2021.102605.
Huang, J., Wang, M., Xu, X., Jie, B., and Zhang, D. (2020). A novel node-level structure embedding and alignment representation of structural networks for brain disease analysis. Medical Image Analysis, 65:101755-110767. DOI: 10.1016/j.media.2020.101755.
Ilin, R., Watson, T., and Kozma, R. (2017). Abstraction hierarchy in deep learning neural networks. In International Joint Conference on Neural Networks, pages 768-774, Anchorage, AK, USA. IEEE. DOI: 10.1109/IJCNN.2017.7965929.
Isella, L., Stehlé, J., Barrat, A., Cattuto, C., Pinton, J.-F., and Van den Broeck, W. (2011). What's in a crowd? analysis of face-to-face behavioral networks. Journal of theoretical biology, 271(1):166-180. DOI: 10.1016/j.jtbi.2010.11.033.
Ji, W., Li, X., Wei, L., Wu, F., and Zhuang, Y. (2020). Context-aware graph label propagation network for saliency detection. IEEE Transactions on Image Processing, 29:8177-8186. DOI: 10.1109/TIP.2020.3002083.
Jing, Y., Wang, J., Wang, W., Wang, L., and Tan, T. (2020). Relational graph neural network for situation recognition. Pattern Recognition, 108:107544. DOI: 10.1016/j.patcog.2020.107544.
Kiran, B. R. and Serra, J. (2015). Braids of partitions. In Mathematical Morphology and Its Applications to Signal and Image Processing, pages 217-228. Springer International Publishing. DOI: 10.1007/978-3-319-18720-4_19.
Krishnammal, P. M., Therase, L. M., Devi, E. A., and Joany, R. M. (2022). Wavelets and convolutional neural networks-based automatic segmentation and prediction of mri brain images. In IOT with Smart Systems, pages 229-241, Singapore. Springer. DOI: 10.1007/978-981-16-3945-6_23.
Kurtz, C., Naegel, B., and Passat, N. (2014). Connected filtering based on multivalued component-trees. IEEE Transactions on Image Processing, 23:5152-5164. DOI: 10.1109/TIP.2014.2362053.
Kurzweil, R. (2013). How to create a mind: the secret of human thought revealed. Penguin Books. DOI: 10.5860/choice.50-6167.
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017). Feature pyramid networks for object detection. In IEEE Conference on Computer Vision and Pattern Recognition, pages 936-944, Honolulu, HI, USA. IEEE. DOI: 10.1109/CVPR.2017.106.
Liu, Y., Cheng, M.-M., Hu, X., Bian, J.-W., Zhang, L., Bai, X., and Tang, J. (2019). Richer convolutional features for edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41:1939-1946. DOI: 10.1109/TPAMI.2018.2878849.
Luo, W., Zhang, C., Zhang, X., and Wu, H. (2019). Improving action recognition with the graph-neural-network-based interaction reasoning. In IEEE Visual Communications and Image Processing, pages 1-4. IEEE. DOI: 10.1109/VCIP47243.2019.8965768.
Maia, D. S., Pham, M.-T., Aptoula, E., Guiotte, F., and Lefevre, S. (2021). Classification of remote sensing data with morphological attribute profiles: a decade of advances. IEEE Geoscience and Remote Sensing Magazine, 9:43-71. DOI: 10.1109/MGRS.2021.3051859.
Makarov, I., Kiselev, D., Nikitinsky, N., and Subelj, L. (2021). Survey on graph embeddings and their applications to machine learning problems on graphs. PeerJ Computer Science, 7. DOI: 10.7717/peerj-cs.357.
Makrogiannis, S., Annasamudram, N., Wang, Y., Miranda, H., and Zheng, K. (2021). A system for spatio-temporal cell detection and segmentation in time-lapse microscopy. In IEEE International Conference on Bioinformatics and Biomedicine, pages 2266-2273, Houston, TX, USA. IEEE. DOI: 10.1109/BIBM52615.2021.9669421.
Maninis, K.-K., Pont-Tuset, J., Arbelaez, P., and Gool, L. V. (2018). Convolutional oriented boundaries: from image segmentation to high-level tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40:819-833. DOI: 10.1109/TPAMI.2017.2700300.
Mansilla, L. A. C. and Miranda, P. A. V. (2016). Oriented image foresting transform segmentation: connectivity constraints with adjustable width. In Conference on Graphics, Patterns and Images, pages 289-296. IEEE. DOI: 10.1109/SIBGRAPI.2016.047.
Marr, D. (1982). Vision: a computational investigation into the human representation and processing of visual information. MIT Press. DOI: 10.7551/mitpress/9780262514620.001.0001.
Martin, D., Fowlkes, C., Tal, D., and Malik, J. (2001). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings Eighth IEEE International Conference on Computer Vision, pages 416-423. IEEE Comput. Soc. DOI: 10.1109/ICCV.2001.937655.
Meyer, F. (2001). Hierarchies of partitions and morphological segmentation. In Scale-Space and Morphology in Computer Vision, volume 2106, pages 161-182. Springer Berlin Heidelberg. DOI: 10.1007/3-540-47778-0_14.
Micheli, A. (2009). Neural network for graphs: a contextual constructive approach. IEEE Transactions on Neural Networks, 20(3):498-511. DOI: 10.1109/tnn.2008.2010350.
Mihalcea, R. and Radev, D. (2012). Graph-based natural language processing and information retrieval. DOI: 10.1017/cbo9780511976247.
Najman, L. and Couprie, M. (2006). Building the component tree in quasi-linear time. IEEE Transactions on Image Processing, 15:3531-3539. DOI: 10.1109/TIP.2006.877518.
Najman, L. and Schmitt, M. (1996). Geodesic saliency of watershed contours and hierarchical segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18:1163-1173. DOI: 10.1109/34.546254.
Najman, L. and Talbot, H. (2013). Mathematical morphology: from theory to applications. John Wiley & Sons, Inc., 1 edition. DOI: 10.1002/9781118600788.
Newman, M. (2018). Networks. Oxford university press. Book.
Nguyen, T. T., Krishnakumari, P., Calvert, S. C., Vu, H. L., and van Lint, H. (2019). Feature extraction and clustering analysis of highway congestion. Transportation Research Part C: Emerging Technologies, 100:238-258. DOI: 10.1016/j.trc.2019.01.017.
Ortega, A., Frossard, P., Kovačević, J., Moura, J. M., and Vandergheynst, P. (2018). Graph signal processing: Overview, challenges, and applications. Proceedings of the IEEE, 106(5):808-828. DOI: 10.1109/jproc.2018.2820126.
Padilla, F. J. A., Romaniuk, B., Naegel, B., Servagi-Vernat, S., Morland, D., Papathanassiou, D., and Passat, N. (2021). Random walkers on morphological trees: a segmentation paradigm. Pattern Recognition Letters, 141:16-22. DOI: 10.1016/j.patrec.2020.11.001.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: machine learning in python. Journal of Machine Learning Research, 12:2825-2830. DOI: 10.48550/arxiv.1201.0490.
Perozzi, B., Al-Rfou, R., and Skiena, S. (2014). Deepwalk: online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 701-710. ACM. DOI: 10.1145/2623330.2623732.
Perret, B., Chierchia, G., Cousty, J., Guimarães, S. F., Kenmochi, Y., and Najman, L. (2019a). Higra: hierarchical graph analysis. SoftwareX, 10:100335. DOI: 10.1016/j.softx.2019.100335.
Perret, B., Cousty, J., Guimaraes, S. J. F., and Maia, D. S. (2018). Evaluation of hierarchical watersheds. IEEE Transactions on Image Processing, 27:1676-1688. DOI: 10.1109/TIP.2017.2779604.
Perret, B., Cousty, J., Guimarães, S. J. F., Kenmochi, Y., and Najman, L. (2019b). Removing non-significant regions in hierarchical clustering and segmentation. Pattern Recognition Letters, 128:433-439. DOI: 10.1016/j.patrec.2019.10.008.
Qi, X., Liao, R., Jia, J., Fidler, S., and Urtasun, R. (2017). 3d graph neural networks for rgbd semantic segmentation. In IEEE International Conference on Computer Vision, pages 5209-5218. IEEE. DOI: 10.1109/ICCV.2017.556.
Sakarya, U. and Telatar, Z. (2010). Video scene detection using graph-based representations. Signal Processing: Image Communication, 25(10):774-783. DOI: 10.1016/j.image.2010.10.001.
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G. (2009). The graph neural network model. IEEE Transactions on Neural Networks, 20:61-80. DOI: 10.1109/TNN.2008.2005605.
Selvan, R., Kipf, T., Welling, M., Juarez, A. G.-U., Pedersen, J. H., Petersen, J., and de Bruijne, M. (2020). Graph refinement based airway extraction using mean-field networks and graph neural networks. Medical Image Analysis, 64:101751. DOI: 10.1016/j.media.2020.101751.
Serna, A. and Marcotegui, B. (2014). Detection, segmentation and classification of 3d urban objects using mathematical morphology and supervised learning. ISPRS Journal of Photogrammetry and Remote Sensing, 93:243-255. DOI: 10.1016/j.isprsjprs.2014.03.015.
Soille, P. (2008). Constrained connectivity for hierarchical image partitioning and simplification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30:1132-1145. DOI: 10.1109/TPAMI.2007.70817.
Soille, P. and Najman, L. (2012). On morphological hierarchical representations for image processing and spatial data clustering. In Applications of Discrete Geometry and Mathematical Morphology, pages 43-67. Springer Berlin Heidelberg. DOI: 10.1007/978-3-642-32313-3_4.
Sokal, R. R. and Rohlf, J. (1962). The comparison of dendrograms by objective methods. TAXON, 11:33-40. DOI: 10.2307/1217208.
Sun, L., Huo, Q., Jia, W., and Chen, K. (2015). A robust approach for text detection from natural scene images. Pattern Recognition, 48:2906-2920. DOI: 10.1016/j.patcog.2015.04.002.
Taylor, C. J. (2013). Towards fast and accurate segmentation. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1916-1922. IEEE. DOI: 10.1109/CVPR.2013.250.
Tochon, G., Mura, M. D., Veganzones, M. A., Valero, S., Salembierr, P., and Chanussot, J. (2018). Advances in utilization of hierarchical representations in remote sensing data analysis. In Reference Module in Earth Systems and Environmental Sciences, pages 77-107. Elsevier. DOI: 10.1016/B978-0-12-409548-9.10340-9.
Tong, H., He, J., Li, M., Zhang, C., and Ma, W.-Y. (2005). Graph based multi-modality learning. In Proceedings of the 13th annual ACM international conference on Multimedia, pages 862-871. DOI: 10.1145/1101149.1101337.
Vachier, C. and Meyer, F. (1995). Extinction value: a new measurement of persistence. In IEEE Workshop on nonlinear signal and image processing, volume 1, pages 254-257. Neos Marmaras Greece. Available at:[link].
Wang, D., Cui, P., and Zhu, W. (2016). Structural deep network embedding. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1225-1234. ACM. DOI: 10.1145/2939672.2939753.
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., and Yu, P. S. (2020). A comprehensive survey on graph neural networks. IEEE transactions on neural networks and learning systems, 32(1):4-24. DOI: 10.1109/tnnls.2020.2978386.
Wyner, A., Olson, M., Bleich, J., and Mease, D. (2017). Explaining the success of adaboost and random forests as interpolating classifiers. The Journal of Machine Learning Research, 18:1558-1590. DOI: 10.5555/3122009.3153004.
Xu, C., Whitt, S., and Corso, J. J. (2013). Flattening supervoxel hierarchies by the uniform entropy slice. In IEEE International Conference on Computer Vision, pages 2240-2247. IEEE. DOI: 10.1109/ICCV.2013.279.
Xu, C., Xiong, C., and Corso, J. J. (2012). Streaming hierarchical video segmentation. In European Conference on Computer Vision, pages 626-639. Springer Berlin Heidelberg. DOI: 10.1007/978-3-642-33783-3_45.
Yang, G., Cao, J., Chen, Z., Guo, J., and Li, J. (2020). Graph-based neural networks for explainable image privacy inference. Pattern Recognition, 105:107360. DOI: 10.1016/j.patcog.2020.107360.
Zhang, J., Tsai, P.-H., and Tsai, M.-H. (2024). Semantic2graph: graph-based multi-modal feature fusion for action segmentation in videos. Applied Intelligence, 54(2):2084-2099. DOI: 10.1007/s10489-023-05259-z.
Zitnik, M., Nguyen, F., Wang, B., Leskovec, J., Goldenberg, A., and Hoffman, M. M. (2019). Machine learning for integrating data in biology and medicine: Principles, practice, and opportunities. Information Fusion, 50:71-91. DOI: 10.1016/j.inffus.2018.09.012.
Zwettler, G. and Backfrieder, W. (2015). Evolution strategy classification utilizing meta features and domain-specific statistical a priori models for fully-automated and entire segmentation of medical datasets in 3d radiology. In International Conference on Computing and Communications Technologies, pages 12-18, Chennai, India. IEEE. DOI: 10.1109/ICCCT2.2015.7292712.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Raquel Almeida, Laurent Amsaleg, Zenilton Kleber G. do Patrocínio Júnior, Ewa Kijak, Simon Malinowski, Silvio Jamil Ferzoli Guimarães
This work is licensed under a Creative Commons Attribution 4.0 International License.
