Toward Advancing License Plate Super-Resolution in Real-World Scenarios: A Dataset and Benchmark

Valfride Nascimento; Gabriel E. Lima; Rafael O. Ribeiro; William Robson Schwartz; Rayson Laroca; David Menotti

doi:10.5753/jbcs.2025.5159

Authors

Valfride Nascimento Federal University of Paraná https://orcid.org/0000-0002-7416-613X
Gabriel E. Lima Federal University of Paraná https://orcid.org/0009-0009-7599-8550
Rafael O. Ribeiro Brazilian Federal Police https://orcid.org/0000-0002-6381-3469
William Robson Schwartz Federal University of Minas Gerais https://orcid.org/0000-0003-1449-8834
Rayson Laroca Pontifical Catholic University of Paraná, Federal University of Paraná https://orcid.org/0000-0003-1943-2711
David Menotti Federal University of Paraná https://orcid.org/0000-0003-2430-2030

DOI:

https://doi.org/10.5753/jbcs.2025.5159

Keywords:

Ensemble, License Plate Recognition, Super-Resolution, UFPR-SR-Plates dataset

Abstract

Recent advancements in super-resolution for License Plate Recognition (LPR) have sought to address challenges posed by low-resolution (LR) and degraded images in surveillance, traffic monitoring, and forensic applications. However, existing studies have relied on private datasets and simplistic degradation models. To address this gap, we introduce UFPR-SR-Plates, a novel dataset containing 10,000 tracks with 100,000 paired low and high-resolution license plate images captured under real-world conditions. We establish a benchmark using multiple sequential LR and high-resolution (HR) images per vehicle – five of each – and two state-of-the-art models for super-resolution of license plates. We also investigate three fusion strategies to evaluate how combining predictions from a leading Optical Character Recognition (OCR) model for multiple super-resolved license plates enhances overall performance. Our findings demonstrate that super-resolution significantly boosts LPR performance, with further improvements observed when applying majority vote-based fusion techniques. Specifically, the Layout-Aware and Character-Driven Network (LCDNet) model combined with the Majority Vote by Character Position (MVCP) strategy led to the highest recognition rates, increasing from 1.7% with low-resolution images to 31.1% with super-resolution, and up to 44.7% when combining OCR outputs from five super-resolved images. These findings underscore the critical role of super-resolution and temporal information in enhancing LPR accuracy under real-world, adverse conditions. The proposed dataset is publicly available to support further research and can be accessed at: https://valfride.github.io/nascimento2024toward/.

Downloads

Download data is not yet available.

References

AlHalawani, S., Benjdira, B., Ammar, A., Koubaa, A., and Ali, A. M. (2024). DiffPlate: A diffusion model for super-resolution of license plate images. Electronics, 13(13):2670. DOI: 10.3390/electronics13132670.

Angelika Mulia, D., Safitri, S., and Gede Putra Kusuma Negara, I. (2024). YOLOv8 and Faster R-CNN performance evaluation with super-resolution in license plate recognition. International Journal of Computing and Digital Systems, 16(1):365-375. DOI: 10.12785/ijcds/160129.

Chen, S.-L., Liu, Q., Chen, F., and Yin, X.-C. (2023). End-to-end multi-line license plate recognition with cascaded perception. In International Conference on Document Analysis and Recognition (ICDAR), pages 274-289. DOI: 10.1007/978-3-031-41734-4_17.

Ghamrawi, N. and McCallum, A. (2005). Collective multi-label classification. In Proceedings of the 14th ACM international conference on Information and knowledge management, pages 195-200. DOI: 10.1145/1099554.109959.

Gonçalves, G. R. et al. (2018). Real-time automatic license plate recognition through deep multi-task networks. In Conference on Graphics, Patterns and Images (SIBGRAPI), pages 110-117. DOI: 10.1109/SIBGRAPI.2018.00021.

Gonçalves, G. R. et al. (2019). Multi-task learning for low-resolution license plate recognition. In Iberoamerican Congress on Pattern Recognition (CIARP), pages 251-261. DOI: 10.1007/978-3-030-33904-3_23.

Hamdi, A., Chan, Y. K., and Koo, V. C. (2021). A new image enhancement and super resolution technique for license plate recognition. Heliyon, 7(11). DOI: 10.1016/j.heliyon.2021.e08341.

Hijji, M., Khan, A., Alwakeel, M. M., Harrabi, R., Aradah, F., Cheikh, F. A., Sajjad, M., and Muhammad, K. (2023). Intelligent image super-resolution for vehicle license plate in surveillance applications. Mathematics, 11(4):892. DOI: 10.3390/math11040892.

Jia, W. and Xie, M. (2023). An efficient license plate detection approach with deep convolutional neural networks in unconstrained scenarios. IEEE Access, 11:85626-85639. DOI: 10.1109/ACCESS.2023.3301122.

Johnson, J., Alahi, A., and Fei-Fei, L. (2016). Perceptual losses for real-time style transfer and super-resolution. In European Conference on Computer Vision (ECCV), pages 694-711. DOI: 10.1007/978-3-319-46475-6_43.

Ke, X. et al. (2023). An ultra-fast automatic license plate recognition approach for unconstrained scenarios. IEEE Transactions on Intelligent Transportation Systems, 24(5):5172-5185. DOI: 10.1109/TITS.2023.3237581.

Kim, D., Kim, J., and Park, E. (2024). AFA-Net: Adaptive feature attention network in image deblurring and super-resolution for improving license plate recognition. Computer Vision and Image Understanding, 238:103879. DOI: 10.1016/j.cviu.2023.103879.

Kittler, J., Hatef, M., Duin, R. P., and Matas, J. (1998). On combining classifiers. IEEE transactions on pattern analysis and machine intelligence, 20(3):226-239. DOI: 10.1109/34.667881.

Laroca, R., dos Santos, M., and Menotti, D. (2025). Improving small drone detection through multi-scale processing and data augmentation. In International Joint Conference on Neural Networks (IJCNN), pages 1-8. DOI: 10.48550/arXiv.2504.19347.

Laroca, R., Estevam, V., Britto Jr., A. S., Minetto, R., and Menotti, D. (2023a). Do we train on test data? The impact of near-duplicates on license plate recognition. In International Joint Conference on Neural Networks (IJCNN), pages 1-8. DOI: 10.1109/IJCNN54540.2023.10191584.

Laroca, R., Zanlorensi, L. A., Estevam, V., Minetto, R., and Menotti, D. (2023b). Leveraging model fusion for improved license plate recognition. In Iberoamerican Congress on Pattern Recognition, pages 60-75. DOI: 10.1007/978-3-031-49249-5_5.

Laroca, R., Zanlorensi, L. A., Gonçalves, G. R., Todt, E., Schwartz, W. R., and Menotti, D. (2021). An efficient and layout-independent automatic license plate recognition system based on the YOLO detector. IET Intelligent Transport Systems, 15(4):483-503. DOI: 10.1049/itr2.12030.

Laroca, R. et al. (2022). On the cross-dataset generalization in license plate recognition. In International Conference on Computer Vision Theory and Applications (VISAPP), pages 166-178. DOI: 10.5220/0010846800003124.

Ledig, C. et al. (2017). Photo-realistic single image super-resolution using a generative adversarial network. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 105-114. DOI: 10.1109/CVPR.2017.19.

Lee, S., Kim, J.-H., and Heo, J.-P. (2020). Super-resolution of license plate images via character-based perceptual loss. In IEEE International Conference on Big Data and Smart Computing (BigComp), pages 560-563. DOI: 10.1109/BigComp48618.2020.000-1.

Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., and Timofte, R. (2021). Swinir: Image restoration using swin transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1833-1844. DOI: 10.1109/ICCVW54120.2021.00210.

Lima, G. E. et al. (2024). Toward enhancing vehicle color recognition in adverse conditions: A dataset and benchmark. In Conference on Graphics, Patterns and Images (SIBGRAPI), pages 1-6. DOI: 10.1109/SIBGRAPI62404.2024.10716307.

Lin, M., Liu, L., Wang, F., Li, J., and Pan, J. (2021). License plate image reconstruction based on generative adversarial networks. Remote Sensing, 13(15):3018. DOI: 10.3390/rs13153018.

Liu, A., Liu, Y., Gu, J., Qiao, Y., and Dong, C. (2023). Blind image super-resolution: A survey and beyond. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(5):5461-5480. DOI: 10.1109/TPAMI.2022.3203009.

Liu, Q., Liu, Y., Chen, S.-L., Zhang, T.-H., Chen, F., and Yin, X.-C. (2024a). Improving multi-type license plate recognition via learning globally and contrastively. IEEE Transactions on Intelligent Transportation Systems, pages 1-11. Early Access. DOI: 10.1109/TITS.2024.3365537.

Liu, Y.-Y., Liu, Q., Chen, S.-L., Chen, F., and Yin, X.-C. (2024b). Irregular license plate recognition via global information integration. In International Conference on Multimedia Modeling, pages 325-339. DOI: 10.1007/978-3-031-53308-2_24.

Luo, X., Huang, Y., and Miao, W. (2024). Real-world license plate image super-resolution via domain-specific degradation modeling. In IEEE Conference on Artificial Intelligence (CAI), pages 1175-1180. IEEE. DOI: 10.1109/CAI59869.2024.00210.

Maier, A., Moussa, D., Spruck, A., Seiler, J., and Riess, C. (2022). Reliability scoring for the recognition of degraded license plates. In IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), pages 1-8. DOI: 10.1109/AVSS56176.2022.9959390.

Mehri, A., Ardakani, P. B., and Sappa, A. D. (2021). MPRNet: Multi-path residual network for lightweight image super resolution. In IEEE Winter Conference on Applications of Computer Vision (WACV), pages 2703-2712. DOI: 10.1109/WACV48630.2021.00275.

Moussa, D. et al. (2022). Forensic license plate recognition with compression-informed transformers. In IEEE International Conference on Image Processing (ICIP), pages 406-410. DOI: 10.1109/ICIP46576.2022.9897178.

Nascimento, V., Laroca, R., and Menotti, D. (2024a). Super-resolution towards license plate recognition. In Anais do XXXVII Concurso de Teses e DissertaÃ§Ãµes (CTD) do Congresso da Sociedade Brasileira de ComputaÃ§Ã£o (CSBC), pages 78-87. DOI: 10.5753/ctd.2024.1999.

Nascimento, V., Laroca, R., Ribeiro, R. O., Schwartz, W. R., and Menotti, D. (2024b). Enhancing license plate super-resolution: A layout-aware and character-driven approach. Conference on Graphics, Patterns and Images (SIBGRAPI), pages 1-6. DOI: 10.1109/SIBGRAPI62404.2024.10716303.

Nascimento, V. et al. (2022). Combining attention module and pixel shuffle for license plate super-resolution. In Conference on Graphics, Patterns and Images (SIBGRAPI), pages 228-233. DOI: 10.1109/SIBGRAPI55357.2022.9991753.

Nascimento, V. et al. (2023). Super-resolution of license plate images using attention modules and sub-pixel convolution layers. Computers & Graphics, 113:69-76. DOI: 10.1016/j.cag.2023.05.005.

Oliveira, I. O. et al. (2021). Vehicle-Rear: A new dataset to explore feature fusion for vehicle identification using convolutional neural networks. IEEE Access, 9:101065-101077. DOI: 10.1109/ACCESS.2021.3097964.

Pan, S., Chen, S.-B., and Luo, B. (2023). A super-resolution-based license plate recognition method for remote surveillance. Journal of Visual Communication and Image Representation, 94:103844. DOI: 10.1016/j.jvcir.2023.103844.

Pan, Y., Tang, J., and Tjahjadi, T. (2024). LPSRGAN: Generative adversarial networks for super-resolution of license plate image. Neurocomputing, 580:127426. DOI: 10.1016/j.neucom.2024.127426.

Prabhavalkar, R., Hori, T., Sainath, T. N., Schlüter, R., and Watanabe, S. (2023). End-to-end speech recognition: A survey. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 32:325-351. DOI: 10.1109/TASLP.2023.3328283.

Presidência da República (2014). Lei nº 9.503, de 23 de setembro de 1997. Código de Trânsito Brasileiro. [link].

Rao, Z. et al. (2024). License plate recognition system in unconstrained scenes via a new image correction scheme and improved CRNN. Expert Systems with Applications, 243:122878. DOI: 10.1016/j.eswa.2023.122878.

Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016). You only look once: Unified, real-time object detection. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 779-788. DOI: 10.1109/CVPR.2016.91.

Reul, C., Springmann, U., Wick, C., and Puppe, F. (2018). Improving ocr accuracy on early printed books by utilizing cross fold training and voting. In 2018 13th IAPR International Workshop on Document Analysis Systems (DAS), pages 423-428. IEEE. DOI: 10.1109/DAS.2018.30.

Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D. J., and Norouzi, M. (2023). Image super-resolution via iterative refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(4):4713-4726. DOI: 10.1109/TPAMI.2022.3204461.

Schirrmacher, F., Lorch, B., Maier, A., and Riess, C. (2023). Benchmarking probabilistic deep learning methods for license plate recognition. IEEE Transactions on Intelligent Transportation Systems, 24(9):9203-9216. DOI: 10.1109/TITS.2023.3278533.

Sendjasni, A. and Larabi, M.-C. (2024). Embedding similarity learning for extreme license plate super-resolution. In 2024 IEEE 26th International Workshop on Multimedia Signal Processing (MMSP), pages 1-6. IEEE. DOI: 10.1109/MMSP61759.2024.10743401.

Shi, B., Yang, M., Wang, X., Lyu, P., Yao, C., and Bai, X. (2019). ASTER: An attentional scene text recognizer with flexible rectification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(9):2035-2048. DOI: 10.1109/TPAMI.2018.2848939.

Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., and Wang, Z. (2016). Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1874-1883. DOI: 10.48550/arXiv.1609.05158.

Silva, S. M. and Jung, C. R. (2020). Real-time license plate detection and recognition using deep convolutional neural networks. Journal of Visual Communication and Image Representation, page 102773. DOI: 10.1016/j.jvcir.2020.102773.

Silva, S. M. and Jung, C. R. (2022). A flexible approach for automatic license plate recognition in unconstrained scenarios. IEEE Transactions on Intelligent Transportation Systems, 23(6):5693-5703. DOI: 10.1109/TITS.2021.3055946.

Ultralytics (2023). YOLOv8. Available at: [link] Accessed: 2024-06-28.

Wang, X., Xie, L., Dong, C., and Shan, Y. (2021). Real-esrgan: Training real-world blind super-resolution with pure synthetic data. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1905-1914. DOI: 10.1109/ICCVW54120.2021.00217.

Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., and Change Loy, C. (2018). Esrgan: Enhanced super-resolution generative adversarial networks. In Proceedings of the European conference on computer vision (ECCV) workshops, pages 0-0. DOI: 10.1007/978-3-030-11021-5_5.

Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Qiao, Y., and Loy, C. C. (2019). ESRGAN: Enhanced super-resolution generative adversarial networks. In European Conference on Computer Vision (ECCV), pages 63-79. DOI: 10.1007/978-3-030-11021-5_5.

Wang, Y., Bian, Z.-P., Zhou, Y., and Chau, L.-P. (2022). Rethinking and designing a high-performing automatic license plate recognition approach. IEEE Transactions on Intelligent Transportation Systems, 23(7):8868-8880. DOI: 10.1109/TITS.2021.3087158.

Wei, C., Han, F., Fan, Z., Shi, L., and Peng, C. (2024). Efficient license plate recognition in unconstrained scenarios. Journal of Visual Communication and Image Representation, 104:104314. DOI: 10.1016/j.jvcir.2024.104314.

Xu, Z., Yang, W., Meng, A., Lu, N., Huang, H., Ying, C., and Huang, L. (2018). Towards end-to-end license plate detection and recognition: A large dataset and baseline. In European Conference on Computer Vision (ECCV), pages 261-277. DOI: 10.1007/978-3-030-01261-8_16.

Zhang, R. et al. (2018). The unreasonable effectiveness of deep features as a perceptual metric. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 586-595. DOI: 10.1109/CVPR.2018.00068.

Zhao, H.-h. and Liu, H. (2020). Multiple classifiers fusion and cnn feature extraction for handwritten digits recognition. Granular Computing, 5(3):411-418. DOI: 10.1007/s41066-019-00158-6.

Zhou, Z.-H. (2025). Ensemble methods: foundations and algorithms. CRC press. Book.