Transfer-Guided Hyperparameter Optimization for Hybrid Compact Convolutional Transformers in Handwritten Recognition

Muhammad Munsarif; Richmond Ampah-Mensah

doi:10.5753/jbcs.2026.8588

Authors

Muhammad Munsarif Universitas Muhammadiyah Semarang https://orcid.org/0000-0002-3009-4234
Richmond Ampah-Mensah Universitas Muhammadiyah Semarang https://orcid.org/0009-0007-2124-3262

DOI:

https://doi.org/10.5753/jbcs.2026.8588

Keywords:

Handwritten recognition, Compact Convolutional Transformer, Hyperparameter transfer, Genetic algorithm, CCT-LSTM, CCT-GRU

Abstract

Handwritten recognition remains sensitive to model architecture and hyperparameter selection. This study proposes a two-stage transfer-guided strategy in which CNN genetic algorithm (CNN-GA) hyperparameters are selected first and then transferred to Compact Convolutional Transformer (CCT), CCT-LSTM, CCT-biLSTM, and CCT-GRU models. The models are evaluated with 2 and 7 transformer encoder layers on an English handwriting dataset derived from NIST Special Database 19 and on MNIST. The reported CNN-GA baseline reaches 97.77% on English handwriting, while CCT-based models reach 98.96%–99.07% with 2 layers and 99.46%–99.63% with 7 layers. CCT-GRU gives the highest reported 2-layer English result at 99.07%, and CCT-LSTM gives the highest reported 7-layer result at 99.63%. On MNIST, plain CCT reaches the highest reported result, 99.55% with 7 layers. The findings show that hyperparameter transfer is useful, but recurrent hybridization remains dataset- and depth-dependent.

Downloads

Download data is not yet available.

References

Agrawal, V., Jagtap, J., and Kantipudi, M. V. V. P. (2024a). Exploration of advancements in handwritten document recognition techniques. Intelligent Systems with Applications, 22:200358. DOI: 10.1016/J.ISWA.2024.200358.

Agrawal, V., Jagtap, J., Patil, S., and Kotecha, K. (2024b). Performance analysis of hybrid deep learning framework using a vision transformer and convolutional neural network for handwritten digit recognition. MethodsX, 12:102554. DOI: 10.1016/J.MEX.2024.102554.

Baldominos, A., Saez, Y., and Isasi, P. (2019). A survey of handwritten character recognition with mnist and emnist. Applied Sciences, 9(15):3169. DOI: 10.3390/APP9153169.

Berriche, L., Alqahtani, A., and Rekik, S. (2024). Hybrid arabic handwritten character segmentation using cnn and graph theory algorithm. Journal of King Saud University - Computer and Information Sciences, 36(1):101872. DOI: 10.1016/J.JKSUCI.2023.101872.

Bordelon, B., Noci, L., Li, M., Hanin, B., and Pehlevan, C. (2023). Depthwise hyperparameter transfer in residual networks: Dynamics and scaling limit. arXiv. DOI: 10.48550/arXiv.2309.16620.

Chitty-Venkata, K. T., Emani, M., Vishwanath, V., and Somani, A. K. (2022). Neural architecture search for transformers: A survey. IEEE Access, 10:108374-108412. DOI: 10.1109/ACCESS.2022.3212767.

Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using rnn encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, pages 1724-1734. DOI: 10.3115/v1/d14-1179.

Cohen, G., Afshar, S., Tapson, J., and van Schaik, A. (2017). Emnist: An extension of mnist to handwritten letters. arXiv. DOI: 10.48550/arXiv.1702.05373.

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., and Houlsby, N. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv. DOI: 10.48550/arXiv.2010.11929.

Fatyanosa, T. N. and Aritsugi, M. (2021). An automatic convolutional neural network optimization using a diversity-guided genetic algorithm. IEEE Access, 9:91410-91426. DOI: 10.1109/ACCESS.2021.3091729.

Garrido-Munoz, C., Rios-Vila, A., and Calvo-Zaragoza, J. (2025). Handwritten text recognition: A survey. arXiv. DOI: 10.48550/arXiv.2502.08417.

Hassani, A., Walton, S., Shah, N., Abuduweili, A., Li, J., and Shi, H. (2021). Escaping the big data paradigm with compact transformers. arXiv. DOI: 10.48550/arXiv.2104.05704.

Li, Y., Chen, D., Tang, T., and Shen, X. (2025). Htr-vt: Handwritten text recognition with vision transformer. Pattern Recognition, 158:110967. DOI: 10.1016/J.PATCOG.2024.110967.

Munsarif, M., Noersasongko, E., Andono, P. N., and Soeleman, M. A. (2023). The evaluation of convolutional neural network and genetic algorithm performance based on the number of hyperparameters for english handwritten recognition. IAES International Journal of Artificial Intelligence (IJ-AI), 12(3):1250-1259. DOI: 10.11591/ijai.v12.i3.pp1250-1259.

Nguyen, V., Cai, J., and Chu, J. (2019). Hybrid cnn-gru model for high efficient handwritten digit recognition. In ACM International Conference Proceeding Series, pages 66-71. DOI: 10.1145/3357254.3357276.

Qian, X. et al. (2023). Knowledge transfer evolutionary search for lightweight neural architecture with dynamic inference. Pattern Recognition, 143:109790. DOI: 10.1016/J.PATCOG.2023.109790.

Raiaan, M. A. K. et al. (2024). A systematic review of hyperparameter optimization techniques in convolutional neural networks. Decision Analytics Journal, 11:100470. DOI: 10.1016/J.DAJOUR.2024.100470.

Sun, Y., Xue, B., Zhang, M., Yen, G. G., and Lv, J. (2020). Automatically designing cnn architectures using the genetic algorithm for image classification. IEEE Transactions on Cybernetics, 50(9):3840-3854. DOI: 10.1109/TCYB.2020.2983860.

Yang, G. et al. (2022). Tensor programs v: Tuning large neural networks via zero-shot hyperparameter transfer. arXiv. DOI: 10.48550/arXiv.2203.03466.

Zhan, Z.-H., Li, J.-Y., and Zhang, J. (2022). Evolutionary deep learning: A survey. Neurocomputing, 483:42-58. DOI: 10.1016/J.NEUCOM.2022.01.099.