A Simple U-Diffusion Inpainting Structure
DOI:
https://doi.org/10.5753/jbcs.2026.4907Keywords:
LDMM, Multiscale, Inpainting, LLEAbstract
The inpainting problem is addressed in this work through a very simplified version of the approach based on low-dimensional manifold model (LDMM), in which the actual working principle of the LDMM is put into evidence, namely, the simulated diffusion of image pixels that takes place in a manifold from which patches are drawn to form a given image. The simplicity of this principle is translated into a straightforward algorithm that borrows ideas from the Locally Linear Embedding (LLE) method, commonly used for dimensionality reduction and data visualization. The equivalence between this much simpler algorithm and the original LDMM is illustrated through visual inspection and experimental measurements of peak signal-to-noise ratios. By maintaining the key components of LDMM while reducing conceptual and computational complexity, the proposed method offers a streamlined solution for image inpainting tasks. Additionally, a (U-shaped) multi-scale use of the proposed algorithm is presented as a significantly better initializer for missing pixels, thus reducing the number of algorithmic iterations for convergence.
Downloads
References
Adam, R. D., Peter, P., and Weickert, J. (2017). Denoising by inpainting. In Scale Space and Variational Methods in Computer Vision: 6th International Conference, SSVM 2017, Kolding, Denmark, June 4-8, 2017, Proceedings 6, pages 121-132. Springer. DOI: 10.1007/978-3-319-58771-4_10.
Adler, A., Emiya, V., Jafari, M. G., Elad, M., Gribonval, R., and Plumbley, M. D. (2011). Audio inpainting. IEEE Transactions on Audio, Speech, and Language Processing, 20(3):922-932. DOI: 10.1109/TASL.2011.2168211.
Burger, H. C., Schuler, C. J., and Harmeling, S. (2012). Image denoising: Can plain neural networks compete with bm3d? In 2012 IEEE conference on computer vision and pattern recognition, pages 2392-2399. IEEE. DOI: 10.1109/CVPR.2012.6247952.
Carlsson, G., Ishkhanov, T., De Silva, V., and Zomorodian, A. (2008). On the local behavior of spaces of natural images. International journal of computer vision, 76(1):1-12. DOI: 10.1007/s11263-007-0056-x.
Donoho, D. L. (2006). For most large underdetermined systems of equations, the minimal l1-norm near-solution approximates the sparsest near-solution. Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences, 59(7):907-934. DOI: 10.1002/cpa.20132.
Gadylshin, K., Silvestrov, I., and Bakulin, A. (2020). Inpainting of local wavefront attributes using artificial intelligence for enhancement of massive 3-d pre-stack seismic data. Geophysical Journal International, 223(3):1888-1898. DOI: 10.1190/segam2019-3214642.1.
Getreuer, P. (2012). Total variation inpainting using split bregman. Image Processing On Line, 2:147-157. DOI: 10.5201/IPOL.2012.G-TVI.
Gilboa, G. and Osher, S. (2009). Nonlocal operators with applications to image processing. Multiscale Modeling & Simulation, 7(3):1005-1028. DOI: 10.1137/0706985.
Lee, A. B., Pedersen, K. S., and Mumford, D. (2003). The nonlinear statistics of high-contrast patches in natural images. International Journal of Computer Vision, 54(1):83-103. DOI: 10.1023/a:1023705401078.
Li, J., Huang, C., Chan, R., Feng, H., Ng, M. K., and Zeng, T. (2023). Spherical image inpainting with frame transformation and data-driven prior deep networks. SIAM Journal on Imaging Sciences, 16(3):1177-1194. DOI: 10.1137/22M152462X.
Li, Z., Shi, Z., and Sun, J. (2017). Point integral method for solving poisson-type equations on manifolds from point clouds with convergence guarantees. Communications in Computational Physics, 22(1):228-258. DOI: 10.4208/cicp.111015.250716a.
Montalvão, J., Canuto, J., and Miranda, L. (2020). Bias-compensated estimator for intrinsic dimension and differential entropy. Journal of Communication and Information Systems, 35(1):300-310. DOI: 10.14209/jcis.2020.30.
Osher, S., Shi, Z., and Zhu, W. (2017). Low dimensional manifold model for image processing. SIAM Journal on Imaging Sciences, 10(4):1669-1690. DOI: 10.1137/16M1058686.
Pan, J., Hu, Z., Su, Z., and Yang, M.-H. (2014). Deblurring text images via l0-regularized intensity and gradient prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2901-2908. DOI: 10.1109/cvpr.2014.371.
Peyré, G. (2009). Manifold models for signals and images. Computer vision and image understanding, 113(2):249-260. DOI: 10.1016/j.cviu.2008.09.003.
Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention-MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pages 234-241. Springer. DOI: 10.1007/978-3-319-24574-4_28.
Roweis, S. T. and Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. science, 290(5500):2323-2326. DOI: 10.1126/science.290.5500.2323.
Saad, Y. and Schultz, M. H. (1986). Gmres: A generalized minimal residual algorithm for solving nonsymmetric linear systems. SIAM Journal on scientific and statistical computing, 7(3):856-869. DOI: 10.1137/0907058.
Tauböck, G., Rajbamshi, S., and Balazs, P. (2020). Dictionary learning for sparse audio inpainting. IEEE Journal of Selected Topics in Signal Processing, 15(1):104-119. DOI: 10.1109/JSTSP.2020.3046422.
Ulyanov, D., Vedaldi, A., and Lempitsky, V. (2018). Deep image prior. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 9446-9454. DOI: 10.1109/cvpr.2018.00984.
Vishwakarma, A. (2023). Denoising and inpainting of sonar images using convolutional sparse representation. IEEE Transactions on Instrumentation and Measurement, 72:1-9. DOI: 10.1109/TIM.2023.3246527.
Yao, D., McLaughlin, S., and Altmann, Y. (2022). Patch-based image restoration using expectation propagation. SIAM Journal on Imaging Sciences, 15(1):192-227. DOI: 10.1137/21M1427541.
Ye, J. C., Han, Y., and Cha, E. (2018). Deep convolutional framelets: A general deep learning framework for inverse problems. SIAM Journal on Imaging Sciences, 11(2):991-1048. DOI: 10.1137/17M1141771.
Yin, R., Gao, T., Lu, Y. M., and Daubechies, I. (2017). A tale of two bases: Local-nonlocal regularization on image patches with convolution framelets. SIAM Journal on Imaging Sciences, 10(2):711-750. DOI: 0.1137/16M1091447.
Zhang, D., Liang, Z., Yang, G., Li, Q., Li, L., and Sun, X. (2018). A robust forgery detection algorithm for object removal by exemplar-based image inpainting. Multimedia Tools and Applications, 77:11823-11842. DOI: 10.1007/s11042-017-4829-0.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Jugurta Montalvão, Gabriel F.A. Bastos, Israel J. Santos Filho
This work is licensed under a Creative Commons Attribution 4.0 International License.
