AShE – A Shadow Estimator for Augmented Reality Systems on Mobile Platforms
DOI:
https://doi.org/10.5753/jis.2026.6127Keywords:
Augmented Reality, Lighting Etimation, 3D ReconstructionAbstract
This work presents a geometric method for estimating the directional light vector in a scene, enabling realistic shadow generation in augmented reality through image segmentation and inverse rendering. The solution is designed for mobile devices, eliminating the need for specialized hardware and machine learning-based algorithms. The method employs standard cameras and fiducial markers to project realistic shadows on virtual objects, ensuring coherent visual integration with the real environment. The experiments demonstrated that the proposed approach achieved an average angular error of 11.42° for synthetic datasets, effectively estimating scene illumination and generating visually convincing shadow projections. Qualitative tests indicate that the system performs well under various lighting conditions, although it faces limitations in scenarios with translucent objects or diffuse lighting. The results suggest that this solution can serve as an efficient and accessible alternative for augmented reality applications on mobile devices, enhancing immersion and realism without requiring complex computational infrastructure.
Downloads
References
Azinovic, D., Li, T.-M., Kaplanyan, A., and Nießner, M. (2019). Inverse path tracing for joint material and lighting estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2447–2456. DOI: https://doi.org/10.1109/CVPR.2019.00255.
Azuma and Ronald, T. (1997). A survey of augmented reality. Presence: Teleoperators & Virtual Environments, 6(4):355–385. DOI: https://doi.org/10.1162/pres.1997.6.4.355.
Boom, B. J., Orts-Escolano, S., Ning, X. X., McDonagh, S., Sandilands, P., and Fisher, R. B. (2017). Interactive light source position estimation for augmented reality with an rgb-d camera. Computer Animation and Virtual Worlds, 28(1):e1686. DOI: https://doi.org/10.1002/cav.1686.
Cao, X. and Foroosh, H. (2007). Camera calibration and light source orientation from solar shadows. Computer Vision and Image Understanding, 105(1):60–72. DOI: https://doi.org/10.1016/j.cviu.2006.08.003.
Castro, T. K., Figueiredo, L. H., and Velho, L. (2012). Realistic shadows for mobile augmented reality. In Symposium on Virtual and Augmented Reality, pages 36–45. DOI: https://doi.org/10.1109/SVR.2012.9.
Chotikakamthorn, N. (2015). Near point light source location estimation from shadow edge correspondence. In 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM), pages 30–35. IEEE. DOI: https://doi.org/10.1109/iccis.2015.7274543.
Gruber, L., Richter-Trummer, T., and Schmalstieg, D. (2012). Real-time photometric registration from arbitrary geometry. In 2012 IEEE international symposium on mixed and augmented reality (ISMAR), pages 119–128. IEEE. DOI: https://doi.org/10.1109/ISMAR.2012.6402548.
Kán, P. and Kafumann, H. (2019). Deeplight: light source estimation for augmented reality using deep learning. The Visual Computer, 35(6):873–883. DOI: https://doi.org/10.1007/s00371-019-01666-x.
Kato, H., Beker, D., Morariu, M., Ando, T., Matsuoka, T., Kehl, W., and Gaidon, A. (2020). Differentiable rendering: A survey. arXiv preprint arXiv:2006.12057. DOI: https://doi.org/10.48550/arXiv.2006.12057.
Kato, H. and Billinghurst, M. (1999). Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR’99), pages 85–94. IEEE. DOI: https://doi.org/10.1109/IWAR.1999.803809.
Koc, E. and Balcisoy, S. (2013). Estimation of environmental lighting from known geometries for mobile augmented reality. In 2013 International Conference on Cyberworlds, pages 132–139. IEEE. DOI: https://doi.org/10.1109/CW.2013.65.
LeGendre, C., Ma, W.-C., Fyffe, G., Flynn, J., Charbonnel, L., Busch, J., and Debevec, P. (2019). Deeplight: Learning illumination for unconstrained mobile mixed reality. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5918–5928. DOI: https://doi.org/10.1109/CVPR.2019.00607.
Liu, D. S.-M. and Wu, S.-J. (2022). Light direction estimation and hand touchable interaction for augmented reality. Virtual Reality, 26(3):1155–1172. DOI: https://doi.org/10.1007/s10055-022-00624-8.
Lopez-Moreno, J., Garces, E., Hadap, S., Reinhard, E., and Gutierrez, D. (2013). Multiple light source estimation in a single image. In Computer Graphics Forum, volume 32, pages 170–182. Wiley Online Library. DOI: https://doi.org/10.1111/cgf.12195.
Macqueen, J. (1967). Some methods for classification and analysis of multivariate observations. In Proceedings of 5-th Berkeley Symposium on Mathematical Statistics and Probability/University of California Press.
Marques, B. A. D., Clua, E. W. G., Montenegro, A. A., and Vasconcelos, C. N. (2022). Spatially and color consistent environment lighting estimation using deep neural networks for mixed reality. Computers & Graphics, 102:257–268. DOI: https://doi.org/10.1016/j.cag.2021.08.007.
Nguyen, R. M. and Le, M. N. (2012). Light source estimation from a single image. In 2012 12th International Conference on Control Automation Robotics & Vision (ICARCV), pages 1358–1363. IEEE. DOI: https://doi.org/10.1109/ICARCV.2012.6485343.
Otsu, N. (1979). A threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics, 9(1):62–66. DOI: https://doi.org/10.1109/TSMC.1979.4310076.
Rother, C., Kolmogorov, V., and Blake, A. (2004). Grabcut: interactive foreground extraction using iterated graph cuts. ACM transactions on graphics (TOG), 23(3):309–314. DOI: https://doi.org/10.1145/1015706.1015720.
Tao, X., Cao, J., Hong, Y., and Niu, L. (2024). Shadow generation with decomposed mask prediction and attentive shadow filling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 5198–5206. DOI: https://doi.org/10.1609/aaai.v38i6.28326.
Wang, Y. and Samaras, D. (2003). Estimation of multiple directional light sources for synthesis of augmented reality images. Graphical Models, 65(4):185–205. DOI: https://doi.org/10.1016/S1524-0703(03)00043-2.
Yu, Y. and Smith, W. A. (2019). Inverserendernet: Learning single image inverse rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3155–3164. DOI: https://doi.org/10.48550/arXiv.1811.12328.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 André Luís Braga Dutra, Rodrigo Luis de Souza da Silva
This work is licensed under a Creative Commons Attribution 4.0 International License.
JIS is free of charge for authors and readers, and all papers published by JIS follow the Creative Commons Attribution 4.0 International (CC BY 4.0) license.
