INTUITIVE GESTURE-DRIVEN AUGMENTED REALITY FOR USER INTERACTION IN LOW-COST AR SYSTEMS

Authors

Keywords:

Augmented Reality, Gesture Recognition, Human-computer Interaction, Low-Cost AR, MediaPipe, OpenCV

Abstract

Augmented Reality (AR) offers promising applications in education, healthcare, and industry, but its adoption in low-resource settings is hindered by expensive hardware and complex interaction methods. This study presents a lightweight, gesture-driven AR interface designed to enable intuitive user interaction on low-cost devices. Using standard webcams and open-source tools, MediaPipe for hand landmark detection and OpenCV for visual overlays, a rule-based system was developed to recognize three core gestures (open palm, fist, point) and trigger real-time AR overlays. The system was optimized for latency and frame rate, achieving 81.8% accuracy in gesture recognition during user testing with 20 participants. Real-time performance averaged 44.8 ms latency and 24.6 FPS, demonstrating responsive feedback on entry-level hardware. Despite limitations such as variable lighting and lack of depth sensing, the system proved intuitive and effective for immersive interaction. This work contributes a scalable AR interaction model that enhances accessibility in educational and low-cost environments.

Dimensions

Azuma, R. T. (1997). A survey of augmented reality. Presence: Teleoperators and Virtual Environments, 6(4), 355–385. https://doi.org/10.1162/pres.1997.6.4.355

Billinghurst, M., Clark, A., and Lee, G. (2015). A survey of augmented reality. Foundations and Trends® in Human–Computer Interaction, 8(2–3), 73–272. https://doi.org/10.1561/1100000049

Cao, J., Lam, K.-Y., Lee, L.-H., Liu, X., Hui, P., and Su, X. (2022). Mobile augmented reality: User interfaces, frameworks, and intelligence. ACM Computing Surveys, 55(9). https://doi.org/10.1145/3557999

Carmigniani, J., Furht, B., Anisetti, M., Ceravolo, P., Damiani, E., and Ivkovic, M. (2011). Augmented reality technologies, systems and applications. Multimedia Tools and Applications, 51(1), 341–377. https://doi.org/10.1007/s11042-010-0660-6

Ghazwani, Y., and Smith, S. (2020). Interaction in augmented reality. In Proceedings of the 2020 4th International Conference on Virtual and Augmented Reality Simulations*. https://doi.org/10.1145/3385378.3385384

Guna, J., Jakus, G., Pogačnik, M., Tomažič, S., and Sodnik, J. (2014). An analysis of the precision and reliability of the Leap Motion sensor and its suitability for static and dynamic tracking. Sensors, 14(2), 3702–3720. https://doi.org/10.3390/s140203702

Lee, K., Lee, J., and Lee, J. Y. (2018). Comparative study on augmented reality approaches for interactive learning. Interactive Learning Environments, 26(4), 516–529. https://doi.org/10.1080/10494820.2017.1337038

Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., ... and Grundmann, M. (2019). MediaPipe: A framework for building perception pipelines. arXiv Preprint arXiv:1906.08172. https://arxiv.org/abs/1906.08172

Majdoub Bhiri, N., Ameur, S., Alouani, I., Mahjoub, M. A., and Ben Khalifa, A. (2023). Hand gesture recognition with focus on leap motion: An overview, real-world challenges, and future directions. Expert Systems with Applications, 226, 120125. https://doi.org/10.1016/j.eswa.2023.120125

Pavlovic, V. I., Sharma, R., and Huang, T. S. (1997). Visual interpretation of hand gestures for human-computer interaction: A review. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(7), 677–695. https://doi.org/10.1109/34.598226

Rautaray, S. S., and Agrawal, A. (2015). Vision based hand gesture recognition for human computer interaction: A survey. Artificial Intelligence Review, 43(1), 1–54. https://doi.org/10.1007/s10462-012-9356-9

Sanna, A., Lamberti, F., Paravati, G., and Manuri, F. (2013). A Kinect-based natural interface for quadrotor control. Entertainment Computing, 4(3), 179–186. https://doi.org/10.1016/j.entcom.2013.01.001

Shen, J., Dudley, J., Mo, G., and Kristensson, P. O. (2022). Gesture spotter: A rapid prototyping tool for key gesture spotting in virtual and augmented reality applications. IEEE Transactions on Visualization and Computer Graphics, 28(11), 3618–3628. https://doi.org/10.1109/tvcg.2022.3203004

Syed, T. A., and et al. (2023). In-depth review of augmented reality: Tracking technologies, development tools, AR displays, collaborative AR, and security concerns. Sensors, 23(1), 146. https://www.mdpi.com/1424-8220/23/1/146

Tati, S., Sahu, N. K., and Rath, S. K. (2019). A robust hand gesture recognition system using MediaPipe framework. International Journal of Interactive Multimedia and Artificial Intelligence, 5(7), 52–59. https://doi.org/10.9781/ijimai.2019.07.003

Torres, W., and et al. (2024). A framework for real-time gestural recognition and augmented reality for industrial applications. Sensors, 24(8), 2407. https://doi.org/10.3390/s24082407

Vertucci, R., D’Onofrio, S., Ricciardi, S., and De Nino, M. (2023). History of augmented reality. In Springer Handbooks (pp. 35–50). https://doi.org/10.1007/978-3-030-67822-7_2

Wang, X., Kim, M. J., Love, P. E., and Kang, S. C. (2013). Augmented reality in built environment: Classification and implications for future research. Automation in Construction, 32, 1–13. https://doi.org/10.1016/j.autcon.2012.11.021

Zhang, X., and Zhu, S. (2019). Real-time hand gesture recognition based on deep features and support vector machines. IEEE Access, 7, 103138–103145. https://doi.org/10.1109/ACCESS.2019.2931064

Zhou, Z., and Deng, Z. (2017). Rule-based hand gesture recognition for human–computer interaction. Multimedia Tools and Applications, 76(19), 20067–20083. https://doi.org/10.1007/s11042-016-4028-5

Published

05-10-2025

How to Cite

Muhammad-Bello, B. L., Ndubuisi, A. V., Suleiman, M. A., & Salihu, I. A. (2025). INTUITIVE GESTURE-DRIVEN AUGMENTED REALITY FOR USER INTERACTION IN LOW-COST AR SYSTEMS. FUDMA JOURNAL OF SCIENCES, 9(10), 174-180. https://doi.org/10.33003/fjs-2025-0910-3908

Issue

Vol. 9 No. 10 (2025): FUDMA Journal of Sciences - Vol. 9 No. 10

Section

Research Articles
Copyright & Licensing

Copyright (c) 2025 Bilkisu Larai Muhammad-Bello, Awoke Victor Ndubuisi, Muhammad Aliyu Suleiman, Ibrahim Anka Salihu

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

How to Cite

Muhammad-Bello, B. L., Ndubuisi, A. V., Suleiman, M. A., & Salihu, I. A. (2025). INTUITIVE GESTURE-DRIVEN AUGMENTED REALITY FOR USER INTERACTION IN LOW-COST AR SYSTEMS. FUDMA JOURNAL OF SCIENCES, 9(10), 174-180. https://doi.org/10.33003/fjs-2025-0910-3908

Most read articles by the same author(s)