ENHANCING AI-TOUCH FREE HAND GESTURE BASED HUMAN COMPUTER INTERACTION
DOI:
https://doi.org/10.63458/ijerst.v2i3.89Keywords:
Touch-Free Human-Computer Interaction (HCI),Artificial Intelligence(AI), Hand Gestures, Webcam Vision Control System, OpenCV, TensorFlow, Machine Learning Model, System Control, Interactive Computing, Gesture Recognition, Computer Vision, Accessibility, PythonAbstract
In the era of touch-based Human-Computer Interaction (HCI) systems, there is a growing need to develop touch-free
methodologies that improve user interaction. This research leverages rapid advancements in Artificial Intelligence (AI)
technology, with the primary goal of enhancing touch-free HCI through the integration of AI-based hand gestures. The system
aims to facilitate control without the need for traditional peripherals, such as a keyboard or mouse. Our system marks a significant
advancement in reducing reliance on conventional input devices, utilizing a webcam as the primary input device. The approach
combines several Python packages, including OpenCV, MediaPy, AutoGUI, PyTorch, and TensorFlow, to track hand movements
and execute system actions. This paper introduces a camera-based vision control system that uses a sophisticated hand gesture
algorithm, powered by machine learning models, to interpret finger gestures for managing various system functions. The system
enables users to control the system cursor using natural hand and finger gestures, allowing precise control over cursor movements,
clicks, scrolling, application launches, and the execution of keyboard shortcuts. This technological advancement significantly
enhances human-computer interaction, providing a more interactive and user-friendly computing experience. By seamlessly
integrating OpenCV, MediaPy, AutoGUI, PyTorch, TensorFlow, and a robust machine learning model, our approach
demonstrates innovation in touch-free HCI. It also establishes a comprehensive framework for the development of future
interactive systems. The utilization of these Python tools enhances the efficiency and scalability of the proposed system, making
it a promising step toward the evolution of intuitive and accessible computing interfaces
References
Mhetar,A.,Sriroop,B.K., Kavya,A.G.S., Nayak,R., Javali, R., Suma, K. V. Virtual mouse. In International Conference on Circuits, Communication, Control and Computing, pp. 69-72. IEEE. 2014.
Tsai,Tsung-Han, Chih-ChiHuang, Kung-Long Zhang. ‘Embedded virtual mouse system by using hand gesture recognition.’ 2015 IEEE International Conference on Consumer Electronics-Taiwan. IEEE, 2015.
Varun, K.S, Puneeth, Virtual mouse implementation using open CV. In 2019 3rd International Conference on Trends in Electronics and Informatics (ICOEI) (pp. 435-438). IEEE. 2019.
Farooq, Javeria, and Muhaddisa Barat Ali. ‘Real time hand gesture recognition for computer interaction.’ 2014 International Conference on Robotics and Emerging Allied Technologies in Engineering (iCREATE). IEEE, 2014.
Shibly, Kabid Hassan, Samrat Kumar Dey, Md Aminul Islam, and Shahriar Iftekhar Showrav. ‘Design and development of hand gesture based virtual mouse.’In 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology(ICASERT),pp.1-5.IEEE, 2019.
Joshi, Hritik, Ratnesh Litoriya, and Dharmendra Mangal.’Design of a Virtual Mouse Using Gesture Recognition and Machine Learning.’2022.
Haria, A., Subramanian, A., Asokkumar, N., Poddar, S., Nayak, J. S. ‘Hand gesture recognition for human computer interaction’. Procedia computer science, No.115, pp.367-374.
Sharma, Ram Pratap, Gyanendra K.Verma. ‘Human computer interaction using hand gesture.’Procedia Computer Science , No. 54 pp.721-727.2015.
Robertson, Paul, Robert Laddaga, MaxVan Kleek. "Virtual mouse vision-based interface.’ Proceedings of the 9th international conference on Intelligent user interfaces’. 2004.
Tran, Dinh-Son, Ngoc-Huynh Ho, Hyung-Jeong Yang, Eu-Tteum Baek, Soo-Hyung Kim, and Gueesang Lee."Real-time hand gestures potting and recognition using RGB-D camera and 3D convolutional neural network."AppliedSciences 10, no. 2 , pp.722. 2020.
Mhetar, Ashish, et al. "Virtual mouse." International Conference on Circuits, Communication, Control and Computing. IEEE, 2014.
Matlani, Roshnee, Roshan Dadlani, Sharv Dumbre, Shruti Mishra ,AbhaTewari."Virtual Mouse using Hand Gestures." In 2021 International Conference on Technological Advancements and Innovations(ICTAI),pp. 340-345.IEEE, 2021.
