Integrating AI and IoT in STEM Education Through a Gesture-Control Project

Abstract

This paper presents the development and deployment of a hand gesture-controlled lighting system specifically designed for STEM education at the undergraduate level. The primary objective of this study is to demonstrate a practical framework for integrating complex AI and IoT concepts through a hands-on, constructionist learning approach. The project combines affordable microcontroller hardware, specifically the ESP32-CAM and ESP32 Dev Board, with the MediaPipe framework and Internet of Things (IoT) protocols to transform human hand movements into interactive visual effects. By utilizing MediaPipe for real-time hand landmark detection and implementing explicit geometric rules for gesture classification, the project provides students with direct experience in hardware-software integration and distributed system control. The study details the technical architecture, including robust debouncing mechanisms to ensure operational stability. Furthermore, the system supports versatile deployment options, such as standalone executable files for Windows, enhancing classroom accessibility. This "white-box" design principle facilitates a deeper understanding of embedded programming and practical AI applications. Ultimately, the project serves as a comprehensive educational tool that successfully bridges theoretical knowledge and applied STEM skills in modern engineering curricula.