Publication Search

Gesture  recognition  technology  is  used  to  detect  movements  through  image processing,   enabling  computers  or digital devices to understand and interpret human  body  movements  as  input  or  commands.   This  technology  has  great potential  to bridge communication between the deaf community and individuals without   hearing   impairments,    enhancing  interaction  and  enriching  mutual understanding between the two.  However,  the accuracy ofgesture recognition is often  affected  by variations in the distance between hand landmarks.  Based on this problem,  this research proposes a methodfor stabilizing the measurement of distances between landmark points  in gesture recognition through a polynomial regression  approach.   Specifically,   the  distance  between  hand  landmarks  is calculated and stabilized using polynomial  regression to improve the accuracy of gesture recognition.  This method is implemented using the MediaPipeframework to detect and track hands in real-time,  and the OpenCV library to manage video. The  research  results  show  that  this  approach  can  significantly  improve  the stability  and accuracy  of gesture detection.   The developed system successfully detects gestures for  letters A  through F with a high accuracy  rate,  averaging above 98,3%.  The use ofpolynomial regression helps enhance detection accuracy by reducing noise in the landmark data.

Communication is an essential human right, yet a significant communication gap persists between individuals with sensory disabilities, specifically the deaf and speech-impaired, and the general public. While many technological solutions have been proposed to translate sign language, existing models primarily rely on heavy deep learning architectures such as Convolutional Neural Networks (CNN) or Recurrent Neural Networks (RNN/LSTM). These models often demand high computational power, leading to latency and limiting real-time application on standard devices. This study proposes a lightweight, fast, and highly responsive sign language translation system specifically designed to recognize static alphabets (A-Z) and single-character air writing. The system utilizes MediaPipe for hand tracking, where feature extraction is intelligently processed by calculating the relative spatial coordinates of fingertips to the wrist, reducing dependency on raw camera coordinates. Classification is performed using a Support Vector Machine (SVM) with a Radial Basis Function (RBF) kernel, prioritizing computational efficiency without sacrificing accuracy. To enhance user experience, the system introduces three key novelties: smart relative feature extraction, an anti-duplication hold system with a 1-second timer to prevent input spamming, and a non-blocking multithreaded audio execution (Daemon Thread) utilizing Google Text-to-Speech (gTTS), ensuring the webcam feed remains fluid during audio playback. Additionally, an alternative air-writing mode is integrated, utilizing geometric heuristics and PyTesseract OCR to read single drawn letters in the air. The results indicate that the proposed system operates swiftly and efficiently, bridging the communication barrier with a hardware-friendly approach.

Indonesian Sign Language (BISINDO) serves as a primary communication medium for the deaf community; however, limited public understanding often creates barriers during daily interactions. This study aims to develop a real-time BISINDO word-level translation system using hand landmark extraction and temporal modeling with Long Short-Term Memory (LSTM). The system employs MediaPipe Hands to detect 21 hand landmarks per frame, which are then processed as sequential motion patterns to classify five BISINDO words: saya, terima kasih, maaf, nama, and kamu. A total of 250 gesture samples were recorded under controlled lighting conditions as the primary dataset. The processed sequences were used to train the LSTM model, which was subsequently integrated with an ESP32 microcontroller and a DFPlayer Mini module to produce direct audio output. Experimental results show that the model achieved an average accuracy of 86%, with precision and recall values ranging from 0.81 to 0.94. The confusion matrix analysis indicates that most gestures were correctly classified, although some errors occurred in gestures with similar initial motion trajectories. Integration testing demonstrated an average system latency of 3.8 seconds and an audio output success rate of 85%. These findings indicate that the proposed system is capable of translating BISINDO word-level gestures accurately, responsively, and consistently in real-time conditions. This study provides a strong foundation for the broader development of sign language translation systems, with potential enhancements in vocabulary expansion, multi-user datasets, and hardware optimization for deployment in real-world environments.

Seiring berkembangnya teknologi interaksi manusia dan komputer, manusia bisa berinteraksi dengan komputer tanpa harus menggunakan alat fisik seperti mouse. Salah satu caranya adalah dengan menggerakkan jari tangan di depan kamera (webcam), dan gerakan itu bisa dikenali untuk mengontrol komputer. Penelitian ini membuat sistem mouse virtual yang bisa mendeteksi gerakan jari secara waktu nyata (real-time) dan mengubahnya menjadi perintah untuk menggerakkan kursor, gerakan klik kanan, klik kiri, menggulir halaman (scroll), dan mengatur volume suara. Untuk mendeteksi 21 titik di tangan yang membantu sistem mengenali gerakan tangan, digunakan MediaPipe Hand Tracking. Dan untuk pengolahan gambar guna mengaktifkan fungsi mouse virtual seperti klik dan gerakan kursor, digunakan OpenCV (Open Computer Vision). Pengujian dilakukan pada jarak 30 cm dan 50 cm dari webcam, dengan pencahayaan dari bohlam LED sebesar 5 watt. Hasilnya, sistem bekerja sangat baik pada jarak 30 cm dengan tingkat keberhasilan 100%. Sedangkan pada jarak 50 cm, kinerja sistem sedikit menurun, terutama untuk klik kiri (70%) dan gerak kursor (80%). Secara keseluruhan, sistem ini berhasil menjalankan fungsi dengan baik, dengan tingkat keberhasilan rata-rata 95,8%. Sistem ini bisa dikembangkan lagi agar lebih maksimal untuk berbagai kebutuhan.

This study aims to develop an intuitive and efficient smart home control system by utilizing hand tracking and speech recognition technologies. These technologies employ the OpenCV, Mediapipe, PyAudio, and Speech Recognition libraries to recognize hand gestures and voice commands in real-time. The system is developed using a Raspberry Pi connected to a webcam and microphone as input devices, and a relay to control electronic appliances. The results show a high accuracy rate at optimal light intensity for hand tracking and a specific distance for speech recognition. This system is implemented in an IoT environment to control devices such as lights and door locks. The research is expected to contribute to the development of smarter and more user-friendly smart homes.

Introducing alphabetical sign language is necessary to bridge communication between deaf and hard-of-hearing people and their surrounding environment. This research aims to develop a sign language alphabet letter detection system based on American Sign Language (ASL). The research methods include data collection, feature extraction with OpenCV and Mediapipe, model development with Random Forest algorithm, and real-time system testing. The test results show that the developed system can achieve 97% prediction accuracy in recognizing hand patterns that represent ASL letters. The system uses a webcam as real-time input, providing accurate responses in various environmental conditions. This research contributes significantly to developing communication support technology for the deaf community, with implications for increased inclusivity and social engagement.

This research focuses on the use of hand tracking technology in a drawing program based on the MediaPipe framework. The aim of this study is to develop a digital drawing system that can track hand movements in real-time without additional input devices like a mouse or stylus. This technology utilizes computer vision algorithms to detect and track the user's hand movements, which are then translated into strokes on the screen. The study employs a descriptive-qualitative method with a software experimentation approach. The results show that the system has a high level of accuracy and is responsive to hand movements, providing a more natural and intuitive user experience. The implications of this research are significant in supporting technology-based educational and creative applications.