Analisis Temporal Gerakan Kata BISINDO Menggunakan Landmark Tangan dan LSTM dengan Keluaran Suara Berbasis ESP32 Secara Real-time

I Gusti Agung Made Yoga Mahaputra ; I Gusti Agung Made Yoga Mahaputra ; Putri Alit Widyastuti Santiary ; I Ketut Swardika

Jurnal Elektronika dan Komputer 📅 2025 Vol 18, No 2 🔗 DOI: 10.51903/elkom.v18i2.3258

Abstract

Indonesian Sign Language (BISINDO) serves as a primary communication medium for the deaf community; however, limited public understanding often creates barriers during daily interactions. This study aims to develop a real-time BISINDO word-level translation system using hand landmark extraction and temporal modeling with Long Short-Term Memory (LSTM). The system employs MediaPipe Hands to detect 21 hand landmarks per frame, which are then processed as sequential motion patterns to classify five BISINDO words: saya, terima kasih, maaf, nama, and kamu. A total of 250 gesture samples were recorded under controlled lighting conditions as the primary dataset. The processed sequences were used to train the LSTM model, which was subsequently integrated with an ESP32 microcontroller and a DFPlayer Mini module to produce direct audio output. Experimental results show that the model achieved an average accuracy of 86%, with precision and recall values ranging from 0.81 to 0.94. The confusion matrix analysis indicates that most gestures were correctly classified, although some errors occurred in gestures with similar initial motion trajectories. Integration testing demonstrated an average system latency of 3.8 seconds and an audio output success rate of 85%. These findings indicate that the proposed system is capable of translating BISINDO word-level gestures accurately, responsively, and consistently in real-time conditions. This study provides a strong foundation for the broader development of sign language translation systems, with potential enhancements in vocabulary expansion, multi-user datasets, and hardware optimization for deployment in real-world environments.

Keywords

BISINDO; Hand Landmark; LSTM

How to Cite

I Gusti Agung Made Yoga Mahaputra, et al. (2025). Analisis Temporal Gerakan Kata BISINDO Menggunakan Landmark Tangan dan LSTM dengan Keluaran Suara Berbasis ESP32 Secara Real-time. Jurnal Elektronika dan Komputer, 18(2). https://doi.org/10.51903/elkom.v18i2.3258

I Gusti Agung Made Yoga Mahaputra; I Gusti Agung Made Yoga Mahaputra; Putri Alit Widyastuti Santiary; I Ketut Swardika, "Analisis Temporal Gerakan Kata BISINDO Menggunakan Landmark Tangan dan LSTM dengan Keluaran Suara Berbasis ESP32 Secara Real-time," Jurnal Elektronika dan Komputer, vol. 18, no. 2, 2025.

I Gusti Agung Made Yoga Mahaputra; I Gusti Agung Made Yoga Mahaputra; Putri Alit Widyastuti Santiary; I Ketut Swardika. "Analisis Temporal Gerakan Kata BISINDO Menggunakan Landmark Tangan dan LSTM dengan Keluaran Suara Berbasis ESP32 Secara Real-time." Jurnal Elektronika dan Komputer, vol. 18, no. 2, 2025.

I Gusti Agung Made Yoga Mahaputra, et al. (2025) 'Analisis Temporal Gerakan Kata BISINDO Menggunakan Landmark Tangan dan LSTM dengan Keluaran Suara Berbasis ESP32 Secara Real-time', Jurnal Elektronika dan Komputer, 18(2). doi: 10.51903/elkom.v18i2.3258.

I Gusti Agung Made Yoga Mahaputra; I Gusti Agung Made Yoga Mahaputra; Putri Alit Widyastuti Santiary; I Ketut Swardika. Analisis Temporal Gerakan Kata BISINDO Menggunakan Landmark Tangan dan LSTM dengan Keluaran Suara Berbasis ESP32 Secara Real-time. Jurnal Elektronika dan Komputer. 2025;18(2).

Analisis Kinerja Metode Long Short-Term Memory (LSTM) dalam Klasifikasi Sentimen Ulasan Pengguna Shopee

Muhimatul Ifadah; Muhimatul Ifadah; Bambang Irawan

Klasifikasi Jenis Bunga Menggunakan Algoritma Convolutional Neural Network (CNN)

Ade Irgi Firdaus; Ade Irgi Firdaus; Dwi Okta Djoas; Riefaldi Diofano Saputra; Indry Anggraeny; Hilda Apriliya Ningsih

Prediksi Harga Saham Menggunakan ARIMA Outlier sebagai Pendekatan Awal Menuju Analisis AI Keuangan

Adam, Cindi; Adam, Cindi; Idhom, Mohammad; Trimono, Trimono

Analisis Performansi Pendekatan Machine Learning Pada Deteksi Penyakit Daun Tanaman Kopi

Purnomo, Rosyana Fitria; Purnomo, Rosyana Fitria; Yodhi Yuniarthe; Hilda Dwi Yunita; Fatimah Fahurian; Ahmad Ikhwan

Hybrid Apriori–MST Framework for Revealing the Structural Backbone of Consumer Purchase Patterns

Marta Dinata, Riadi; Kurniawan Atmadja; Marhaeni Mahaeni; Lely Mustika

Rancang Bangun Antena Mikrostrip Patch Triangular Pada Frekuensi 1800 Mhz Sebagai Penguat Jaringan Internet Di Kebun Raya Balikpapan

Nurmeiliana Putri; Maria Ulfah; Fathur Zaini Rachman; Maria Ulfah

Tren Sitasi Jurnal