Abstract

Background: People suffering from hearing and speaking disabilities have a few ways of communicating with other people. One of these is to communicate through the use of sign language.

Objective: Developing a system for sign language recognition becomes essential for the deaf as well as a mute person. The recognition system acts as a translator between a disabled and an able person. This eliminates the hindrances in the exchange of ideas. Most of the existing systems are very poorly designed with limited support for the needs of their day to day facilities.

Methods: The proposed system embedded with gesture recognition capability has been introduced here, which extracts signs from a video sequence and displays them on screen. On the other hand, a speech to text, as well as text to speech system, is also introduced to further facilitate the grieved people. To get the best out of the human computer relationship, the proposed solution consists of various cutting-edge technologies and Machine Learning based sign recognition models, which have been trained by using Tensor Flow and Keras library.

Results: The proposed architecture works better than several gesture recognition techniques like background elimination and conversion to HSV because of the sharply defined image provided to the model for classification. The results of testing indicate reliable recognition systems with high accuracy that includes most of the essential and necessary features for any deaf and dumb person in his/her day to day tasks.

Conclusion: It is the need of current technological advances to develop reliable solutions which can be deployed to assist deaf and dumb people to adjust to normal life. Instead of focusing on a standalone technology, a plethora of them have been introduced in this proposed work. The proposed Sign Recognition System is based on feature extraction and classification. The trained model helps in the identification of different gestures.

Keywords: Tactile sign language, gesture recognition system, machine learning, speech to text, text to speech, technological aid.

Graphical Abstract