Start Submission Become a Reviewer

Reading: A Multi-Modular Approach for Sign Language and Speech Recognition for Deaf-Mute People

Download

A- A+
Alt. Display

Section I

A Multi-Modular Approach for Sign Language and Speech Recognition for Deaf-Mute People

Authors:

D. T. D. M. Dahanayaka ,

The Open University of Sri Lanka, Nugegoda, LK
About D. T. D. M.

Department of Mechanical Engineering

 

AMIE (SL), BTec (Eng)(Hons)(OUSL)

X close

B. G. D. A. Madhusanka,

The Open University of Sri Lanka, Nugegoda, LK
About B. G. D. A.

Lecturer, Department of Mechanical Engineering

 

BTech (Eng)(Hons) (OUSL), MSc (Moratuwa)

X close

I. U. Atthanayake

The Open University of Sri Lanka, LK
About I. U.

Head of the Department, Senior Lecturer (GI)

 

B.Sc. Eng. (Hons)(Peradeniya), Mphil (UOM), PhD(Warwick)

X close

Abstract

Deaf and Mute people cannot communicate efficiently to express their feelings to ordinary people. The common method these people use for communication is the sign language. But these sign languages are not very familiar to ordinary people. Therefore, effective communication between deaf and mute people and ordinary people is seriously affected. This paper presents the development of an Android mobile application to translate sign language into speech-language for ordinary people, and speech into text for deaf and mute people using Convolution Neural Network (CNN). The study focuses on vision-based Sign Language Recognition (SLR) and Automatic Speech Recognition (ASR) mobile application. The main challenging tasks were audio classification and image classification. Therefore, CNN was used to train audio clips and images. Mel-frequency Cepstral Coefficient (MFCC) approach was used for ASR. The mobile application was developed by Python programming and Android Studio. After developing the application, testing was done for letters A and C, and these letters were identified with 95% accuracy.
How to Cite: Dahanayaka, D.T.D.M., Madhusanka, B.G.D.A. and Atthanayake, I.U., 2021. A Multi-Modular Approach for Sign Language and Speech Recognition for Deaf-Mute People. Engineer: Journal of the Institution of Engineers, Sri Lanka, 54(4), pp.97–105. DOI: http://doi.org/10.4038/engineer.v54i4.7474
Published on 30 Dec 2021.
Peer Reviewed

Downloads

  • PDF (EN)

    comments powered by Disqus