Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model
Every one of us has a unique manner of communicating to explore the world, and such communication helps to interpret life. Sign language is the popular language of communication for hearing and speech-disabled people. When a sign language user interacts with a non-sign language user, it becomes difficult for a signer to express themselves to another person. A sign language recognition system can help a signer to interpret the sign of a non-sign language user. This study presents a sign language recognition system that is capable of recognizing Arabic Sign Language from recorded RGB videos. To achieve this, two datasets were considered, such as (1) the raw dataset and (2) the face-hand region-based segmented dataset produced from the raw dataset. Moreover, operational layer-based multi-layer perceptron "SelfMLP" is proposed in this study to build CNN-LSTM-SelfMLP models for Arabic Sign Language recognition. MobileNetV2 and ResNet18-based CNN backbones and three SelfMLPs were used to construct six different models of CNN-LSTM-SelfMLP architecture for performance comparison of Arabic Sign Language recognition. This study examined the signer-independent mode to deal with real-time application circumstances. As a result, MobileNetV2-LSTM-SelfMLP on the segmented dataset achieved the best accuracy of 87.69% with 88.57% precision, 87.69% recall, 87.72% F1 score, and 99.75% specificity. Overall, face-hand region-based segmentation and SelfMLP-infused MobileNetV2-LSTM-SelfMLP surpassed the previous findings on Arabic Sign Language recognition by 10.970% accuracy.
| Medienart: |
E-Artikel |
|---|
| Erscheinungsjahr: |
2023 |
|---|---|
| Erschienen: |
2023 |
| Enthalten in: |
Zur Gesamtaufnahme - volume:23 |
|---|---|
| Enthalten in: |
Sensors (Basel, Switzerland) - 23(2023), 16 vom: 14. Aug. |
| Sprache: |
Englisch |
|---|
| Beteiligte Personen: |
Podder, Kanchon Kanti [VerfasserIn] |
|---|
| Links: |
|---|
| Themen: |
Arabic Sign Language |
|---|
| Anmerkungen: |
Date Completed 28.08.2023 Date Revised 29.08.2023 published: Electronic Citation Status MEDLINE |
|---|
| doi: |
10.3390/s23167156 |
|---|
| funding: |
|
|---|---|
| Förderinstitution / Projekttitel: |
|
| PPN (Katalog-ID): |
NLM361298749 |
|---|
| LEADER | 01000naa a22002652 4500 | ||
|---|---|---|---|
| 001 | NLM361298749 | ||
| 003 | DE-627 | ||
| 005 | 20231226084839.0 | ||
| 007 | cr uuu---uuuuu | ||
| 008 | 231226s2023 xx |||||o 00| ||eng c | ||
| 024 | 7 | |a 10.3390/s23167156 |2 doi | |
| 028 | 5 | 2 | |a pubmed24n1204.xml |
| 035 | |a (DE-627)NLM361298749 | ||
| 035 | |a (NLM)37631693 | ||
| 035 | |a (PII)7156 | ||
| 040 | |a DE-627 |b ger |c DE-627 |e rakwb | ||
| 041 | |a eng | ||
| 100 | 1 | |a Podder, Kanchon Kanti |e verfasserin |4 aut | |
| 245 | 1 | 0 | |a Signer-Independent Arabic Sign Language Recognition System Using Deep Learning Model |
| 264 | 1 | |c 2023 | |
| 336 | |a Text |b txt |2 rdacontent | ||
| 337 | |a ƒaComputermedien |b c |2 rdamedia | ||
| 338 | |a ƒa Online-Ressource |b cr |2 rdacarrier | ||
| 500 | |a Date Completed 28.08.2023 | ||
| 500 | |a Date Revised 29.08.2023 | ||
| 500 | |a published: Electronic | ||
| 500 | |a Citation Status MEDLINE | ||
| 520 | |a Every one of us has a unique manner of communicating to explore the world, and such communication helps to interpret life. Sign language is the popular language of communication for hearing and speech-disabled people. When a sign language user interacts with a non-sign language user, it becomes difficult for a signer to express themselves to another person. A sign language recognition system can help a signer to interpret the sign of a non-sign language user. This study presents a sign language recognition system that is capable of recognizing Arabic Sign Language from recorded RGB videos. To achieve this, two datasets were considered, such as (1) the raw dataset and (2) the face-hand region-based segmented dataset produced from the raw dataset. Moreover, operational layer-based multi-layer perceptron "SelfMLP" is proposed in this study to build CNN-LSTM-SelfMLP models for Arabic Sign Language recognition. MobileNetV2 and ResNet18-based CNN backbones and three SelfMLPs were used to construct six different models of CNN-LSTM-SelfMLP architecture for performance comparison of Arabic Sign Language recognition. This study examined the signer-independent mode to deal with real-time application circumstances. As a result, MobileNetV2-LSTM-SelfMLP on the segmented dataset achieved the best accuracy of 87.69% with 88.57% precision, 87.69% recall, 87.72% F1 score, and 99.75% specificity. Overall, face-hand region-based segmentation and SelfMLP-infused MobileNetV2-LSTM-SelfMLP surpassed the previous findings on Arabic Sign Language recognition by 10.970% accuracy | ||
| 650 | 4 | |a Journal Article | |
| 650 | 4 | |a Arabic Sign Language | |
| 650 | 4 | |a MediaPipe | |
| 650 | 4 | |a deep learning | |
| 650 | 4 | |a dynamic sign language | |
| 650 | 4 | |a segmentation | |
| 700 | 1 | |a Ezeddin, Maymouna |e verfasserin |4 aut | |
| 700 | 1 | |a Chowdhury, Muhammad E H |e verfasserin |4 aut | |
| 700 | 1 | |a Sumon, Md Shaheenur Islam |e verfasserin |4 aut | |
| 700 | 1 | |a Tahir, Anas M |e verfasserin |4 aut | |
| 700 | 1 | |a Ayari, Mohamed Arselene |e verfasserin |4 aut | |
| 700 | 1 | |a Dutta, Proma |e verfasserin |4 aut | |
| 700 | 1 | |a Khandakar, Amith |e verfasserin |4 aut | |
| 700 | 1 | |a Mahbub, Zaid Bin |e verfasserin |4 aut | |
| 700 | 1 | |a Kadir, Muhammad Abdul |e verfasserin |4 aut | |
| 773 | 0 | 8 | |i Enthalten in |t Sensors (Basel, Switzerland) |d 2007 |g 23(2023), 16 vom: 14. Aug. |w (DE-627)NLM187985170 |x 1424-8220 |7 nnns |
| 773 | 1 | 8 | |g volume:23 |g year:2023 |g number:16 |g day:14 |g month:08 |
| 856 | 4 | 0 | |u http://dx.doi.org/10.3390/s23167156 |3 Volltext |
| 912 | |a GBV_USEFLAG_A | ||
| 912 | |a GBV_NLM | ||
| 951 | |a AR | ||
| 952 | |d 23 |j 2023 |e 16 |b 14 |c 08 | ||