Empowering Diversity in Education: A Web-Based Tool for Real-Time Sign Language Detection
Keywords:
Diversity in Education, Artificial Intelligence, Machine Learning, Sign Language Detection
Abstract
People worldwide have severe hearing loss and there is little to no technological involvement to empower diversity and inclusion in the education system. This study aims to provide an understanding of the needs of the community of such individuals and a possible solution to the problem using current advances in artificial intelligence technologies. In particular, we will present a solution to the problem through a real-time sign language detection web application. First, the structural and functional aspects of the application are described, followed by the software architecture needs. The technological aspect of the application is entirely defined by expressing all the significant implementations of each programming language contained behind the software diagrams and architecture. This study will provide the latest technologies in the market combined with a well-planned and developed image detection application that is scalable, portable, maintainable, and efficient in empowering diversity in education. This technologically up-to-date application is delivered as a full-stack web application. The application produces results that waver between 85%-95% quality off detection; it consists of a bi-partite backend, one for creating and compiling models, namely Python, TensorFlow, and Jupyter Notebooks, and the other part held by Node.JS to power the front-end built with React and TensorFlow's JS. This study presents a fully developed web-based application that has the potential to impact and improve diversity in education significantly.
Downloads
Download data is not yet available.
Metrics
Metrics Loading ...
PlumX Statistics
References
1. Alex Banks, E. P. (2020, April 22). Learning React: Modern Patterns for Developing React Apps. O'Reilly Media. Retrieved April 22, 2024, from https://react.dev: https://react.dev/
2. Canadian. (2019, September). Canadian Museum for Human Rights. Retrieved 2019, from Language rights are human rights: humanrights.ca/story
3. Casciaro, M., & Mammino, L. (2020). Node.js Design Patterns - Third Edition: Design and Implement Production-Grade Node.js Applications Using Proven Patterns and Techniques. Packt Publishing.
4. Changhwan, N. (2023, October 23). Sign language translator AI. Retrieved August 05, 2024, from Google Play: https://play.google.com/store/apps/details?id=com.siyaroll.sltranslator&hl=en&pli=1
5. Chemistry, R. S. (2016, August 23). 3D graphene adds dimension to deaf-mute communication. Retrieved from https://www.chemistryworld.com/news/3d-graphene-adds-dimension-to-deaf-mute-communication/1017315.article
6. depositphotos.com. (2024, January 21). Set of deaf and dumb language. Retrieved January 21, 2024, from Set of deaf and dumb language: https://depositphotos.com/photo/set-deaf-dumb-language-tattooed-male-hands-latin-alphabet-isolated-231581414.html
7. Eriglen, G., Alda, K., & Bruno, G. (2015). A Real-Time Vision-Based System for Recognition of Static Dactyls of Albanian Alphabet. In F. A. Deaf, Deaf People in Albania.
8. Finnish Association of the Deaf. (2016). Deaf People in Albania in 2015. A survey study. (I. L. Rainò, Ed.) Albania: Gent Grafik.
9. Fitzpatrick, D. M. (2022, October 14). Create GUI Applications with Python & Qt5: The hands-on guide to making apps with Python. Kindle edition. Retrieved from Create GUI Applications with Python & Qt5: The hands-on guide to making apps with Python
10. Gates, B. (2023, December 26). AI is about to supercharge the innovation pipeline. AI is about to supercharge the innovation pipeline. LinkedIn. Retrieved from https://www.linkedin.com/pulse/ai-supercharge-innovation-pipeline-bill-gates-rsmqc/?trackingId=Cm2uoSiCR7SsoMQRVMAtZg%3D%3D
11. GBD 2019 Hearing Loss Collaborators. (2021, March 13). Hearing loss prevalence and years lived with disability, 1990–2019: findings from the Global Burden of Disease Study 2019. 397(10278), pp. 996-1009. Retrieved from www.ncbi.nlm.nih.gov: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7960691/#bib12
12. Géron, A. (2019, April 22). Hands-On-Machine-Learning-with-Scikit-Learn-Keras-and-Tensorflow. O'Reilly. Retrieved 2024, from https://www.tensorflow.org/: https://www.tensorflow.org/resources/learn-ml?gclid=Cj0KCQjwpeaYBhDXARIsAEzItbGEA2Ut3ClPj2xxqmCL9eToD5IIINVXVm4-u5nZ5ntmsUO6NqCI1fYaAp1YEALw_wcB
13. INSTAT. (2019). http://www.instat.gov.al/media/3706. Retrieved April 23, 2024, from http://www.instat.gov.al/: http://www.instat.gov.al/
14. INSTAT. (2019). Profili i Personave me Aftësi të kufizuar në Shqipëri. Instat, https://www.instat.gov.al/media/3706/profili_i__personave_me_aft_si_t__kufizuar_n__shqip_ri.pdf. Retrieved 22 April 2024, from http://www.instat.gov.al/media/3706
15. Kolivand, H., Joudaki, S., Sunar, M., & Tully, D. (2021). A new framework for sign language alphabet hand posture recognition using geometrical features through artificial neural network (part 1). Neural Computing and Applications, 33, pp. 4945-4963.
16. Lee, C., Ng, K., Chen, C.-H., Lau, H., Chung, S., & Tsoi, T. (2021). American sign language recognition and training method with recurrent neural network. Expert Systems with Applications, 167, p. 114403.
17. Lin, T. (2024, February 29). LabelImg: The Ultimate Tool for Efficient Data Annotation. Retrieved August 08, 2024, from https://github.com/HumanSignal/labelImg
18. Mario Casciaro, L. M. (2020, April 22). Node.js Design Patterns - Third edition: Design and implement production-grade Node.js applications using proven patterns and techniques. Packt Publishing. Retrieved April 22, 2024, from https://nodejs.org: https://nodejs.org/en/
19. mozilla.org. (2024). MVC. Retrieved from https://developer.mozilla.org/en-US/docs/Glossary
20. O.Leka, S. H. (2023, July). Oresti9919 - SignLanguageApp. Retrieved from https://github.com/Oresti9919/SignLanguageApp/tree/main
21. Omnisoftex. (2023, January 28). Envatomarket. Retrieved August 05, 2024, from codecanyon: https://codecanyon.net/item/image-pro-text-to-sign-sign-to-text/43087647
22. Osmani, A. (2023, April 22). Learning JavaScript Design Patterns: A JavaScript and React Developer's Guide. O'Reilly Media. Retrieved April 22, 2024, from https://www.javascript.com/: https://www.javascript.com/resources
23. Pablo, M. G. (2022, March 32). Create a simple sign language recognition app using teachable machine, Monaca, Vue.js and Framework7. Retrieved August 5, 2024, from Medium: https://medium.com/the-web-tub/create-a-simple-sign-language-recognition-app-using-teachable-machine-monaca-vue-js-31072c2456a4
24. Rabie, A. R., Ahmed, Y. K., Y, K., Eissa, J. A., Sharif, H., Ahmad, T. A., & HKamberaj, H. (2022). Convolution neural network based automatic localization of landmarks on lateral x-ray images. Multimed Tools Appl, 81, 37403–37415.
25. Ramalho, L. (2022, April 22). Fluent Python. O'Reilly Media. Retrieved April 22, 2024, from https://www.python.org: https://www.python.org/doc/essays/blurb/
26. RedMonk. (2024, March 8). The RedMonk Programming Language Rankings: January 2024. Retrieved from https://redmonk.com/sogrady/2024/03/08/language-rankings-1-24/
27. Sahoo, J., Prakash, A., Pławiak, P., & Samantray, S. (2022). Real-time hand gesture recognition using fine-tuned convolutional neural network. 22(3), p. 706.
28. Trends, G. (2024, January 28). Pytorch-TensorFlow. Retrieved from https://trends.google.com/trends/explore?geo=US&q=pytorch,tensorflow
29. WHO. (2024, February 2). Deafness and hearing loss. Retrieved from www.who.int: https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss
30. WHO. (2024, April 23). https://www.who.int/news-room/facts-in-pictures/detail/deafness. Retrieved April 23, 2024, from https://www.who.int: https://www.who.int/
31. Built With (2024, February). React Usage Statistics. Retrieved from https://trends.builtwith.com/javascript/React
32. Yankiver, D. (2021, May 31). 5 Major Companies That Use Node.js and Why - Nerd For Tech - Medium. Retrieved from https://medium.com/nerd-for-tech/5-major-companies-that-use-node-js-and-why-211fb5cc267d
2. Canadian. (2019, September). Canadian Museum for Human Rights. Retrieved 2019, from Language rights are human rights: humanrights.ca/story
3. Casciaro, M., & Mammino, L. (2020). Node.js Design Patterns - Third Edition: Design and Implement Production-Grade Node.js Applications Using Proven Patterns and Techniques. Packt Publishing.
4. Changhwan, N. (2023, October 23). Sign language translator AI. Retrieved August 05, 2024, from Google Play: https://play.google.com/store/apps/details?id=com.siyaroll.sltranslator&hl=en&pli=1
5. Chemistry, R. S. (2016, August 23). 3D graphene adds dimension to deaf-mute communication. Retrieved from https://www.chemistryworld.com/news/3d-graphene-adds-dimension-to-deaf-mute-communication/1017315.article
6. depositphotos.com. (2024, January 21). Set of deaf and dumb language. Retrieved January 21, 2024, from Set of deaf and dumb language: https://depositphotos.com/photo/set-deaf-dumb-language-tattooed-male-hands-latin-alphabet-isolated-231581414.html
7. Eriglen, G., Alda, K., & Bruno, G. (2015). A Real-Time Vision-Based System for Recognition of Static Dactyls of Albanian Alphabet. In F. A. Deaf, Deaf People in Albania.
8. Finnish Association of the Deaf. (2016). Deaf People in Albania in 2015. A survey study. (I. L. Rainò, Ed.) Albania: Gent Grafik.
9. Fitzpatrick, D. M. (2022, October 14). Create GUI Applications with Python & Qt5: The hands-on guide to making apps with Python. Kindle edition. Retrieved from Create GUI Applications with Python & Qt5: The hands-on guide to making apps with Python
10. Gates, B. (2023, December 26). AI is about to supercharge the innovation pipeline. AI is about to supercharge the innovation pipeline. LinkedIn. Retrieved from https://www.linkedin.com/pulse/ai-supercharge-innovation-pipeline-bill-gates-rsmqc/?trackingId=Cm2uoSiCR7SsoMQRVMAtZg%3D%3D
11. GBD 2019 Hearing Loss Collaborators. (2021, March 13). Hearing loss prevalence and years lived with disability, 1990–2019: findings from the Global Burden of Disease Study 2019. 397(10278), pp. 996-1009. Retrieved from www.ncbi.nlm.nih.gov: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7960691/#bib12
12. Géron, A. (2019, April 22). Hands-On-Machine-Learning-with-Scikit-Learn-Keras-and-Tensorflow. O'Reilly. Retrieved 2024, from https://www.tensorflow.org/: https://www.tensorflow.org/resources/learn-ml?gclid=Cj0KCQjwpeaYBhDXARIsAEzItbGEA2Ut3ClPj2xxqmCL9eToD5IIINVXVm4-u5nZ5ntmsUO6NqCI1fYaAp1YEALw_wcB
13. INSTAT. (2019). http://www.instat.gov.al/media/3706. Retrieved April 23, 2024, from http://www.instat.gov.al/: http://www.instat.gov.al/
14. INSTAT. (2019). Profili i Personave me Aftësi të kufizuar në Shqipëri. Instat, https://www.instat.gov.al/media/3706/profili_i__personave_me_aft_si_t__kufizuar_n__shqip_ri.pdf. Retrieved 22 April 2024, from http://www.instat.gov.al/media/3706
15. Kolivand, H., Joudaki, S., Sunar, M., & Tully, D. (2021). A new framework for sign language alphabet hand posture recognition using geometrical features through artificial neural network (part 1). Neural Computing and Applications, 33, pp. 4945-4963.
16. Lee, C., Ng, K., Chen, C.-H., Lau, H., Chung, S., & Tsoi, T. (2021). American sign language recognition and training method with recurrent neural network. Expert Systems with Applications, 167, p. 114403.
17. Lin, T. (2024, February 29). LabelImg: The Ultimate Tool for Efficient Data Annotation. Retrieved August 08, 2024, from https://github.com/HumanSignal/labelImg
18. Mario Casciaro, L. M. (2020, April 22). Node.js Design Patterns - Third edition: Design and implement production-grade Node.js applications using proven patterns and techniques. Packt Publishing. Retrieved April 22, 2024, from https://nodejs.org: https://nodejs.org/en/
19. mozilla.org. (2024). MVC. Retrieved from https://developer.mozilla.org/en-US/docs/Glossary
20. O.Leka, S. H. (2023, July). Oresti9919 - SignLanguageApp. Retrieved from https://github.com/Oresti9919/SignLanguageApp/tree/main
21. Omnisoftex. (2023, January 28). Envatomarket. Retrieved August 05, 2024, from codecanyon: https://codecanyon.net/item/image-pro-text-to-sign-sign-to-text/43087647
22. Osmani, A. (2023, April 22). Learning JavaScript Design Patterns: A JavaScript and React Developer's Guide. O'Reilly Media. Retrieved April 22, 2024, from https://www.javascript.com/: https://www.javascript.com/resources
23. Pablo, M. G. (2022, March 32). Create a simple sign language recognition app using teachable machine, Monaca, Vue.js and Framework7. Retrieved August 5, 2024, from Medium: https://medium.com/the-web-tub/create-a-simple-sign-language-recognition-app-using-teachable-machine-monaca-vue-js-31072c2456a4
24. Rabie, A. R., Ahmed, Y. K., Y, K., Eissa, J. A., Sharif, H., Ahmad, T. A., & HKamberaj, H. (2022). Convolution neural network based automatic localization of landmarks on lateral x-ray images. Multimed Tools Appl, 81, 37403–37415.
25. Ramalho, L. (2022, April 22). Fluent Python. O'Reilly Media. Retrieved April 22, 2024, from https://www.python.org: https://www.python.org/doc/essays/blurb/
26. RedMonk. (2024, March 8). The RedMonk Programming Language Rankings: January 2024. Retrieved from https://redmonk.com/sogrady/2024/03/08/language-rankings-1-24/
27. Sahoo, J., Prakash, A., Pławiak, P., & Samantray, S. (2022). Real-time hand gesture recognition using fine-tuned convolutional neural network. 22(3), p. 706.
28. Trends, G. (2024, January 28). Pytorch-TensorFlow. Retrieved from https://trends.google.com/trends/explore?geo=US&q=pytorch,tensorflow
29. WHO. (2024, February 2). Deafness and hearing loss. Retrieved from www.who.int: https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss
30. WHO. (2024, April 23). https://www.who.int/news-room/facts-in-pictures/detail/deafness. Retrieved April 23, 2024, from https://www.who.int: https://www.who.int/
31. Built With (2024, February). React Usage Statistics. Retrieved from https://trends.builtwith.com/javascript/React
32. Yankiver, D. (2021, May 31). 5 Major Companies That Use Node.js and Why - Nerd For Tech - Medium. Retrieved from https://medium.com/nerd-for-tech/5-major-companies-that-use-node-js-and-why-211fb5cc267d
Published
2024-08-31
How to Cite
Meda, S., Oresti Leka, O., & Kamberaj, H. (2024). Empowering Diversity in Education: A Web-Based Tool for Real-Time Sign Language Detection. European Scientific Journal, ESJ, 20(24), 29. https://doi.org/10.19044/esj.2024.v20n24p29
Section
ESJ Natural/Life/Medical Sciences
Copyright (c) 2024 Shefqet Meda, Oresti Oresti Leka, Hiqmet Kamberaj
This work is licensed under a Creative Commons Attribution 4.0 International License.
.jpg)
