<i>Learn2Write</i>: Augmented Reality and Machine Learning-Based Mobile App to Learn Writing
Augmented reality (AR) has been widely used in education, particularly for child education. This paper presents the design and implementation of a novel mobile app, <i>Learn2Write</i>, using machine learning techniques and augmented reality to teach alphabet writing. The app has two main...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-12-01
|
| Series: | Computers |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2073-431X/11/1/4 |
| _version_ | 1797494950395379712 |
|---|---|
| author | Md. Nahidul Islam Opu Md. Rakibul Islam Muhammad Ashad Kabir Md. Sabir Hossain Mohammad Mainul Islam |
| author_facet | Md. Nahidul Islam Opu Md. Rakibul Islam Muhammad Ashad Kabir Md. Sabir Hossain Mohammad Mainul Islam |
| author_sort | Md. Nahidul Islam Opu |
| collection | DOAJ |
| description | Augmented reality (AR) has been widely used in education, particularly for child education. This paper presents the design and implementation of a novel mobile app, <i>Learn2Write</i>, using machine learning techniques and augmented reality to teach alphabet writing. The app has two main features: (i) guided learning to teach users how to write the alphabet and (ii) on-screen and AR-based handwriting testing using machine learning. A learner needs to write on the mobile screen in on-screen testing, whereas AR-based testing allows one to evaluate writing on paper or a board in a real world environment. We implement a novel approach to use machine learning for AR-based testing to detect an alphabet written on a board or paper. It detects the handwritten alphabet using our developed machine learning model. After that, a 3D model of that alphabet appears on the screen with its pronunciation/sound. The key benefit of our approach is that it allows the learner to use a handwritten alphabet. As we have used marker-less augmented reality, it does not require a static image as a marker. The app was built with ARCore SDK for Unity. We further evaluated and quantified the performance of our app on multiple devices. |
| first_indexed | 2024-03-10T01:41:36Z |
| format | Article |
| id | doaj.art-a36795a021d14722b4969db8c0688174 |
| institution | Directory Open Access Journal |
| issn | 2073-431X |
| language | English |
| last_indexed | 2024-03-10T01:41:36Z |
| publishDate | 2021-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Computers |
| spelling | doaj.art-a36795a021d14722b4969db8c06881742023-11-23T13:23:25ZengMDPI AGComputers2073-431X2021-12-01111410.3390/computers11010004<i>Learn2Write</i>: Augmented Reality and Machine Learning-Based Mobile App to Learn WritingMd. Nahidul Islam Opu0Md. Rakibul Islam1Muhammad Ashad Kabir2Md. Sabir Hossain3Mohammad Mainul Islam4Department of Computer Science and Engineering, Chittagong University of Engineering and Technology, Chottogram 4349, BangladeshDepartment of Computer Science and Engineering, Chittagong University of Engineering and Technology, Chottogram 4349, BangladeshData Science Research Unit, School of Computing, Mathematics and Engineering, Charles Sturt University, Bathurst, NSW 2795, AustraliaDepartment of Computer Science and Engineering, Chittagong University of Engineering and Technology, Chottogram 4349, BangladeshVerizon Media Australia, Sydney, NSW 2015, AustraliaAugmented reality (AR) has been widely used in education, particularly for child education. This paper presents the design and implementation of a novel mobile app, <i>Learn2Write</i>, using machine learning techniques and augmented reality to teach alphabet writing. The app has two main features: (i) guided learning to teach users how to write the alphabet and (ii) on-screen and AR-based handwriting testing using machine learning. A learner needs to write on the mobile screen in on-screen testing, whereas AR-based testing allows one to evaluate writing on paper or a board in a real world environment. We implement a novel approach to use machine learning for AR-based testing to detect an alphabet written on a board or paper. It detects the handwritten alphabet using our developed machine learning model. After that, a 3D model of that alphabet appears on the screen with its pronunciation/sound. The key benefit of our approach is that it allows the learner to use a handwritten alphabet. As we have used marker-less augmented reality, it does not require a static image as a marker. The app was built with ARCore SDK for Unity. We further evaluated and quantified the performance of our app on multiple devices.https://www.mdpi.com/2073-431X/11/1/4mobile appaugmented realitymachine learningalphabet learninghandwriting recognition |
| spellingShingle | Md. Nahidul Islam Opu Md. Rakibul Islam Muhammad Ashad Kabir Md. Sabir Hossain Mohammad Mainul Islam <i>Learn2Write</i>: Augmented Reality and Machine Learning-Based Mobile App to Learn Writing Computers mobile app augmented reality machine learning alphabet learning handwriting recognition |
| title | <i>Learn2Write</i>: Augmented Reality and Machine Learning-Based Mobile App to Learn Writing |
| title_full | <i>Learn2Write</i>: Augmented Reality and Machine Learning-Based Mobile App to Learn Writing |
| title_fullStr | <i>Learn2Write</i>: Augmented Reality and Machine Learning-Based Mobile App to Learn Writing |
| title_full_unstemmed | <i>Learn2Write</i>: Augmented Reality and Machine Learning-Based Mobile App to Learn Writing |
| title_short | <i>Learn2Write</i>: Augmented Reality and Machine Learning-Based Mobile App to Learn Writing |
| title_sort | i learn2write i augmented reality and machine learning based mobile app to learn writing |
| topic | mobile app augmented reality machine learning alphabet learning handwriting recognition |
| url | https://www.mdpi.com/2073-431X/11/1/4 |
| work_keys_str_mv | AT mdnahidulislamopu ilearn2writeiaugmentedrealityandmachinelearningbasedmobileapptolearnwriting AT mdrakibulislam ilearn2writeiaugmentedrealityandmachinelearningbasedmobileapptolearnwriting AT muhammadashadkabir ilearn2writeiaugmentedrealityandmachinelearningbasedmobileapptolearnwriting AT mdsabirhossain ilearn2writeiaugmentedrealityandmachinelearningbasedmobileapptolearnwriting AT mohammadmainulislam ilearn2writeiaugmentedrealityandmachinelearningbasedmobileapptolearnwriting |