Towards Digital Twins of 3D Reconstructed Apparel Models with an End-to-End Mobile Visualization
Digital twin technologies are still developing and are being increasingly leveraged to facilitate daily life activities. This study presents a novel approach for leveraging the capability of mobile devices for photo collection, cloud processing, and deep learning-based 3D generation, with seamless d...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-07-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/15/8571 |
| _version_ | 1797587127671717888 |
|---|---|
| author | Surasachai Doungtap Jirayu Petchhan Varinya Phanichraksaphong Jenq-Haur Wang |
| author_facet | Surasachai Doungtap Jirayu Petchhan Varinya Phanichraksaphong Jenq-Haur Wang |
| author_sort | Surasachai Doungtap |
| collection | DOAJ |
| description | Digital twin technologies are still developing and are being increasingly leveraged to facilitate daily life activities. This study presents a novel approach for leveraging the capability of mobile devices for photo collection, cloud processing, and deep learning-based 3D generation, with seamless display in virtual reality (VR) wearables. The purpose of our study is to provide a system that makes use of cloud computing resources to offload the resource-intensive activities of 3D reconstruction and deep-learning-based scene interpretation. We establish an end-to-end pipeline from 2D to 3D reconstruction, which automatically builds accurate 3D models from collected photographs using sophisticated deep-learning techniques. These models are then converted to a VR-compatible format, allowing for immersive and interactive experiences on wearable devices. Our findings attest to the completion of 3D entities regenerated by the CAP–UDF model using ShapeNetCars and Deep Fashion 3D datasets with a discrepancy in L2 Chamfer distance of only 0.089 and 0.129, respectively. Furthermore, the demonstration of the end-to-end process from 2D capture to 3D visualization on VR occurs continuously. |
| first_indexed | 2024-03-11T00:32:50Z |
| format | Article |
| id | doaj.art-f70e294f2a8440e79826f8c8a1112d5f |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-11T00:32:50Z |
| publishDate | 2023-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-f70e294f2a8440e79826f8c8a1112d5f2023-11-18T22:34:39ZengMDPI AGApplied Sciences2076-34172023-07-011315857110.3390/app13158571Towards Digital Twins of 3D Reconstructed Apparel Models with an End-to-End Mobile VisualizationSurasachai Doungtap0Jirayu Petchhan1Varinya Phanichraksaphong2Jenq-Haur Wang3International Graduate Program of Electrical Engineering and Computer Science, National Taipei University of Technology, Taipei 10608, TaiwanDepartment of Electrical Engineering, National Taiwan University of Science and Technology, Taipei 106, TaiwanInternational Graduate Program of Electrical Engineering and Computer Science, National Taipei University of Technology, Taipei 10608, TaiwanDepartment of Computer Science and Information Engineering, National Taipei University of Technology, Taipei 10608, TaiwanDigital twin technologies are still developing and are being increasingly leveraged to facilitate daily life activities. This study presents a novel approach for leveraging the capability of mobile devices for photo collection, cloud processing, and deep learning-based 3D generation, with seamless display in virtual reality (VR) wearables. The purpose of our study is to provide a system that makes use of cloud computing resources to offload the resource-intensive activities of 3D reconstruction and deep-learning-based scene interpretation. We establish an end-to-end pipeline from 2D to 3D reconstruction, which automatically builds accurate 3D models from collected photographs using sophisticated deep-learning techniques. These models are then converted to a VR-compatible format, allowing for immersive and interactive experiences on wearable devices. Our findings attest to the completion of 3D entities regenerated by the CAP–UDF model using ShapeNetCars and Deep Fashion 3D datasets with a discrepancy in L2 Chamfer distance of only 0.089 and 0.129, respectively. Furthermore, the demonstration of the end-to-end process from 2D capture to 3D visualization on VR occurs continuously.https://www.mdpi.com/2076-3417/13/15/8571apparel industrydigital twinsvirtual reality3D reconstructionCAP–UDF |
| spellingShingle | Surasachai Doungtap Jirayu Petchhan Varinya Phanichraksaphong Jenq-Haur Wang Towards Digital Twins of 3D Reconstructed Apparel Models with an End-to-End Mobile Visualization Applied Sciences apparel industry digital twins virtual reality 3D reconstruction CAP–UDF |
| title | Towards Digital Twins of 3D Reconstructed Apparel Models with an End-to-End Mobile Visualization |
| title_full | Towards Digital Twins of 3D Reconstructed Apparel Models with an End-to-End Mobile Visualization |
| title_fullStr | Towards Digital Twins of 3D Reconstructed Apparel Models with an End-to-End Mobile Visualization |
| title_full_unstemmed | Towards Digital Twins of 3D Reconstructed Apparel Models with an End-to-End Mobile Visualization |
| title_short | Towards Digital Twins of 3D Reconstructed Apparel Models with an End-to-End Mobile Visualization |
| title_sort | towards digital twins of 3d reconstructed apparel models with an end to end mobile visualization |
| topic | apparel industry digital twins virtual reality 3D reconstruction CAP–UDF |
| url | https://www.mdpi.com/2076-3417/13/15/8571 |
| work_keys_str_mv | AT surasachaidoungtap towardsdigitaltwinsof3dreconstructedapparelmodelswithanendtoendmobilevisualization AT jirayupetchhan towardsdigitaltwinsof3dreconstructedapparelmodelswithanendtoendmobilevisualization AT varinyaphanichraksaphong towardsdigitaltwinsof3dreconstructedapparelmodelswithanendtoendmobilevisualization AT jenqhaurwang towardsdigitaltwinsof3dreconstructedapparelmodelswithanendtoendmobilevisualization |