Mobile Augmented Reality for Low-End Devices Based on Planar Surface Recognition and Optimized Vertex Data Rendering
Mobile Augmented Reality (MAR) is designed to keep pace with high-end mobile computing and their powerful sensors. This evolution excludes users with low-end devices and network constraints. This article presents ModAR, a hybrid Android prototype that expands the MAR experience to the aforementioned...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-09-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/11/18/8750 |
| _version_ | 1797520322663022592 |
|---|---|
| author | Styliani Verykokou Argyro-Maria Boutsi Charalabos Ioannidis |
| author_facet | Styliani Verykokou Argyro-Maria Boutsi Charalabos Ioannidis |
| author_sort | Styliani Verykokou |
| collection | DOAJ |
| description | Mobile Augmented Reality (MAR) is designed to keep pace with high-end mobile computing and their powerful sensors. This evolution excludes users with low-end devices and network constraints. This article presents ModAR, a hybrid Android prototype that expands the MAR experience to the aforementioned target group. It combines feature-based image matching and pose estimation with fast rendering of 3D textured models. Planar objects of the real environment are used as pattern images for overlaying users’ meshes or the app’s default ones. Since ModAR is based on the OpenCV C++ library at Android NDK and OpenGL ES 2.0 graphics API, there are no dependencies on additional software, operating system version or model-specific hardware. The developed 3D graphics engine implements optimized vertex-data rendering with a combination of data grouping, synchronization, sub-texture compression and instancing for limited CPU/GPU resources and a single-threaded approach. It achieves up to 3× speed-up compared to standard index rendering, and AR overlay of a 50 K vertices 3D model in less than 30 s. Several deployment scenarios on pose estimation demonstrate that the oriented FAST detector with an upper threshold of features per frame combined with the ORB descriptor yield best results in terms of robustness and efficiency, achieving a 90% reduction of image matching time compared to the time required by the AGAST detector and the BRISK descriptor, corresponding to pattern recognition accuracy of above 90% for a wide range of scale changes, regardless of any in-plane rotations and partial occlusions of the pattern. |
| first_indexed | 2024-03-10T07:55:07Z |
| format | Article |
| id | doaj.art-6b8bb86ed6164f75abd4cdcfef3d57c2 |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-10T07:55:07Z |
| publishDate | 2021-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-6b8bb86ed6164f75abd4cdcfef3d57c22023-11-22T11:57:39ZengMDPI AGApplied Sciences2076-34172021-09-011118875010.3390/app11188750Mobile Augmented Reality for Low-End Devices Based on Planar Surface Recognition and Optimized Vertex Data RenderingStyliani Verykokou0Argyro-Maria Boutsi1Charalabos Ioannidis2Laboratory of Photogrammetry, Zografou Campus, School of Rural, Surveying and Geoinformatics Engineering, National Technical University of Athens, 15780 Athens, GreeceLaboratory of Photogrammetry, Zografou Campus, School of Rural, Surveying and Geoinformatics Engineering, National Technical University of Athens, 15780 Athens, GreeceLaboratory of Photogrammetry, Zografou Campus, School of Rural, Surveying and Geoinformatics Engineering, National Technical University of Athens, 15780 Athens, GreeceMobile Augmented Reality (MAR) is designed to keep pace with high-end mobile computing and their powerful sensors. This evolution excludes users with low-end devices and network constraints. This article presents ModAR, a hybrid Android prototype that expands the MAR experience to the aforementioned target group. It combines feature-based image matching and pose estimation with fast rendering of 3D textured models. Planar objects of the real environment are used as pattern images for overlaying users’ meshes or the app’s default ones. Since ModAR is based on the OpenCV C++ library at Android NDK and OpenGL ES 2.0 graphics API, there are no dependencies on additional software, operating system version or model-specific hardware. The developed 3D graphics engine implements optimized vertex-data rendering with a combination of data grouping, synchronization, sub-texture compression and instancing for limited CPU/GPU resources and a single-threaded approach. It achieves up to 3× speed-up compared to standard index rendering, and AR overlay of a 50 K vertices 3D model in less than 30 s. Several deployment scenarios on pose estimation demonstrate that the oriented FAST detector with an upper threshold of features per frame combined with the ORB descriptor yield best results in terms of robustness and efficiency, achieving a 90% reduction of image matching time compared to the time required by the AGAST detector and the BRISK descriptor, corresponding to pattern recognition accuracy of above 90% for a wide range of scale changes, regardless of any in-plane rotations and partial occlusions of the pattern.https://www.mdpi.com/2076-3417/11/18/8750mobile augmented realitypattern recognitionvertex-based renderinggeometric instancingcamera pose estimation3D rendering |
| spellingShingle | Styliani Verykokou Argyro-Maria Boutsi Charalabos Ioannidis Mobile Augmented Reality for Low-End Devices Based on Planar Surface Recognition and Optimized Vertex Data Rendering Applied Sciences mobile augmented reality pattern recognition vertex-based rendering geometric instancing camera pose estimation 3D rendering |
| title | Mobile Augmented Reality for Low-End Devices Based on Planar Surface Recognition and Optimized Vertex Data Rendering |
| title_full | Mobile Augmented Reality for Low-End Devices Based on Planar Surface Recognition and Optimized Vertex Data Rendering |
| title_fullStr | Mobile Augmented Reality for Low-End Devices Based on Planar Surface Recognition and Optimized Vertex Data Rendering |
| title_full_unstemmed | Mobile Augmented Reality for Low-End Devices Based on Planar Surface Recognition and Optimized Vertex Data Rendering |
| title_short | Mobile Augmented Reality for Low-End Devices Based on Planar Surface Recognition and Optimized Vertex Data Rendering |
| title_sort | mobile augmented reality for low end devices based on planar surface recognition and optimized vertex data rendering |
| topic | mobile augmented reality pattern recognition vertex-based rendering geometric instancing camera pose estimation 3D rendering |
| url | https://www.mdpi.com/2076-3417/11/18/8750 |
| work_keys_str_mv | AT stylianiverykokou mobileaugmentedrealityforlowenddevicesbasedonplanarsurfacerecognitionandoptimizedvertexdatarendering AT argyromariaboutsi mobileaugmentedrealityforlowenddevicesbasedonplanarsurfacerecognitionandoptimizedvertexdatarendering AT charalabosioannidis mobileaugmentedrealityforlowenddevicesbasedonplanarsurfacerecognitionandoptimizedvertexdatarendering |