ConvNeXtPose: A Fast Accurate Method for 3D Human Pose Estimation and Its AR Fitness Application in Mobile Devices
In general, 3D human-pose estimation requires high-performance computing resources. Existing methods working on mobile devices trade off accuracy in return for increased efficiency, often making the estimation accuracy far from sufficient for developing serious applications. In this paper, we presen...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10288440/ |
| _version_ | 1797644918574809088 |
|---|---|
| author | Hong Son Nguyen Myounggon Kim Changbin Im Sanghoon Han JungHyun Han |
| author_facet | Hong Son Nguyen Myounggon Kim Changbin Im Sanghoon Han JungHyun Han |
| author_sort | Hong Son Nguyen |
| collection | DOAJ |
| description | In general, 3D human-pose estimation requires high-performance computing resources. Existing methods working on mobile devices trade off accuracy in return for increased efficiency, often making the estimation accuracy far from sufficient for developing serious applications. In this paper, we present a mobile 3D human-pose estimation model, achieving real-time performances with a well-designed balance between efficiency and accuracy. As the backbone, our model leverages the cutting-edge ConvNeXt architecture, renowned for its feature extraction capabilities. We enhance its performance through strategic architectural modifications and incorporation of depthwise separable convolutions in the upsampling module. The experiments made with the Human3.6M dataset show that the accuracy delivered by our model is comparable to that of the state-of-the-art models, consuming significantly fewer computational resources. To showcase the practicality of our model, we present a prototype of an AR fitness application. Built upon our 3D human pose estimation model, it helps trainees recreate trainers’ poses from reference images. The effectiveness of the application is validated via experiments and evaluations. The source code can be found at: <uri>https://github.com/medialab-ku/ConvNeXtPose</uri>. |
| first_indexed | 2024-03-11T14:38:22Z |
| format | Article |
| id | doaj.art-c48294aa64dd424283ae637ebac555c4 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-11T14:38:22Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-c48294aa64dd424283ae637ebac555c42023-10-30T23:00:34ZengIEEEIEEE Access2169-35362023-01-011111739311740210.1109/ACCESS.2023.332634310288440ConvNeXtPose: A Fast Accurate Method for 3D Human Pose Estimation and Its AR Fitness Application in Mobile DevicesHong Son Nguyen0https://orcid.org/0009-0001-5843-2274Myounggon Kim1https://orcid.org/0000-0002-0767-4812Changbin Im2https://orcid.org/0009-0008-3612-5944Sanghoon Han3https://orcid.org/0009-0001-8873-1611JungHyun Han4https://orcid.org/0000-0001-6438-2974Department of Computer Science and Engineering, Korea University, Seoul, South KoreaDepartment of Computer Science and Engineering, Korea University, Seoul, South KoreaDepartment of Computer Science and Engineering, Korea University, Seoul, South KoreaDepartment of Computer Science and Engineering, Korea University, Seoul, South KoreaDepartment of Computer Science and Engineering, Korea University, Seoul, South KoreaIn general, 3D human-pose estimation requires high-performance computing resources. Existing methods working on mobile devices trade off accuracy in return for increased efficiency, often making the estimation accuracy far from sufficient for developing serious applications. In this paper, we present a mobile 3D human-pose estimation model, achieving real-time performances with a well-designed balance between efficiency and accuracy. As the backbone, our model leverages the cutting-edge ConvNeXt architecture, renowned for its feature extraction capabilities. We enhance its performance through strategic architectural modifications and incorporation of depthwise separable convolutions in the upsampling module. The experiments made with the Human3.6M dataset show that the accuracy delivered by our model is comparable to that of the state-of-the-art models, consuming significantly fewer computational resources. To showcase the practicality of our model, we present a prototype of an AR fitness application. Built upon our 3D human pose estimation model, it helps trainees recreate trainers’ poses from reference images. The effectiveness of the application is validated via experiments and evaluations. The source code can be found at: <uri>https://github.com/medialab-ku/ConvNeXtPose</uri>.https://ieeexplore.ieee.org/document/10288440/Augmented reality3D human pose estimationpose correctionpose matching |
| spellingShingle | Hong Son Nguyen Myounggon Kim Changbin Im Sanghoon Han JungHyun Han ConvNeXtPose: A Fast Accurate Method for 3D Human Pose Estimation and Its AR Fitness Application in Mobile Devices IEEE Access Augmented reality 3D human pose estimation pose correction pose matching |
| title | ConvNeXtPose: A Fast Accurate Method for 3D Human Pose Estimation and Its AR Fitness Application in Mobile Devices |
| title_full | ConvNeXtPose: A Fast Accurate Method for 3D Human Pose Estimation and Its AR Fitness Application in Mobile Devices |
| title_fullStr | ConvNeXtPose: A Fast Accurate Method for 3D Human Pose Estimation and Its AR Fitness Application in Mobile Devices |
| title_full_unstemmed | ConvNeXtPose: A Fast Accurate Method for 3D Human Pose Estimation and Its AR Fitness Application in Mobile Devices |
| title_short | ConvNeXtPose: A Fast Accurate Method for 3D Human Pose Estimation and Its AR Fitness Application in Mobile Devices |
| title_sort | convnextpose a fast accurate method for 3d human pose estimation and its ar fitness application in mobile devices |
| topic | Augmented reality 3D human pose estimation pose correction pose matching |
| url | https://ieeexplore.ieee.org/document/10288440/ |
| work_keys_str_mv | AT hongsonnguyen convnextposeafastaccuratemethodfor3dhumanposeestimationanditsarfitnessapplicationinmobiledevices AT myounggonkim convnextposeafastaccuratemethodfor3dhumanposeestimationanditsarfitnessapplicationinmobiledevices AT changbinim convnextposeafastaccuratemethodfor3dhumanposeestimationanditsarfitnessapplicationinmobiledevices AT sanghoonhan convnextposeafastaccuratemethodfor3dhumanposeestimationanditsarfitnessapplicationinmobiledevices AT junghyunhan convnextposeafastaccuratemethodfor3dhumanposeestimationanditsarfitnessapplicationinmobiledevices |