Model transfer from 2D to 3D study for boxing pose estimation
IntroductionBoxing as a sport is growing on Chinese campuses, resulting in a coaching shortage. The human pose estimation technology can be employed to estimate boxing poses and teach interns to relieve the shortage. Currently, 3D cameras can provide more depth information than 2D cameras. It can po...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2023-03-01
|
| Series: | Frontiers in Neurorobotics |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fnbot.2023.1148545/full |
| _version_ | 1797867240683470848 |
|---|---|
| author | Jianchu Lin Xiaolong Xie Wangping Wu Shengpeng Xu Chunyan Liu Toshboev Hudoyberdi Xiaobing Chen |
| author_facet | Jianchu Lin Xiaolong Xie Wangping Wu Shengpeng Xu Chunyan Liu Toshboev Hudoyberdi Xiaobing Chen |
| author_sort | Jianchu Lin |
| collection | DOAJ |
| description | IntroductionBoxing as a sport is growing on Chinese campuses, resulting in a coaching shortage. The human pose estimation technology can be employed to estimate boxing poses and teach interns to relieve the shortage. Currently, 3D cameras can provide more depth information than 2D cameras. It can potentially improve the estimation. However, the input channels are inconsistent between 2D and 3D images, and there is a lack of detailed analysis about the key point location, which indicates the network design for improving the human pose estimation technology.MethodTherefore, a model transfer with channel patching was implemented to solve the problems of channel inconsistency. The differences between the key points were analyzed. Three popular and highly structured 2D models of OpenPose (OP), stacked Hourglass (HG), and High Resolution (HR) networks were employed. Ways of reusing RGB channels were investigated to fill up the depth channel. Then, their performances were investigated to find out the limitations of each network structure.Results and discussionThe results show that model transfer learning by the mean way of RGB channels patching the lacking channel can improve the average accuracies of pose key points from 1 to 20% than without transfer. 3D accuracies are 0.3 to 0.5% higher than 2D baselines. The stacked structure of the network shows better on hip and knee points than the parallel structure, although the parallel design shows much better on the residue points. As a result, the model transfer can practically fulfill boxing pose estimation from 2D to 3D. |
| first_indexed | 2024-04-09T23:37:09Z |
| format | Article |
| id | doaj.art-4948d7be36cf4aa5b8ec3021e271d480 |
| institution | Directory Open Access Journal |
| issn | 1662-5218 |
| language | English |
| last_indexed | 2024-04-09T23:37:09Z |
| publishDate | 2023-03-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Neurorobotics |
| spelling | doaj.art-4948d7be36cf4aa5b8ec3021e271d4802023-03-20T04:32:41ZengFrontiers Media S.A.Frontiers in Neurorobotics1662-52182023-03-011710.3389/fnbot.2023.11485451148545Model transfer from 2D to 3D study for boxing pose estimationJianchu LinXiaolong XieWangping WuShengpeng XuChunyan LiuToshboev HudoyberdiXiaobing ChenIntroductionBoxing as a sport is growing on Chinese campuses, resulting in a coaching shortage. The human pose estimation technology can be employed to estimate boxing poses and teach interns to relieve the shortage. Currently, 3D cameras can provide more depth information than 2D cameras. It can potentially improve the estimation. However, the input channels are inconsistent between 2D and 3D images, and there is a lack of detailed analysis about the key point location, which indicates the network design for improving the human pose estimation technology.MethodTherefore, a model transfer with channel patching was implemented to solve the problems of channel inconsistency. The differences between the key points were analyzed. Three popular and highly structured 2D models of OpenPose (OP), stacked Hourglass (HG), and High Resolution (HR) networks were employed. Ways of reusing RGB channels were investigated to fill up the depth channel. Then, their performances were investigated to find out the limitations of each network structure.Results and discussionThe results show that model transfer learning by the mean way of RGB channels patching the lacking channel can improve the average accuracies of pose key points from 1 to 20% than without transfer. 3D accuracies are 0.3 to 0.5% higher than 2D baselines. The stacked structure of the network shows better on hip and knee points than the parallel structure, although the parallel design shows much better on the residue points. As a result, the model transfer can practically fulfill boxing pose estimation from 2D to 3D.https://www.frontiersin.org/articles/10.3389/fnbot.2023.1148545/fullboxing robotcomputer visionhuman pose estimation3D model transfernegative transfer |
| spellingShingle | Jianchu Lin Xiaolong Xie Wangping Wu Shengpeng Xu Chunyan Liu Toshboev Hudoyberdi Xiaobing Chen Model transfer from 2D to 3D study for boxing pose estimation Frontiers in Neurorobotics boxing robot computer vision human pose estimation 3D model transfer negative transfer |
| title | Model transfer from 2D to 3D study for boxing pose estimation |
| title_full | Model transfer from 2D to 3D study for boxing pose estimation |
| title_fullStr | Model transfer from 2D to 3D study for boxing pose estimation |
| title_full_unstemmed | Model transfer from 2D to 3D study for boxing pose estimation |
| title_short | Model transfer from 2D to 3D study for boxing pose estimation |
| title_sort | model transfer from 2d to 3d study for boxing pose estimation |
| topic | boxing robot computer vision human pose estimation 3D model transfer negative transfer |
| url | https://www.frontiersin.org/articles/10.3389/fnbot.2023.1148545/full |
| work_keys_str_mv | AT jianchulin modeltransferfrom2dto3dstudyforboxingposeestimation AT xiaolongxie modeltransferfrom2dto3dstudyforboxingposeestimation AT wangpingwu modeltransferfrom2dto3dstudyforboxingposeestimation AT shengpengxu modeltransferfrom2dto3dstudyforboxingposeestimation AT chunyanliu modeltransferfrom2dto3dstudyforboxingposeestimation AT toshboevhudoyberdi modeltransferfrom2dto3dstudyforboxingposeestimation AT xiaobingchen modeltransferfrom2dto3dstudyforboxingposeestimation |