Human-pose estimation based on weak supervision
Background: In computer vision, simultaneously estimating human pose, shape, and clothing is a practical issue in real life, but remains a challenging task owing to the variety of clothing, complexity of deformation, shortage of large-scale datasets, and difficulty in estimating clothing style. Meth...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
KeAi Communications Co., Ltd.
2023-08-01
|
| Series: | Virtual Reality & Intelligent Hardware |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2096579622000857 |
| _version_ | 1797737376510902272 |
|---|---|
| author | Xiaoyan Hu Xizhao Bao Guoli Wei Zhaoyu Li |
| author_facet | Xiaoyan Hu Xizhao Bao Guoli Wei Zhaoyu Li |
| author_sort | Xiaoyan Hu |
| collection | DOAJ |
| description | Background: In computer vision, simultaneously estimating human pose, shape, and clothing is a practical issue in real life, but remains a challenging task owing to the variety of clothing, complexity of deformation, shortage of large-scale datasets, and difficulty in estimating clothing style. Methods: We propose a multistage weakly supervised method that makes full use of data with less labeled information for learning to estimate human body shape, pose, and clothing deformation. In the first stage, the SMPL human-body model parameters were regressed using the multi-view 2D key points of the human body. Using multi-view information as weakly supervised information can avoid the deep ambiguity problem of a single view, obtain a more accurate human posture, and access supervisory information easily. In the second stage, clothing is represented by a PCAbased model that uses two-dimensional key points of clothing as supervised information to regress the parameters. In the third stage, we predefine an embedding graph for each type of clothing to describe the deformation. Then, the mask information of the clothing is used to further adjust the deformation of the clothing. To facilitate training, we constructed a multi-view synthetic dataset that included BCNet and SURREAL. Results: The Experiments show that the accuracy of our method reaches the same level as that of SOTA methods using strong supervision information while only using weakly supervised information. Because this study uses only weakly supervised information, which is much easier to obtain, it has the advantage of utilizing existing data as training data. Experiments on the DeepFashion2 dataset show that our method can make full use of the existing weak supervision information for fine-tuning on a dataset with little supervision information, compared with the strong supervision information that cannot be trained or adjusted owing to the lack of exact annotation information. Conclusions: Our weak supervision method can accurately estimate human body size, pose, and several common types of clothing and overcome the issues of the current shortage of clothing data. |
| first_indexed | 2024-03-12T13:27:49Z |
| format | Article |
| id | doaj.art-c6b5a22c727c430799b16b3acdefbbba |
| institution | Directory Open Access Journal |
| issn | 2096-5796 |
| language | English |
| last_indexed | 2024-03-12T13:27:49Z |
| publishDate | 2023-08-01 |
| publisher | KeAi Communications Co., Ltd. |
| record_format | Article |
| series | Virtual Reality & Intelligent Hardware |
| spelling | doaj.art-c6b5a22c727c430799b16b3acdefbbba2023-08-25T04:24:09ZengKeAi Communications Co., Ltd.Virtual Reality & Intelligent Hardware2096-57962023-08-0154366377Human-pose estimation based on weak supervisionXiaoyan Hu0Xizhao Bao1Guoli Wei2Zhaoyu Li3School of Artificial Intelligence, Beijing Normal University, Beijing 100875, ChinaSchool of Artificial Intelligence, Beijing Normal University, Beijing 100875, ChinaSchool of Artificial Intelligence, Beijing Normal University, Beijing 100875, ChinaCorresponding author.; School of Artificial Intelligence, Beijing Normal University, Beijing 100875, ChinaBackground: In computer vision, simultaneously estimating human pose, shape, and clothing is a practical issue in real life, but remains a challenging task owing to the variety of clothing, complexity of deformation, shortage of large-scale datasets, and difficulty in estimating clothing style. Methods: We propose a multistage weakly supervised method that makes full use of data with less labeled information for learning to estimate human body shape, pose, and clothing deformation. In the first stage, the SMPL human-body model parameters were regressed using the multi-view 2D key points of the human body. Using multi-view information as weakly supervised information can avoid the deep ambiguity problem of a single view, obtain a more accurate human posture, and access supervisory information easily. In the second stage, clothing is represented by a PCAbased model that uses two-dimensional key points of clothing as supervised information to regress the parameters. In the third stage, we predefine an embedding graph for each type of clothing to describe the deformation. Then, the mask information of the clothing is used to further adjust the deformation of the clothing. To facilitate training, we constructed a multi-view synthetic dataset that included BCNet and SURREAL. Results: The Experiments show that the accuracy of our method reaches the same level as that of SOTA methods using strong supervision information while only using weakly supervised information. Because this study uses only weakly supervised information, which is much easier to obtain, it has the advantage of utilizing existing data as training data. Experiments on the DeepFashion2 dataset show that our method can make full use of the existing weak supervision information for fine-tuning on a dataset with little supervision information, compared with the strong supervision information that cannot be trained or adjusted owing to the lack of exact annotation information. Conclusions: Our weak supervision method can accurately estimate human body size, pose, and several common types of clothing and overcome the issues of the current shortage of clothing data.http://www.sciencedirect.com/science/article/pii/S2096579622000857Human pose estimationClothing estimationWeak supervision |
| spellingShingle | Xiaoyan Hu Xizhao Bao Guoli Wei Zhaoyu Li Human-pose estimation based on weak supervision Virtual Reality & Intelligent Hardware Human pose estimation Clothing estimation Weak supervision |
| title | Human-pose estimation based on weak supervision |
| title_full | Human-pose estimation based on weak supervision |
| title_fullStr | Human-pose estimation based on weak supervision |
| title_full_unstemmed | Human-pose estimation based on weak supervision |
| title_short | Human-pose estimation based on weak supervision |
| title_sort | human pose estimation based on weak supervision |
| topic | Human pose estimation Clothing estimation Weak supervision |
| url | http://www.sciencedirect.com/science/article/pii/S2096579622000857 |
| work_keys_str_mv | AT xiaoyanhu humanposeestimationbasedonweaksupervision AT xizhaobao humanposeestimationbasedonweaksupervision AT guoliwei humanposeestimationbasedonweaksupervision AT zhaoyuli humanposeestimationbasedonweaksupervision |