Human Pose Estimation via Dynamic Information Transfer
This paper presents a multi-task learning framework, called the dynamic information transfer network (DITN). We mainly focused on improving the pose estimation with the spatial relationship of the adjacent joints. To benefit from the explicit structural knowledge, we constructed two branches with a...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/12/3/695 |
| _version_ | 1797624780561580032 |
|---|---|
| author | Yihang Li Qingxuan Shi Jingya Song Fang Yang |
| author_facet | Yihang Li Qingxuan Shi Jingya Song Fang Yang |
| author_sort | Yihang Li |
| collection | DOAJ |
| description | This paper presents a multi-task learning framework, called the dynamic information transfer network (DITN). We mainly focused on improving the pose estimation with the spatial relationship of the adjacent joints. To benefit from the explicit structural knowledge, we constructed two branches with a shared backbone to localize the human joints and bones, respectively. Since related tasks share a high-level representation, we leveraged the bone information to refine the joint localization via dynamic information transfer. In detail, we extracted the dynamic parameters from the bone branch and used them to make the network learn constraint relationships via dynamic convolution. Moreover, attention blocks were added after the information transfer to balance the information across different granularity levels and induce the network to focus on the informative regions. The experimental results demonstrated the effectiveness of the DITN, which achieved 90.8% PCKh@0.5 on MPII and 75.0% AP on COCO. The qualitative results on the MPII and COCO datasets showed that the DITN achieved better performance, especially on heavily occluded or easily confusable joint localization. |
| first_indexed | 2024-03-11T09:47:19Z |
| format | Article |
| id | doaj.art-da7fe8b21bf745eeacc661f5425f93dc |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-11T09:47:19Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-da7fe8b21bf745eeacc661f5425f93dc2023-11-16T16:30:10ZengMDPI AGElectronics2079-92922023-01-0112369510.3390/electronics12030695Human Pose Estimation via Dynamic Information TransferYihang Li0Qingxuan Shi1Jingya Song2Fang Yang3School of Cyber Security and Computer, Hebei University, Baoding 071002, ChinaSchool of Cyber Security and Computer, Hebei University, Baoding 071002, ChinaSchool of Cyber Security and Computer, Hebei University, Baoding 071002, ChinaSchool of Cyber Security and Computer, Hebei University, Baoding 071002, ChinaThis paper presents a multi-task learning framework, called the dynamic information transfer network (DITN). We mainly focused on improving the pose estimation with the spatial relationship of the adjacent joints. To benefit from the explicit structural knowledge, we constructed two branches with a shared backbone to localize the human joints and bones, respectively. Since related tasks share a high-level representation, we leveraged the bone information to refine the joint localization via dynamic information transfer. In detail, we extracted the dynamic parameters from the bone branch and used them to make the network learn constraint relationships via dynamic convolution. Moreover, attention blocks were added after the information transfer to balance the information across different granularity levels and induce the network to focus on the informative regions. The experimental results demonstrated the effectiveness of the DITN, which achieved 90.8% PCKh@0.5 on MPII and 75.0% AP on COCO. The qualitative results on the MPII and COCO datasets showed that the DITN achieved better performance, especially on heavily occluded or easily confusable joint localization.https://www.mdpi.com/2079-9292/12/3/695computer visionpose estimationmulti-task learningdynamic information transfer |
| spellingShingle | Yihang Li Qingxuan Shi Jingya Song Fang Yang Human Pose Estimation via Dynamic Information Transfer Electronics computer vision pose estimation multi-task learning dynamic information transfer |
| title | Human Pose Estimation via Dynamic Information Transfer |
| title_full | Human Pose Estimation via Dynamic Information Transfer |
| title_fullStr | Human Pose Estimation via Dynamic Information Transfer |
| title_full_unstemmed | Human Pose Estimation via Dynamic Information Transfer |
| title_short | Human Pose Estimation via Dynamic Information Transfer |
| title_sort | human pose estimation via dynamic information transfer |
| topic | computer vision pose estimation multi-task learning dynamic information transfer |
| url | https://www.mdpi.com/2079-9292/12/3/695 |
| work_keys_str_mv | AT yihangli humanposeestimationviadynamicinformationtransfer AT qingxuanshi humanposeestimationviadynamicinformationtransfer AT jingyasong humanposeestimationviadynamicinformationtransfer AT fangyang humanposeestimationviadynamicinformationtransfer |