Improved Human-Object Interaction Detection Through On-the-Fly Stacked Generalization
Human-object interaction (HOI) detection, which finds the relationships between humans and objects, is an important research area, but current HOI detection performance is unsatisfactory. One of the main problems is that CNN-based HOI detection algorithms fail to predict correct outputs for unseen t...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9360596/ |
| _version_ | 1828909871831973888 |
|---|---|
| author | Geonu Lee Kimin Yun Jungchan Cho |
| author_facet | Geonu Lee Kimin Yun Jungchan Cho |
| author_sort | Geonu Lee |
| collection | DOAJ |
| description | Human-object interaction (HOI) detection, which finds the relationships between humans and objects, is an important research area, but current HOI detection performance is unsatisfactory. One of the main problems is that CNN-based HOI detection algorithms fail to predict correct outputs for unseen test data based on a limited number of available training examples. Herein, we propose a novel framework for HOI detection called the on-the-fly stacked generalization deep neural network (OSGNet). OSGNet consists of three main components: (1) feature extraction modules, (2) HOI relationship detection networks, and (3) a meta-learner for combining the outputs of sub-models. Here, components (1) and (2) are considered to be sub-models. Any task-based feature extraction modules, such as classification or human pose estimation modules, can be used as sub-models. To achieve on-the-fly stacked generalization, the sub-models and meta-learner are trained simultaneously. The sub-models are trained to provide complementary information, and the meta-learner improves the generalization performance for unseen test data. Extensive experiments demonstrate that the proposed method achieves state-of-the-art accuracy, particularly in cases involving rare classes. |
| first_indexed | 2024-12-13T18:34:01Z |
| format | Article |
| id | doaj.art-d66a53405ab447ab8a0344d867df743b |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-13T18:34:01Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-d66a53405ab447ab8a0344d867df743b2022-12-21T23:35:25ZengIEEEIEEE Access2169-35362021-01-019342513426310.1109/ACCESS.2021.30612089360596Improved Human-Object Interaction Detection Through On-the-Fly Stacked GeneralizationGeonu Lee0https://orcid.org/0000-0003-2518-8364Kimin Yun1https://orcid.org/0000-0002-4493-9437Jungchan Cho2https://orcid.org/0000-0002-3859-1702College of Information Technology, Gachon University, Seongnam, South KoreaArtificial Intelligence Research Laboratory, Electronics and Telecommunications Research Institute, Daejeon, South KoreaCollege of Information Technology, Gachon University, Seongnam, South KoreaHuman-object interaction (HOI) detection, which finds the relationships between humans and objects, is an important research area, but current HOI detection performance is unsatisfactory. One of the main problems is that CNN-based HOI detection algorithms fail to predict correct outputs for unseen test data based on a limited number of available training examples. Herein, we propose a novel framework for HOI detection called the on-the-fly stacked generalization deep neural network (OSGNet). OSGNet consists of three main components: (1) feature extraction modules, (2) HOI relationship detection networks, and (3) a meta-learner for combining the outputs of sub-models. Here, components (1) and (2) are considered to be sub-models. Any task-based feature extraction modules, such as classification or human pose estimation modules, can be used as sub-models. To achieve on-the-fly stacked generalization, the sub-models and meta-learner are trained simultaneously. The sub-models are trained to provide complementary information, and the meta-learner improves the generalization performance for unseen test data. Extensive experiments demonstrate that the proposed method achieves state-of-the-art accuracy, particularly in cases involving rare classes.https://ieeexplore.ieee.org/document/9360596/Deep learninghuman-object interactionhuman pose estimationaction recognition |
| spellingShingle | Geonu Lee Kimin Yun Jungchan Cho Improved Human-Object Interaction Detection Through On-the-Fly Stacked Generalization IEEE Access Deep learning human-object interaction human pose estimation action recognition |
| title | Improved Human-Object Interaction Detection Through On-the-Fly Stacked Generalization |
| title_full | Improved Human-Object Interaction Detection Through On-the-Fly Stacked Generalization |
| title_fullStr | Improved Human-Object Interaction Detection Through On-the-Fly Stacked Generalization |
| title_full_unstemmed | Improved Human-Object Interaction Detection Through On-the-Fly Stacked Generalization |
| title_short | Improved Human-Object Interaction Detection Through On-the-Fly Stacked Generalization |
| title_sort | improved human object interaction detection through on the fly stacked generalization |
| topic | Deep learning human-object interaction human pose estimation action recognition |
| url | https://ieeexplore.ieee.org/document/9360596/ |
| work_keys_str_mv | AT geonulee improvedhumanobjectinteractiondetectionthroughontheflystackedgeneralization AT kiminyun improvedhumanobjectinteractiondetectionthroughontheflystackedgeneralization AT jungchancho improvedhumanobjectinteractiondetectionthroughontheflystackedgeneralization |