Polygonized Silhouettes and Polygon Coding Based Feature Representation for Human Action Recognition
The characteristics of human silhouette shape can be used for action recognition and classification. In this paper, a novel feature extraction method for the silhouette-based classification of human actions in videos is proposed. The proposed method is based on polygonization of silhouette images an...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10144743/ |
| _version_ | 1797804045741588480 |
|---|---|
| author | Ogul Gocmen Murat Emin Akata |
| author_facet | Ogul Gocmen Murat Emin Akata |
| author_sort | Ogul Gocmen |
| collection | DOAJ |
| description | The characteristics of human silhouette shape can be used for action recognition and classification. In this paper, a novel feature extraction method for the silhouette-based classification of human actions in videos is proposed. The proposed method is based on polygonization of silhouette images and coding. Since conventional silhouette generation methods do not satisfy the integrity of silhouettes, Yolact++ is modified as a silhouette generator. Our innovative approach Yolact++ masks are used as silhouettes to overcome this problem. For this purpose, a new image form called Poly Silhouette (PoS), a new Polygonization (PoG) algorithm and a new Polygon Coding (PoC) algorithm have been developed. The polygonization step is based on, but is not similar to curve and image polygonization. It is fast, adaptable, and accurate on the contour coordinates of the PoS images. PoCs were generated by projecting each edge vector generated from the corner coordinates of the PoS onto the angular areas and codes for the PoS were formed. These codes are grouped into k-mers similar to genetic algorithms and are used as features. The proposed innovative feature extraction method guarantees that feature vectors of equal length are generated from any action video. Thus, no additional action is required to overcome the dimensionality problem. By using different k-mer lengths, the classification accuracy of the method versus computation time was analyzed and depicted in figures. The method developed was tested on HMDB51 & UCF101 datasets: for SVM 20.98%, 1.63% for k-NN 4.96%, 6.83%, respectively, and significant improvements were achieved. |
| first_indexed | 2024-03-13T05:31:14Z |
| format | Article |
| id | doaj.art-31c6026099a042818a2ff20f9da6ec58 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-13T05:31:14Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-31c6026099a042818a2ff20f9da6ec582023-06-14T23:00:18ZengIEEEIEEE Access2169-35362023-01-0111570215703610.1109/ACCESS.2023.328345810144743Polygonized Silhouettes and Polygon Coding Based Feature Representation for Human Action RecognitionOgul Gocmen0https://orcid.org/0000-0001-9059-9800Murat Emin Akata1Department of Computer Engineering, Engineering Faculty, Baskent University, Ankara, TurkeyDepartment of Computer Engineering, Engineering Faculty, Baskent University, Ankara, TurkeyThe characteristics of human silhouette shape can be used for action recognition and classification. In this paper, a novel feature extraction method for the silhouette-based classification of human actions in videos is proposed. The proposed method is based on polygonization of silhouette images and coding. Since conventional silhouette generation methods do not satisfy the integrity of silhouettes, Yolact++ is modified as a silhouette generator. Our innovative approach Yolact++ masks are used as silhouettes to overcome this problem. For this purpose, a new image form called Poly Silhouette (PoS), a new Polygonization (PoG) algorithm and a new Polygon Coding (PoC) algorithm have been developed. The polygonization step is based on, but is not similar to curve and image polygonization. It is fast, adaptable, and accurate on the contour coordinates of the PoS images. PoCs were generated by projecting each edge vector generated from the corner coordinates of the PoS onto the angular areas and codes for the PoS were formed. These codes are grouped into k-mers similar to genetic algorithms and are used as features. The proposed innovative feature extraction method guarantees that feature vectors of equal length are generated from any action video. Thus, no additional action is required to overcome the dimensionality problem. By using different k-mer lengths, the classification accuracy of the method versus computation time was analyzed and depicted in figures. The method developed was tested on HMDB51 & UCF101 datasets: for SVM 20.98%, 1.63% for k-NN 4.96%, 6.83%, respectively, and significant improvements were achieved.https://ieeexplore.ieee.org/document/10144743/Polygonized silhouettespolygon codinghuman action recognition |
| spellingShingle | Ogul Gocmen Murat Emin Akata Polygonized Silhouettes and Polygon Coding Based Feature Representation for Human Action Recognition IEEE Access Polygonized silhouettes polygon coding human action recognition |
| title | Polygonized Silhouettes and Polygon Coding Based Feature Representation for Human Action Recognition |
| title_full | Polygonized Silhouettes and Polygon Coding Based Feature Representation for Human Action Recognition |
| title_fullStr | Polygonized Silhouettes and Polygon Coding Based Feature Representation for Human Action Recognition |
| title_full_unstemmed | Polygonized Silhouettes and Polygon Coding Based Feature Representation for Human Action Recognition |
| title_short | Polygonized Silhouettes and Polygon Coding Based Feature Representation for Human Action Recognition |
| title_sort | polygonized silhouettes and polygon coding based feature representation for human action recognition |
| topic | Polygonized silhouettes polygon coding human action recognition |
| url | https://ieeexplore.ieee.org/document/10144743/ |
| work_keys_str_mv | AT ogulgocmen polygonizedsilhouettesandpolygoncodingbasedfeaturerepresentationforhumanactionrecognition AT murateminakata polygonizedsilhouettesandpolygoncodingbasedfeaturerepresentationforhumanactionrecognition |