GeoConv: geodesic guided convolution for facial action unit recognition
Automatic facial action unit (AU) recognition has attracted great attention but still remains a challenging task, as subtle changes of local facial muscles are difficult to thoroughly capture. Most existing AU recognition approaches leverage geometry information in a straightforward 2D or 3D manner,...
| Main Authors: | , , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2023
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/172657 |
| _version_ | 1811691320761122816 |
|---|---|
| author | Chen, Yuedong Song, Guoxian Shao, Zhiwen Cai, Jianfei Cham, Tat-Jen Zheng, Jianmin |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Chen, Yuedong Song, Guoxian Shao, Zhiwen Cai, Jianfei Cham, Tat-Jen Zheng, Jianmin |
| author_sort | Chen, Yuedong |
| collection | NTU |
| description | Automatic facial action unit (AU) recognition has attracted great attention but still remains a challenging task, as subtle changes of local facial muscles are difficult to thoroughly capture. Most existing AU recognition approaches leverage geometry information in a straightforward 2D or 3D manner, which either ignore 3D manifold information or suffer from high computational costs. In this paper, we propose a novel geodesic guided convolution (GeoConv) for AU recognition by embedding 3D manifold information into 2D convolutions. Specifically, the kernel of GeoConv is weighted by our introduced geodesic weights, which are negatively correlated to geodesic distances on a coarsely reconstructed 3D morphable face model. Moreover, based on GeoConv, we further develop an end-to-end trainable framework named GeoCNN for AU recognition. Extensive experiments on BP4D and DISFA benchmarks show that our approach significantly outperforms the state-of-the-art AU recognition methods. |
| first_indexed | 2024-10-01T06:18:01Z |
| format | Journal Article |
| id | ntu-10356/172657 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T06:18:01Z |
| publishDate | 2023 |
| record_format | dspace |
| spelling | ntu-10356/1726572023-12-19T02:24:07Z GeoConv: geodesic guided convolution for facial action unit recognition Chen, Yuedong Song, Guoxian Shao, Zhiwen Cai, Jianfei Cham, Tat-Jen Zheng, Jianmin School of Computer Science and Engineering Engineering::Computer science and engineering::Computing methodologies::Pattern recognition Geodesic Guided Convolution 3D Morphable Face Model Automatic facial action unit (AU) recognition has attracted great attention but still remains a challenging task, as subtle changes of local facial muscles are difficult to thoroughly capture. Most existing AU recognition approaches leverage geometry information in a straightforward 2D or 3D manner, which either ignore 3D manifold information or suffer from high computational costs. In this paper, we propose a novel geodesic guided convolution (GeoConv) for AU recognition by embedding 3D manifold information into 2D convolutions. Specifically, the kernel of GeoConv is weighted by our introduced geodesic weights, which are negatively correlated to geodesic distances on a coarsely reconstructed 3D morphable face model. Moreover, based on GeoConv, we further develop an end-to-end trainable framework named GeoCNN for AU recognition. Extensive experiments on BP4D and DISFA benchmarks show that our approach significantly outperforms the state-of-the-art AU recognition methods. National Research Foundation (NRF) This research is supported by the National Research Foundation, Singapore under its International Research Centres in Singapore Funding Initiative. This research is also supported in part by Monash FIT Start-up Grant. 2023-12-19T02:24:07Z 2023-12-19T02:24:07Z 2022 Journal Article Chen, Y., Song, G., Shao, Z., Cai, J., Cham, T. & Zheng, J. (2022). GeoConv: geodesic guided convolution for facial action unit recognition. Pattern Recognition, 122, 108355-. https://dx.doi.org/10.1016/j.patcog.2021.108355 0031-3203 https://hdl.handle.net/10356/172657 10.1016/j.patcog.2021.108355 2-s2.0-85118707359 122 108355 en Pattern Recognition © 2021 Elsevier Ltd. All rights reserved. |
| spellingShingle | Engineering::Computer science and engineering::Computing methodologies::Pattern recognition Geodesic Guided Convolution 3D Morphable Face Model Chen, Yuedong Song, Guoxian Shao, Zhiwen Cai, Jianfei Cham, Tat-Jen Zheng, Jianmin GeoConv: geodesic guided convolution for facial action unit recognition |
| title | GeoConv: geodesic guided convolution for facial action unit recognition |
| title_full | GeoConv: geodesic guided convolution for facial action unit recognition |
| title_fullStr | GeoConv: geodesic guided convolution for facial action unit recognition |
| title_full_unstemmed | GeoConv: geodesic guided convolution for facial action unit recognition |
| title_short | GeoConv: geodesic guided convolution for facial action unit recognition |
| title_sort | geoconv geodesic guided convolution for facial action unit recognition |
| topic | Engineering::Computer science and engineering::Computing methodologies::Pattern recognition Geodesic Guided Convolution 3D Morphable Face Model |
| url | https://hdl.handle.net/10356/172657 |
| work_keys_str_mv | AT chenyuedong geoconvgeodesicguidedconvolutionforfacialactionunitrecognition AT songguoxian geoconvgeodesicguidedconvolutionforfacialactionunitrecognition AT shaozhiwen geoconvgeodesicguidedconvolutionforfacialactionunitrecognition AT caijianfei geoconvgeodesicguidedconvolutionforfacialactionunitrecognition AT chamtatjen geoconvgeodesicguidedconvolutionforfacialactionunitrecognition AT zhengjianmin geoconvgeodesicguidedconvolutionforfacialactionunitrecognition |