Facial Expression Recognition with Geometric Scattering on 3D Point Clouds
As one of the pioneering data representations, the point cloud has shown its straightforward capacity to depict fine geometry in many applications, including computer graphics, molecular structurology, modern sensing signal processing, and more. However, unlike computer graphs obtained with auxiliar...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2022-10-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/22/21/8293 |
| _version_ | 1797466541837516800 |
|---|---|
| author | Yi He Keren Fu Peng Cheng Jianwei Zhang |
| author_facet | Yi He Keren Fu Peng Cheng Jianwei Zhang |
| author_sort | Yi He |
| collection | DOAJ |
| description | As one of the pioneering data representations, the point cloud has shown its straightforward capacity to depict fine geometry in many applications, including computer graphics, molecular structurology, modern sensing signal processing, and more. However, unlike computer graphs obtained with auxiliary regularization techniques or from syntheses, raw sensor/scanner (metric) data often contain natural random noise caused by multiple extrinsic factors, especially in the case of high-speed imaging scenarios. On the other hand, grid-like imaging techniques (e.g., RGB images or video frames) tend to entangle interesting aspects with environmental variations such as pose/illuminations with Euclidean sampling/processing pipelines. As one such typical problem, 3D Facial Expression Recognition (3D FER) has been developed into a new stage, with remaining difficulties involving the implementation of efficient feature abstraction methods for high dimensional observations and of stabilizing methods to obtain adequate robustness in cases of random exterior variations. In this paper, a localized and smoothed overlapping kernel is proposed to extract discriminative inherent geometric features. By association between the induced deformation stability and certain types of exterior perturbations through manifold scattering transform, we provide a novel framework that directly consumes point cloud coordinates for FER while requiring no predefined meshes or other features/signals. As a result, our compact framework achieves <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>78.33</mn><mo>%</mo></mrow></semantics></math></inline-formula> accuracy on the Bosphorus dataset for expression recognition challenge and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>77.55</mn><mo>%</mo></mrow></semantics></math></inline-formula> on 3D-BUFE. |
| first_indexed | 2024-03-09T18:40:18Z |
| format | Article |
| id | doaj.art-9f3ce12e6d5b40c09693a71dad25878f |
| institution | Directory Open Access Journal |
| issn | 1424-8220 |
| language | English |
| last_indexed | 2024-03-09T18:40:18Z |
| publishDate | 2022-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj.art-9f3ce12e6d5b40c09693a71dad25878f2023-11-24T06:45:56ZengMDPI AGSensors1424-82202022-10-012221829310.3390/s22218293Facial Expression Recognition with Geometric Scattering on 3D Point CloudsYi He0Keren Fu1Peng Cheng2Jianwei Zhang3National Key Laboratory of Fundamental Science on Synthetic Vision, Chengdu 610065, ChinaCollege of Computer Science, Sichuan University, Chengdu 610065, ChinaSchool of Aeronautics and Astronautics, Sichuan University, Chengdu 610065, ChinaCollege of Computer Science, Sichuan University, Chengdu 610065, ChinaAs one of the pioneering data representations, the point cloud has shown its straightforward capacity to depict fine geometry in many applications, including computer graphics, molecular structurology, modern sensing signal processing, and more. However, unlike computer graphs obtained with auxiliary regularization techniques or from syntheses, raw sensor/scanner (metric) data often contain natural random noise caused by multiple extrinsic factors, especially in the case of high-speed imaging scenarios. On the other hand, grid-like imaging techniques (e.g., RGB images or video frames) tend to entangle interesting aspects with environmental variations such as pose/illuminations with Euclidean sampling/processing pipelines. As one such typical problem, 3D Facial Expression Recognition (3D FER) has been developed into a new stage, with remaining difficulties involving the implementation of efficient feature abstraction methods for high dimensional observations and of stabilizing methods to obtain adequate robustness in cases of random exterior variations. In this paper, a localized and smoothed overlapping kernel is proposed to extract discriminative inherent geometric features. By association between the induced deformation stability and certain types of exterior perturbations through manifold scattering transform, we provide a novel framework that directly consumes point cloud coordinates for FER while requiring no predefined meshes or other features/signals. As a result, our compact framework achieves <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>78.33</mn><mo>%</mo></mrow></semantics></math></inline-formula> accuracy on the Bosphorus dataset for expression recognition challenge and <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>77.55</mn><mo>%</mo></mrow></semantics></math></inline-formula> on 3D-BUFE.https://www.mdpi.com/1424-8220/22/21/82933D facial expression recognitiongeometric scatteringpoint clouds |
| spellingShingle | Yi He Keren Fu Peng Cheng Jianwei Zhang Facial Expression Recognition with Geometric Scattering on 3D Point Clouds Sensors 3D facial expression recognition geometric scattering point clouds |
| title | Facial Expression Recognition with Geometric Scattering on 3D Point Clouds |
| title_full | Facial Expression Recognition with Geometric Scattering on 3D Point Clouds |
| title_fullStr | Facial Expression Recognition with Geometric Scattering on 3D Point Clouds |
| title_full_unstemmed | Facial Expression Recognition with Geometric Scattering on 3D Point Clouds |
| title_short | Facial Expression Recognition with Geometric Scattering on 3D Point Clouds |
| title_sort | facial expression recognition with geometric scattering on 3d point clouds |
| topic | 3D facial expression recognition geometric scattering point clouds |
| url | https://www.mdpi.com/1424-8220/22/21/8293 |
| work_keys_str_mv | AT yihe facialexpressionrecognitionwithgeometricscatteringon3dpointclouds AT kerenfu facialexpressionrecognitionwithgeometricscatteringon3dpointclouds AT pengcheng facialexpressionrecognitionwithgeometricscatteringon3dpointclouds AT jianweizhang facialexpressionrecognitionwithgeometricscatteringon3dpointclouds |