Conditional adversarial synthesis of 3D facial action unit
Employing deep learning-based approaches for fine-grained facial expression analysis, such as those involving the estimation of Action Unit (AU) intensities, is difficult due to the lack of a large-scale dataset of real faces with sufficiently diverse AU labels for training. In this paper, we consid...
| Autores principales: | , , , , |
|---|---|
| Otros Autores: | |
| Formato: | Journal Article |
| Lenguaje: | English |
| Publicado: |
2020
|
| Materias: | |
| Acceso en línea: | https://hdl.handle.net/10356/138268 |
| _version_ | 1826117330788155392 |
|---|---|
| author | Liu, Zhilei Song, Guoxian Cai, Jianfei Cham, Tat-Jen Zhang, Juyong |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Liu, Zhilei Song, Guoxian Cai, Jianfei Cham, Tat-Jen Zhang, Juyong |
| author_sort | Liu, Zhilei |
| collection | NTU |
| description | Employing deep learning-based approaches for fine-grained facial expression analysis, such as those involving the estimation of Action Unit (AU) intensities, is difficult due to the lack of a large-scale dataset of real faces with sufficiently diverse AU labels for training. In this paper, we consider how AU-level facial image synthesis can be used to substantially augment such a dataset. We propose an AU synthesis framework that combines the well-known 3D Morphable Model (3DMM), which intrinsically disentangles expression parameters from other face attributes, with models that adversarially generate 3DMM expression parameters conditioned on given target AU labels, in contrast to the more conventional approach of generating facial images directly. In this way, we are able to synthesize new combinations of expression parameters and facial images from desired AU labels. Extensive quantitative and qualitative results on the benchmark DISFA dataset and BP4D dataset demonstrate the effectiveness of our method on 3DMM facial expression parameter synthesis and data augmentation for deep learning-based AU intensity estimation. |
| first_indexed | 2024-10-01T04:25:51Z |
| format | Journal Article |
| id | ntu-10356/138268 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T04:25:51Z |
| publishDate | 2020 |
| record_format | dspace |
| spelling | ntu-10356/1382682020-04-30T01:46:04Z Conditional adversarial synthesis of 3D facial action unit Liu, Zhilei Song, Guoxian Cai, Jianfei Cham, Tat-Jen Zhang, Juyong School of Computer Science and Engineering Institute for Media Innovation (IMI) Engineering::Computer science and engineering FACS Action Unit Synthesis Employing deep learning-based approaches for fine-grained facial expression analysis, such as those involving the estimation of Action Unit (AU) intensities, is difficult due to the lack of a large-scale dataset of real faces with sufficiently diverse AU labels for training. In this paper, we consider how AU-level facial image synthesis can be used to substantially augment such a dataset. We propose an AU synthesis framework that combines the well-known 3D Morphable Model (3DMM), which intrinsically disentangles expression parameters from other face attributes, with models that adversarially generate 3DMM expression parameters conditioned on given target AU labels, in contrast to the more conventional approach of generating facial images directly. In this way, we are able to synthesize new combinations of expression parameters and facial images from desired AU labels. Extensive quantitative and qualitative results on the benchmark DISFA dataset and BP4D dataset demonstrate the effectiveness of our method on 3DMM facial expression parameter synthesis and data augmentation for deep learning-based AU intensity estimation. NRF (Natl Research Foundation, S’pore) MOE (Min. of Education, S’pore) 2020-04-30T01:46:03Z 2020-04-30T01:46:03Z 2019 Journal Article Liu, Z., Song, G., Cai, J., Cham, T.-J., & Zhang, J. (2019). Conditional adversarial synthesis of 3D facial action unit. Neurocomputing, 355, 200-208. doi:10.1016/j.neucom.2019.05.003 0925-2312 https://hdl.handle.net/10356/138268 10.1016/j.neucom.2019.05.003 2-s2.0-85065603291 355 200 208 en Neurocomputing © 2019 Elsevier B.V. All rights reserved. |
| spellingShingle | Engineering::Computer science and engineering FACS Action Unit Synthesis Liu, Zhilei Song, Guoxian Cai, Jianfei Cham, Tat-Jen Zhang, Juyong Conditional adversarial synthesis of 3D facial action unit |
| title | Conditional adversarial synthesis of 3D facial action unit |
| title_full | Conditional adversarial synthesis of 3D facial action unit |
| title_fullStr | Conditional adversarial synthesis of 3D facial action unit |
| title_full_unstemmed | Conditional adversarial synthesis of 3D facial action unit |
| title_short | Conditional adversarial synthesis of 3D facial action unit |
| title_sort | conditional adversarial synthesis of 3d facial action unit |
| topic | Engineering::Computer science and engineering FACS Action Unit Synthesis |
| url | https://hdl.handle.net/10356/138268 |
| work_keys_str_mv | AT liuzhilei conditionaladversarialsynthesisof3dfacialactionunit AT songguoxian conditionaladversarialsynthesisof3dfacialactionunit AT caijianfei conditionaladversarialsynthesisof3dfacialactionunit AT chamtatjen conditionaladversarialsynthesisof3dfacialactionunit AT zhangjuyong conditionaladversarialsynthesisof3dfacialactionunit |