Data-Driven Advancements in Lip Motion Analysis: A Review
This work reviews the dataset-driven advancements that have occurred in the area of lip motion analysis, particularly visual lip-reading and visual lip motion authentication, in the deep learning era. We provide an analysis of datasets and their usage, creation, and associated challenges. Future res...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-11-01
|
| Series: | Electronics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2079-9292/12/22/4698 |
| _version_ | 1797459470590148608 |
|---|---|
| author | Shad Torrie Andrew Sumsion Dah-Jye Lee Zheng Sun |
| author_facet | Shad Torrie Andrew Sumsion Dah-Jye Lee Zheng Sun |
| author_sort | Shad Torrie |
| collection | DOAJ |
| description | This work reviews the dataset-driven advancements that have occurred in the area of lip motion analysis, particularly visual lip-reading and visual lip motion authentication, in the deep learning era. We provide an analysis of datasets and their usage, creation, and associated challenges. Future research can utilize this work as a guide for selecting appropriate datasets and as a source of insights for creating new and innovative datasets. Large and varied datasets are vital to a successful deep learning system. There have been many incredible advancements made in these fields due to larger datasets. There are indications that even larger, more varied datasets would result in further improvement upon existing systems. We highlight the datasets that brought about the progression in lip-reading systems from digit- to word-level lip-reading, and then from word- to sentence-level lip-reading. Through an in-depth analysis of lip-reading system results, we show that datasets with large amounts of diversity increase results immensely. We then discuss the next step for lip-reading systems to move from sentence- to dialogue-level lip-reading and emphasize that new datasets are required to make this transition possible. We then explore lip motion authentication datasets. While lip motion authentication has been well researched, it is not very unified on a particular implementation, and there is no benchmark dataset to compare the various methods. As was seen in the lip-reading analysis, large, diverse datasets are required to evaluate the robustness and accuracy of new methods attempted by researchers. These large datasets have pushed the work in the visual lip-reading realm. Due to the lack of large, diverse, and publicly accessible datasets, visual lip motion authentication research has struggled to validate results and real-world applications. A new benchmark dataset is required to unify the studies in this area such that they can be compared to previous methods as well as validate new methods more effectively. |
| first_indexed | 2024-03-09T16:51:48Z |
| format | Article |
| id | doaj.art-7e7e888643fd4a8ba94e56f92506542c |
| institution | Directory Open Access Journal |
| issn | 2079-9292 |
| language | English |
| last_indexed | 2024-03-09T16:51:48Z |
| publishDate | 2023-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Electronics |
| spelling | doaj.art-7e7e888643fd4a8ba94e56f92506542c2023-11-24T14:39:44ZengMDPI AGElectronics2079-92922023-11-011222469810.3390/electronics12224698Data-Driven Advancements in Lip Motion Analysis: A ReviewShad Torrie0Andrew Sumsion1Dah-Jye Lee2Zheng Sun3Department of Electrical and Computer Engineering, Brigham Young University, Provo, UT 84602, USADepartment of Electrical and Computer Engineering, Brigham Young University, Provo, UT 84602, USADepartment of Electrical and Computer Engineering, Brigham Young University, Provo, UT 84602, USADepartment of Electrical and Computer Engineering, Brigham Young University, Provo, UT 84602, USAThis work reviews the dataset-driven advancements that have occurred in the area of lip motion analysis, particularly visual lip-reading and visual lip motion authentication, in the deep learning era. We provide an analysis of datasets and their usage, creation, and associated challenges. Future research can utilize this work as a guide for selecting appropriate datasets and as a source of insights for creating new and innovative datasets. Large and varied datasets are vital to a successful deep learning system. There have been many incredible advancements made in these fields due to larger datasets. There are indications that even larger, more varied datasets would result in further improvement upon existing systems. We highlight the datasets that brought about the progression in lip-reading systems from digit- to word-level lip-reading, and then from word- to sentence-level lip-reading. Through an in-depth analysis of lip-reading system results, we show that datasets with large amounts of diversity increase results immensely. We then discuss the next step for lip-reading systems to move from sentence- to dialogue-level lip-reading and emphasize that new datasets are required to make this transition possible. We then explore lip motion authentication datasets. While lip motion authentication has been well researched, it is not very unified on a particular implementation, and there is no benchmark dataset to compare the various methods. As was seen in the lip-reading analysis, large, diverse datasets are required to evaluate the robustness and accuracy of new methods attempted by researchers. These large datasets have pushed the work in the visual lip-reading realm. Due to the lack of large, diverse, and publicly accessible datasets, visual lip motion authentication research has struggled to validate results and real-world applications. A new benchmark dataset is required to unify the studies in this area such that they can be compared to previous methods as well as validate new methods more effectively.https://www.mdpi.com/2079-9292/12/22/4698lip readingmachine visionbiometricsdatasetsdeep learning |
| spellingShingle | Shad Torrie Andrew Sumsion Dah-Jye Lee Zheng Sun Data-Driven Advancements in Lip Motion Analysis: A Review Electronics lip reading machine vision biometrics datasets deep learning |
| title | Data-Driven Advancements in Lip Motion Analysis: A Review |
| title_full | Data-Driven Advancements in Lip Motion Analysis: A Review |
| title_fullStr | Data-Driven Advancements in Lip Motion Analysis: A Review |
| title_full_unstemmed | Data-Driven Advancements in Lip Motion Analysis: A Review |
| title_short | Data-Driven Advancements in Lip Motion Analysis: A Review |
| title_sort | data driven advancements in lip motion analysis a review |
| topic | lip reading machine vision biometrics datasets deep learning |
| url | https://www.mdpi.com/2079-9292/12/22/4698 |
| work_keys_str_mv | AT shadtorrie datadrivenadvancementsinlipmotionanalysisareview AT andrewsumsion datadrivenadvancementsinlipmotionanalysisareview AT dahjyelee datadrivenadvancementsinlipmotionanalysisareview AT zhengsun datadrivenadvancementsinlipmotionanalysisareview |