Multimodel deception detection - are you telling a lie?
Deception detection plays a crucial role across various fields, evolving from traditional physical polygraphs to today’s machine learning techniques to analyze deceptive behaviors. Fraud can be detected through multiple modalities, including heart rate, EEG, blood pressure, facial micro-expressions,...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis-Master by Coursework |
| Language: | English |
| Published: |
Nanyang Technological University
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/181486 |
| _version_ | 1826127107461218304 |
|---|---|
| author | Yuan, Weiyun |
| author2 | Alex Chichung Kot |
| author_facet | Alex Chichung Kot Yuan, Weiyun |
| author_sort | Yuan, Weiyun |
| collection | NTU |
| description | Deception detection plays a crucial role across various fields, evolving from traditional physical polygraphs to today’s machine learning techniques to analyze deceptive behaviors. Fraud can be detected through multiple modalities, including heart rate, EEG, blood pressure, facial micro-expressions, and voice changes. This project introduces a multimodal deception detection system that utilizes two primary modalities: facial micro-expressions and voice. It integrates 2D and 3D ResNet models, trained on spectral data and video frames. Un- like most similar projects that primarily utilize Western face databases for train- ing, this project specifically focuses on deception detection among Asian populations, employing the ROSE Lab Vision2 dataset. This dataset encompasses three domains: China, India, and Malaysia. To enhance the baseline accuracy, the project employs a pre-training of multimodel using contrastive learning. Contrastive learning is employed to ascertain the correspondence between video and audio by training on the Asian Speaker dataset. This method enhances the model’s ability to discern the behavioral characteristics of Asians, and the trained weights are subsequently loaded into the fraud detection task to improve the prediction performance of the system. |
| first_indexed | 2025-03-09T14:28:40Z |
| format | Thesis-Master by Coursework |
| id | ntu-10356/181486 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2025-03-09T14:28:40Z |
| publishDate | 2024 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1814862024-12-06T15:49:15Z Multimodel deception detection - are you telling a lie? Yuan, Weiyun Alex Chichung Kot School of Electrical and Electronic Engineering DSO Rapid-Rich Object Search (ROSE) Lab EACKOT@ntu.edu.sg Computer and Information Science Engineering Machine learning Multimodel deception detection Contrastive learning Deception detection plays a crucial role across various fields, evolving from traditional physical polygraphs to today’s machine learning techniques to analyze deceptive behaviors. Fraud can be detected through multiple modalities, including heart rate, EEG, blood pressure, facial micro-expressions, and voice changes. This project introduces a multimodal deception detection system that utilizes two primary modalities: facial micro-expressions and voice. It integrates 2D and 3D ResNet models, trained on spectral data and video frames. Un- like most similar projects that primarily utilize Western face databases for train- ing, this project specifically focuses on deception detection among Asian populations, employing the ROSE Lab Vision2 dataset. This dataset encompasses three domains: China, India, and Malaysia. To enhance the baseline accuracy, the project employs a pre-training of multimodel using contrastive learning. Contrastive learning is employed to ascertain the correspondence between video and audio by training on the Asian Speaker dataset. This method enhances the model’s ability to discern the behavioral characteristics of Asians, and the trained weights are subsequently loaded into the fraud detection task to improve the prediction performance of the system. Master's degree 2024-12-04T05:51:49Z 2024-12-04T05:51:49Z 2024 Thesis-Master by Coursework Yuan, W. (2024). Multimodel deception detection - are you telling a lie?. Master's thesis, Nanyang Technological University, Singapore. https://hdl.handle.net/10356/181486 https://hdl.handle.net/10356/181486 en application/pdf Nanyang Technological University |
| spellingShingle | Computer and Information Science Engineering Machine learning Multimodel deception detection Contrastive learning Yuan, Weiyun Multimodel deception detection - are you telling a lie? |
| title | Multimodel deception detection - are you telling a lie? |
| title_full | Multimodel deception detection - are you telling a lie? |
| title_fullStr | Multimodel deception detection - are you telling a lie? |
| title_full_unstemmed | Multimodel deception detection - are you telling a lie? |
| title_short | Multimodel deception detection - are you telling a lie? |
| title_sort | multimodel deception detection are you telling a lie |
| topic | Computer and Information Science Engineering Machine learning Multimodel deception detection Contrastive learning |
| url | https://hdl.handle.net/10356/181486 |
| work_keys_str_mv | AT yuanweiyun multimodeldeceptiondetectionareyoutellingalie |