Analysis of Deep Learning-Based Decision-Making in an Emotional Spontaneous Speech Task
In this work, we present an approach to understand the computational methods and decision-making involved in the identification of emotions in spontaneous speech. The selected task consists of Spanish TV debates, which entail a high level of complexity as well as additional subjectivity in the human...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/13/2/980 |
| _version_ | 1797446555814330368 |
|---|---|
| author | Mikel de Velasco Raquel Justo Asier López Zorrilla María Inés Torres |
| author_facet | Mikel de Velasco Raquel Justo Asier López Zorrilla María Inés Torres |
| author_sort | Mikel de Velasco |
| collection | DOAJ |
| description | In this work, we present an approach to understand the computational methods and decision-making involved in the identification of emotions in spontaneous speech. The selected task consists of Spanish TV debates, which entail a high level of complexity as well as additional subjectivity in the human perception-based annotation procedure. A simple convolutional neural model is proposed, and its behaviour is analysed to explain its decision-making. The proposed model slightly outperforms commonly used CNN architectures such as VGG16, while being much lighter. Internal layer-by-layer transformations of the input spectrogram are visualised and analysed. Finally, a class model visualisation is proposed as a simple interpretation approach whose usefulness is assessed in the work. |
| first_indexed | 2024-03-09T13:43:11Z |
| format | Article |
| id | doaj.art-52f9b8c87551460eb2113493259e7f8b |
| institution | Directory Open Access Journal |
| issn | 2076-3417 |
| language | English |
| last_indexed | 2024-03-09T13:43:11Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj.art-52f9b8c87551460eb2113493259e7f8b2023-11-30T21:04:27ZengMDPI AGApplied Sciences2076-34172023-01-0113298010.3390/app13020980Analysis of Deep Learning-Based Decision-Making in an Emotional Spontaneous Speech TaskMikel de Velasco0Raquel Justo1Asier López Zorrilla2María Inés Torres3Department of Electricity and Electronics, Faculty of Science and Technology, University of the Basque Country UPV/EHU, 48940 Leioa, SpainDepartment of Electricity and Electronics, Faculty of Science and Technology, University of the Basque Country UPV/EHU, 48940 Leioa, SpainDepartment of Electricity and Electronics, Faculty of Science and Technology, University of the Basque Country UPV/EHU, 48940 Leioa, SpainDepartment of Electricity and Electronics, Faculty of Science and Technology, University of the Basque Country UPV/EHU, 48940 Leioa, SpainIn this work, we present an approach to understand the computational methods and decision-making involved in the identification of emotions in spontaneous speech. The selected task consists of Spanish TV debates, which entail a high level of complexity as well as additional subjectivity in the human perception-based annotation procedure. A simple convolutional neural model is proposed, and its behaviour is analysed to explain its decision-making. The proposed model slightly outperforms commonly used CNN architectures such as VGG16, while being much lighter. Internal layer-by-layer transformations of the input spectrogram are visualised and analysed. Finally, a class model visualisation is proposed as a simple interpretation approach whose usefulness is assessed in the work.https://www.mdpi.com/2076-3417/13/2/980emotion detectionspeech processingexplainable artificial intelligencemachine learning |
| spellingShingle | Mikel de Velasco Raquel Justo Asier López Zorrilla María Inés Torres Analysis of Deep Learning-Based Decision-Making in an Emotional Spontaneous Speech Task Applied Sciences emotion detection speech processing explainable artificial intelligence machine learning |
| title | Analysis of Deep Learning-Based Decision-Making in an Emotional Spontaneous Speech Task |
| title_full | Analysis of Deep Learning-Based Decision-Making in an Emotional Spontaneous Speech Task |
| title_fullStr | Analysis of Deep Learning-Based Decision-Making in an Emotional Spontaneous Speech Task |
| title_full_unstemmed | Analysis of Deep Learning-Based Decision-Making in an Emotional Spontaneous Speech Task |
| title_short | Analysis of Deep Learning-Based Decision-Making in an Emotional Spontaneous Speech Task |
| title_sort | analysis of deep learning based decision making in an emotional spontaneous speech task |
| topic | emotion detection speech processing explainable artificial intelligence machine learning |
| url | https://www.mdpi.com/2076-3417/13/2/980 |
| work_keys_str_mv | AT mikeldevelasco analysisofdeeplearningbaseddecisionmakinginanemotionalspontaneousspeechtask AT raqueljusto analysisofdeeplearningbaseddecisionmakinginanemotionalspontaneousspeechtask AT asierlopezzorrilla analysisofdeeplearningbaseddecisionmakinginanemotionalspontaneousspeechtask AT mariainestorres analysisofdeeplearningbaseddecisionmakinginanemotionalspontaneousspeechtask |