Complex Emotion Profiling: An Incremental Active Learning Based Approach With Sparse Annotations
Generally, in-the-wild emotions are complex in nature. They often occur in combinations of multiple basic emotions, such as fear, happy, disgust, anger, sadness and surprise. Unlike the basic emotions, annotation of complex emotions, such as pain, is a time-consuming and expensive exercise. Moreover...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2020-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9165088/ |
| _version_ | 1830193348216881152 |
|---|---|
| author | Selvarajah Thuseethan Sutharshan Rajasegarar John Yearwood |
| author_facet | Selvarajah Thuseethan Sutharshan Rajasegarar John Yearwood |
| author_sort | Selvarajah Thuseethan |
| collection | DOAJ |
| description | Generally, in-the-wild emotions are complex in nature. They often occur in combinations of multiple basic emotions, such as fear, happy, disgust, anger, sadness and surprise. Unlike the basic emotions, annotation of complex emotions, such as pain, is a time-consuming and expensive exercise. Moreover, there is an increasing demand for profiling such complex emotions as they are useful in many real-world application domains, such as medical, psychology, security and computer science. The traditional emotion recognition systems require a significant amount of annotated training samples to understand the complex emotions. This limits the direct applicability of those methods for complex emotion detection from images and videos. Therefore, it is important to learn the profile of the in-the-wild complex emotions accurately using limited annotated samples. In this paper, we propose a deep framework to incrementally and actively profile in-the-wild complex emotions, from sparse data. Our approach consists of three major components, namely a pre-processing unit, an optimization unit and an active learning unit. The pre-processing unit removes the variations present in the complex emotion images extracted from an uncontrolled environment. Our novel incremental active learning algorithm along with an optimization unit effectively predicts the complex emotions present in-the-wild. Evaluation using multiple complex emotions benchmark datasets reveals that our proposed approach performs close to the human perception capability in effectively profiling complex emotions. Further, our proposed approach shows a significant performance enhancement, in comparison with the state-of-the-art deep networks and other benchmark complex emotion profiling approaches. |
| first_indexed | 2024-12-18T00:11:35Z |
| format | Article |
| id | doaj.art-0786791df8414057ae3e1ca5327529a1 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-18T00:11:35Z |
| publishDate | 2020-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-0786791df8414057ae3e1ca5327529a12022-12-21T21:27:40ZengIEEEIEEE Access2169-35362020-01-01814771114772710.1109/ACCESS.2020.30159179165088Complex Emotion Profiling: An Incremental Active Learning Based Approach With Sparse AnnotationsSelvarajah Thuseethan0https://orcid.org/0000-0001-6378-9940Sutharshan Rajasegarar1https://orcid.org/0000-0002-6559-6736John Yearwood2School of Information Technology, Deakin University, Geelong, VIC, AustraliaSchool of Information Technology, Deakin University, Geelong, VIC, AustraliaSchool of Information Technology, Deakin University, Geelong, VIC, AustraliaGenerally, in-the-wild emotions are complex in nature. They often occur in combinations of multiple basic emotions, such as fear, happy, disgust, anger, sadness and surprise. Unlike the basic emotions, annotation of complex emotions, such as pain, is a time-consuming and expensive exercise. Moreover, there is an increasing demand for profiling such complex emotions as they are useful in many real-world application domains, such as medical, psychology, security and computer science. The traditional emotion recognition systems require a significant amount of annotated training samples to understand the complex emotions. This limits the direct applicability of those methods for complex emotion detection from images and videos. Therefore, it is important to learn the profile of the in-the-wild complex emotions accurately using limited annotated samples. In this paper, we propose a deep framework to incrementally and actively profile in-the-wild complex emotions, from sparse data. Our approach consists of three major components, namely a pre-processing unit, an optimization unit and an active learning unit. The pre-processing unit removes the variations present in the complex emotion images extracted from an uncontrolled environment. Our novel incremental active learning algorithm along with an optimization unit effectively predicts the complex emotions present in-the-wild. Evaluation using multiple complex emotions benchmark datasets reveals that our proposed approach performs close to the human perception capability in effectively profiling complex emotions. Further, our proposed approach shows a significant performance enhancement, in comparison with the state-of-the-art deep networks and other benchmark complex emotion profiling approaches.https://ieeexplore.ieee.org/document/9165088/Active learningcomplex emotionsemotion recognitionincremental learningsparse data |
| spellingShingle | Selvarajah Thuseethan Sutharshan Rajasegarar John Yearwood Complex Emotion Profiling: An Incremental Active Learning Based Approach With Sparse Annotations IEEE Access Active learning complex emotions emotion recognition incremental learning sparse data |
| title | Complex Emotion Profiling: An Incremental Active Learning Based Approach With Sparse Annotations |
| title_full | Complex Emotion Profiling: An Incremental Active Learning Based Approach With Sparse Annotations |
| title_fullStr | Complex Emotion Profiling: An Incremental Active Learning Based Approach With Sparse Annotations |
| title_full_unstemmed | Complex Emotion Profiling: An Incremental Active Learning Based Approach With Sparse Annotations |
| title_short | Complex Emotion Profiling: An Incremental Active Learning Based Approach With Sparse Annotations |
| title_sort | complex emotion profiling an incremental active learning based approach with sparse annotations |
| topic | Active learning complex emotions emotion recognition incremental learning sparse data |
| url | https://ieeexplore.ieee.org/document/9165088/ |
| work_keys_str_mv | AT selvarajahthuseethan complexemotionprofilinganincrementalactivelearningbasedapproachwithsparseannotations AT sutharshanrajasegarar complexemotionprofilinganincrementalactivelearningbasedapproachwithsparseannotations AT johnyearwood complexemotionprofilinganincrementalactivelearningbasedapproachwithsparseannotations |