Predicting Perceived Emotions in Animated GIFs with 3D Convolutional Neural Networks
© 2016 IEEE. Animated GIFs are widely used on the Internet to express emotions, but their automatic analysis is largely unexplored before. To help with the search and recommendation of GIFs, we aim to predict their emotions perceived by humans based on their contents. Since previous solutions to thi...
| Main Authors: | , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/138085 |
| _version_ | 1826208802017378304 |
|---|---|
| author | Chen, Weixuan Picard, Rosalind W. |
| author2 | Massachusetts Institute of Technology. Media Laboratory |
| author_facet | Massachusetts Institute of Technology. Media Laboratory Chen, Weixuan Picard, Rosalind W. |
| author_sort | Chen, Weixuan |
| collection | MIT |
| description | © 2016 IEEE. Animated GIFs are widely used on the Internet to express emotions, but their automatic analysis is largely unexplored before. To help with the search and recommendation of GIFs, we aim to predict their emotions perceived by humans based on their contents. Since previous solutions to this problem only utilize image-based features and lose all the motion information, we propose to use 3D convolutional neural networks (CNNs) to extract spatiotemporal features from GIFs. We evaluate our methodology on a crowd-sourcing platform called GIFGIF with more than 6000 animated GIFs, and achieve a better accuracy then any previous approach in predicting crowd-sourced intensity scores of 17 emotions. It is also found that our trained model can be used to distinguish and cluster emotions in terms of valence and risk perception. |
| first_indexed | 2024-09-23T14:12:32Z |
| format | Article |
| id | mit-1721.1/138085 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T14:12:32Z |
| publishDate | 2021 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1380852021-11-10T03:17:35Z Predicting Perceived Emotions in Animated GIFs with 3D Convolutional Neural Networks Chen, Weixuan Picard, Rosalind W. Massachusetts Institute of Technology. Media Laboratory © 2016 IEEE. Animated GIFs are widely used on the Internet to express emotions, but their automatic analysis is largely unexplored before. To help with the search and recommendation of GIFs, we aim to predict their emotions perceived by humans based on their contents. Since previous solutions to this problem only utilize image-based features and lose all the motion information, we propose to use 3D convolutional neural networks (CNNs) to extract spatiotemporal features from GIFs. We evaluate our methodology on a crowd-sourcing platform called GIFGIF with more than 6000 animated GIFs, and achieve a better accuracy then any previous approach in predicting crowd-sourced intensity scores of 17 emotions. It is also found that our trained model can be used to distinguish and cluster emotions in terms of valence and risk perception. 2021-11-09T21:46:56Z 2021-11-09T21:46:56Z 2016-12 2019-07-31T16:10:50Z Article http://purl.org/eprint/type/ConferencePaper https://hdl.handle.net/1721.1/138085 Chen, Weixuan and Picard, Rosalind W. 2016. "Predicting Perceived Emotions in Animated GIFs with 3D Convolutional Neural Networks." en 10.1109/ism.2016.0081 Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) MIT web domain |
| spellingShingle | Chen, Weixuan Picard, Rosalind W. Predicting Perceived Emotions in Animated GIFs with 3D Convolutional Neural Networks |
| title | Predicting Perceived Emotions in Animated GIFs with 3D Convolutional Neural Networks |
| title_full | Predicting Perceived Emotions in Animated GIFs with 3D Convolutional Neural Networks |
| title_fullStr | Predicting Perceived Emotions in Animated GIFs with 3D Convolutional Neural Networks |
| title_full_unstemmed | Predicting Perceived Emotions in Animated GIFs with 3D Convolutional Neural Networks |
| title_short | Predicting Perceived Emotions in Animated GIFs with 3D Convolutional Neural Networks |
| title_sort | predicting perceived emotions in animated gifs with 3d convolutional neural networks |
| url | https://hdl.handle.net/1721.1/138085 |
| work_keys_str_mv | AT chenweixuan predictingperceivedemotionsinanimatedgifswith3dconvolutionalneuralnetworks AT picardrosalindw predictingperceivedemotionsinanimatedgifswith3dconvolutionalneuralnetworks |