Deep-learning based affective video analysis and synthesis

The major challenge in computational creativity within the context of audio-visual analysis, is the difficulty in extracting high quality content from large quantities of video footage. Current development focuses on using submodular optimization of frame-based quality-aware relevance model to creat...

Full description

Bibliographic Details
Main Author: Tan, Christopher Say Wei
Other Authors: Yu Han
Format: Final Year Project (FYP)
Language:English
Published: Nanyang Technological University 2020
Subjects:
Online Access:https://hdl.handle.net/10356/144505
_version_ 1826111436478218240
author Tan, Christopher Say Wei
author2 Yu Han
author_facet Yu Han
Tan, Christopher Say Wei
author_sort Tan, Christopher Say Wei
collection NTU
description The major challenge in computational creativity within the context of audio-visual analysis, is the difficulty in extracting high quality content from large quantities of video footage. Current development focuses on using submodular optimization of frame-based quality-aware relevance model to create summaries which are both diverse and representative of the entire video footage. Our work complements on existing work on query-adaptive video summarization, where we implement the Natural Language Toolkit and Rapid Automatic Keyword Extraction algorithm to extract keywords for query generation. The query is used in the Quality-Aware Relevance Estimation model for thumbnail selection. The generated thumbnails will identify key scenes in the video footage which will be subsequently summarized and merged by weighted sampling of the key scenes to the length of a short summary. We found that our video summary has more related scenes, higher average similarity score with key words compared to baseline, and it also improves on the average qualitative aspects of the summary.
first_indexed 2024-10-01T02:50:41Z
format Final Year Project (FYP)
id ntu-10356/144505
institution Nanyang Technological University
language English
last_indexed 2024-10-01T02:50:41Z
publishDate 2020
publisher Nanyang Technological University
record_format dspace
spelling ntu-10356/1445052020-11-10T04:54:56Z Deep-learning based affective video analysis and synthesis Tan, Christopher Say Wei Yu Han School of Computer Science and Engineering han.yu@ntu.edu.sg Engineering::Computer science and engineering The major challenge in computational creativity within the context of audio-visual analysis, is the difficulty in extracting high quality content from large quantities of video footage. Current development focuses on using submodular optimization of frame-based quality-aware relevance model to create summaries which are both diverse and representative of the entire video footage. Our work complements on existing work on query-adaptive video summarization, where we implement the Natural Language Toolkit and Rapid Automatic Keyword Extraction algorithm to extract keywords for query generation. The query is used in the Quality-Aware Relevance Estimation model for thumbnail selection. The generated thumbnails will identify key scenes in the video footage which will be subsequently summarized and merged by weighted sampling of the key scenes to the length of a short summary. We found that our video summary has more related scenes, higher average similarity score with key words compared to baseline, and it also improves on the average qualitative aspects of the summary. Bachelor of Engineering (Computer Science) 2020-11-10T04:54:56Z 2020-11-10T04:54:56Z 2020 Final Year Project (FYP) https://hdl.handle.net/10356/144505 en SCE19-0630 application/pdf Nanyang Technological University
spellingShingle Engineering::Computer science and engineering
Tan, Christopher Say Wei
Deep-learning based affective video analysis and synthesis
title Deep-learning based affective video analysis and synthesis
title_full Deep-learning based affective video analysis and synthesis
title_fullStr Deep-learning based affective video analysis and synthesis
title_full_unstemmed Deep-learning based affective video analysis and synthesis
title_short Deep-learning based affective video analysis and synthesis
title_sort deep learning based affective video analysis and synthesis
topic Engineering::Computer science and engineering
url https://hdl.handle.net/10356/144505
work_keys_str_mv AT tanchristophersaywei deeplearningbasedaffectivevideoanalysisandsynthesis