Discovering thematic visual objects in unconstrained videos
Over the last decade, with the popularization of camera-equipped devices, there has been an explosive growth of video data. Despite the diverse visual contents, there are usually some thematic objects in these videos. As the key objects to be presented, thematic objects appear frequently and occ...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
2018
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/10356/75713 |
| _version_ | 1811688400396222464 |
|---|---|
| author | Yang, Jiong |
| author2 | Yuan Junsong |
| author_facet | Yuan Junsong Yang, Jiong |
| author_sort | Yang, Jiong |
| collection | NTU |
| description | Over the last decade, with the popularization of camera-equipped devices,
there has been an explosive growth of video data. Despite the diverse
visual contents, there are usually some thematic objects in these videos. As
the key objects to be presented, thematic objects appear frequently and
occupy highlighted positions in the video scenes, thus retain our impression
after watching the videos, such as the bride and the groom in wedding
ceremony videos, the birthday girl in birthday party videos, or product
logo in commercial videos. Automatically discovering and localizing these
thematic objects can benefit many real-world applications, such as video
summarization, search, and labeling. However, this task is challenging as
there is no prior information or initialization about the thematic objects.
Moreover, there is usually background clutter, occlusions, or camera motions
accompanying the targets. In this thesis, a systematic study is conducted on
the automatic discovery and localization of thematic objects in videos.
We have studied this problem under various settings, including automatic
discovery and localization of the thematic object in single videos, automatic
discovery and segmentation of the thematic object in single videos, and automatic
thematic action discovery and localization in collections of videos. In
the absence of category-specific supervision and manual initialization, various
category-independent cues have been explored to discover and localize the
thematic objects. These include the spatiotemporal saliency to highlight
regions with salient appearance or motion with respect to the background,
temporal smoothness of spatial locations and appearance variations along
the object moving trajectory, and global appearance consistency of the object
throughout its presence. When the discovery is performed in video collections
instead of single videos, the semantic similarities in terms of appearance
and/or motion patterns of the objects between different videos are also important.
Novel techniques are proposed in this thesis to improve the reliability
and efficiency of these cues as well as how they can be better explored to
improve the discovery and localization performance. Extensive evaluations
on both benchmarking as well as newly proposed datasets demonstrate the
usefulness of these proposed methods as well as their superiority over existing
approaches. |
| first_indexed | 2024-10-01T05:31:36Z |
| format | Thesis |
| id | ntu-10356/75713 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T05:31:36Z |
| publishDate | 2018 |
| record_format | dspace |
| spelling | ntu-10356/757132021-03-20T13:18:50Z Discovering thematic visual objects in unconstrained videos Yang, Jiong Yuan Junsong Interdisciplinary Graduate School (IGS) DRNTU::Engineering::Electrical and electronic engineering Over the last decade, with the popularization of camera-equipped devices, there has been an explosive growth of video data. Despite the diverse visual contents, there are usually some thematic objects in these videos. As the key objects to be presented, thematic objects appear frequently and occupy highlighted positions in the video scenes, thus retain our impression after watching the videos, such as the bride and the groom in wedding ceremony videos, the birthday girl in birthday party videos, or product logo in commercial videos. Automatically discovering and localizing these thematic objects can benefit many real-world applications, such as video summarization, search, and labeling. However, this task is challenging as there is no prior information or initialization about the thematic objects. Moreover, there is usually background clutter, occlusions, or camera motions accompanying the targets. In this thesis, a systematic study is conducted on the automatic discovery and localization of thematic objects in videos. We have studied this problem under various settings, including automatic discovery and localization of the thematic object in single videos, automatic discovery and segmentation of the thematic object in single videos, and automatic thematic action discovery and localization in collections of videos. In the absence of category-specific supervision and manual initialization, various category-independent cues have been explored to discover and localize the thematic objects. These include the spatiotemporal saliency to highlight regions with salient appearance or motion with respect to the background, temporal smoothness of spatial locations and appearance variations along the object moving trajectory, and global appearance consistency of the object throughout its presence. When the discovery is performed in video collections instead of single videos, the semantic similarities in terms of appearance and/or motion patterns of the objects between different videos are also important. Novel techniques are proposed in this thesis to improve the reliability and efficiency of these cues as well as how they can be better explored to improve the discovery and localization performance. Extensive evaluations on both benchmarking as well as newly proposed datasets demonstrate the usefulness of these proposed methods as well as their superiority over existing approaches. Doctor of Philosophy (IGS) 2018-06-10T15:17:15Z 2018-06-10T15:17:15Z 2018 Thesis Yang, J. (2018). Discovering thematic visual objects in unconstrained videos. Doctoral thesis, Nanyang Technological University, Singapore. http://hdl.handle.net/10356/75713 10.32657/10356/75713 en 184 p. application/pdf |
| spellingShingle | DRNTU::Engineering::Electrical and electronic engineering Yang, Jiong Discovering thematic visual objects in unconstrained videos |
| title | Discovering thematic visual objects in unconstrained videos |
| title_full | Discovering thematic visual objects in unconstrained videos |
| title_fullStr | Discovering thematic visual objects in unconstrained videos |
| title_full_unstemmed | Discovering thematic visual objects in unconstrained videos |
| title_short | Discovering thematic visual objects in unconstrained videos |
| title_sort | discovering thematic visual objects in unconstrained videos |
| topic | DRNTU::Engineering::Electrical and electronic engineering |
| url | http://hdl.handle.net/10356/75713 |
| work_keys_str_mv | AT yangjiong discoveringthematicvisualobjectsinunconstrainedvideos |