Video summarization via multiview representative selection
Video contents are inherently heterogeneous. To exploit different feature modalities in a diverse video collection for video summarization, we propose to formulate the task as a multiview representative selection problem. The goal is to select visual elements that are representative of a video consi...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2019
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/106096 http://hdl.handle.net/10220/48870 http://dx.doi.org/10.1109/TIP.2017.2789332 |
| _version_ | 1811694247344078848 |
|---|---|
| author | Meng, Jingjing Wang, Suchen Wang, Hongxing Yuan, Junsong Tan, Yap-Peng |
| author2 | School of Electrical and Electronic Engineering |
| author_facet | School of Electrical and Electronic Engineering Meng, Jingjing Wang, Suchen Wang, Hongxing Yuan, Junsong Tan, Yap-Peng |
| author_sort | Meng, Jingjing |
| collection | NTU |
| description | Video contents are inherently heterogeneous. To exploit different feature modalities in a diverse video collection for video summarization, we propose to formulate the task as a multiview representative selection problem. The goal is to select visual elements that are representative of a video consistently across different views (i.e., feature modalities). We present in this paper the multiview sparse dictionary selection with centroid co-regularization method, which optimizes the representative selection in each view, and enforces that the view-specific selections to be similar by regularizing them towards a consensus selection. We also introduce a diversity regularizer to favor a selection of diverse representatives. The problem can be efficiently solved by an alternating minimizing optimization with the fast iterative shrinkage thresholding algorithm. Experiments on synthetic data and benchmark video datasets validate the effectiveness of the proposed approach for video summarization, in comparison with other video summarization methods and representative selection methods such as K-medoids, sparse dictionary selection, and multiview clustering. |
| first_indexed | 2024-10-01T07:04:32Z |
| format | Journal Article |
| id | ntu-10356/106096 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T07:04:32Z |
| publishDate | 2019 |
| record_format | dspace |
| spelling | ntu-10356/1060962019-12-06T22:04:29Z Video summarization via multiview representative selection Meng, Jingjing Wang, Suchen Wang, Hongxing Yuan, Junsong Tan, Yap-Peng School of Electrical and Electronic Engineering Rapid-Rich Object Search Lab Video Summarization Multi-view DRNTU::Engineering::Electrical and electronic engineering Video contents are inherently heterogeneous. To exploit different feature modalities in a diverse video collection for video summarization, we propose to formulate the task as a multiview representative selection problem. The goal is to select visual elements that are representative of a video consistently across different views (i.e., feature modalities). We present in this paper the multiview sparse dictionary selection with centroid co-regularization method, which optimizes the representative selection in each view, and enforces that the view-specific selections to be similar by regularizing them towards a consensus selection. We also introduce a diversity regularizer to favor a selection of diverse representatives. The problem can be efficiently solved by an alternating minimizing optimization with the fast iterative shrinkage thresholding algorithm. Experiments on synthetic data and benchmark video datasets validate the effectiveness of the proposed approach for video summarization, in comparison with other video summarization methods and representative selection methods such as K-medoids, sparse dictionary selection, and multiview clustering. MOE (Min. of Education, S’pore) Accepted version 2019-06-20T06:01:31Z 2019-12-06T22:04:29Z 2019-06-20T06:01:31Z 2019-12-06T22:04:29Z 2018 Journal Article Meng, J., Wang, S., Wang, H., Yuan, J., & Tan, Y.-P. (2018). Video summarization via multiview representative selection. IEEE Transactions on Image Processing, 27(5), 2134-2145. doi:10.1109/TIP.2017.2789332 1057-7149 https://hdl.handle.net/10356/106096 http://hdl.handle.net/10220/48870 http://dx.doi.org/10.1109/TIP.2017.2789332 en IEEE Transactions on Image Processing © 2018 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. The published version is available at: https://doi.org/10.1109/TIP.2017.2789332 13 p. application/pdf |
| spellingShingle | Video Summarization Multi-view DRNTU::Engineering::Electrical and electronic engineering Meng, Jingjing Wang, Suchen Wang, Hongxing Yuan, Junsong Tan, Yap-Peng Video summarization via multiview representative selection |
| title | Video summarization via multiview representative selection |
| title_full | Video summarization via multiview representative selection |
| title_fullStr | Video summarization via multiview representative selection |
| title_full_unstemmed | Video summarization via multiview representative selection |
| title_short | Video summarization via multiview representative selection |
| title_sort | video summarization via multiview representative selection |
| topic | Video Summarization Multi-view DRNTU::Engineering::Electrical and electronic engineering |
| url | https://hdl.handle.net/10356/106096 http://hdl.handle.net/10220/48870 http://dx.doi.org/10.1109/TIP.2017.2789332 |
| work_keys_str_mv | AT mengjingjing videosummarizationviamultiviewrepresentativeselection AT wangsuchen videosummarizationviamultiviewrepresentativeselection AT wanghongxing videosummarizationviamultiviewrepresentativeselection AT yuanjunsong videosummarizationviamultiviewrepresentativeselection AT tanyappeng videosummarizationviamultiviewrepresentativeselection |