Optimal region selection for stereoscopic video subtitle insertion
Stereoscopic subtitle insertion is a fundamental and essential element in stereoscopic film and TV industry. However, little work has been dedicated to the optimal region selection for stereoscopic subtitle insertion. In addition, there is no public database reported for the performance evaluation o...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Journal Article |
| Language: | English |
| Published: |
2020
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/140153 |
| _version_ | 1826126174967824384 |
|---|---|
| author | Yue, Guanghui Hou, Chunping Lei, Jianjun Fang, Yuming Lin, Weisi |
| author2 | School of Computer Science and Engineering |
| author_facet | School of Computer Science and Engineering Yue, Guanghui Hou, Chunping Lei, Jianjun Fang, Yuming Lin, Weisi |
| author_sort | Yue, Guanghui |
| collection | NTU |
| description | Stereoscopic subtitle insertion is a fundamental and essential element in stereoscopic film and TV industry. However, little work has been dedicated to the optimal region selection for stereoscopic subtitle insertion. In addition, there is no public database reported for the performance evaluation of it. In this paper, we build the first large-scale video database (TJU3D) for stereoscopic video subtitle insertion, which includes 50 video sequences with rich screen scenes. Compared with 2D subtitle region selection, there are several problems we have to consider in stereoscopic subtitle region selection: 1) the subtitle should avoid depth cue collision and occlusion from objects in stereoscopic video sequences; 2) the disparity value of the subtitle must be minimized to reduce visual discomfort; and 3) the temporal coherence constraint must be considered during region selection for subtitles in video sequences. By considering these constraints, we propose an optimal region selection algorithm for stereoscopic subtitle insertion. First, we compute the disparity map of each video frame in video sequences. For each frame, the optimal position and disparity value of the subtitle are determined by a subtitle region selection algorithm, which contains two parts (i.e., the coarse selection and fine selection). After that, by considering the temporal consistency between adjacent frames, the position and disparity value of each frame are further classified and processed in order to avoid the subtitle jitter. We evaluate the proposed method on TJU3D video database through two visual discomfort prediction metrics and one subjective experiment. To further verify the effectiveness of the proposed method, we also validate the performance of the proposed method on video comfort assessment database, i.e., IEEE-SA Stereo Database. Experimental results demonstrate that the visual discomfort is greatly reduced when using the proposed method compared with the basic method. |
| first_indexed | 2024-10-01T06:48:23Z |
| format | Journal Article |
| id | ntu-10356/140153 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T06:48:23Z |
| publishDate | 2020 |
| record_format | dspace |
| spelling | ntu-10356/1401532020-05-27T02:46:46Z Optimal region selection for stereoscopic video subtitle insertion Yue, Guanghui Hou, Chunping Lei, Jianjun Fang, Yuming Lin, Weisi School of Computer Science and Engineering Engineering::Computer science and engineering Stereoscopic Subtitle Visual Discomfort Stereoscopic subtitle insertion is a fundamental and essential element in stereoscopic film and TV industry. However, little work has been dedicated to the optimal region selection for stereoscopic subtitle insertion. In addition, there is no public database reported for the performance evaluation of it. In this paper, we build the first large-scale video database (TJU3D) for stereoscopic video subtitle insertion, which includes 50 video sequences with rich screen scenes. Compared with 2D subtitle region selection, there are several problems we have to consider in stereoscopic subtitle region selection: 1) the subtitle should avoid depth cue collision and occlusion from objects in stereoscopic video sequences; 2) the disparity value of the subtitle must be minimized to reduce visual discomfort; and 3) the temporal coherence constraint must be considered during region selection for subtitles in video sequences. By considering these constraints, we propose an optimal region selection algorithm for stereoscopic subtitle insertion. First, we compute the disparity map of each video frame in video sequences. For each frame, the optimal position and disparity value of the subtitle are determined by a subtitle region selection algorithm, which contains two parts (i.e., the coarse selection and fine selection). After that, by considering the temporal consistency between adjacent frames, the position and disparity value of each frame are further classified and processed in order to avoid the subtitle jitter. We evaluate the proposed method on TJU3D video database through two visual discomfort prediction metrics and one subjective experiment. To further verify the effectiveness of the proposed method, we also validate the performance of the proposed method on video comfort assessment database, i.e., IEEE-SA Stereo Database. Experimental results demonstrate that the visual discomfort is greatly reduced when using the proposed method compared with the basic method. 2020-05-27T02:46:45Z 2020-05-27T02:46:45Z 2017 Journal Article Yue, G., Hou, C., Lei, J., Fang, Y., & Lin, W. (2018). Optimal region selection for stereoscopic video subtitle insertion. IEEE Transactions on Circuits and Systems for Video Technology, 28(11), 3141-3153. doi:10.1109/TCSVT.2017.2739756 1051-8215 https://hdl.handle.net/10356/140153 10.1109/TCSVT.2017.2739756 2-s2.0-85028457246 11 28 3141 3153 en IEEE Transactions on Circuits and Systems for Video Technology © 2017 IEEE. All rights reserved. |
| spellingShingle | Engineering::Computer science and engineering Stereoscopic Subtitle Visual Discomfort Yue, Guanghui Hou, Chunping Lei, Jianjun Fang, Yuming Lin, Weisi Optimal region selection for stereoscopic video subtitle insertion |
| title | Optimal region selection for stereoscopic video subtitle insertion |
| title_full | Optimal region selection for stereoscopic video subtitle insertion |
| title_fullStr | Optimal region selection for stereoscopic video subtitle insertion |
| title_full_unstemmed | Optimal region selection for stereoscopic video subtitle insertion |
| title_short | Optimal region selection for stereoscopic video subtitle insertion |
| title_sort | optimal region selection for stereoscopic video subtitle insertion |
| topic | Engineering::Computer science and engineering Stereoscopic Subtitle Visual Discomfort |
| url | https://hdl.handle.net/10356/140153 |
| work_keys_str_mv | AT yueguanghui optimalregionselectionforstereoscopicvideosubtitleinsertion AT houchunping optimalregionselectionforstereoscopicvideosubtitleinsertion AT leijianjun optimalregionselectionforstereoscopicvideosubtitleinsertion AT fangyuming optimalregionselectionforstereoscopicvideosubtitleinsertion AT linweisi optimalregionselectionforstereoscopicvideosubtitleinsertion |