Compressed video action recognition with refined motion vectors
Although CNN has reached satisfactory performance in image-related tasks, using CNN to process videos is much more challenging due to the enormous size of raw video streams. In this Final Year Project, we propose to use motion vectors and residuals from modern video compression techniques to effecti...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project (FYP) |
| Language: | English |
| Published: |
2019
|
| Subjects: | |
| Online Access: | http://hdl.handle.net/10356/78259 |
| _version_ | 1826123581894950912 |
|---|---|
| author | Yu, Shining |
| author2 | Jiang Xudong |
| author_facet | Jiang Xudong Yu, Shining |
| author_sort | Yu, Shining |
| collection | NTU |
| description | Although CNN has reached satisfactory performance in image-related tasks, using CNN to process videos is much more challenging due to the enormous size of raw video streams. In this Final Year Project, we propose to use motion vectors and residuals from modern video compression techniques to effectively learn representation of the raw frames and greatly remove the temporal redundancy, giving faster video processing model. Compressed Video Action Recognition (CoViAR) has explored to directly use compressed video to train the deep neural network, where the motion vectors was utilized to present temporal information. However, motion vector is designed for minimizing video size where precious motion information are not obligatory. Compared with optical flow, motion vectors contain noisy and unreliable motion information. Inspired by the mechanism of video compression codecs, we propose an approach to refine the motion vectors where unreliable movement will be removed while temporal information is largely reserved. We prove that replacing the original motion vector with refined one and using the same network as CoViAR has achieved state-of-art performance on the UCF-101 and HMDB-51 with negligible efficiency degrades comparing with original CoViAR. |
| first_indexed | 2024-10-01T06:07:00Z |
| format | Final Year Project (FYP) |
| id | ntu-10356/78259 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T06:07:00Z |
| publishDate | 2019 |
| record_format | dspace |
| spelling | ntu-10356/782592023-07-07T17:43:11Z Compressed video action recognition with refined motion vectors Yu, Shining Jiang Xudong School of Electrical and Electronic Engineering DRNTU::Engineering::Electrical and electronic engineering Although CNN has reached satisfactory performance in image-related tasks, using CNN to process videos is much more challenging due to the enormous size of raw video streams. In this Final Year Project, we propose to use motion vectors and residuals from modern video compression techniques to effectively learn representation of the raw frames and greatly remove the temporal redundancy, giving faster video processing model. Compressed Video Action Recognition (CoViAR) has explored to directly use compressed video to train the deep neural network, where the motion vectors was utilized to present temporal information. However, motion vector is designed for minimizing video size where precious motion information are not obligatory. Compared with optical flow, motion vectors contain noisy and unreliable motion information. Inspired by the mechanism of video compression codecs, we propose an approach to refine the motion vectors where unreliable movement will be removed while temporal information is largely reserved. We prove that replacing the original motion vector with refined one and using the same network as CoViAR has achieved state-of-art performance on the UCF-101 and HMDB-51 with negligible efficiency degrades comparing with original CoViAR. Bachelor of Engineering (Electrical and Electronic Engineering) 2019-06-14T04:06:30Z 2019-06-14T04:06:30Z 2019 Final Year Project (FYP) http://hdl.handle.net/10356/78259 en Nanyang Technological University 62 p. application/pdf |
| spellingShingle | DRNTU::Engineering::Electrical and electronic engineering Yu, Shining Compressed video action recognition with refined motion vectors |
| title | Compressed video action recognition with refined motion vectors |
| title_full | Compressed video action recognition with refined motion vectors |
| title_fullStr | Compressed video action recognition with refined motion vectors |
| title_full_unstemmed | Compressed video action recognition with refined motion vectors |
| title_short | Compressed video action recognition with refined motion vectors |
| title_sort | compressed video action recognition with refined motion vectors |
| topic | DRNTU::Engineering::Electrical and electronic engineering |
| url | http://hdl.handle.net/10356/78259 |
| work_keys_str_mv | AT yushining compressedvideoactionrecognitionwithrefinedmotionvectors |