Video Reflection Removal Through Spatio-Temporal Optimization
© 2017 IEEE. Reflections can obstruct content during video capture and hence their removal is desirable. Current removal techniques are designed for still images, extracting only one reflection (foreground) and one background layer from the input. When extended to videos, unpleasant artifacts such a...
| Main Authors: | , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021
|
| Online Access: | https://hdl.handle.net/1721.1/137933 |
| _version_ | 1811074394891485184 |
|---|---|
| author | Nandoriya, Ajay Elgharib, Mohamed Kim, Changil Hefeeda, Mohamed Matusik, Wojciech |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Nandoriya, Ajay Elgharib, Mohamed Kim, Changil Hefeeda, Mohamed Matusik, Wojciech |
| author_sort | Nandoriya, Ajay |
| collection | MIT |
| description | © 2017 IEEE. Reflections can obstruct content during video capture and hence their removal is desirable. Current removal techniques are designed for still images, extracting only one reflection (foreground) and one background layer from the input. When extended to videos, unpleasant artifacts such as temporal flickering and incomplete separation are generated. We present a technique for video reflection removal by jointly solving for motion and separation. The novelty of our work is in our optimization formulation as well as the motion initialization strategy. We present a novel spatiotemporal optimization that takes n frames as input and directly estimates 2n frames as output, n for each layer. We aim to fully utilize spatio-temporal information in our objective terms. Our motion initialization is based on iterative frame-to-frame alignment instead of the direct alignment used by current approaches. We compare against advanced video extensions of the state of the art, and we significantly reduce temporal flickering and improve separation. In addition, we reduce image blur and recover moving objects more accurately. We validate our approach through subjective and objective evaluations on real and controlled data. |
| first_indexed | 2024-09-23T09:48:46Z |
| format | Article |
| id | mit-1721.1/137933 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T09:48:46Z |
| publishDate | 2021 |
| publisher | IEEE |
| record_format | dspace |
| spelling | mit-1721.1/1379332023-02-08T21:09:36Z Video Reflection Removal Through Spatio-Temporal Optimization Nandoriya, Ajay Elgharib, Mohamed Kim, Changil Hefeeda, Mohamed Matusik, Wojciech Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory © 2017 IEEE. Reflections can obstruct content during video capture and hence their removal is desirable. Current removal techniques are designed for still images, extracting only one reflection (foreground) and one background layer from the input. When extended to videos, unpleasant artifacts such as temporal flickering and incomplete separation are generated. We present a technique for video reflection removal by jointly solving for motion and separation. The novelty of our work is in our optimization formulation as well as the motion initialization strategy. We present a novel spatiotemporal optimization that takes n frames as input and directly estimates 2n frames as output, n for each layer. We aim to fully utilize spatio-temporal information in our objective terms. Our motion initialization is based on iterative frame-to-frame alignment instead of the direct alignment used by current approaches. We compare against advanced video extensions of the state of the art, and we significantly reduce temporal flickering and improve separation. In addition, we reduce image blur and recover moving objects more accurately. We validate our approach through subjective and objective evaluations on real and controlled data. 2021-11-09T15:48:12Z 2021-11-09T15:48:12Z 2017-10 2019-06-21T15:54:13Z Article http://purl.org/eprint/type/ConferencePaper https://hdl.handle.net/1721.1/137933 Nandoriya, Ajay, Elgharib, Mohamed, Kim, Changil, Hefeeda, Mohamed and Matusik, Wojciech. 2017. "Video Reflection Removal Through Spatio-Temporal Optimization." en 10.1109/iccv.2017.264 Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf IEEE MIT web domain |
| spellingShingle | Nandoriya, Ajay Elgharib, Mohamed Kim, Changil Hefeeda, Mohamed Matusik, Wojciech Video Reflection Removal Through Spatio-Temporal Optimization |
| title | Video Reflection Removal Through Spatio-Temporal Optimization |
| title_full | Video Reflection Removal Through Spatio-Temporal Optimization |
| title_fullStr | Video Reflection Removal Through Spatio-Temporal Optimization |
| title_full_unstemmed | Video Reflection Removal Through Spatio-Temporal Optimization |
| title_short | Video Reflection Removal Through Spatio-Temporal Optimization |
| title_sort | video reflection removal through spatio temporal optimization |
| url | https://hdl.handle.net/1721.1/137933 |
| work_keys_str_mv | AT nandoriyaajay videoreflectionremovalthroughspatiotemporaloptimization AT elgharibmohamed videoreflectionremovalthroughspatiotemporaloptimization AT kimchangil videoreflectionremovalthroughspatiotemporaloptimization AT hefeedamohamed videoreflectionremovalthroughspatiotemporaloptimization AT matusikwojciech videoreflectionremovalthroughspatiotemporaloptimization |