Data Driven 2-D-to-3-D Video Conversion for Soccer
© 1999-2012 IEEE. A wide adoption of 3-D videos is hindered by the lack of high-quality 3-D content. One promising solution to this problem is through data-driven 2-D-To-3-D video conversion. Such approaches are based on learning depth maps from a large dataset of 2-D+Depth images. However, current...
| Main Authors: | , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | English |
| Published: |
Institute of Electrical and Electronics Engineers (IEEE)
2021
|
| Online Access: | https://hdl.handle.net/1721.1/134843 |
| _version_ | 1826206104042864640 |
|---|---|
| author | Calagari, Kiana Elgharib, Mohamed Didyk, Piotr Kaspar, Alexandre Matusik, Wojciech Hefeeda, Mohamed |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Calagari, Kiana Elgharib, Mohamed Didyk, Piotr Kaspar, Alexandre Matusik, Wojciech Hefeeda, Mohamed |
| author_sort | Calagari, Kiana |
| collection | MIT |
| description | © 1999-2012 IEEE. A wide adoption of 3-D videos is hindered by the lack of high-quality 3-D content. One promising solution to this problem is through data-driven 2-D-To-3-D video conversion. Such approaches are based on learning depth maps from a large dataset of 2-D+Depth images. However, current conversion methods, while general, produce low-quality results with artifacts that are not acceptable to many viewers. We propose a novel, data-driven method for 2-D-To-3-D video conversion. Our method transfers the depth gradients from a large database of 2-D+Depth images. Capturing 2-D+Depth databases, however, are complex and costly, especially for outdoor sports games. We address this problem by creating a synthetic database from computer games and showing that this synthetic database can effectively be used to convert real videos. We propose a spatio-Temporal method to ensure the smoothness of the generated depth within individual frames and across successive frames. In addition, we present an object boundary detection method customized for 2-D-To-3-D conversion systems, which produces clear depth boundaries for players. We implement our method and validate it by conducting user studies that evaluate depth perception and visual comfort of the converted 3-D videos. We show that our method produces high-quality 3-D videos that are almost indistinguishable from videos shot by stereo cameras. In addition, our method significantly outperforms the current state-of-The-Art methods. For example, up to 20% improvement in the perceived depth is achieved by our method, which translates to improving the mean opinion score from good to excellent. |
| first_indexed | 2024-09-23T13:24:09Z |
| format | Article |
| id | mit-1721.1/134843 |
| institution | Massachusetts Institute of Technology |
| language | English |
| last_indexed | 2024-09-23T13:24:09Z |
| publishDate | 2021 |
| publisher | Institute of Electrical and Electronics Engineers (IEEE) |
| record_format | dspace |
| spelling | mit-1721.1/1348432023-11-03T15:26:41Z Data Driven 2-D-to-3-D Video Conversion for Soccer Calagari, Kiana Elgharib, Mohamed Didyk, Piotr Kaspar, Alexandre Matusik, Wojciech Hefeeda, Mohamed Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory © 1999-2012 IEEE. A wide adoption of 3-D videos is hindered by the lack of high-quality 3-D content. One promising solution to this problem is through data-driven 2-D-To-3-D video conversion. Such approaches are based on learning depth maps from a large dataset of 2-D+Depth images. However, current conversion methods, while general, produce low-quality results with artifacts that are not acceptable to many viewers. We propose a novel, data-driven method for 2-D-To-3-D video conversion. Our method transfers the depth gradients from a large database of 2-D+Depth images. Capturing 2-D+Depth databases, however, are complex and costly, especially for outdoor sports games. We address this problem by creating a synthetic database from computer games and showing that this synthetic database can effectively be used to convert real videos. We propose a spatio-Temporal method to ensure the smoothness of the generated depth within individual frames and across successive frames. In addition, we present an object boundary detection method customized for 2-D-To-3-D conversion systems, which produces clear depth boundaries for players. We implement our method and validate it by conducting user studies that evaluate depth perception and visual comfort of the converted 3-D videos. We show that our method produces high-quality 3-D videos that are almost indistinguishable from videos shot by stereo cameras. In addition, our method significantly outperforms the current state-of-The-Art methods. For example, up to 20% improvement in the perceived depth is achieved by our method, which translates to improving the mean opinion score from good to excellent. 2021-10-27T20:09:26Z 2021-10-27T20:09:26Z 2018 2019-06-21T16:15:24Z Article http://purl.org/eprint/type/ConferencePaper https://hdl.handle.net/1721.1/134843 en 10.1109/TMM.2017.2748458 IEEE Transactions on Multimedia Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Institute of Electrical and Electronics Engineers (IEEE) MIT web domain |
| spellingShingle | Calagari, Kiana Elgharib, Mohamed Didyk, Piotr Kaspar, Alexandre Matusik, Wojciech Hefeeda, Mohamed Data Driven 2-D-to-3-D Video Conversion for Soccer |
| title | Data Driven 2-D-to-3-D Video Conversion for Soccer |
| title_full | Data Driven 2-D-to-3-D Video Conversion for Soccer |
| title_fullStr | Data Driven 2-D-to-3-D Video Conversion for Soccer |
| title_full_unstemmed | Data Driven 2-D-to-3-D Video Conversion for Soccer |
| title_short | Data Driven 2-D-to-3-D Video Conversion for Soccer |
| title_sort | data driven 2 d to 3 d video conversion for soccer |
| url | https://hdl.handle.net/1721.1/134843 |
| work_keys_str_mv | AT calagarikiana datadriven2dto3dvideoconversionforsoccer AT elgharibmohamed datadriven2dto3dvideoconversionforsoccer AT didykpiotr datadriven2dto3dvideoconversionforsoccer AT kasparalexandre datadriven2dto3dvideoconversionforsoccer AT matusikwojciech datadriven2dto3dvideoconversionforsoccer AT hefeedamohamed datadriven2dto3dvideoconversionforsoccer |