Gradient-based 2D-to-3D Conversion for Soccer Videos
A wide spread adoption of 3D videos and technologies is hindered by the lack of high-quality 3D content. One promising solution to address this problem is to use automated 2D-to-3D conversion. However, current conversion methods, while general, produce low-quality results with artifacts that are not...
| Main Authors: | , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Language: | en_US |
| Published: |
Association for Computing Machinery (ACM)
2015
|
| Online Access: | http://hdl.handle.net/1721.1/99743 https://orcid.org/0000-0002-6090-5392 https://orcid.org/0000-0003-0212-5643 |
| _version_ | 1811074468393517056 |
|---|---|
| author | Calagari, Kiana Elgharib, Mohamed Didyk, Piotr Kaspar, Alexandre Matusik, Wojciech Hefeeda, Mohamed |
| author2 | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory |
| author_facet | Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Calagari, Kiana Elgharib, Mohamed Didyk, Piotr Kaspar, Alexandre Matusik, Wojciech Hefeeda, Mohamed |
| author_sort | Calagari, Kiana |
| collection | MIT |
| description | A wide spread adoption of 3D videos and technologies is hindered by the lack of high-quality 3D content. One promising solution to address this problem is to use automated 2D-to-3D conversion. However, current conversion methods, while general, produce low-quality results with artifacts that are not acceptable to many viewers. We address this problem by showing how to construct a high-quality, domain-specific conversion method for soccer videos. We propose a novel, data-driven method that generates stereoscopic frames by transferring depth information from similar frames in a database of 3D stereoscopic videos. Creating a database of 3D stereoscopic videos with accurate depth is, however, very difficult. One of the key findings in this paper is showing that computer generated content in current sports computer games can be used to generate high-quality 3D video reference database for 2D-to-3D conversion methods. Once we retrieve similar 3D video frames, our technique transfers depth gradients to the target frame while respecting object boundaries. It then computes depth maps from the gradients, and generates the output stereoscopic video. We implement our method and validate it by conducting user-studies that evaluate depth perception and visual comfort of the converted 3D videos. We show that our method produces high-quality 3D videos that are almost indistinguishable from videos shot by stereo cameras. In addition, our method significantly outperforms the current state-of-the-art method. For example, up to 20% improvement in the perceived depth is achieved by our method, which translates to improving the mean opinion score from Good to Excellent. |
| first_indexed | 2024-09-23T09:49:55Z |
| format | Article |
| id | mit-1721.1/99743 |
| institution | Massachusetts Institute of Technology |
| language | en_US |
| last_indexed | 2024-09-23T09:49:55Z |
| publishDate | 2015 |
| publisher | Association for Computing Machinery (ACM) |
| record_format | dspace |
| spelling | mit-1721.1/997432022-09-30T17:05:49Z Gradient-based 2D-to-3D Conversion for Soccer Videos Calagari, Kiana Elgharib, Mohamed Didyk, Piotr Kaspar, Alexandre Matusik, Wojciech Hefeeda, Mohamed Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science Kaspar, Alexandre Matusik, Wojciech A wide spread adoption of 3D videos and technologies is hindered by the lack of high-quality 3D content. One promising solution to address this problem is to use automated 2D-to-3D conversion. However, current conversion methods, while general, produce low-quality results with artifacts that are not acceptable to many viewers. We address this problem by showing how to construct a high-quality, domain-specific conversion method for soccer videos. We propose a novel, data-driven method that generates stereoscopic frames by transferring depth information from similar frames in a database of 3D stereoscopic videos. Creating a database of 3D stereoscopic videos with accurate depth is, however, very difficult. One of the key findings in this paper is showing that computer generated content in current sports computer games can be used to generate high-quality 3D video reference database for 2D-to-3D conversion methods. Once we retrieve similar 3D video frames, our technique transfers depth gradients to the target frame while respecting object boundaries. It then computes depth maps from the gradients, and generates the output stereoscopic video. We implement our method and validate it by conducting user-studies that evaluate depth perception and visual comfort of the converted 3D videos. We show that our method produces high-quality 3D videos that are almost indistinguishable from videos shot by stereo cameras. In addition, our method significantly outperforms the current state-of-the-art method. For example, up to 20% improvement in the perceived depth is achieved by our method, which translates to improving the mean opinion score from Good to Excellent. Qatar Computing Research Institute-CSAIL Partnership National Science Foundation (U.S.) (Grant IIS-1111415) 2015-11-09T13:09:54Z 2015-11-09T13:09:54Z 2015-10 Article http://purl.org/eprint/type/ConferencePaper 978-1-4503-3459-4 http://hdl.handle.net/1721.1/99743 Calagari, Kiana, Mohamed Elgharib, Piotr Didyk, Alexandre Kaspar, Wojciech Matusik, and Mohamed Hefeeda. "Gradient-based 2D-to-3D Conversion for Soccer Videos." 23rd ACM International Conference on Multimedia (October 2015). https://orcid.org/0000-0002-6090-5392 https://orcid.org/0000-0003-0212-5643 en_US http://acmmm.hosting.acm.org/2015/wp-content/uploads/102617-ACM-MM15-d5web.pdf Proceedings of the 23rd ACM International Conference on Multimedia Creative Commons Attribution-Noncommercial-Share Alike http://creativecommons.org/licenses/by-nc-sa/4.0/ application/pdf Association for Computing Machinery (ACM) Kaspar |
| spellingShingle | Calagari, Kiana Elgharib, Mohamed Didyk, Piotr Kaspar, Alexandre Matusik, Wojciech Hefeeda, Mohamed Gradient-based 2D-to-3D Conversion for Soccer Videos |
| title | Gradient-based 2D-to-3D Conversion for Soccer Videos |
| title_full | Gradient-based 2D-to-3D Conversion for Soccer Videos |
| title_fullStr | Gradient-based 2D-to-3D Conversion for Soccer Videos |
| title_full_unstemmed | Gradient-based 2D-to-3D Conversion for Soccer Videos |
| title_short | Gradient-based 2D-to-3D Conversion for Soccer Videos |
| title_sort | gradient based 2d to 3d conversion for soccer videos |
| url | http://hdl.handle.net/1721.1/99743 https://orcid.org/0000-0002-6090-5392 https://orcid.org/0000-0003-0212-5643 |
| work_keys_str_mv | AT calagarikiana gradientbased2dto3dconversionforsoccervideos AT elgharibmohamed gradientbased2dto3dconversionforsoccervideos AT didykpiotr gradientbased2dto3dconversionforsoccervideos AT kasparalexandre gradientbased2dto3dconversionforsoccervideos AT matusikwojciech gradientbased2dto3dconversionforsoccervideos AT hefeedamohamed gradientbased2dto3dconversionforsoccervideos |