Real-time multi-GPU-based 8KVR stitching and streaming on 5G MEC/Cloud environments
In this study, we propose a multi-GPU-based 8KVR stitching system that operates in real time on both local and cloud machine environments. The proposed system first obtains multiple 4 K video inputs, decodes them, and generates a stitched 8KVR video stream in real time. The generated 8KVR video stre...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Electronics and Telecommunications Research Institute (ETRI)
2021-11-01
|
| Series: | ETRI Journal |
| Subjects: | |
| Online Access: | https://doi.org/10.4218/etrij.2021-0210 |
| _version_ | 1818365149053976576 |
|---|---|
| author | HeeKyung Lee Gi-Mun Um Seong Yong Lim Jeongil Seo Moonsung Gwak |
| author_facet | HeeKyung Lee Gi-Mun Um Seong Yong Lim Jeongil Seo Moonsung Gwak |
| author_sort | HeeKyung Lee |
| collection | DOAJ |
| description | In this study, we propose a multi-GPU-based 8KVR stitching system that operates in real time on both local and cloud machine environments. The proposed system first obtains multiple 4 K video inputs, decodes them, and generates a stitched 8KVR video stream in real time. The generated 8KVR video stream can be downloaded and rendered omnidirectionally in player apps on smartphones, tablets, and head-mounted displays. To speed up processing, we adopt group-of-pictures-based distributed decoding/encoding and buffering with the NV12 format, along with multi-GPU-based parallel processing. Furthermore, we develop several algorithms such as equirectangular projection-based color correction, real-time CG overlay, and object motion-based seam estimation and correction, to improve the stitching quality. From experiments in both local and cloud machine environments, we confirm the feasibility of the proposed 8KVR stitching system with stitching speed of up to 83.7 fps for six-channel and 62.7 fps for eight-channel inputs. In addition, in an 8KVR live streaming test on the 5G MEC/cloud, the proposed system achieves stable performances with 8 K@30 fps in both indoor and outdoor environments, even during motion. |
| first_indexed | 2024-12-13T22:15:40Z |
| format | Article |
| id | doaj.art-81e2fa1490624e01a9ab3f3da3479ca5 |
| institution | Directory Open Access Journal |
| issn | 1225-6463 |
| language | English |
| last_indexed | 2024-12-13T22:15:40Z |
| publishDate | 2021-11-01 |
| publisher | Electronics and Telecommunications Research Institute (ETRI) |
| record_format | Article |
| series | ETRI Journal |
| spelling | doaj.art-81e2fa1490624e01a9ab3f3da3479ca52022-12-21T23:29:34ZengElectronics and Telecommunications Research Institute (ETRI)ETRI Journal1225-64632021-11-01441627210.4218/etrij.2021-021010.4218/etrij.2021-0210Real-time multi-GPU-based 8KVR stitching and streaming on 5G MEC/Cloud environmentsHeeKyung LeeGi-Mun UmSeong Yong LimJeongil SeoMoonsung GwakIn this study, we propose a multi-GPU-based 8KVR stitching system that operates in real time on both local and cloud machine environments. The proposed system first obtains multiple 4 K video inputs, decodes them, and generates a stitched 8KVR video stream in real time. The generated 8KVR video stream can be downloaded and rendered omnidirectionally in player apps on smartphones, tablets, and head-mounted displays. To speed up processing, we adopt group-of-pictures-based distributed decoding/encoding and buffering with the NV12 format, along with multi-GPU-based parallel processing. Furthermore, we develop several algorithms such as equirectangular projection-based color correction, real-time CG overlay, and object motion-based seam estimation and correction, to improve the stitching quality. From experiments in both local and cloud machine environments, we confirm the feasibility of the proposed 8KVR stitching system with stitching speed of up to 83.7 fps for six-channel and 62.7 fps for eight-channel inputs. In addition, in an 8KVR live streaming test on the 5G MEC/cloud, the proposed system achieves stable performances with 8 K@30 fps in both indoor and outdoor environments, even during motion.https://doi.org/10.4218/etrij.2021-02105g8kvrcloudlive streamingsmartphone camera |
| spellingShingle | HeeKyung Lee Gi-Mun Um Seong Yong Lim Jeongil Seo Moonsung Gwak Real-time multi-GPU-based 8KVR stitching and streaming on 5G MEC/Cloud environments ETRI Journal 5g 8kvr cloud live streaming smartphone camera |
| title | Real-time multi-GPU-based 8KVR stitching and streaming on 5G MEC/Cloud environments |
| title_full | Real-time multi-GPU-based 8KVR stitching and streaming on 5G MEC/Cloud environments |
| title_fullStr | Real-time multi-GPU-based 8KVR stitching and streaming on 5G MEC/Cloud environments |
| title_full_unstemmed | Real-time multi-GPU-based 8KVR stitching and streaming on 5G MEC/Cloud environments |
| title_short | Real-time multi-GPU-based 8KVR stitching and streaming on 5G MEC/Cloud environments |
| title_sort | real time multi gpu based 8kvr stitching and streaming on 5g mec cloud environments |
| topic | 5g 8kvr cloud live streaming smartphone camera |
| url | https://doi.org/10.4218/etrij.2021-0210 |
| work_keys_str_mv | AT heekyunglee realtimemultigpubased8kvrstitchingandstreamingon5gmeccloudenvironments AT gimunum realtimemultigpubased8kvrstitchingandstreamingon5gmeccloudenvironments AT seongyonglim realtimemultigpubased8kvrstitchingandstreamingon5gmeccloudenvironments AT jeongilseo realtimemultigpubased8kvrstitchingandstreamingon5gmeccloudenvironments AT moonsunggwak realtimemultigpubased8kvrstitchingandstreamingon5gmeccloudenvironments |