Real-time distance field acceleration based free-viewpoint video synthesis for large sports fields
Abstract Free-viewpoint video allows the user to view objects from any virtual perspective, creating an immersive visual experience. This technology enhances the interactivity and freedom of multimedia performances. However, many free-viewpoint video synthesis methods hardly satisfy the requirement...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2024-01-01
|
| Series: | Computational Visual Media |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s41095-022-0323-3 |
| _version_ | 1797362998804742144 |
|---|---|
| author | Yanran Dai Jing Li Yuqi Jiang Haidong Qin Bang Liang Shikuan Hong Haozhe Pan Tao Yang |
| author_facet | Yanran Dai Jing Li Yuqi Jiang Haidong Qin Bang Liang Shikuan Hong Haozhe Pan Tao Yang |
| author_sort | Yanran Dai |
| collection | DOAJ |
| description | Abstract Free-viewpoint video allows the user to view objects from any virtual perspective, creating an immersive visual experience. This technology enhances the interactivity and freedom of multimedia performances. However, many free-viewpoint video synthesis methods hardly satisfy the requirement to work in real time with high precision, particularly for sports fields having large areas and numerous moving objects. To address these issues, we propose a free-viewpoint video synthesis method based on distance field acceleration. The central idea is to fuse multi-view distance field information and use it to adjust the search step size adaptively. Adaptive step size search is used in two ways: for fast estimation of multi-object three-dimensional surfaces, and synthetic view rendering based on global occlusion judgement. We have implemented our ideas using parallel computing for interactive display, using CUDA and OpenGL frameworks, and have used real-world and simulated experimental datasets for evaluation. The results show that the proposed method can render free-viewpoint videos with multiple objects on large sports fields at 25 fps. Furthermore, the visual quality of our synthetic novel viewpoint images exceeds that of state-of-the-art neural-rendering-based methods. |
| first_indexed | 2024-03-08T16:15:17Z |
| format | Article |
| id | doaj.art-71550b1aef3245f9818809ba3480b707 |
| institution | Directory Open Access Journal |
| issn | 2096-0433 2096-0662 |
| language | English |
| last_indexed | 2024-03-08T16:15:17Z |
| publishDate | 2024-01-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Computational Visual Media |
| spelling | doaj.art-71550b1aef3245f9818809ba3480b7072024-01-07T12:39:09ZengSpringerOpenComputational Visual Media2096-04332096-06622024-01-0110233135310.1007/s41095-022-0323-3Real-time distance field acceleration based free-viewpoint video synthesis for large sports fieldsYanran Dai0Jing Li1Yuqi Jiang2Haidong Qin3Bang Liang4Shikuan Hong5Haozhe Pan6Tao Yang7School of Telecommunications Engineering, Xidian UniversitySchool of Telecommunications Engineering, Xidian UniversitySchool of Telecommunications Engineering, Xidian UniversityNational Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, SAIIP, the School of Computer Science, Northwestern Polytechnical UniversityNational Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, SAIIP, the School of Computer Science, Northwestern Polytechnical UniversitySchool of Telecommunications Engineering, Xidian UniversitySchool of Telecommunications Engineering, Xidian UniversityNational Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, SAIIP, the School of Computer Science, Northwestern Polytechnical UniversityAbstract Free-viewpoint video allows the user to view objects from any virtual perspective, creating an immersive visual experience. This technology enhances the interactivity and freedom of multimedia performances. However, many free-viewpoint video synthesis methods hardly satisfy the requirement to work in real time with high precision, particularly for sports fields having large areas and numerous moving objects. To address these issues, we propose a free-viewpoint video synthesis method based on distance field acceleration. The central idea is to fuse multi-view distance field information and use it to adjust the search step size adaptively. Adaptive step size search is used in two ways: for fast estimation of multi-object three-dimensional surfaces, and synthetic view rendering based on global occlusion judgement. We have implemented our ideas using parallel computing for interactive display, using CUDA and OpenGL frameworks, and have used real-world and simulated experimental datasets for evaluation. The results show that the proposed method can render free-viewpoint videos with multiple objects on large sports fields at 25 fps. Furthermore, the visual quality of our synthetic novel viewpoint images exceeds that of state-of-the-art neural-rendering-based methods.https://doi.org/10.1007/s41095-022-0323-3free-viewpoint videoview synthesiscamera arraydistance fieldsports video |
| spellingShingle | Yanran Dai Jing Li Yuqi Jiang Haidong Qin Bang Liang Shikuan Hong Haozhe Pan Tao Yang Real-time distance field acceleration based free-viewpoint video synthesis for large sports fields Computational Visual Media free-viewpoint video view synthesis camera array distance field sports video |
| title | Real-time distance field acceleration based free-viewpoint video synthesis for large sports fields |
| title_full | Real-time distance field acceleration based free-viewpoint video synthesis for large sports fields |
| title_fullStr | Real-time distance field acceleration based free-viewpoint video synthesis for large sports fields |
| title_full_unstemmed | Real-time distance field acceleration based free-viewpoint video synthesis for large sports fields |
| title_short | Real-time distance field acceleration based free-viewpoint video synthesis for large sports fields |
| title_sort | real time distance field acceleration based free viewpoint video synthesis for large sports fields |
| topic | free-viewpoint video view synthesis camera array distance field sports video |
| url | https://doi.org/10.1007/s41095-022-0323-3 |
| work_keys_str_mv | AT yanrandai realtimedistancefieldaccelerationbasedfreeviewpointvideosynthesisforlargesportsfields AT jingli realtimedistancefieldaccelerationbasedfreeviewpointvideosynthesisforlargesportsfields AT yuqijiang realtimedistancefieldaccelerationbasedfreeviewpointvideosynthesisforlargesportsfields AT haidongqin realtimedistancefieldaccelerationbasedfreeviewpointvideosynthesisforlargesportsfields AT bangliang realtimedistancefieldaccelerationbasedfreeviewpointvideosynthesisforlargesportsfields AT shikuanhong realtimedistancefieldaccelerationbasedfreeviewpointvideosynthesisforlargesportsfields AT haozhepan realtimedistancefieldaccelerationbasedfreeviewpointvideosynthesisforlargesportsfields AT taoyang realtimedistancefieldaccelerationbasedfreeviewpointvideosynthesisforlargesportsfields |