From a Visual Scene to a Virtual Representation: A Cross-Domain Review
The widespread use of smartphones and other low-cost equipment as recording devices, the massive growth in bandwidth, and the ever-growing demand for new applications with enhanced capabilities, made visual data a must in several scenarios, including surveillance, sports, retail, entertainment, and...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10145441/ |
| _version_ | 1797803155861274624 |
|---|---|
| author | Americo Pereira Pedro Carvalho Nuno Pereira Paula Viana Luis Corte-Real |
| author_facet | Americo Pereira Pedro Carvalho Nuno Pereira Paula Viana Luis Corte-Real |
| author_sort | Americo Pereira |
| collection | DOAJ |
| description | The widespread use of smartphones and other low-cost equipment as recording devices, the massive growth in bandwidth, and the ever-growing demand for new applications with enhanced capabilities, made visual data a must in several scenarios, including surveillance, sports, retail, entertainment, and intelligent vehicles. Despite significant advances in analyzing and extracting data from images and video, there is a lack of solutions able to analyze and semantically describe the information in the visual scene so that it can be efficiently used and repurposed. Scientific contributions have focused on individual aspects or addressing specific problems and application areas, and no cross-domain solution is available to implement a complete system that enables information passing between cross-cutting algorithms. This paper analyses the problem from an end-to-end perspective, i.e., from the visual scene analysis to the representation of information in a virtual environment, including how the extracted data can be described and stored. A simple processing pipeline is introduced to set up a structure for discussing challenges and opportunities in different steps of the entire process, allowing to identify current gaps in the literature. The work reviews various technologies specifically from the perspective of their applicability to an end-to-end pipeline for scene analysis and synthesis, along with an extensive analysis of datasets for relevant tasks. |
| first_indexed | 2024-03-13T05:16:36Z |
| format | Article |
| id | doaj.art-d5bdb09b99ce4f0c8ec3e193d4467896 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-03-13T05:16:36Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-d5bdb09b99ce4f0c8ec3e193d44678962023-06-15T23:00:42ZengIEEEIEEE Access2169-35362023-01-0111579165793310.1109/ACCESS.2023.328349510145441From a Visual Scene to a Virtual Representation: A Cross-Domain ReviewAmerico Pereira0https://orcid.org/0000-0002-1939-2126Pedro Carvalho1https://orcid.org/0000-0003-4983-4316Nuno Pereira2Paula Viana3https://orcid.org/0000-0001-8447-2360Luis Corte-Real4https://orcid.org/0000-0003-2116-7056Centre for Telecommunications and Multimedia, Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), Porto, PortugalCentre for Telecommunications and Multimedia, Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), Porto, PortugalCentre for Telecommunications and Multimedia, Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), Porto, PortugalCentre for Telecommunications and Multimedia, Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), Porto, PortugalCentre for Telecommunications and Multimedia, Institute for Systems and Computer Engineering, Technology and Science (INESC TEC), Porto, PortugalThe widespread use of smartphones and other low-cost equipment as recording devices, the massive growth in bandwidth, and the ever-growing demand for new applications with enhanced capabilities, made visual data a must in several scenarios, including surveillance, sports, retail, entertainment, and intelligent vehicles. Despite significant advances in analyzing and extracting data from images and video, there is a lack of solutions able to analyze and semantically describe the information in the visual scene so that it can be efficiently used and repurposed. Scientific contributions have focused on individual aspects or addressing specific problems and application areas, and no cross-domain solution is available to implement a complete system that enables information passing between cross-cutting algorithms. This paper analyses the problem from an end-to-end perspective, i.e., from the visual scene analysis to the representation of information in a virtual environment, including how the extracted data can be described and stored. A simple processing pipeline is introduced to set up a structure for discussing challenges and opportunities in different steps of the entire process, allowing to identify current gaps in the literature. The work reviews various technologies specifically from the perspective of their applicability to an end-to-end pipeline for scene analysis and synthesis, along with an extensive analysis of datasets for relevant tasks.https://ieeexplore.ieee.org/document/10145441/Computer visiondatasetsscene analysisscene reconstructionvisual scene understanding |
| spellingShingle | Americo Pereira Pedro Carvalho Nuno Pereira Paula Viana Luis Corte-Real From a Visual Scene to a Virtual Representation: A Cross-Domain Review IEEE Access Computer vision datasets scene analysis scene reconstruction visual scene understanding |
| title | From a Visual Scene to a Virtual Representation: A Cross-Domain Review |
| title_full | From a Visual Scene to a Virtual Representation: A Cross-Domain Review |
| title_fullStr | From a Visual Scene to a Virtual Representation: A Cross-Domain Review |
| title_full_unstemmed | From a Visual Scene to a Virtual Representation: A Cross-Domain Review |
| title_short | From a Visual Scene to a Virtual Representation: A Cross-Domain Review |
| title_sort | from a visual scene to a virtual representation a cross domain review |
| topic | Computer vision datasets scene analysis scene reconstruction visual scene understanding |
| url | https://ieeexplore.ieee.org/document/10145441/ |
| work_keys_str_mv | AT americopereira fromavisualscenetoavirtualrepresentationacrossdomainreview AT pedrocarvalho fromavisualscenetoavirtualrepresentationacrossdomainreview AT nunopereira fromavisualscenetoavirtualrepresentationacrossdomainreview AT paulaviana fromavisualscenetoavirtualrepresentationacrossdomainreview AT luiscortereal fromavisualscenetoavirtualrepresentationacrossdomainreview |