A Physics-Driven CNN Model for Real-Time Sea Waves 3D Reconstruction
One of the most promising techniques for the analysis of Spatio-Temporal ocean wave fields is stereo vision. Indeed, the reconstruction accuracy and resolution typically outperform other approaches like radars, satellites, etc. However, it is computationally expensive so its application is typically...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-09-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/13/18/3780 |
| _version_ | 1827681377049378816 |
|---|---|
| author | Mara Pistellato Filippo Bergamasco Andrea Torsello Francesco Barbariol Jeseon Yoo Jin-Yong Jeong Alvise Benetazzo |
| author_facet | Mara Pistellato Filippo Bergamasco Andrea Torsello Francesco Barbariol Jeseon Yoo Jin-Yong Jeong Alvise Benetazzo |
| author_sort | Mara Pistellato |
| collection | DOAJ |
| description | One of the most promising techniques for the analysis of Spatio-Temporal ocean wave fields is stereo vision. Indeed, the reconstruction accuracy and resolution typically outperform other approaches like radars, satellites, etc. However, it is computationally expensive so its application is typically restricted to the analysis of short pre-recorded sequences. What prevents such methodology from being truly real-time is the final 3D surface estimation from a scattered, non-equispaced point cloud. Recently, we studied a novel approach exploiting the temporal dependence of subsequent frames to iteratively update the wave spectrum over time. Albeit substantially faster, the unpredictable convergence time of the optimization involved still prevents its usage as a continuously running remote sensing infrastructure. In this work, we build upon the same idea, but investigating the feasibility of a fully data-driven Machine Learning (ML) approach. We designed a novel Convolutional Neural Network that learns how to produce an accurate surface from the scattered elevation data of three subsequent frames. The key idea is to embed the linear dispersion relation into the model itself to physically relate the sparse points observed at different times. Assuming that the scattered data are uniformly distributed in the spatial domain, this has the same effect of increasing the sample density of each single frame. Experiments demonstrate how the proposed technique, even if trained with purely synthetic data, can produce accurate and physically consistent surfaces at five frames per second on a modern PC. |
| first_indexed | 2024-03-10T07:14:17Z |
| format | Article |
| id | doaj.art-403c69f2167845efbdb18a1a7d236517 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T07:14:17Z |
| publishDate | 2021-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-403c69f2167845efbdb18a1a7d2365172023-11-22T15:08:15ZengMDPI AGRemote Sensing2072-42922021-09-011318378010.3390/rs13183780A Physics-Driven CNN Model for Real-Time Sea Waves 3D ReconstructionMara Pistellato0Filippo Bergamasco1Andrea Torsello2Francesco Barbariol3Jeseon Yoo4Jin-Yong Jeong5Alvise Benetazzo6Department of Environmental Sciences, Informatics and Statistics-Ca’ Foscari University of Venice, Dorsoduro 3246, 30123 Venice, ItalyDepartment of Environmental Sciences, Informatics and Statistics-Ca’ Foscari University of Venice, Dorsoduro 3246, 30123 Venice, ItalyDepartment of Environmental Sciences, Informatics and Statistics-Ca’ Foscari University of Venice, Dorsoduro 3246, 30123 Venice, ItalyIstituto di Scienze Marine (ISMAR), Consiglio Nazionale delle Ricerche (CNR), 30122 Venice, ItalyKorea Institute of Ocean Science and Technology (KIOST), Pusan 49111, KoreaKorea Institute of Ocean Science and Technology (KIOST), Pusan 49111, KoreaIstituto di Scienze Marine (ISMAR), Consiglio Nazionale delle Ricerche (CNR), 30122 Venice, ItalyOne of the most promising techniques for the analysis of Spatio-Temporal ocean wave fields is stereo vision. Indeed, the reconstruction accuracy and resolution typically outperform other approaches like radars, satellites, etc. However, it is computationally expensive so its application is typically restricted to the analysis of short pre-recorded sequences. What prevents such methodology from being truly real-time is the final 3D surface estimation from a scattered, non-equispaced point cloud. Recently, we studied a novel approach exploiting the temporal dependence of subsequent frames to iteratively update the wave spectrum over time. Albeit substantially faster, the unpredictable convergence time of the optimization involved still prevents its usage as a continuously running remote sensing infrastructure. In this work, we build upon the same idea, but investigating the feasibility of a fully data-driven Machine Learning (ML) approach. We designed a novel Convolutional Neural Network that learns how to produce an accurate surface from the scattered elevation data of three subsequent frames. The key idea is to embed the linear dispersion relation into the model itself to physically relate the sparse points observed at different times. Assuming that the scattered data are uniformly distributed in the spatial domain, this has the same effect of increasing the sample density of each single frame. Experiments demonstrate how the proposed technique, even if trained with purely synthetic data, can produce accurate and physically consistent surfaces at five frames per second on a modern PC.https://www.mdpi.com/2072-4292/13/18/3780sea-waveswave fieldssurface reconstructionConvolutional Neural Networksdepth completion |
| spellingShingle | Mara Pistellato Filippo Bergamasco Andrea Torsello Francesco Barbariol Jeseon Yoo Jin-Yong Jeong Alvise Benetazzo A Physics-Driven CNN Model for Real-Time Sea Waves 3D Reconstruction Remote Sensing sea-waves wave fields surface reconstruction Convolutional Neural Networks depth completion |
| title | A Physics-Driven CNN Model for Real-Time Sea Waves 3D Reconstruction |
| title_full | A Physics-Driven CNN Model for Real-Time Sea Waves 3D Reconstruction |
| title_fullStr | A Physics-Driven CNN Model for Real-Time Sea Waves 3D Reconstruction |
| title_full_unstemmed | A Physics-Driven CNN Model for Real-Time Sea Waves 3D Reconstruction |
| title_short | A Physics-Driven CNN Model for Real-Time Sea Waves 3D Reconstruction |
| title_sort | physics driven cnn model for real time sea waves 3d reconstruction |
| topic | sea-waves wave fields surface reconstruction Convolutional Neural Networks depth completion |
| url | https://www.mdpi.com/2072-4292/13/18/3780 |
| work_keys_str_mv | AT marapistellato aphysicsdrivencnnmodelforrealtimeseawaves3dreconstruction AT filippobergamasco aphysicsdrivencnnmodelforrealtimeseawaves3dreconstruction AT andreatorsello aphysicsdrivencnnmodelforrealtimeseawaves3dreconstruction AT francescobarbariol aphysicsdrivencnnmodelforrealtimeseawaves3dreconstruction AT jeseonyoo aphysicsdrivencnnmodelforrealtimeseawaves3dreconstruction AT jinyongjeong aphysicsdrivencnnmodelforrealtimeseawaves3dreconstruction AT alvisebenetazzo aphysicsdrivencnnmodelforrealtimeseawaves3dreconstruction AT marapistellato physicsdrivencnnmodelforrealtimeseawaves3dreconstruction AT filippobergamasco physicsdrivencnnmodelforrealtimeseawaves3dreconstruction AT andreatorsello physicsdrivencnnmodelforrealtimeseawaves3dreconstruction AT francescobarbariol physicsdrivencnnmodelforrealtimeseawaves3dreconstruction AT jeseonyoo physicsdrivencnnmodelforrealtimeseawaves3dreconstruction AT jinyongjeong physicsdrivencnnmodelforrealtimeseawaves3dreconstruction AT alvisebenetazzo physicsdrivencnnmodelforrealtimeseawaves3dreconstruction |