Selective sensor fusion for neural visual-inertial odometry
Deep learning approaches for Visual-Inertial Odometry (VIO) have proven successful, but they rarely focus on incorporating robust fusion strategies for dealing with imperfect input sensory data. We propose a novel end-to-end selective sensor fusion framework for monocular VIO, which fuses monocular...
| Egile Nagusiak: | , , , , , , |
|---|---|
| Formatua: | Conference item |
| Hizkuntza: | English |
| Argitaratua: |
IEEE
2019
|
| _version_ | 1826310544953442304 |
|---|---|
| author | Chen, C Rosa, S Miao, Y Lu, CX Wu, W Markham, A Trigoni, N |
| author_facet | Chen, C Rosa, S Miao, Y Lu, CX Wu, W Markham, A Trigoni, N |
| author_sort | Chen, C |
| collection | OXFORD |
| description | Deep learning approaches for Visual-Inertial Odometry (VIO) have proven successful, but they rarely focus on incorporating robust fusion strategies for dealing with imperfect input sensory data. We propose a novel end-to-end selective sensor fusion framework for monocular VIO, which fuses monocular images and inertial measurements in order to estimate the trajectory whilst improving robustness to real-life issues, such as missing and corrupted data or bad sensor synchronization. In particular, we propose two fusion modalities based on different masking strategies: deterministic soft fusion and stochastic hard fusion, and we compare with previously proposed direct fusion baselines. During testing, the network is able to selectively process the features of the available sensor modalities and produce a trajectory at scale. We present a thorough investigation on the performances on three public autonomous driving, Micro Aerial Vehicle (MAV) and hand-held VIO datasets. The results demonstrate the effectiveness of the fusion strategies, which offer better performances compared to direct fusion, particularly in presence of corrupted data. In addition, we study the interpretability of the fusion networks by visualising the masking layers in different scenarios and with varying data corruption, revealing interesting correlations between the fusion networks and imperfect sensory input data. |
| first_indexed | 2024-03-07T07:53:33Z |
| format | Conference item |
| id | oxford-uuid:cc9b3e5f-a737-4b4d-91c9-1ffed0cebaf0 |
| institution | University of Oxford |
| language | English |
| last_indexed | 2024-03-07T07:53:33Z |
| publishDate | 2019 |
| publisher | IEEE |
| record_format | dspace |
| spelling | oxford-uuid:cc9b3e5f-a737-4b4d-91c9-1ffed0cebaf02023-08-01T13:51:29ZSelective sensor fusion for neural visual-inertial odometryConference itemhttp://purl.org/coar/resource_type/c_5794uuid:cc9b3e5f-a737-4b4d-91c9-1ffed0cebaf0EnglishSymplectic ElementsIEEE2019Chen, CRosa, SMiao, YLu, CXWu, WMarkham, ATrigoni, NDeep learning approaches for Visual-Inertial Odometry (VIO) have proven successful, but they rarely focus on incorporating robust fusion strategies for dealing with imperfect input sensory data. We propose a novel end-to-end selective sensor fusion framework for monocular VIO, which fuses monocular images and inertial measurements in order to estimate the trajectory whilst improving robustness to real-life issues, such as missing and corrupted data or bad sensor synchronization. In particular, we propose two fusion modalities based on different masking strategies: deterministic soft fusion and stochastic hard fusion, and we compare with previously proposed direct fusion baselines. During testing, the network is able to selectively process the features of the available sensor modalities and produce a trajectory at scale. We present a thorough investigation on the performances on three public autonomous driving, Micro Aerial Vehicle (MAV) and hand-held VIO datasets. The results demonstrate the effectiveness of the fusion strategies, which offer better performances compared to direct fusion, particularly in presence of corrupted data. In addition, we study the interpretability of the fusion networks by visualising the masking layers in different scenarios and with varying data corruption, revealing interesting correlations between the fusion networks and imperfect sensory input data. |
| spellingShingle | Chen, C Rosa, S Miao, Y Lu, CX Wu, W Markham, A Trigoni, N Selective sensor fusion for neural visual-inertial odometry |
| title | Selective sensor fusion for neural visual-inertial odometry |
| title_full | Selective sensor fusion for neural visual-inertial odometry |
| title_fullStr | Selective sensor fusion for neural visual-inertial odometry |
| title_full_unstemmed | Selective sensor fusion for neural visual-inertial odometry |
| title_short | Selective sensor fusion for neural visual-inertial odometry |
| title_sort | selective sensor fusion for neural visual inertial odometry |
| work_keys_str_mv | AT chenc selectivesensorfusionforneuralvisualinertialodometry AT rosas selectivesensorfusionforneuralvisualinertialodometry AT miaoy selectivesensorfusionforneuralvisualinertialodometry AT lucx selectivesensorfusionforneuralvisualinertialodometry AT wuw selectivesensorfusionforneuralvisualinertialodometry AT markhama selectivesensorfusionforneuralvisualinertialodometry AT trigonin selectivesensorfusionforneuralvisualinertialodometry |