Direct Aerial Visual Geolocalization Using Deep Neural Networks
Unmanned aerial vehicles (UAVs) must keep track of their location in order to maintain flight plans. Currently, this task is almost entirely performed by a combination of Inertial Measurement Units (IMUs) and reference to GNSS (Global Navigation Satellite System). Navigation by GNSS, however, is not...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-10-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/13/19/4017 |
| _version_ | 1827680549778489344 |
|---|---|
| author | Winthrop Harvey Chase Rainwater Jackson Cothren |
| author_facet | Winthrop Harvey Chase Rainwater Jackson Cothren |
| author_sort | Winthrop Harvey |
| collection | DOAJ |
| description | Unmanned aerial vehicles (UAVs) must keep track of their location in order to maintain flight plans. Currently, this task is almost entirely performed by a combination of Inertial Measurement Units (IMUs) and reference to GNSS (Global Navigation Satellite System). Navigation by GNSS, however, is not always reliable, due to various causes both natural (reflection and blockage from objects, technical fault, inclement weather) and artificial (GPS spoofing and denial). In such GPS-denied situations, it is desirable to have additional methods for aerial geolocalization. One such method is visual geolocalization, where aircraft use their ground facing cameras to localize and navigate. The state of the art in many ground-level image processing tasks involve the use of Convolutional Neural Networks (CNNs). We present here a study of how effectively a modern CNN designed for visual classification can be applied to the problem of Absolute Visual Geolocalization (AVL, localization without a prior location estimate). An Xception based architecture is trained from scratch over a >1000 km<sup>2</sup> section of Washington County, Arkansas to directly regress latitude and longitude from images from different orthorectified high-altitude survey flights. It achieves average localization accuracy on unseen image sets over the same region from different years and seasons with as low as 115 m average error, which localizes to 0.004% of the training area, or about 8% of the width of the 1.5 × 1.5 km input image. This demonstrates that CNNs are expressive enough to encode robust landscape information for geolocalization over large geographic areas. Furthermore, discussed are methods of providing uncertainty for CNN regression outputs, and future areas of potential improvement for use of deep neural networks in visual geolocalization. |
| first_indexed | 2024-03-10T06:52:14Z |
| format | Article |
| id | doaj.art-8be232b1195c4bc5b288f6db0ab9e371 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T06:52:14Z |
| publishDate | 2021-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-8be232b1195c4bc5b288f6db0ab9e3712023-11-22T16:44:16ZengMDPI AGRemote Sensing2072-42922021-10-011319401710.3390/rs13194017Direct Aerial Visual Geolocalization Using Deep Neural NetworksWinthrop Harvey0Chase Rainwater1Jackson Cothren2Department of Industrial Engineering, University of Arkansas, Fayetteville, AR 72701, USADepartment of Industrial Engineering, University of Arkansas, Fayetteville, AR 72701, USADepartment of Geosciences, University of Arkansas, Fayetteville, AR 72701, USAUnmanned aerial vehicles (UAVs) must keep track of their location in order to maintain flight plans. Currently, this task is almost entirely performed by a combination of Inertial Measurement Units (IMUs) and reference to GNSS (Global Navigation Satellite System). Navigation by GNSS, however, is not always reliable, due to various causes both natural (reflection and blockage from objects, technical fault, inclement weather) and artificial (GPS spoofing and denial). In such GPS-denied situations, it is desirable to have additional methods for aerial geolocalization. One such method is visual geolocalization, where aircraft use their ground facing cameras to localize and navigate. The state of the art in many ground-level image processing tasks involve the use of Convolutional Neural Networks (CNNs). We present here a study of how effectively a modern CNN designed for visual classification can be applied to the problem of Absolute Visual Geolocalization (AVL, localization without a prior location estimate). An Xception based architecture is trained from scratch over a >1000 km<sup>2</sup> section of Washington County, Arkansas to directly regress latitude and longitude from images from different orthorectified high-altitude survey flights. It achieves average localization accuracy on unseen image sets over the same region from different years and seasons with as low as 115 m average error, which localizes to 0.004% of the training area, or about 8% of the width of the 1.5 × 1.5 km input image. This demonstrates that CNNs are expressive enough to encode robust landscape information for geolocalization over large geographic areas. Furthermore, discussed are methods of providing uncertainty for CNN regression outputs, and future areas of potential improvement for use of deep neural networks in visual geolocalization.https://www.mdpi.com/2072-4292/13/19/4017geolocalizationvisual localizationabsolute visual geolocalizationdroneUAVmachine learning |
| spellingShingle | Winthrop Harvey Chase Rainwater Jackson Cothren Direct Aerial Visual Geolocalization Using Deep Neural Networks Remote Sensing geolocalization visual localization absolute visual geolocalization drone UAV machine learning |
| title | Direct Aerial Visual Geolocalization Using Deep Neural Networks |
| title_full | Direct Aerial Visual Geolocalization Using Deep Neural Networks |
| title_fullStr | Direct Aerial Visual Geolocalization Using Deep Neural Networks |
| title_full_unstemmed | Direct Aerial Visual Geolocalization Using Deep Neural Networks |
| title_short | Direct Aerial Visual Geolocalization Using Deep Neural Networks |
| title_sort | direct aerial visual geolocalization using deep neural networks |
| topic | geolocalization visual localization absolute visual geolocalization drone UAV machine learning |
| url | https://www.mdpi.com/2072-4292/13/19/4017 |
| work_keys_str_mv | AT winthropharvey directaerialvisualgeolocalizationusingdeepneuralnetworks AT chaserainwater directaerialvisualgeolocalizationusingdeepneuralnetworks AT jacksoncothren directaerialvisualgeolocalizationusingdeepneuralnetworks |