A news picture geo-localization pipeline based on deep learning and street view images
Numerous news or event pictures are taken and shared on the internet every day that have abundant information worth being mined, but only a small fraction of them are geotagged. The visual content of the news image hints at clues of the geographical location because they are usually taken at the sit...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Taylor & Francis Group
2022-12-01
|
| Series: | International Journal of Digital Earth |
| Subjects: | |
| Online Access: | http://dx.doi.org/10.1080/17538947.2022.2121437 |
| _version_ | 1797678498715795456 |
|---|---|
| author | Tianyou Chu Yumin Chen Heng Su Zhenzhen Xu Guodong Chen Annan Zhou |
| author_facet | Tianyou Chu Yumin Chen Heng Su Zhenzhen Xu Guodong Chen Annan Zhou |
| author_sort | Tianyou Chu |
| collection | DOAJ |
| description | Numerous news or event pictures are taken and shared on the internet every day that have abundant information worth being mined, but only a small fraction of them are geotagged. The visual content of the news image hints at clues of the geographical location because they are usually taken at the site of the incident, which provides a prerequisite for geo-localization. This paper proposes an automated pipeline based on deep learning for the geo-localization of news pictures in a large-scale urban environment using geotagged street view images as a reference dataset. The approach obtains location information by constructing an attention-based feature extraction network. Then, the image features are aggregated, and the candidate street view image results are retrieved by the selective matching kernel function. Finally, the coordinates of the news images are estimated by the kernel density prediction method. The pipeline is tested in the news pictures in Hong Kong. In the comparison experiments, the proposed pipeline shows stable performance and generalizability in the large-scale urban environment. In addition, the performance analysis of components in the pipeline shows the ability to recognize localization features of partial areas in pictures and the effectiveness of the proposed solution in news picture geo-localization. |
| first_indexed | 2024-03-11T23:00:38Z |
| format | Article |
| id | doaj.art-8e17c866d0294832a35980ebb4c387bb |
| institution | Directory Open Access Journal |
| issn | 1753-8947 1753-8955 |
| language | English |
| last_indexed | 2024-03-11T23:00:38Z |
| publishDate | 2022-12-01 |
| publisher | Taylor & Francis Group |
| record_format | Article |
| series | International Journal of Digital Earth |
| spelling | doaj.art-8e17c866d0294832a35980ebb4c387bb2023-09-21T14:57:11ZengTaylor & Francis GroupInternational Journal of Digital Earth1753-89471753-89552022-12-011511485150510.1080/17538947.2022.21214372121437A news picture geo-localization pipeline based on deep learning and street view imagesTianyou Chu0Yumin Chen1Heng Su2Zhenzhen Xu3Guodong Chen4Annan Zhou5School of Resource and Environment Science, Wuhan UniversitySchool of Resource and Environment Science, Wuhan UniversitySchool of Resource and Environment Science, Wuhan UniversitySchool of Resource and Environment Science, Wuhan UniversitySchool of Resource and Environment Science, Wuhan UniversitySchool of Resource and Environment Science, Wuhan UniversityNumerous news or event pictures are taken and shared on the internet every day that have abundant information worth being mined, but only a small fraction of them are geotagged. The visual content of the news image hints at clues of the geographical location because they are usually taken at the site of the incident, which provides a prerequisite for geo-localization. This paper proposes an automated pipeline based on deep learning for the geo-localization of news pictures in a large-scale urban environment using geotagged street view images as a reference dataset. The approach obtains location information by constructing an attention-based feature extraction network. Then, the image features are aggregated, and the candidate street view image results are retrieved by the selective matching kernel function. Finally, the coordinates of the news images are estimated by the kernel density prediction method. The pipeline is tested in the news pictures in Hong Kong. In the comparison experiments, the proposed pipeline shows stable performance and generalizability in the large-scale urban environment. In addition, the performance analysis of components in the pipeline shows the ability to recognize localization features of partial areas in pictures and the effectiveness of the proposed solution in news picture geo-localization.http://dx.doi.org/10.1080/17538947.2022.2121437street view imagesgeo-localizationimage retrievalsocial media |
| spellingShingle | Tianyou Chu Yumin Chen Heng Su Zhenzhen Xu Guodong Chen Annan Zhou A news picture geo-localization pipeline based on deep learning and street view images International Journal of Digital Earth street view images geo-localization image retrieval social media |
| title | A news picture geo-localization pipeline based on deep learning and street view images |
| title_full | A news picture geo-localization pipeline based on deep learning and street view images |
| title_fullStr | A news picture geo-localization pipeline based on deep learning and street view images |
| title_full_unstemmed | A news picture geo-localization pipeline based on deep learning and street view images |
| title_short | A news picture geo-localization pipeline based on deep learning and street view images |
| title_sort | news picture geo localization pipeline based on deep learning and street view images |
| topic | street view images geo-localization image retrieval social media |
| url | http://dx.doi.org/10.1080/17538947.2022.2121437 |
| work_keys_str_mv | AT tianyouchu anewspicturegeolocalizationpipelinebasedondeeplearningandstreetviewimages AT yuminchen anewspicturegeolocalizationpipelinebasedondeeplearningandstreetviewimages AT hengsu anewspicturegeolocalizationpipelinebasedondeeplearningandstreetviewimages AT zhenzhenxu anewspicturegeolocalizationpipelinebasedondeeplearningandstreetviewimages AT guodongchen anewspicturegeolocalizationpipelinebasedondeeplearningandstreetviewimages AT annanzhou anewspicturegeolocalizationpipelinebasedondeeplearningandstreetviewimages AT tianyouchu newspicturegeolocalizationpipelinebasedondeeplearningandstreetviewimages AT yuminchen newspicturegeolocalizationpipelinebasedondeeplearningandstreetviewimages AT hengsu newspicturegeolocalizationpipelinebasedondeeplearningandstreetviewimages AT zhenzhenxu newspicturegeolocalizationpipelinebasedondeeplearningandstreetviewimages AT guodongchen newspicturegeolocalizationpipelinebasedondeeplearningandstreetviewimages AT annanzhou newspicturegeolocalizationpipelinebasedondeeplearningandstreetviewimages |