A Novel Deep Learning Approach Using Contextual Embeddings for Toponym Resolution
This article describes a novel approach for toponym resolution with deep neural networks. The proposed approach does not involve matching references in the text against entries in a gazetteer, instead directly predicting geo-spatial coordinates. Multiple inputs are considered in the neural network a...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2021-12-01
|
| Series: | ISPRS International Journal of Geo-Information |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2220-9964/11/1/28 |
| _version_ | 1797493535110332416 |
|---|---|
| author | Ana Bárbara Cardoso Bruno Martins Jacinto Estima |
| author_facet | Ana Bárbara Cardoso Bruno Martins Jacinto Estima |
| author_sort | Ana Bárbara Cardoso |
| collection | DOAJ |
| description | This article describes a novel approach for toponym resolution with deep neural networks. The proposed approach does not involve matching references in the text against entries in a gazetteer, instead directly predicting geo-spatial coordinates. Multiple inputs are considered in the neural network architecture (e.g., the surrounding words are considered in combination with the toponym to disambiguate), using pre-trained contextual word embeddings (i.e., ELMo or BERT) as well as bi-directional Long Short-Term Memory units, which are both regularly used for modeling textual data. The intermediate representations are then used to predict a probability distribution over possible geo-spatial regions, and finally to predict the coordinates for the input toponym. The proposed model was tested on three datasets used on previous toponym resolution studies, specifically the (i) <i>War of the Rebellion</i>, (ii) <i>Local–Global Lexicon</i>, and (iii) <i>SpatialML</i> corpora. Moreover, we evaluated the effect of using (i) geophysical terrain properties as external information, including information on elevation or terrain development, among others, and (ii) additional data collected from Wikipedia articles, to further help with the training of the model. The obtained results show improvements using the proposed method, when compared to previous approaches, and specifically when BERT embeddings and additional data are involved. |
| first_indexed | 2024-03-10T01:21:23Z |
| format | Article |
| id | doaj.art-d70188213d4545da893ea303ca2c5df1 |
| institution | Directory Open Access Journal |
| issn | 2220-9964 |
| language | English |
| last_indexed | 2024-03-10T01:21:23Z |
| publishDate | 2021-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | ISPRS International Journal of Geo-Information |
| spelling | doaj.art-d70188213d4545da893ea303ca2c5df12023-11-23T13:59:57ZengMDPI AGISPRS International Journal of Geo-Information2220-99642021-12-011112810.3390/ijgi11010028A Novel Deep Learning Approach Using Contextual Embeddings for Toponym ResolutionAna Bárbara Cardoso0Bruno Martins1Jacinto Estima2INESC-ID, Instituto Superior Técnico, Universidade de Lisboa, 1649-004 Lisboa, PortugalINESC-ID, Instituto Superior Técnico, Universidade de Lisboa, 1649-004 Lisboa, PortugalINESC-ID, Faculdade de Design, Tecnologia e Comunicação (IADE), Universidade Europeia, 1200-649 Lisboa, PortugalThis article describes a novel approach for toponym resolution with deep neural networks. The proposed approach does not involve matching references in the text against entries in a gazetteer, instead directly predicting geo-spatial coordinates. Multiple inputs are considered in the neural network architecture (e.g., the surrounding words are considered in combination with the toponym to disambiguate), using pre-trained contextual word embeddings (i.e., ELMo or BERT) as well as bi-directional Long Short-Term Memory units, which are both regularly used for modeling textual data. The intermediate representations are then used to predict a probability distribution over possible geo-spatial regions, and finally to predict the coordinates for the input toponym. The proposed model was tested on three datasets used on previous toponym resolution studies, specifically the (i) <i>War of the Rebellion</i>, (ii) <i>Local–Global Lexicon</i>, and (iii) <i>SpatialML</i> corpora. Moreover, we evaluated the effect of using (i) geophysical terrain properties as external information, including information on elevation or terrain development, among others, and (ii) additional data collected from Wikipedia articles, to further help with the training of the model. The obtained results show improvements using the proposed method, when compared to previous approaches, and specifically when BERT embeddings and additional data are involved.https://www.mdpi.com/2220-9964/11/1/28geographical text analysisresolving toponyms in textual documentsdeep learning for NLPcontextual word embeddingsmachine learning with neural networks |
| spellingShingle | Ana Bárbara Cardoso Bruno Martins Jacinto Estima A Novel Deep Learning Approach Using Contextual Embeddings for Toponym Resolution ISPRS International Journal of Geo-Information geographical text analysis resolving toponyms in textual documents deep learning for NLP contextual word embeddings machine learning with neural networks |
| title | A Novel Deep Learning Approach Using Contextual Embeddings for Toponym Resolution |
| title_full | A Novel Deep Learning Approach Using Contextual Embeddings for Toponym Resolution |
| title_fullStr | A Novel Deep Learning Approach Using Contextual Embeddings for Toponym Resolution |
| title_full_unstemmed | A Novel Deep Learning Approach Using Contextual Embeddings for Toponym Resolution |
| title_short | A Novel Deep Learning Approach Using Contextual Embeddings for Toponym Resolution |
| title_sort | novel deep learning approach using contextual embeddings for toponym resolution |
| topic | geographical text analysis resolving toponyms in textual documents deep learning for NLP contextual word embeddings machine learning with neural networks |
| url | https://www.mdpi.com/2220-9964/11/1/28 |
| work_keys_str_mv | AT anabarbaracardoso anoveldeeplearningapproachusingcontextualembeddingsfortoponymresolution AT brunomartins anoveldeeplearningapproachusingcontextualembeddingsfortoponymresolution AT jacintoestima anoveldeeplearningapproachusingcontextualembeddingsfortoponymresolution AT anabarbaracardoso noveldeeplearningapproachusingcontextualembeddingsfortoponymresolution AT brunomartins noveldeeplearningapproachusingcontextualembeddingsfortoponymresolution AT jacintoestima noveldeeplearningapproachusingcontextualembeddingsfortoponymresolution |