Inferring socioeconomic environment from built environment characteristics based street view images: An approach of Seq2Seq method
The street view image (SVI) and the point of interest (POI) are data sources with different modalities and representing different urban environments, respectively. These two types of data may not fully cover a city. Inferring from one type of data to another will enrich our means of sensing cities a...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-09-01
|
| Series: | International Journal of Applied Earth Observations and Geoinformation |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1569843223002820 |
| _version_ | 1797677664829440000 |
|---|---|
| author | Yan Zhang Fan Zhang Libo Fang Nengcheng Chen |
| author_facet | Yan Zhang Fan Zhang Libo Fang Nengcheng Chen |
| author_sort | Yan Zhang |
| collection | DOAJ |
| description | The street view image (SVI) and the point of interest (POI) are data sources with different modalities and representing different urban environments, respectively. These two types of data may not fully cover a city. Inferring from one type of data to another will enrich our means of sensing cities and facilitate a deeper understanding of the place. Intuitively, by analyzing visual elements of the urban built environment such as buildings, roads, vehicles, and other features captured in photos of their surroundings, we can infer valuable information about the most likely POI types. Traditional approaches are mainly based on probabilistic statistical models. Such models lack a holistic understanding of the scenario and accurate modeling of the non-linear relationship between the built and socio-economic environments. In response, a Seq2Seq framework is proposed to “translate” and create a bridge between those two data. Experiments in Wuhan demonstrated the high performance of our method (accuracy = 0.770, recall = 0.773, f1= 0.771). And this work allows us to better understand the “many-to-many” relationship between the two environments (data), effectively enriching our means of perceiving cities. It also has potential applications in location awareness and visual navigation. |
| first_indexed | 2024-03-11T22:48:33Z |
| format | Article |
| id | doaj.art-3ef4b510d6f74e1b8822b7e786b983af |
| institution | Directory Open Access Journal |
| issn | 1569-8432 |
| language | English |
| last_indexed | 2024-03-11T22:48:33Z |
| publishDate | 2023-09-01 |
| publisher | Elsevier |
| record_format | Article |
| series | International Journal of Applied Earth Observations and Geoinformation |
| spelling | doaj.art-3ef4b510d6f74e1b8822b7e786b983af2023-09-22T04:38:16ZengElsevierInternational Journal of Applied Earth Observations and Geoinformation1569-84322023-09-01123103458Inferring socioeconomic environment from built environment characteristics based street view images: An approach of Seq2Seq methodYan Zhang0Fan Zhang1Libo Fang2Nengcheng Chen3National engineering research center of geographic information system, China University of Geosciences, Wuhan 430074, China; State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan 430079, ChinaDepartment of Civil and Environmental Engineering, The Hong Kong University of Science and Technology, 999077, Hong Kong, ChinaHunan Architectural Design Institute Group Co., Ltd, Changsha, 410208, ChinaNational engineering research center of geographic information system, China University of Geosciences, Wuhan 430074, China; State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan University, Wuhan 430079, China; Correspondence to: 430074, Lumo Road 388, Wuhan, China.The street view image (SVI) and the point of interest (POI) are data sources with different modalities and representing different urban environments, respectively. These two types of data may not fully cover a city. Inferring from one type of data to another will enrich our means of sensing cities and facilitate a deeper understanding of the place. Intuitively, by analyzing visual elements of the urban built environment such as buildings, roads, vehicles, and other features captured in photos of their surroundings, we can infer valuable information about the most likely POI types. Traditional approaches are mainly based on probabilistic statistical models. Such models lack a holistic understanding of the scenario and accurate modeling of the non-linear relationship between the built and socio-economic environments. In response, a Seq2Seq framework is proposed to “translate” and create a bridge between those two data. Experiments in Wuhan demonstrated the high performance of our method (accuracy = 0.770, recall = 0.773, f1= 0.771). And this work allows us to better understand the “many-to-many” relationship between the two environments (data), effectively enriching our means of perceiving cities. It also has potential applications in location awareness and visual navigation.http://www.sciencedirect.com/science/article/pii/S1569843223002820Social sensingSeq2SeqMulti-modalLSTMGeoAI |
| spellingShingle | Yan Zhang Fan Zhang Libo Fang Nengcheng Chen Inferring socioeconomic environment from built environment characteristics based street view images: An approach of Seq2Seq method International Journal of Applied Earth Observations and Geoinformation Social sensing Seq2Seq Multi-modal LSTM GeoAI |
| title | Inferring socioeconomic environment from built environment characteristics based street view images: An approach of Seq2Seq method |
| title_full | Inferring socioeconomic environment from built environment characteristics based street view images: An approach of Seq2Seq method |
| title_fullStr | Inferring socioeconomic environment from built environment characteristics based street view images: An approach of Seq2Seq method |
| title_full_unstemmed | Inferring socioeconomic environment from built environment characteristics based street view images: An approach of Seq2Seq method |
| title_short | Inferring socioeconomic environment from built environment characteristics based street view images: An approach of Seq2Seq method |
| title_sort | inferring socioeconomic environment from built environment characteristics based street view images an approach of seq2seq method |
| topic | Social sensing Seq2Seq Multi-modal LSTM GeoAI |
| url | http://www.sciencedirect.com/science/article/pii/S1569843223002820 |
| work_keys_str_mv | AT yanzhang inferringsocioeconomicenvironmentfrombuiltenvironmentcharacteristicsbasedstreetviewimagesanapproachofseq2seqmethod AT fanzhang inferringsocioeconomicenvironmentfrombuiltenvironmentcharacteristicsbasedstreetviewimagesanapproachofseq2seqmethod AT libofang inferringsocioeconomicenvironmentfrombuiltenvironmentcharacteristicsbasedstreetviewimagesanapproachofseq2seqmethod AT nengchengchen inferringsocioeconomicenvironmentfrombuiltenvironmentcharacteristicsbasedstreetviewimagesanapproachofseq2seqmethod |