Deep Learning Approach for Classifying the Built Year and Structure of Individual Buildings by Automatically Linking Street View Images and GIS Building Data
The built year and structure of individual buildings are crucial factors for estimating and assessing potential earthquake and tsunami damage. Recent advances in sensing and analysis technologies allow the acquisition of high-resolution street view images (SVIs) that present new possibilities for re...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2023-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10018300/ |
| _version_ | 1811168717075120128 |
|---|---|
| author | Yoshiki Ogawa Chenbo Zhao Takuya Oki Shenglong Chen Yoshihide Sekimoto |
| author_facet | Yoshiki Ogawa Chenbo Zhao Takuya Oki Shenglong Chen Yoshihide Sekimoto |
| author_sort | Yoshiki Ogawa |
| collection | DOAJ |
| description | The built year and structure of individual buildings are crucial factors for estimating and assessing potential earthquake and tsunami damage. Recent advances in sensing and analysis technologies allow the acquisition of high-resolution street view images (SVIs) that present new possibilities for research and development. In this study, we developed a model to estimate the built year and structure of a building using omnidirectional SVIs captured using an onboard camera. We used geographic information system (GIS) building data and SVIs to generate an annotated built-year and structure dataset by developing a method to automatically combine the GIS data with images of individual buildings cropped through object detection. Furthermore, we trained a deep learning model to classify the built year and structure of buildings using the annotated image dataset based on a deep convolutional neural network (DCNN) and a vision transformer (ViT). The results showed that SVI accurately predicts the built year and structure of individual buildings using ViT (overall accuracies for structure = 0.94 [three classes] and 0.96 [two classes] and for age = 0.68 [six classes] and 0.90 [three classes]). Compared with DCNN-based networks, the proposed Swin transformer based on ViT architectures effectively improves prediction accuracy. The results indicate that multiple high-resolution images can be obtained for individual buildings using SVI, and the proposed method is an effective approach for classifying structures and determining building age. The automatic, accurate, and large-scale mapping of the built year and structure of individual buildings can help develop specific disaster prevention measures. |
| first_indexed | 2024-04-10T16:31:26Z |
| format | Article |
| id | doaj.art-8a9b36e8ba5542f1a008d91570fd9162 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-04-10T16:31:26Z |
| publishDate | 2023-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-8a9b36e8ba5542f1a008d91570fd91622023-02-09T00:00:46ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352023-01-01161740175510.1109/JSTARS.2023.323750910018300Deep Learning Approach for Classifying the Built Year and Structure of Individual Buildings by Automatically Linking Street View Images and GIS Building DataYoshiki Ogawa0https://orcid.org/0000-0002-1987-4520Chenbo Zhao1https://orcid.org/0000-0001-9476-7787Takuya Oki2https://orcid.org/0000-0002-4848-0707Shenglong Chen3https://orcid.org/0000-0003-4252-2846Yoshihide Sekimoto4Center for Spatial Information Science, University of Tokyo, Tokyo, JapanDepartment of Civil Engineering, University of Tokyo, Tokyo, JapanSchool of Environment and Society, Tokyo Institute of Technology, Tokyo, JapanDepartment of Civil Engineering, University of Tokyo, Tokyo, JapanCenter for Spatial Information Science, University of Tokyo, Tokyo, JapanThe built year and structure of individual buildings are crucial factors for estimating and assessing potential earthquake and tsunami damage. Recent advances in sensing and analysis technologies allow the acquisition of high-resolution street view images (SVIs) that present new possibilities for research and development. In this study, we developed a model to estimate the built year and structure of a building using omnidirectional SVIs captured using an onboard camera. We used geographic information system (GIS) building data and SVIs to generate an annotated built-year and structure dataset by developing a method to automatically combine the GIS data with images of individual buildings cropped through object detection. Furthermore, we trained a deep learning model to classify the built year and structure of buildings using the annotated image dataset based on a deep convolutional neural network (DCNN) and a vision transformer (ViT). The results showed that SVI accurately predicts the built year and structure of individual buildings using ViT (overall accuracies for structure = 0.94 [three classes] and 0.96 [two classes] and for age = 0.68 [six classes] and 0.90 [three classes]). Compared with DCNN-based networks, the proposed Swin transformer based on ViT architectures effectively improves prediction accuracy. The results indicate that multiple high-resolution images can be obtained for individual buildings using SVI, and the proposed method is an effective approach for classifying structures and determining building age. The automatic, accurate, and large-scale mapping of the built year and structure of individual buildings can help develop specific disaster prevention measures.https://ieeexplore.ieee.org/document/10018300/Building identificationdeep learningobject detectionstreet view images (SVIs)Swin transformer |
| spellingShingle | Yoshiki Ogawa Chenbo Zhao Takuya Oki Shenglong Chen Yoshihide Sekimoto Deep Learning Approach for Classifying the Built Year and Structure of Individual Buildings by Automatically Linking Street View Images and GIS Building Data IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Building identification deep learning object detection street view images (SVIs) Swin transformer |
| title | Deep Learning Approach for Classifying the Built Year and Structure of Individual Buildings by Automatically Linking Street View Images and GIS Building Data |
| title_full | Deep Learning Approach for Classifying the Built Year and Structure of Individual Buildings by Automatically Linking Street View Images and GIS Building Data |
| title_fullStr | Deep Learning Approach for Classifying the Built Year and Structure of Individual Buildings by Automatically Linking Street View Images and GIS Building Data |
| title_full_unstemmed | Deep Learning Approach for Classifying the Built Year and Structure of Individual Buildings by Automatically Linking Street View Images and GIS Building Data |
| title_short | Deep Learning Approach for Classifying the Built Year and Structure of Individual Buildings by Automatically Linking Street View Images and GIS Building Data |
| title_sort | deep learning approach for classifying the built year and structure of individual buildings by automatically linking street view images and gis building data |
| topic | Building identification deep learning object detection street view images (SVIs) Swin transformer |
| url | https://ieeexplore.ieee.org/document/10018300/ |
| work_keys_str_mv | AT yoshikiogawa deeplearningapproachforclassifyingthebuiltyearandstructureofindividualbuildingsbyautomaticallylinkingstreetviewimagesandgisbuildingdata AT chenbozhao deeplearningapproachforclassifyingthebuiltyearandstructureofindividualbuildingsbyautomaticallylinkingstreetviewimagesandgisbuildingdata AT takuyaoki deeplearningapproachforclassifyingthebuiltyearandstructureofindividualbuildingsbyautomaticallylinkingstreetviewimagesandgisbuildingdata AT shenglongchen deeplearningapproachforclassifyingthebuiltyearandstructureofindividualbuildingsbyautomaticallylinkingstreetviewimagesandgisbuildingdata AT yoshihidesekimoto deeplearningapproachforclassifyingthebuiltyearandstructureofindividualbuildingsbyautomaticallylinkingstreetviewimagesandgisbuildingdata |