Technical Solution Discussion for Key Challenges of Operational Convolutional Neural Network-Based Building-Damage Assessment from Satellite Imagery: Perspective from Benchmark xBD Dataset
Earth Observation satellite imaging helps building diagnosis during a disaster. Several models are put forward on the xBD dataset, which can be divided into two levels: the building level and the pixel level. Models from two levels evolve into several versions that will be reviewed in this paper. Th...
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-11-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/12/22/3808 |
| _version_ | 1797547292613410816 |
|---|---|
| author | Jinhua Su Yanbing Bai Xingrui Wang Dong Lu Bo Zhao Hanfang Yang Erick Mas Shunichi Koshimura |
| author_facet | Jinhua Su Yanbing Bai Xingrui Wang Dong Lu Bo Zhao Hanfang Yang Erick Mas Shunichi Koshimura |
| author_sort | Jinhua Su |
| collection | DOAJ |
| description | Earth Observation satellite imaging helps building diagnosis during a disaster. Several models are put forward on the xBD dataset, which can be divided into two levels: the building level and the pixel level. Models from two levels evolve into several versions that will be reviewed in this paper. There are four key challenges hindering researchers from moving forward on this task, and this paper tries to give technical solutions. First, metrics on different levels could not be compared directly. We put forward a fairer metric and give a method to convert between metrics of two levels. Secondly, drone images may be another important source, but drone data may have only a post-disaster image. This paper shows and compares methods of directly detecting and generating. Thirdly, the class imbalance is a typical feature of the xBD dataset and leads to a bad F1 score for minor damage and major damage. This paper provides four specific data resampling strategies, which are Main-Label Over-Sampling (MLOS), Discrimination After Cropping (DAC), Dilation of Area with Minority (DAM) and Synthetic Minority Over-Sampling Technique (SMOTE), as well as cost-sensitive re-weighting schemes. Fourthly, faster prediction meets the need for a real-time situation. This paper recommends three specific methods, feature-map subtraction, parameter sharing, and knowledge distillation. Finally, we developed our AI-driven Damage Diagnose Platform (ADDP). This paper introduces the structure of ADDP and technical details. Customized settings, interface preview, and upload and download satellite images are major services our platform provides. |
| first_indexed | 2024-03-10T14:42:17Z |
| format | Article |
| id | doaj.art-bb6edc2acdcb4a8d8c84fdbc1a0fb10f |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T14:42:17Z |
| publishDate | 2020-11-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-bb6edc2acdcb4a8d8c84fdbc1a0fb10f2023-11-20T21:44:54ZengMDPI AGRemote Sensing2072-42922020-11-011222380810.3390/rs12223808Technical Solution Discussion for Key Challenges of Operational Convolutional Neural Network-Based Building-Damage Assessment from Satellite Imagery: Perspective from Benchmark xBD DatasetJinhua Su0Yanbing Bai1Xingrui Wang2Dong Lu3Bo Zhao4Hanfang Yang5Erick Mas6Shunichi Koshimura7Center for Applied Statistics, School of Statistics, Renmin University of China, Beijing 100872, ChinaCenter for Applied Statistics, School of Statistics, Renmin University of China, Beijing 100872, ChinaCenter for Applied Statistics, School of Statistics, Renmin University of China, Beijing 100872, ChinaThe School of Computer Science and Engineering, The University of New South Wales, Sydney, NSW 2052, AustraliaSchool of Informatics, The University of Edinburgh, Edinburgh EH8 9AB, UKCenter for Applied Statistics, School of Statistics, Renmin University of China, Beijing 100872, ChinaInternational Research Institute of Disaster Science, Tohoku University, Sendai 980-8572, JapanInternational Research Institute of Disaster Science, Tohoku University, Sendai 980-8572, JapanEarth Observation satellite imaging helps building diagnosis during a disaster. Several models are put forward on the xBD dataset, which can be divided into two levels: the building level and the pixel level. Models from two levels evolve into several versions that will be reviewed in this paper. There are four key challenges hindering researchers from moving forward on this task, and this paper tries to give technical solutions. First, metrics on different levels could not be compared directly. We put forward a fairer metric and give a method to convert between metrics of two levels. Secondly, drone images may be another important source, but drone data may have only a post-disaster image. This paper shows and compares methods of directly detecting and generating. Thirdly, the class imbalance is a typical feature of the xBD dataset and leads to a bad F1 score for minor damage and major damage. This paper provides four specific data resampling strategies, which are Main-Label Over-Sampling (MLOS), Discrimination After Cropping (DAC), Dilation of Area with Minority (DAM) and Synthetic Minority Over-Sampling Technique (SMOTE), as well as cost-sensitive re-weighting schemes. Fourthly, faster prediction meets the need for a real-time situation. This paper recommends three specific methods, feature-map subtraction, parameter sharing, and knowledge distillation. Finally, we developed our AI-driven Damage Diagnose Platform (ADDP). This paper introduces the structure of ADDP and technical details. Customized settings, interface preview, and upload and download satellite images are major services our platform provides.https://www.mdpi.com/2072-4292/12/22/3808convolutional neural networkbuilding-damage assessmentbenchmark xBD datasetdisaster response online platform |
| spellingShingle | Jinhua Su Yanbing Bai Xingrui Wang Dong Lu Bo Zhao Hanfang Yang Erick Mas Shunichi Koshimura Technical Solution Discussion for Key Challenges of Operational Convolutional Neural Network-Based Building-Damage Assessment from Satellite Imagery: Perspective from Benchmark xBD Dataset Remote Sensing convolutional neural network building-damage assessment benchmark xBD dataset disaster response online platform |
| title | Technical Solution Discussion for Key Challenges of Operational Convolutional Neural Network-Based Building-Damage Assessment from Satellite Imagery: Perspective from Benchmark xBD Dataset |
| title_full | Technical Solution Discussion for Key Challenges of Operational Convolutional Neural Network-Based Building-Damage Assessment from Satellite Imagery: Perspective from Benchmark xBD Dataset |
| title_fullStr | Technical Solution Discussion for Key Challenges of Operational Convolutional Neural Network-Based Building-Damage Assessment from Satellite Imagery: Perspective from Benchmark xBD Dataset |
| title_full_unstemmed | Technical Solution Discussion for Key Challenges of Operational Convolutional Neural Network-Based Building-Damage Assessment from Satellite Imagery: Perspective from Benchmark xBD Dataset |
| title_short | Technical Solution Discussion for Key Challenges of Operational Convolutional Neural Network-Based Building-Damage Assessment from Satellite Imagery: Perspective from Benchmark xBD Dataset |
| title_sort | technical solution discussion for key challenges of operational convolutional neural network based building damage assessment from satellite imagery perspective from benchmark xbd dataset |
| topic | convolutional neural network building-damage assessment benchmark xBD dataset disaster response online platform |
| url | https://www.mdpi.com/2072-4292/12/22/3808 |
| work_keys_str_mv | AT jinhuasu technicalsolutiondiscussionforkeychallengesofoperationalconvolutionalneuralnetworkbasedbuildingdamageassessmentfromsatelliteimageryperspectivefrombenchmarkxbddataset AT yanbingbai technicalsolutiondiscussionforkeychallengesofoperationalconvolutionalneuralnetworkbasedbuildingdamageassessmentfromsatelliteimageryperspectivefrombenchmarkxbddataset AT xingruiwang technicalsolutiondiscussionforkeychallengesofoperationalconvolutionalneuralnetworkbasedbuildingdamageassessmentfromsatelliteimageryperspectivefrombenchmarkxbddataset AT donglu technicalsolutiondiscussionforkeychallengesofoperationalconvolutionalneuralnetworkbasedbuildingdamageassessmentfromsatelliteimageryperspectivefrombenchmarkxbddataset AT bozhao technicalsolutiondiscussionforkeychallengesofoperationalconvolutionalneuralnetworkbasedbuildingdamageassessmentfromsatelliteimageryperspectivefrombenchmarkxbddataset AT hanfangyang technicalsolutiondiscussionforkeychallengesofoperationalconvolutionalneuralnetworkbasedbuildingdamageassessmentfromsatelliteimageryperspectivefrombenchmarkxbddataset AT erickmas technicalsolutiondiscussionforkeychallengesofoperationalconvolutionalneuralnetworkbasedbuildingdamageassessmentfromsatelliteimageryperspectivefrombenchmarkxbddataset AT shunichikoshimura technicalsolutiondiscussionforkeychallengesofoperationalconvolutionalneuralnetworkbasedbuildingdamageassessmentfromsatelliteimageryperspectivefrombenchmarkxbddataset |