Automatic analysis of social media images to identify disaster type and infer appropriate emergency response
Abstract Social media postings are increasingly being used in modern days disaster management. Along with the textual information, the contexts and cues inherent in the images posted on social media play an important role in identifying appropriate emergency responses to a particular disaster. In th...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SpringerOpen
2021-06-01
|
| Series: | Journal of Big Data |
| Subjects: | |
| Online Access: | https://doi.org/10.1186/s40537-021-00471-5 |
| _version_ | 1818865462218326016 |
|---|---|
| author | Amna Asif Shaheen Khatoon Md Maruf Hasan Majed A. Alshamari Sherif Abdou Khaled Mostafa Elsayed Mohsen Rashwan |
| author_facet | Amna Asif Shaheen Khatoon Md Maruf Hasan Majed A. Alshamari Sherif Abdou Khaled Mostafa Elsayed Mohsen Rashwan |
| author_sort | Amna Asif |
| collection | DOAJ |
| description | Abstract Social media postings are increasingly being used in modern days disaster management. Along with the textual information, the contexts and cues inherent in the images posted on social media play an important role in identifying appropriate emergency responses to a particular disaster. In this paper, we proposed a disaster taxonomy of emergency response and used the same taxonomy with an emergency response pipeline together with deep-learning-based image classification and object identification algorithms to automate the emergency response decision-making process. We used the card sorting method to validate the completeness and correctness of the disaster taxonomy. We also used VGG-16 and You Only Look Once (YOLO) algorithms to analyze disaster-related images and identify disaster types and relevant cues (such as objects that appeared in those images). Furthermore, using decision tables and applied analytic hierarchy processes (AHP), we aligned the intermediate outputs to map a disaster-related image into the disaster taxonomy and determine an appropriate type of emergency response for a given disaster. The proposed approach has been validated using Earthquake, Hurricane, and Typhoon as use cases. The results show that 96% of images were categorized correctly on disaster taxonomy using YOLOv4. The accuracy can be further improved using an incremental training approach. Due to the use of cloud-based deep learning algorithms in image analysis, our approach can potentially be useful to real-time crisis management. The algorithms along with the proposed emergency response pipeline can be further enhanced with other spatiotemporal features extracted from multimedia information posted on social media. |
| first_indexed | 2024-12-19T10:47:56Z |
| format | Article |
| id | doaj.art-61e442eb6f0346db9944458926886531 |
| institution | Directory Open Access Journal |
| issn | 2196-1115 |
| language | English |
| last_indexed | 2024-12-19T10:47:56Z |
| publishDate | 2021-06-01 |
| publisher | SpringerOpen |
| record_format | Article |
| series | Journal of Big Data |
| spelling | doaj.art-61e442eb6f0346db99444589268865312022-12-21T20:25:10ZengSpringerOpenJournal of Big Data2196-11152021-06-018112810.1186/s40537-021-00471-5Automatic analysis of social media images to identify disaster type and infer appropriate emergency responseAmna Asif0Shaheen Khatoon1Md Maruf Hasan2Majed A. Alshamari3Sherif Abdou4Khaled Mostafa Elsayed5Mohsen Rashwan6Information Systems Department, College of Computer Sciences & Information Technology, King Faisal University (KFU)Information Systems Department, College of Computer Sciences & Information Technology, King Faisal University (KFU)Information Systems Department, College of Computer Sciences & Information Technology, King Faisal University (KFU)Information Systems Department, College of Computer Sciences & Information Technology, King Faisal University (KFU)Faculty of Computers and Artificial Intelligence, Cairo UniversityFaculty of Computing & Information, Cairo UniversityFaculty of Engineering, Cairo UniversityAbstract Social media postings are increasingly being used in modern days disaster management. Along with the textual information, the contexts and cues inherent in the images posted on social media play an important role in identifying appropriate emergency responses to a particular disaster. In this paper, we proposed a disaster taxonomy of emergency response and used the same taxonomy with an emergency response pipeline together with deep-learning-based image classification and object identification algorithms to automate the emergency response decision-making process. We used the card sorting method to validate the completeness and correctness of the disaster taxonomy. We also used VGG-16 and You Only Look Once (YOLO) algorithms to analyze disaster-related images and identify disaster types and relevant cues (such as objects that appeared in those images). Furthermore, using decision tables and applied analytic hierarchy processes (AHP), we aligned the intermediate outputs to map a disaster-related image into the disaster taxonomy and determine an appropriate type of emergency response for a given disaster. The proposed approach has been validated using Earthquake, Hurricane, and Typhoon as use cases. The results show that 96% of images were categorized correctly on disaster taxonomy using YOLOv4. The accuracy can be further improved using an incremental training approach. Due to the use of cloud-based deep learning algorithms in image analysis, our approach can potentially be useful to real-time crisis management. The algorithms along with the proposed emergency response pipeline can be further enhanced with other spatiotemporal features extracted from multimedia information posted on social media.https://doi.org/10.1186/s40537-021-00471-5Convolutional neural networksImage classificationObject detectionDisaster management |
| spellingShingle | Amna Asif Shaheen Khatoon Md Maruf Hasan Majed A. Alshamari Sherif Abdou Khaled Mostafa Elsayed Mohsen Rashwan Automatic analysis of social media images to identify disaster type and infer appropriate emergency response Journal of Big Data Convolutional neural networks Image classification Object detection Disaster management |
| title | Automatic analysis of social media images to identify disaster type and infer appropriate emergency response |
| title_full | Automatic analysis of social media images to identify disaster type and infer appropriate emergency response |
| title_fullStr | Automatic analysis of social media images to identify disaster type and infer appropriate emergency response |
| title_full_unstemmed | Automatic analysis of social media images to identify disaster type and infer appropriate emergency response |
| title_short | Automatic analysis of social media images to identify disaster type and infer appropriate emergency response |
| title_sort | automatic analysis of social media images to identify disaster type and infer appropriate emergency response |
| topic | Convolutional neural networks Image classification Object detection Disaster management |
| url | https://doi.org/10.1186/s40537-021-00471-5 |
| work_keys_str_mv | AT amnaasif automaticanalysisofsocialmediaimagestoidentifydisastertypeandinferappropriateemergencyresponse AT shaheenkhatoon automaticanalysisofsocialmediaimagestoidentifydisastertypeandinferappropriateemergencyresponse AT mdmarufhasan automaticanalysisofsocialmediaimagestoidentifydisastertypeandinferappropriateemergencyresponse AT majedaalshamari automaticanalysisofsocialmediaimagestoidentifydisastertypeandinferappropriateemergencyresponse AT sherifabdou automaticanalysisofsocialmediaimagestoidentifydisastertypeandinferappropriateemergencyresponse AT khaledmostafaelsayed automaticanalysisofsocialmediaimagestoidentifydisastertypeandinferappropriateemergencyresponse AT mohsenrashwan automaticanalysisofsocialmediaimagestoidentifydisastertypeandinferappropriateemergencyresponse |