A Multimodal Data Fusion and Deep Learning Framework for Large-Scale Wildfire Surface Fuel Mapping
Accurate estimation of fuels is essential for wildland fire simulations as well as decision-making related to land management. Numerous research efforts have leveraged remote sensing and machine learning for classifying land cover and mapping forest vegetation species. In most cases that focused on...
| Main Authors: | , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Fire |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2571-6255/6/2/36 |
| _version_ | 1827757518571438080 |
|---|---|
| author | Mohamad Alipour Inga La Puma Joshua Picotte Kasra Shamsaei Eric Rowell Adam Watts Branko Kosovic Hamed Ebrahimian Ertugrul Taciroglu |
| author_facet | Mohamad Alipour Inga La Puma Joshua Picotte Kasra Shamsaei Eric Rowell Adam Watts Branko Kosovic Hamed Ebrahimian Ertugrul Taciroglu |
| author_sort | Mohamad Alipour |
| collection | DOAJ |
| description | Accurate estimation of fuels is essential for wildland fire simulations as well as decision-making related to land management. Numerous research efforts have leveraged remote sensing and machine learning for classifying land cover and mapping forest vegetation species. In most cases that focused on surface fuel mapping, the spatial scale of interest was smaller than a few hundred square kilometers; thus, many small-scale site-specific models had to be created to cover the landscape at the national scale. The present work aims to develop a large-scale surface fuel identification model using a custom deep learning framework that can ingest multimodal data. Specifically, we use deep learning to extract information from multispectral signatures, high-resolution imagery, and biophysical climate and terrain data in a way that facilitates their end-to-end training on labeled data. A multi-layer neural network is used with spectral and biophysical data, and a convolutional neural network backbone is used to extract the visual features from high-resolution imagery. A Monte Carlo dropout mechanism was also devised to create a stochastic ensemble of models that can capture classification uncertainties while boosting the prediction performance. To train the system as a proof-of-concept, fuel pseudo-labels were created by a random geospatial sampling of existing fuel maps across California. Application results on independent test sets showed promising fuel identification performance with an overall accuracy ranging from 55% to 75%, depending on the level of granularity of the included fuel types. As expected, including the rare—and possibly less consequential—fuel types reduced the accuracy. On the other hand, the addition of high-resolution imagery improved classification performance at all levels. |
| first_indexed | 2024-03-11T08:50:34Z |
| format | Article |
| id | doaj.art-264b5af38d3241de9f1026fe214f1738 |
| institution | Directory Open Access Journal |
| issn | 2571-6255 |
| language | English |
| last_indexed | 2024-03-11T08:50:34Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Fire |
| spelling | doaj.art-264b5af38d3241de9f1026fe214f17382023-11-16T20:27:00ZengMDPI AGFire2571-62552023-01-01623610.3390/fire6020036A Multimodal Data Fusion and Deep Learning Framework for Large-Scale Wildfire Surface Fuel MappingMohamad Alipour0Inga La Puma1Joshua Picotte2Kasra Shamsaei3Eric Rowell4Adam Watts5Branko Kosovic6Hamed Ebrahimian7Ertugrul Taciroglu8Civil & Environmental Engineering Department, University of Illinois at Urbana-Champaign, Urbana, IL 61801, USAKBR, Contractor to the USGS, Sioux Falls, SD 57198, USAEarth Resources Observation & Science Center, United States Geological Survey, Sioux Falls, SD 57198, USACivil & Environmental Engineering Department, University of Nevada Reno, Reno, NV 89557, USADivision of Atmospheric Sciences, Desert Research Institute, Reno, NV 89512, USAPacific Wildland Fire Sciences Laboratory, United States Forest Service, Wenatchee, WA 98801, USAResearch Applications Laboratory, National Center for Atmospheric Research, Boulder, CO 80305, USACivil & Environmental Engineering Department, University of Nevada Reno, Reno, NV 89557, USACivil & Environmental Engineering Department, University of California Los Angeles, Los Angeles, CA 90095, USAAccurate estimation of fuels is essential for wildland fire simulations as well as decision-making related to land management. Numerous research efforts have leveraged remote sensing and machine learning for classifying land cover and mapping forest vegetation species. In most cases that focused on surface fuel mapping, the spatial scale of interest was smaller than a few hundred square kilometers; thus, many small-scale site-specific models had to be created to cover the landscape at the national scale. The present work aims to develop a large-scale surface fuel identification model using a custom deep learning framework that can ingest multimodal data. Specifically, we use deep learning to extract information from multispectral signatures, high-resolution imagery, and biophysical climate and terrain data in a way that facilitates their end-to-end training on labeled data. A multi-layer neural network is used with spectral and biophysical data, and a convolutional neural network backbone is used to extract the visual features from high-resolution imagery. A Monte Carlo dropout mechanism was also devised to create a stochastic ensemble of models that can capture classification uncertainties while boosting the prediction performance. To train the system as a proof-of-concept, fuel pseudo-labels were created by a random geospatial sampling of existing fuel maps across California. Application results on independent test sets showed promising fuel identification performance with an overall accuracy ranging from 55% to 75%, depending on the level of granularity of the included fuel types. As expected, including the rare—and possibly less consequential—fuel types reduced the accuracy. On the other hand, the addition of high-resolution imagery improved classification performance at all levels.https://www.mdpi.com/2571-6255/6/2/36wildland firefuel mappingremote sensingartificial intelligencemachine learningdeep learning |
| spellingShingle | Mohamad Alipour Inga La Puma Joshua Picotte Kasra Shamsaei Eric Rowell Adam Watts Branko Kosovic Hamed Ebrahimian Ertugrul Taciroglu A Multimodal Data Fusion and Deep Learning Framework for Large-Scale Wildfire Surface Fuel Mapping Fire wildland fire fuel mapping remote sensing artificial intelligence machine learning deep learning |
| title | A Multimodal Data Fusion and Deep Learning Framework for Large-Scale Wildfire Surface Fuel Mapping |
| title_full | A Multimodal Data Fusion and Deep Learning Framework for Large-Scale Wildfire Surface Fuel Mapping |
| title_fullStr | A Multimodal Data Fusion and Deep Learning Framework for Large-Scale Wildfire Surface Fuel Mapping |
| title_full_unstemmed | A Multimodal Data Fusion and Deep Learning Framework for Large-Scale Wildfire Surface Fuel Mapping |
| title_short | A Multimodal Data Fusion and Deep Learning Framework for Large-Scale Wildfire Surface Fuel Mapping |
| title_sort | multimodal data fusion and deep learning framework for large scale wildfire surface fuel mapping |
| topic | wildland fire fuel mapping remote sensing artificial intelligence machine learning deep learning |
| url | https://www.mdpi.com/2571-6255/6/2/36 |
| work_keys_str_mv | AT mohamadalipour amultimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT ingalapuma amultimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT joshuapicotte amultimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT kasrashamsaei amultimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT ericrowell amultimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT adamwatts amultimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT brankokosovic amultimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT hamedebrahimian amultimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT ertugrultaciroglu amultimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT mohamadalipour multimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT ingalapuma multimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT joshuapicotte multimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT kasrashamsaei multimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT ericrowell multimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT adamwatts multimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT brankokosovic multimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT hamedebrahimian multimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping AT ertugrultaciroglu multimodaldatafusionanddeeplearningframeworkforlargescalewildfiresurfacefuelmapping |