Federated Onboard-Ground Station Computing With Weakly Supervised Cascading Pyramid Attention Network for Satellite Image Analysis
With advances in NanoSat (CubeSat) and high-resolution sensors, the amount of raw data to be analyzed by human supervisors has been explosively increasing for satellite image analysis. To reduce the raw data, the satellite onboard AI processing with low-power COTS (Commercial, Off-The-Shelf) HW has...
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2022-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9940296/ |
| _version_ | 1811225520683089920 |
|---|---|
| author | Taewoo Kim Minsu Jeon Changha Lee Junsoo Kim Geonwoo Ko Joo-Young Kim Chan-Hyun Youn |
| author_facet | Taewoo Kim Minsu Jeon Changha Lee Junsoo Kim Geonwoo Ko Joo-Young Kim Chan-Hyun Youn |
| author_sort | Taewoo Kim |
| collection | DOAJ |
| description | With advances in NanoSat (CubeSat) and high-resolution sensors, the amount of raw data to be analyzed by human supervisors has been explosively increasing for satellite image analysis. To reduce the raw data, the satellite onboard AI processing with low-power COTS (Commercial, Off-The-Shelf) HW has emerged from a real satellite mission. It filters the useless data (e.g. cloudy images) that is worthless to supervisors, achieving efficient satellite-ground station communication. In the application for complex object recognition, however, additional explanation is required for the reliability of the AI prediction due to its low performance. Although various explainable AI (XAI) methods for providing human-interpretable explanation have been studied, the pyramid architecture in a deep network leads to the background bias problem which visual explanation only focuses on the background context around the object. Missing the small objects in a tiny region leads to poor explainability although the AI model corrects the object class. To resolve the problems, we propose a novel federated onboard-ground station (FOGS) computing with Cascading Pyramid Attention Network (CPANet) for reliable onboard XAI in object recognition. We present an XAI architecture with a cascading attention mechanism for mitigating the background bias for the onboard processing. By exploiting the localization ability in pyramid feature blocks, we can extract high-quality visual explanation covering the both semantic and small contexts of an object. For enhancing visual explainability of complex satellite images, we also describe a novel computing federation with the ground station and supervisors. In the ground station, active learning-based sample selection and attention refinement scheme with a simple feedback method are conducted to achieve the robustness of explanation and efficient supervisor’s annotation cost, simultaneously. Experiments on various datasets show that the proposed system improves task accuracy in object recognition and accurate visual explanation detecting small contexts of objects even in a peripheral region. Then, our attention refinement mechanism demonstrates that the inconsistent explanation can be efficiently resolved only with very simple selection-based feedback. |
| first_indexed | 2024-04-12T09:08:35Z |
| format | Article |
| id | doaj.art-d4bcbf2aab644f2aa9e4d14b04acbec4 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-04-12T09:08:35Z |
| publishDate | 2022-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-d4bcbf2aab644f2aa9e4d14b04acbec42022-12-22T03:39:02ZengIEEEIEEE Access2169-35362022-01-011011731511733310.1109/ACCESS.2022.32198799940296Federated Onboard-Ground Station Computing With Weakly Supervised Cascading Pyramid Attention Network for Satellite Image AnalysisTaewoo Kim0https://orcid.org/0000-0003-4290-6460Minsu Jeon1https://orcid.org/0000-0002-2739-8149Changha Lee2https://orcid.org/0000-0003-3687-2989Junsoo Kim3https://orcid.org/0000-0001-6680-2602Geonwoo Ko4Joo-Young Kim5https://orcid.org/0000-0003-1099-1496Chan-Hyun Youn6https://orcid.org/0000-0002-3970-7308School of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South KoreaSchool of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South KoreaSchool of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South KoreaSchool of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South KoreaSchool of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South KoreaSchool of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South KoreaSchool of Electrical Engineering, Korea Advanced Institute of Science and Technology, Daejeon, South KoreaWith advances in NanoSat (CubeSat) and high-resolution sensors, the amount of raw data to be analyzed by human supervisors has been explosively increasing for satellite image analysis. To reduce the raw data, the satellite onboard AI processing with low-power COTS (Commercial, Off-The-Shelf) HW has emerged from a real satellite mission. It filters the useless data (e.g. cloudy images) that is worthless to supervisors, achieving efficient satellite-ground station communication. In the application for complex object recognition, however, additional explanation is required for the reliability of the AI prediction due to its low performance. Although various explainable AI (XAI) methods for providing human-interpretable explanation have been studied, the pyramid architecture in a deep network leads to the background bias problem which visual explanation only focuses on the background context around the object. Missing the small objects in a tiny region leads to poor explainability although the AI model corrects the object class. To resolve the problems, we propose a novel federated onboard-ground station (FOGS) computing with Cascading Pyramid Attention Network (CPANet) for reliable onboard XAI in object recognition. We present an XAI architecture with a cascading attention mechanism for mitigating the background bias for the onboard processing. By exploiting the localization ability in pyramid feature blocks, we can extract high-quality visual explanation covering the both semantic and small contexts of an object. For enhancing visual explainability of complex satellite images, we also describe a novel computing federation with the ground station and supervisors. In the ground station, active learning-based sample selection and attention refinement scheme with a simple feedback method are conducted to achieve the robustness of explanation and efficient supervisor’s annotation cost, simultaneously. Experiments on various datasets show that the proposed system improves task accuracy in object recognition and accurate visual explanation detecting small contexts of objects even in a peripheral region. Then, our attention refinement mechanism demonstrates that the inconsistent explanation can be efficiently resolved only with very simple selection-based feedback.https://ieeexplore.ieee.org/document/9940296/XAIvisual explanationsatellite image analysishuman-in-the-loop |
| spellingShingle | Taewoo Kim Minsu Jeon Changha Lee Junsoo Kim Geonwoo Ko Joo-Young Kim Chan-Hyun Youn Federated Onboard-Ground Station Computing With Weakly Supervised Cascading Pyramid Attention Network for Satellite Image Analysis IEEE Access XAI visual explanation satellite image analysis human-in-the-loop |
| title | Federated Onboard-Ground Station Computing With Weakly Supervised Cascading Pyramid Attention Network for Satellite Image Analysis |
| title_full | Federated Onboard-Ground Station Computing With Weakly Supervised Cascading Pyramid Attention Network for Satellite Image Analysis |
| title_fullStr | Federated Onboard-Ground Station Computing With Weakly Supervised Cascading Pyramid Attention Network for Satellite Image Analysis |
| title_full_unstemmed | Federated Onboard-Ground Station Computing With Weakly Supervised Cascading Pyramid Attention Network for Satellite Image Analysis |
| title_short | Federated Onboard-Ground Station Computing With Weakly Supervised Cascading Pyramid Attention Network for Satellite Image Analysis |
| title_sort | federated onboard ground station computing with weakly supervised cascading pyramid attention network for satellite image analysis |
| topic | XAI visual explanation satellite image analysis human-in-the-loop |
| url | https://ieeexplore.ieee.org/document/9940296/ |
| work_keys_str_mv | AT taewookim federatedonboardgroundstationcomputingwithweaklysupervisedcascadingpyramidattentionnetworkforsatelliteimageanalysis AT minsujeon federatedonboardgroundstationcomputingwithweaklysupervisedcascadingpyramidattentionnetworkforsatelliteimageanalysis AT changhalee federatedonboardgroundstationcomputingwithweaklysupervisedcascadingpyramidattentionnetworkforsatelliteimageanalysis AT junsookim federatedonboardgroundstationcomputingwithweaklysupervisedcascadingpyramidattentionnetworkforsatelliteimageanalysis AT geonwooko federatedonboardgroundstationcomputingwithweaklysupervisedcascadingpyramidattentionnetworkforsatelliteimageanalysis AT jooyoungkim federatedonboardgroundstationcomputingwithweaklysupervisedcascadingpyramidattentionnetworkforsatelliteimageanalysis AT chanhyunyoun federatedonboardgroundstationcomputingwithweaklysupervisedcascadingpyramidattentionnetworkforsatelliteimageanalysis |