Training Compact Change Detection Network for Remote Sensing Imagery
Change Detection (CD) is a hot remote sensing topic where the change zones are highlighted by analyzing bi-temporal or multi-temporal images. Recently, Deep learning (DL) paved the road to implement various reliable change detection approaches that overcome traditional CD methods limitation. However...
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9456864/ |
| _version_ | 1819146273676066816 |
|---|---|
| author | Amira S. Mahmoud Sayed A. Mohamed Marwa S. Moustafa Reda A. El-Khorib Hisham M. Abdelsalam Ihab A. El-Khodary |
| author_facet | Amira S. Mahmoud Sayed A. Mohamed Marwa S. Moustafa Reda A. El-Khorib Hisham M. Abdelsalam Ihab A. El-Khodary |
| author_sort | Amira S. Mahmoud |
| collection | DOAJ |
| description | Change Detection (CD) is a hot remote sensing topic where the change zones are highlighted by analyzing bi-temporal or multi-temporal images. Recently, Deep learning (DL) paved the road to implement various reliable change detection approaches that overcome traditional CD methods limitation. However, high performance DL based approaches have explosion number of parameters that demanded extensive computation and memory usage in addition to large volumes of training data. To address this issue, we proposed a teacher-student setting for remote sensing imagery change detection. To distill the knowledge from the over-parameterized Siamese teacher network, we proposed tiny student network that was trained using the obtained categorical distribution of probability from the teacher paired Softmax output at high temperature. Practical Swarm Optimization (PSO) was applied in order to optimally configure student architecture. Finally, ample experiments were conducted on LEVIR-CD dataset. Also, we introduced EGSAR-CD dataset, which contains of a large set of bi-temporal SAR images with 460 image pairs (<inline-formula> <tex-math notation="LaTeX">$256 \times 256$ </tex-math></inline-formula>). Experiment results indicate that we can reach up to <inline-formula> <tex-math notation="LaTeX">$5.4\times $ </tex-math></inline-formula> reduction rate in number of parameters with loss of accuracy between 5% and 6% on the LEVIR-CD and EGSAR-CD datasets utilizing self-knowledge distillation. |
| first_indexed | 2024-12-22T13:11:18Z |
| format | Article |
| id | doaj.art-83a00db3778e49af994052a9cc5b5250 |
| institution | Directory Open Access Journal |
| issn | 2169-3536 |
| language | English |
| last_indexed | 2024-12-22T13:11:18Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj.art-83a00db3778e49af994052a9cc5b52502022-12-21T18:24:44ZengIEEEIEEE Access2169-35362021-01-019903669037810.1109/ACCESS.2021.30897669456864Training Compact Change Detection Network for Remote Sensing ImageryAmira S. Mahmoud0https://orcid.org/0000-0003-2738-0819Sayed A. Mohamed1Marwa S. Moustafa2https://orcid.org/0000-0003-3805-9668Reda A. El-Khorib3Hisham M. Abdelsalam4Ihab A. El-Khodary5National Authority for Remote Sensing and Space Science, Cairo, EgyptNational Authority for Remote Sensing and Space Science, Cairo, EgyptNational Authority for Remote Sensing and Space Science, Cairo, EgyptFaculty of Computers and Artificial Intelligence, Cairo University, Giza, EgyptFaculty of Computers and Artificial Intelligence, Cairo University, Giza, EgyptFaculty of Computers and Artificial Intelligence, Cairo University, Giza, EgyptChange Detection (CD) is a hot remote sensing topic where the change zones are highlighted by analyzing bi-temporal or multi-temporal images. Recently, Deep learning (DL) paved the road to implement various reliable change detection approaches that overcome traditional CD methods limitation. However, high performance DL based approaches have explosion number of parameters that demanded extensive computation and memory usage in addition to large volumes of training data. To address this issue, we proposed a teacher-student setting for remote sensing imagery change detection. To distill the knowledge from the over-parameterized Siamese teacher network, we proposed tiny student network that was trained using the obtained categorical distribution of probability from the teacher paired Softmax output at high temperature. Practical Swarm Optimization (PSO) was applied in order to optimally configure student architecture. Finally, ample experiments were conducted on LEVIR-CD dataset. Also, we introduced EGSAR-CD dataset, which contains of a large set of bi-temporal SAR images with 460 image pairs (<inline-formula> <tex-math notation="LaTeX">$256 \times 256$ </tex-math></inline-formula>). Experiment results indicate that we can reach up to <inline-formula> <tex-math notation="LaTeX">$5.4\times $ </tex-math></inline-formula> reduction rate in number of parameters with loss of accuracy between 5% and 6% on the LEVIR-CD and EGSAR-CD datasets utilizing self-knowledge distillation.https://ieeexplore.ieee.org/document/9456864/Change detection (CD)deep learning (DL)knowledge distillation (KD)Siamese networkteacher-student settingpractical swarm optimization (PSO) |
| spellingShingle | Amira S. Mahmoud Sayed A. Mohamed Marwa S. Moustafa Reda A. El-Khorib Hisham M. Abdelsalam Ihab A. El-Khodary Training Compact Change Detection Network for Remote Sensing Imagery IEEE Access Change detection (CD) deep learning (DL) knowledge distillation (KD) Siamese network teacher-student setting practical swarm optimization (PSO) |
| title | Training Compact Change Detection Network for Remote Sensing Imagery |
| title_full | Training Compact Change Detection Network for Remote Sensing Imagery |
| title_fullStr | Training Compact Change Detection Network for Remote Sensing Imagery |
| title_full_unstemmed | Training Compact Change Detection Network for Remote Sensing Imagery |
| title_short | Training Compact Change Detection Network for Remote Sensing Imagery |
| title_sort | training compact change detection network for remote sensing imagery |
| topic | Change detection (CD) deep learning (DL) knowledge distillation (KD) Siamese network teacher-student setting practical swarm optimization (PSO) |
| url | https://ieeexplore.ieee.org/document/9456864/ |
| work_keys_str_mv | AT amirasmahmoud trainingcompactchangedetectionnetworkforremotesensingimagery AT sayedamohamed trainingcompactchangedetectionnetworkforremotesensingimagery AT marwasmoustafa trainingcompactchangedetectionnetworkforremotesensingimagery AT redaaelkhorib trainingcompactchangedetectionnetworkforremotesensingimagery AT hishammabdelsalam trainingcompactchangedetectionnetworkforremotesensingimagery AT ihabaelkhodary trainingcompactchangedetectionnetworkforremotesensingimagery |