Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector Network
The detection performance of small objects in remote sensing images has not been satisfactory compared to large objects, especially in low-resolution and noisy images. A generative adversarial network (GAN)-based model called enhanced super-resolution GAN (ESRGAN) showed remarkable image enhancement...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-05-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/12/9/1432 |
| _version_ | 1797569148950151168 |
|---|---|
| author | Jakaria Rabbi Nilanjan Ray Matthias Schubert Subir Chowdhury Dennis Chao |
| author_facet | Jakaria Rabbi Nilanjan Ray Matthias Schubert Subir Chowdhury Dennis Chao |
| author_sort | Jakaria Rabbi |
| collection | DOAJ |
| description | The detection performance of small objects in remote sensing images has not been satisfactory compared to large objects, especially in low-resolution and noisy images. A generative adversarial network (GAN)-based model called enhanced super-resolution GAN (ESRGAN) showed remarkable image enhancement performance, but reconstructed images usually miss high-frequency edge information. Therefore, object detection performance showed degradation for small objects on recovered noisy and low-resolution remote sensing images. Inspired by the success of edge enhanced GAN (EEGAN) and ESRGAN, we applied a new edge-enhanced super-resolution GAN (EESRGAN) to improve the quality of remote sensing images and used different detector networks in an end-to-end manner where detector loss was backpropagated into the EESRGAN to improve the detection performance. We proposed an architecture with three components: ESRGAN, EEN, and Detection network. We used residual-in-residual dense blocks (RRDB) for both the ESRGAN and EEN, and for the detector network, we used a faster region-based convolutional network (FRCNN) (two-stage detector) and a single-shot multibox detector (SSD) (one stage detector). Extensive experiments on a public (car overhead with context) dataset and another self-assembled (oil and gas storage tank) satellite dataset showed superior performance of our method compared to the standalone state-of-the-art object detectors. |
| first_indexed | 2024-03-10T20:06:47Z |
| format | Article |
| id | doaj.art-b65d0c9b1499428f85fd66b473d25988 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T20:06:47Z |
| publishDate | 2020-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-b65d0c9b1499428f85fd66b473d259882023-11-19T23:14:19ZengMDPI AGRemote Sensing2072-42922020-05-01129143210.3390/rs12091432Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector NetworkJakaria Rabbi0Nilanjan Ray1Matthias Schubert2Subir Chowdhury3Dennis Chao4Department of Computing Science, 2-32 Athabasca Hall, University of Alberta, Edmonton, AB T6G 2E8, CanadaDepartment of Computing Science, 2-32 Athabasca Hall, University of Alberta, Edmonton, AB T6G 2E8, CanadaInstitute for Informatic, Ludwig-Maximilians-Universität München, Oettingenstraße 67, D-80333 Munich, GermanyAlberta Geological Survey, Alberta Energy Regulator, Edmonton, AB T6B 2X3, CanadaAlberta Geological Survey, Alberta Energy Regulator, Edmonton, AB T6B 2X3, CanadaThe detection performance of small objects in remote sensing images has not been satisfactory compared to large objects, especially in low-resolution and noisy images. A generative adversarial network (GAN)-based model called enhanced super-resolution GAN (ESRGAN) showed remarkable image enhancement performance, but reconstructed images usually miss high-frequency edge information. Therefore, object detection performance showed degradation for small objects on recovered noisy and low-resolution remote sensing images. Inspired by the success of edge enhanced GAN (EEGAN) and ESRGAN, we applied a new edge-enhanced super-resolution GAN (EESRGAN) to improve the quality of remote sensing images and used different detector networks in an end-to-end manner where detector loss was backpropagated into the EESRGAN to improve the detection performance. We proposed an architecture with three components: ESRGAN, EEN, and Detection network. We used residual-in-residual dense blocks (RRDB) for both the ESRGAN and EEN, and for the detector network, we used a faster region-based convolutional network (FRCNN) (two-stage detector) and a single-shot multibox detector (SSD) (one stage detector). Extensive experiments on a public (car overhead with context) dataset and another self-assembled (oil and gas storage tank) satellite dataset showed superior performance of our method compared to the standalone state-of-the-art object detectors.https://www.mdpi.com/2072-4292/12/9/1432object detectionfaster region-based convolutional neural network (FRCNN)single-shot multibox detector (SSD)super-resolutionremote sensing imageryedge enhancement |
| spellingShingle | Jakaria Rabbi Nilanjan Ray Matthias Schubert Subir Chowdhury Dennis Chao Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector Network Remote Sensing object detection faster region-based convolutional neural network (FRCNN) single-shot multibox detector (SSD) super-resolution remote sensing imagery edge enhancement |
| title | Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector Network |
| title_full | Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector Network |
| title_fullStr | Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector Network |
| title_full_unstemmed | Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector Network |
| title_short | Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector Network |
| title_sort | small object detection in remote sensing images with end to end edge enhanced gan and object detector network |
| topic | object detection faster region-based convolutional neural network (FRCNN) single-shot multibox detector (SSD) super-resolution remote sensing imagery edge enhancement |
| url | https://www.mdpi.com/2072-4292/12/9/1432 |
| work_keys_str_mv | AT jakariarabbi smallobjectdetectioninremotesensingimageswithendtoendedgeenhancedganandobjectdetectornetwork AT nilanjanray smallobjectdetectioninremotesensingimageswithendtoendedgeenhancedganandobjectdetectornetwork AT matthiasschubert smallobjectdetectioninremotesensingimageswithendtoendedgeenhancedganandobjectdetectornetwork AT subirchowdhury smallobjectdetectioninremotesensingimageswithendtoendedgeenhancedganandobjectdetectornetwork AT dennischao smallobjectdetectioninremotesensingimageswithendtoendedgeenhancedganandobjectdetectornetwork |