UVid-Net: Enhanced Semantic Segmentation of UAV Aerial Videos by Embedding Temporal Information
Semantic segmentation of aerial videos has been extensively used for decision making in monitoring environmental changes, urban planning, and disaster management. The reliability of these decision support systems is dependent on the accuracy of the video semantic segmentation algorithms. The existin...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2021-01-01
|
| Series: | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9392319/ |
| _version_ | 1818901199243444224 |
|---|---|
| author | S. Girisha Ujjwal Verma M. M. Manohara Pai Radhika M. Pai |
| author_facet | S. Girisha Ujjwal Verma M. M. Manohara Pai Radhika M. Pai |
| author_sort | S. Girisha |
| collection | DOAJ |
| description | Semantic segmentation of aerial videos has been extensively used for decision making in monitoring environmental changes, urban planning, and disaster management. The reliability of these decision support systems is dependent on the accuracy of the video semantic segmentation algorithms. The existing CNN-based video semantic segmentation methods have enhanced the image semantic segmentation methods by incorporating an additional module such as LSTM or optical flow for computing temporal dynamics of the video which is a computational overhead. The proposed research work modifies the CNN architecture by incorporating temporal information to improve the efficiency of video semantic segmentation. In this work, an enhanced encoder–decoder based CNN architecture (UVid-Net) is proposed for unmanned aerial vehicle (UAV) video semantic segmentation. The encoder of the proposed architecture embeds temporal information for temporally consistent labeling. The decoder is enhanced by introducing the feature-refiner module, which aids in accurate localization of the class labels. The proposed UVid-Net architecture for UAV video semantic segmentation is quantitatively evaluated on extended ManipalUAVid dataset. The performance metric mean Intersection over Union of 0.79 has been observed which is significantly greater than the other state-of-the-art algorithms. Further, the proposed work produced promising results even for the pretrained model of UVid-Net on urban street scene by fine tuning the final layer on UAV aerial videos. |
| first_indexed | 2024-12-19T20:15:57Z |
| format | Article |
| id | doaj.art-a59f74bb97f04967a5627ed4538282f5 |
| institution | Directory Open Access Journal |
| issn | 2151-1535 |
| language | English |
| last_indexed | 2024-12-19T20:15:57Z |
| publishDate | 2021-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing |
| spelling | doaj.art-a59f74bb97f04967a5627ed4538282f52022-12-21T20:07:11ZengIEEEIEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing2151-15352021-01-01144115412710.1109/JSTARS.2021.30699099392319UVid-Net: Enhanced Semantic Segmentation of UAV Aerial Videos by Embedding Temporal InformationS. Girisha0https://orcid.org/0000-0003-2582-9600Ujjwal Verma1https://orcid.org/0000-0002-6133-5379M. M. Manohara Pai2https://orcid.org/0000-0003-2164-2945Radhika M. Pai3https://orcid.org/0000-0002-0916-0495Department of Information and Communication Technology, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, IndiaDepartment of Electronics and Communication Engineering, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, IndiaDepartment of Information and Communication Technology, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, IndiaDepartment of Information and Communication Technology, Manipal Institute of Technology, Manipal Academy of Higher Education, Manipal, IndiaSemantic segmentation of aerial videos has been extensively used for decision making in monitoring environmental changes, urban planning, and disaster management. The reliability of these decision support systems is dependent on the accuracy of the video semantic segmentation algorithms. The existing CNN-based video semantic segmentation methods have enhanced the image semantic segmentation methods by incorporating an additional module such as LSTM or optical flow for computing temporal dynamics of the video which is a computational overhead. The proposed research work modifies the CNN architecture by incorporating temporal information to improve the efficiency of video semantic segmentation. In this work, an enhanced encoder–decoder based CNN architecture (UVid-Net) is proposed for unmanned aerial vehicle (UAV) video semantic segmentation. The encoder of the proposed architecture embeds temporal information for temporally consistent labeling. The decoder is enhanced by introducing the feature-refiner module, which aids in accurate localization of the class labels. The proposed UVid-Net architecture for UAV video semantic segmentation is quantitatively evaluated on extended ManipalUAVid dataset. The performance metric mean Intersection over Union of 0.79 has been observed which is significantly greater than the other state-of-the-art algorithms. Further, the proposed work produced promising results even for the pretrained model of UVid-Net on urban street scene by fine tuning the final layer on UAV aerial videos.https://ieeexplore.ieee.org/document/9392319/Deep learningsemantic segmentationtransfer learningU-Netunmanned aerial vehicle (UAV) video |
| spellingShingle | S. Girisha Ujjwal Verma M. M. Manohara Pai Radhika M. Pai UVid-Net: Enhanced Semantic Segmentation of UAV Aerial Videos by Embedding Temporal Information IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing Deep learning semantic segmentation transfer learning U-Net unmanned aerial vehicle (UAV) video |
| title | UVid-Net: Enhanced Semantic Segmentation of UAV Aerial Videos by Embedding Temporal Information |
| title_full | UVid-Net: Enhanced Semantic Segmentation of UAV Aerial Videos by Embedding Temporal Information |
| title_fullStr | UVid-Net: Enhanced Semantic Segmentation of UAV Aerial Videos by Embedding Temporal Information |
| title_full_unstemmed | UVid-Net: Enhanced Semantic Segmentation of UAV Aerial Videos by Embedding Temporal Information |
| title_short | UVid-Net: Enhanced Semantic Segmentation of UAV Aerial Videos by Embedding Temporal Information |
| title_sort | uvid net enhanced semantic segmentation of uav aerial videos by embedding temporal information |
| topic | Deep learning semantic segmentation transfer learning U-Net unmanned aerial vehicle (UAV) video |
| url | https://ieeexplore.ieee.org/document/9392319/ |
| work_keys_str_mv | AT sgirisha uvidnetenhancedsemanticsegmentationofuavaerialvideosbyembeddingtemporalinformation AT ujjwalverma uvidnetenhancedsemanticsegmentationofuavaerialvideosbyembeddingtemporalinformation AT mmmanoharapai uvidnetenhancedsemanticsegmentationofuavaerialvideosbyembeddingtemporalinformation AT radhikampai uvidnetenhancedsemanticsegmentationofuavaerialvideosbyembeddingtemporalinformation |