Unseen Land Cover Classification from High-Resolution Orthophotos Using Integration of Zero-Shot Learning and Convolutional Neural Networks
Zero-shot learning (ZSL) is an approach to classify objects unseen during the training phase and shown to be useful for real-world applications, especially when there is a lack of sufficient training data. Only a limited amount of works has been carried out on ZSL, especially in the field of remote...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2020-05-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-4292/12/10/1676 |
| _version_ | 1827716285759225856 |
|---|---|
| author | Biswajeet Pradhan Husam A. H. Al-Najjar Maher Ibrahim Sameen Ivor Tsang Abdullah M. Alamri |
| author_facet | Biswajeet Pradhan Husam A. H. Al-Najjar Maher Ibrahim Sameen Ivor Tsang Abdullah M. Alamri |
| author_sort | Biswajeet Pradhan |
| collection | DOAJ |
| description | Zero-shot learning (ZSL) is an approach to classify objects unseen during the training phase and shown to be useful for real-world applications, especially when there is a lack of sufficient training data. Only a limited amount of works has been carried out on ZSL, especially in the field of remote sensing. This research investigates the use of a convolutional neural network (CNN) as a feature extraction and classification method for land cover mapping using high-resolution orthophotos. In the feature extraction phase, we used a CNN model with a single convolutional layer to extract discriminative features. In the second phase, we used class attributes learned from the Word2Vec model (pre-trained by Google News) to train a second CNN model that performed class signature prediction by using both the features extracted by the first CNN and class attributes during training and only the features during prediction. We trained and tested our models on datasets collected over two subareas in the Cameron Highlands (training dataset, first test dataset) and Ipoh (second test dataset) in Malaysia. Several experiments have been conducted on the feature extraction and classification models regarding the main parameters, such as the network’s layers and depth, number of filters, and the impact of Gaussian noise. As a result, the best models were selected using various accuracy metrics such as top-k categorical accuracy for k = [1,2,3], Recall, Precision, and F1-score. The best model for feature extraction achieved 0.953 F1-score, 0.941 precision, 0.882 recall for the training dataset and 0.904 F1-score, 0.869 precision, 0.949 recall for the first test dataset, and 0.898 F1-score, 0.870 precision, 0.838 recall for the second test dataset. The best model for classification achieved an average of 0.778 top-one, 0.890 top-two and 0.942 top-three accuracy, 0.798 F1-score, 0.766 recall and 0.838 precision for the first test dataset and 0.737 top-one, 0.906 top-two, 0.924 top-three, 0.729 F1-score, 0.676 recall and 0.790 precision for the second test dataset. The results demonstrated that the proposed ZSL is a promising tool for land cover mapping based on high-resolution photos. |
| first_indexed | 2024-03-10T19:37:24Z |
| format | Article |
| id | doaj.art-225ae7d0ffda4ebebf6171122bc2b601 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-03-10T19:37:24Z |
| publishDate | 2020-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-225ae7d0ffda4ebebf6171122bc2b6012023-11-20T01:31:50ZengMDPI AGRemote Sensing2072-42922020-05-011210167610.3390/rs12101676Unseen Land Cover Classification from High-Resolution Orthophotos Using Integration of Zero-Shot Learning and Convolutional Neural NetworksBiswajeet Pradhan0Husam A. H. Al-Najjar1Maher Ibrahim Sameen2Ivor Tsang3Abdullah M. Alamri4The Centre for Advanced Modelling and Geospatial Information Systems (CAMGIS), Faculty of Engineering and IT, University of Technology Sydney, Sydney, NSW 2007, AustraliaThe Centre for Advanced Modelling and Geospatial Information Systems (CAMGIS), Faculty of Engineering and IT, University of Technology Sydney, Sydney, NSW 2007, AustraliaThe Centre for Advanced Modelling and Geospatial Information Systems (CAMGIS), Faculty of Engineering and IT, University of Technology Sydney, Sydney, NSW 2007, AustraliaCenter for Artificial Intelligence, Faculty of Engineering and IT, University of Technology Sydney, Sydney 2007, AustraliaDepartment of Geology & Geophysics, College of Science, King Saud Univ., P.O. Box 2455, Riyadh 11451, Saudi ArabiaZero-shot learning (ZSL) is an approach to classify objects unseen during the training phase and shown to be useful for real-world applications, especially when there is a lack of sufficient training data. Only a limited amount of works has been carried out on ZSL, especially in the field of remote sensing. This research investigates the use of a convolutional neural network (CNN) as a feature extraction and classification method for land cover mapping using high-resolution orthophotos. In the feature extraction phase, we used a CNN model with a single convolutional layer to extract discriminative features. In the second phase, we used class attributes learned from the Word2Vec model (pre-trained by Google News) to train a second CNN model that performed class signature prediction by using both the features extracted by the first CNN and class attributes during training and only the features during prediction. We trained and tested our models on datasets collected over two subareas in the Cameron Highlands (training dataset, first test dataset) and Ipoh (second test dataset) in Malaysia. Several experiments have been conducted on the feature extraction and classification models regarding the main parameters, such as the network’s layers and depth, number of filters, and the impact of Gaussian noise. As a result, the best models were selected using various accuracy metrics such as top-k categorical accuracy for k = [1,2,3], Recall, Precision, and F1-score. The best model for feature extraction achieved 0.953 F1-score, 0.941 precision, 0.882 recall for the training dataset and 0.904 F1-score, 0.869 precision, 0.949 recall for the first test dataset, and 0.898 F1-score, 0.870 precision, 0.838 recall for the second test dataset. The best model for classification achieved an average of 0.778 top-one, 0.890 top-two and 0.942 top-three accuracy, 0.798 F1-score, 0.766 recall and 0.838 precision for the first test dataset and 0.737 top-one, 0.906 top-two, 0.924 top-three, 0.729 F1-score, 0.676 recall and 0.790 precision for the second test dataset. The results demonstrated that the proposed ZSL is a promising tool for land cover mapping based on high-resolution photos.https://www.mdpi.com/2072-4292/12/10/1676land cover classificationdeep-learningCNNZero-Shot Learningremote sensingorthophotos |
| spellingShingle | Biswajeet Pradhan Husam A. H. Al-Najjar Maher Ibrahim Sameen Ivor Tsang Abdullah M. Alamri Unseen Land Cover Classification from High-Resolution Orthophotos Using Integration of Zero-Shot Learning and Convolutional Neural Networks Remote Sensing land cover classification deep-learning CNN Zero-Shot Learning remote sensing orthophotos |
| title | Unseen Land Cover Classification from High-Resolution Orthophotos Using Integration of Zero-Shot Learning and Convolutional Neural Networks |
| title_full | Unseen Land Cover Classification from High-Resolution Orthophotos Using Integration of Zero-Shot Learning and Convolutional Neural Networks |
| title_fullStr | Unseen Land Cover Classification from High-Resolution Orthophotos Using Integration of Zero-Shot Learning and Convolutional Neural Networks |
| title_full_unstemmed | Unseen Land Cover Classification from High-Resolution Orthophotos Using Integration of Zero-Shot Learning and Convolutional Neural Networks |
| title_short | Unseen Land Cover Classification from High-Resolution Orthophotos Using Integration of Zero-Shot Learning and Convolutional Neural Networks |
| title_sort | unseen land cover classification from high resolution orthophotos using integration of zero shot learning and convolutional neural networks |
| topic | land cover classification deep-learning CNN Zero-Shot Learning remote sensing orthophotos |
| url | https://www.mdpi.com/2072-4292/12/10/1676 |
| work_keys_str_mv | AT biswajeetpradhan unseenlandcoverclassificationfromhighresolutionorthophotosusingintegrationofzeroshotlearningandconvolutionalneuralnetworks AT husamahalnajjar unseenlandcoverclassificationfromhighresolutionorthophotosusingintegrationofzeroshotlearningandconvolutionalneuralnetworks AT maheribrahimsameen unseenlandcoverclassificationfromhighresolutionorthophotosusingintegrationofzeroshotlearningandconvolutionalneuralnetworks AT ivortsang unseenlandcoverclassificationfromhighresolutionorthophotosusingintegrationofzeroshotlearningandconvolutionalneuralnetworks AT abdullahmalamri unseenlandcoverclassificationfromhighresolutionorthophotosusingintegrationofzeroshotlearningandconvolutionalneuralnetworks |