Data Separability Metric to Evaluate Radar Target Recognition
The performance of machine learning-based radar target recognition models is determined by the respective model and data to be analyzed. Currently, radar target recognition performance evaluation is based on accuracy metrics, but this method does not include the evaluation metrics regarding the impa...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
China Science Publishing & Media Ltd. (CSPM)
2023-08-01
|
| Series: | Leida xuebao |
| Subjects: | |
| Online Access: | https://radars.ac.cn/cn/article/doi/10.12000/JR23125 |
| _version_ | 1797421260520554496 |
|---|---|
| author | Weidong JIANG Lingyan XUE Xinyu ZHANG |
| author_facet | Weidong JIANG Lingyan XUE Xinyu ZHANG |
| author_sort | Weidong JIANG |
| collection | DOAJ |
| description | The performance of machine learning-based radar target recognition models is determined by the respective model and data to be analyzed. Currently, radar target recognition performance evaluation is based on accuracy metrics, but this method does not include the evaluation metrics regarding the impact of data quality on recognition performance. Data separability describes the degree of mixture of samples from different categories. Furthermore, the data separability metric is independent of the model recognition process. By incorporating it into the recognition evaluation process, recognition difficulty can be quantified, and a benchmark for recognition results can be provided in advance. Therefore, in this paper, we propose a data separability metric based on the rate-distortion theory. Extensive experiments on multiple simulated datasets demonstrated that the proposed metric can compare the separability of multivariate Gaussian datasets. Furthermore, by combining it with the Gaussian mixture model, the designed metric method could overcome the limitation of the rate-distortion function, capture the data’s local separable characteristics, and improve the evaluation accuracy of the overall data separability. Subsequently, we applied the proposed metric to evaluate the recognition difficulty in real datasets, the results of which validated its strong correlation with average recognition accuracy. In the experiments on evaluating the effectiveness of convolutional neural network modules, we first quantified and analyzed the separability trend of the feature extracted by each module during the testing phase. Further, we incorporated the proposed metric as a feature separability loss during the training phase to participate in the network optimization process, guiding the network to extract a more separable feature. This paper provides a new perspective for evaluating and improving the neural network recognition performance in terms of feature separability. |
| first_indexed | 2024-03-09T07:13:22Z |
| format | Article |
| id | doaj.art-70eb78fc835b4928a9d3c19ee6297288 |
| institution | Directory Open Access Journal |
| issn | 2095-283X |
| language | English |
| last_indexed | 2024-03-09T07:13:22Z |
| publishDate | 2023-08-01 |
| publisher | China Science Publishing & Media Ltd. (CSPM) |
| record_format | Article |
| series | Leida xuebao |
| spelling | doaj.art-70eb78fc835b4928a9d3c19ee62972882023-12-03T08:53:14ZengChina Science Publishing & Media Ltd. (CSPM)Leida xuebao2095-283X2023-08-0112486088110.12000/JR23125R23125Data Separability Metric to Evaluate Radar Target RecognitionWeidong JIANG0Lingyan XUE1Xinyu ZHANG2College of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, ChinaCollege of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, ChinaCollege of Electronic Science and Technology, National University of Defense Technology, Changsha 410073, ChinaThe performance of machine learning-based radar target recognition models is determined by the respective model and data to be analyzed. Currently, radar target recognition performance evaluation is based on accuracy metrics, but this method does not include the evaluation metrics regarding the impact of data quality on recognition performance. Data separability describes the degree of mixture of samples from different categories. Furthermore, the data separability metric is independent of the model recognition process. By incorporating it into the recognition evaluation process, recognition difficulty can be quantified, and a benchmark for recognition results can be provided in advance. Therefore, in this paper, we propose a data separability metric based on the rate-distortion theory. Extensive experiments on multiple simulated datasets demonstrated that the proposed metric can compare the separability of multivariate Gaussian datasets. Furthermore, by combining it with the Gaussian mixture model, the designed metric method could overcome the limitation of the rate-distortion function, capture the data’s local separable characteristics, and improve the evaluation accuracy of the overall data separability. Subsequently, we applied the proposed metric to evaluate the recognition difficulty in real datasets, the results of which validated its strong correlation with average recognition accuracy. In the experiments on evaluating the effectiveness of convolutional neural network modules, we first quantified and analyzed the separability trend of the feature extracted by each module during the testing phase. Further, we incorporated the proposed metric as a feature separability loss during the training phase to participate in the network optimization process, guiding the network to extract a more separable feature. This paper provides a new perspective for evaluating and improving the neural network recognition performance in terms of feature separability.https://radars.ac.cn/cn/article/doi/10.12000/JR23125machine learningradar target recognition evaluationrate distortion theoryrecognition accuracydata separability metric |
| spellingShingle | Weidong JIANG Lingyan XUE Xinyu ZHANG Data Separability Metric to Evaluate Radar Target Recognition Leida xuebao machine learning radar target recognition evaluation rate distortion theory recognition accuracy data separability metric |
| title | Data Separability Metric to Evaluate Radar Target Recognition |
| title_full | Data Separability Metric to Evaluate Radar Target Recognition |
| title_fullStr | Data Separability Metric to Evaluate Radar Target Recognition |
| title_full_unstemmed | Data Separability Metric to Evaluate Radar Target Recognition |
| title_short | Data Separability Metric to Evaluate Radar Target Recognition |
| title_sort | data separability metric to evaluate radar target recognition |
| topic | machine learning radar target recognition evaluation rate distortion theory recognition accuracy data separability metric |
| url | https://radars.ac.cn/cn/article/doi/10.12000/JR23125 |
| work_keys_str_mv | AT weidongjiang dataseparabilitymetrictoevaluateradartargetrecognition AT lingyanxue dataseparabilitymetrictoevaluateradartargetrecognition AT xinyuzhang dataseparabilitymetrictoevaluateradartargetrecognition |