Outlier detection and data filling based on KNN and LOF for power transformer operation data classification
The missing and abnormal data in power transformer operation and monitoring greatly affect the accuracy of fault diagnosis and thus threaten the stable operation of power systems. To conduct outlier detection and improve data quality for safety warning, this paper proposes a transformer operation da...
| Main Authors: | , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2023-09-01
|
| Series: | Energy Reports |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S2352484723004250 |
| _version_ | 1797692018031329280 |
|---|---|
| author | Dexu Zou Yongjian Xiang Tao Zhou Qingjun Peng Weiju Dai Zhihu Hong Yong Shi Shan Wang Jianhua Yin Hao Quan |
| author_facet | Dexu Zou Yongjian Xiang Tao Zhou Qingjun Peng Weiju Dai Zhihu Hong Yong Shi Shan Wang Jianhua Yin Hao Quan |
| author_sort | Dexu Zou |
| collection | DOAJ |
| description | The missing and abnormal data in power transformer operation and monitoring greatly affect the accuracy of fault diagnosis and thus threaten the stable operation of power systems. To conduct outlier detection and improve data quality for safety warning, this paper proposes a transformer operation data preprocessing method based on KNN (K-nearest neighbor) and LOF (local outlier factor) for power transformer operation data classification. Firstly, this paper analyzes the characteristics of transformer operation data. Secondly, the local reachable density of the input data is calculated by LOF algorithm. The local outlier factor score of the data is derived according to the local reachable density, and the abnormal data is output according to the abnormal score. Then, KNN algorithm is utilized to classify the relevant data around the abnormal value and missing value of the transformer. The data are filled or corrected according to the classification results. Thirdly, the elbow method is used to determine the optimal K value and cluster operation data by K-Means algorithm. Finally, the proposed method is applied and verified with real transformer operation data in case study. The results show the method can effectively detect and correct the abnormal and missing data, conduct transformer data cleaning and preprocessing and provide accurate and effective data samples for transformer fault diagnosis. |
| first_indexed | 2024-03-12T02:22:32Z |
| format | Article |
| id | doaj.art-d40c84194be44cd9a9897abc3261b725 |
| institution | Directory Open Access Journal |
| issn | 2352-4847 |
| language | English |
| last_indexed | 2024-03-12T02:22:32Z |
| publishDate | 2023-09-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Energy Reports |
| spelling | doaj.art-d40c84194be44cd9a9897abc3261b7252023-09-06T04:51:36ZengElsevierEnergy Reports2352-48472023-09-019698711Outlier detection and data filling based on KNN and LOF for power transformer operation data classificationDexu Zou0Yongjian Xiang1Tao Zhou2Qingjun Peng3Weiju Dai4Zhihu Hong5Yong Shi6Shan Wang7Jianhua Yin8Hao Quan9Electric Power Research Institute, China Southern Power Grid Yunnan Power Grid Co., Ltd., Kunming 650217, ChinaSchool of Automation, Nanjing University of Science and Technology, Nanjing 210094, ChinaSchool of Automation, Nanjing University of Science and Technology, Nanjing 210094, China; Corresponding author.Electric Power Research Institute, China Southern Power Grid Yunnan Power Grid Co., Ltd., Kunming 650217, ChinaElectric Power Research Institute, China Southern Power Grid Yunnan Power Grid Co., Ltd., Kunming 650217, ChinaElectric Power Research Institute, China Southern Power Grid Yunnan Power Grid Co., Ltd., Kunming 650217, ChinaChina Southern Power Grid Yunnan Power Grid Co., Ltd., Kunming 650217, ChinaElectric Power Research Institute, China Southern Power Grid Yunnan Power Grid Co., Ltd., Kunming 650217, ChinaSchool of Artificial Intelligence, Nanjing University of Information Science and Technology, Nanjing 210044, ChinaSchool of Automation, Nanjing University of Science and Technology, Nanjing 210094, ChinaThe missing and abnormal data in power transformer operation and monitoring greatly affect the accuracy of fault diagnosis and thus threaten the stable operation of power systems. To conduct outlier detection and improve data quality for safety warning, this paper proposes a transformer operation data preprocessing method based on KNN (K-nearest neighbor) and LOF (local outlier factor) for power transformer operation data classification. Firstly, this paper analyzes the characteristics of transformer operation data. Secondly, the local reachable density of the input data is calculated by LOF algorithm. The local outlier factor score of the data is derived according to the local reachable density, and the abnormal data is output according to the abnormal score. Then, KNN algorithm is utilized to classify the relevant data around the abnormal value and missing value of the transformer. The data are filled or corrected according to the classification results. Thirdly, the elbow method is used to determine the optimal K value and cluster operation data by K-Means algorithm. Finally, the proposed method is applied and verified with real transformer operation data in case study. The results show the method can effectively detect and correct the abnormal and missing data, conduct transformer data cleaning and preprocessing and provide accurate and effective data samples for transformer fault diagnosis.http://www.sciencedirect.com/science/article/pii/S2352484723004250Power transformerOutlier detectionData sufficiencyLOFKNN |
| spellingShingle | Dexu Zou Yongjian Xiang Tao Zhou Qingjun Peng Weiju Dai Zhihu Hong Yong Shi Shan Wang Jianhua Yin Hao Quan Outlier detection and data filling based on KNN and LOF for power transformer operation data classification Energy Reports Power transformer Outlier detection Data sufficiency LOF KNN |
| title | Outlier detection and data filling based on KNN and LOF for power transformer operation data classification |
| title_full | Outlier detection and data filling based on KNN and LOF for power transformer operation data classification |
| title_fullStr | Outlier detection and data filling based on KNN and LOF for power transformer operation data classification |
| title_full_unstemmed | Outlier detection and data filling based on KNN and LOF for power transformer operation data classification |
| title_short | Outlier detection and data filling based on KNN and LOF for power transformer operation data classification |
| title_sort | outlier detection and data filling based on knn and lof for power transformer operation data classification |
| topic | Power transformer Outlier detection Data sufficiency LOF KNN |
| url | http://www.sciencedirect.com/science/article/pii/S2352484723004250 |
| work_keys_str_mv | AT dexuzou outlierdetectionanddatafillingbasedonknnandlofforpowertransformeroperationdataclassification AT yongjianxiang outlierdetectionanddatafillingbasedonknnandlofforpowertransformeroperationdataclassification AT taozhou outlierdetectionanddatafillingbasedonknnandlofforpowertransformeroperationdataclassification AT qingjunpeng outlierdetectionanddatafillingbasedonknnandlofforpowertransformeroperationdataclassification AT weijudai outlierdetectionanddatafillingbasedonknnandlofforpowertransformeroperationdataclassification AT zhihuhong outlierdetectionanddatafillingbasedonknnandlofforpowertransformeroperationdataclassification AT yongshi outlierdetectionanddatafillingbasedonknnandlofforpowertransformeroperationdataclassification AT shanwang outlierdetectionanddatafillingbasedonknnandlofforpowertransformeroperationdataclassification AT jianhuayin outlierdetectionanddatafillingbasedonknnandlofforpowertransformeroperationdataclassification AT haoquan outlierdetectionanddatafillingbasedonknnandlofforpowertransformeroperationdataclassification |