Outlier Detection and Smoothing Process for Water Level Data Measured by Ultrasonic Sensor in Stream Flows
Water level data sets acquired by ultrasonic sensors in stream-scale channels exhibit relatively large numbers of outliers that are off the measurement range between the ultrasonic sensor and water surface, as well as data dispersion of approximately 2 cm due to random errors such as water waves. Th...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2019-05-01
|
| Series: | Water |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2073-4441/11/5/951 |
| _version_ | 1811336673663909888 |
|---|---|
| author | Inhyeok Bae Un Ji |
| author_facet | Inhyeok Bae Un Ji |
| author_sort | Inhyeok Bae |
| collection | DOAJ |
| description | Water level data sets acquired by ultrasonic sensors in stream-scale channels exhibit relatively large numbers of outliers that are off the measurement range between the ultrasonic sensor and water surface, as well as data dispersion of approximately 2 cm due to random errors such as water waves. Therefore, this study develops a data processing algorithm for outlier removal and smoothing for water level data measured by ultrasonic sensors to consider these characteristics. The outlier removal process includes an initial cutoff process to remove outliers out of the measurement range and an outlier detection process using modified Z-scores based on the median absolute deviation (MAD) of a robust estimator. In addition, an exponentially weighted moving average (EWMA) method is applied to smooth the processed data. Sensitivity analyses are performed for factors that are subjectively set by the user, including the window size for the MAD outlier detection stage, the rejection criterion for the modified Z-score outlier removal stage, and the smoothing constant for the EWMA smoothing stage, based on four different water level data sets acquired by ultrasonic sensors in stream-scale experiments. |
| first_indexed | 2024-04-13T17:42:59Z |
| format | Article |
| id | doaj.art-0cd26ecfa1e4446d9c8e7f39df276ebf |
| institution | Directory Open Access Journal |
| issn | 2073-4441 |
| language | English |
| last_indexed | 2024-04-13T17:42:59Z |
| publishDate | 2019-05-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Water |
| spelling | doaj.art-0cd26ecfa1e4446d9c8e7f39df276ebf2022-12-22T02:37:06ZengMDPI AGWater2073-44412019-05-0111595110.3390/w11050951w11050951Outlier Detection and Smoothing Process for Water Level Data Measured by Ultrasonic Sensor in Stream FlowsInhyeok Bae0Un Ji1Smart City and Construction Engineering, Korea University of Science and Technology, Goyang-Si 10223, KoreaSmart City and Construction Engineering, Korea University of Science and Technology, Goyang-Si 10223, KoreaWater level data sets acquired by ultrasonic sensors in stream-scale channels exhibit relatively large numbers of outliers that are off the measurement range between the ultrasonic sensor and water surface, as well as data dispersion of approximately 2 cm due to random errors such as water waves. Therefore, this study develops a data processing algorithm for outlier removal and smoothing for water level data measured by ultrasonic sensors to consider these characteristics. The outlier removal process includes an initial cutoff process to remove outliers out of the measurement range and an outlier detection process using modified Z-scores based on the median absolute deviation (MAD) of a robust estimator. In addition, an exponentially weighted moving average (EWMA) method is applied to smooth the processed data. Sensitivity analyses are performed for factors that are subjectively set by the user, including the window size for the MAD outlier detection stage, the rejection criterion for the modified Z-score outlier removal stage, and the smoothing constant for the EWMA smoothing stage, based on four different water level data sets acquired by ultrasonic sensors in stream-scale experiments.https://www.mdpi.com/2073-4441/11/5/951data smoothingexponentially weighted moving averagemedian absolute deviationmodified Z-scoresoutlier detectionultrasonic sensorwater level monitoring |
| spellingShingle | Inhyeok Bae Un Ji Outlier Detection and Smoothing Process for Water Level Data Measured by Ultrasonic Sensor in Stream Flows Water data smoothing exponentially weighted moving average median absolute deviation modified Z-scores outlier detection ultrasonic sensor water level monitoring |
| title | Outlier Detection and Smoothing Process for Water Level Data Measured by Ultrasonic Sensor in Stream Flows |
| title_full | Outlier Detection and Smoothing Process for Water Level Data Measured by Ultrasonic Sensor in Stream Flows |
| title_fullStr | Outlier Detection and Smoothing Process for Water Level Data Measured by Ultrasonic Sensor in Stream Flows |
| title_full_unstemmed | Outlier Detection and Smoothing Process for Water Level Data Measured by Ultrasonic Sensor in Stream Flows |
| title_short | Outlier Detection and Smoothing Process for Water Level Data Measured by Ultrasonic Sensor in Stream Flows |
| title_sort | outlier detection and smoothing process for water level data measured by ultrasonic sensor in stream flows |
| topic | data smoothing exponentially weighted moving average median absolute deviation modified Z-scores outlier detection ultrasonic sensor water level monitoring |
| url | https://www.mdpi.com/2073-4441/11/5/951 |
| work_keys_str_mv | AT inhyeokbae outlierdetectionandsmoothingprocessforwaterleveldatameasuredbyultrasonicsensorinstreamflows AT unji outlierdetectionandsmoothingprocessforwaterleveldatameasuredbyultrasonicsensorinstreamflows |