Automated quality control methods for sensor data: a novel observatory approach
National and international networks and observatories of terrestrial-based sensors are emerging rapidly. As such, there is demand for a standardized approach to data quality control, as well as interoperability of data among sensor networks. The National Ecological Observatory Network (NEON) has beg...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Copernicus Publications
2013-07-01
|
| Series: | Biogeosciences |
| Online Access: | http://www.biogeosciences.net/10/4957/2013/bg-10-4957-2013.pdf |
| _version_ | 1811305260014108672 |
|---|---|
| author | J. R. Taylor H. L. Loescher |
| author_facet | J. R. Taylor H. L. Loescher |
| author_sort | J. R. Taylor |
| collection | DOAJ |
| description | National and international networks and observatories of terrestrial-based sensors are emerging rapidly. As such, there is demand for a standardized approach to data quality control, as well as interoperability of data among sensor networks. The National Ecological Observatory Network (NEON) has begun constructing their first terrestrial observing sites, with 60 locations expected to be distributed across the US by 2017. This will result in over 14 000 automated sensors recording more than > 100 Tb of data per year. These data are then used to create other datasets and subsequent "higher-level" data products. In anticipation of this challenge, an overall data quality assurance plan has been developed and the first suite of data quality control measures defined. This data-driven approach focuses on automated methods for defining a suite of plausibility test parameter thresholds. Specifically, these plausibility tests scrutinize the data range and variance of each measurement type by employing a suite of binary checks. The statistical basis for each of these tests is developed, and the methods for calculating test parameter thresholds are explored here. While these tests have been used elsewhere, we apply them in a novel approach by calculating their relevant test parameter thresholds. Finally, implementing automated quality control is demonstrated with preliminary data from a NEON prototype site. |
| first_indexed | 2024-04-13T08:23:27Z |
| format | Article |
| id | doaj.art-40a2390e6b47482cbd4d1c10f0073687 |
| institution | Directory Open Access Journal |
| issn | 1726-4170 1726-4189 |
| language | English |
| last_indexed | 2024-04-13T08:23:27Z |
| publishDate | 2013-07-01 |
| publisher | Copernicus Publications |
| record_format | Article |
| series | Biogeosciences |
| spelling | doaj.art-40a2390e6b47482cbd4d1c10f00736872022-12-22T02:54:34ZengCopernicus PublicationsBiogeosciences1726-41701726-41892013-07-011074957497110.5194/bg-10-4957-2013Automated quality control methods for sensor data: a novel observatory approachJ. R. TaylorH. L. LoescherNational and international networks and observatories of terrestrial-based sensors are emerging rapidly. As such, there is demand for a standardized approach to data quality control, as well as interoperability of data among sensor networks. The National Ecological Observatory Network (NEON) has begun constructing their first terrestrial observing sites, with 60 locations expected to be distributed across the US by 2017. This will result in over 14 000 automated sensors recording more than > 100 Tb of data per year. These data are then used to create other datasets and subsequent "higher-level" data products. In anticipation of this challenge, an overall data quality assurance plan has been developed and the first suite of data quality control measures defined. This data-driven approach focuses on automated methods for defining a suite of plausibility test parameter thresholds. Specifically, these plausibility tests scrutinize the data range and variance of each measurement type by employing a suite of binary checks. The statistical basis for each of these tests is developed, and the methods for calculating test parameter thresholds are explored here. While these tests have been used elsewhere, we apply them in a novel approach by calculating their relevant test parameter thresholds. Finally, implementing automated quality control is demonstrated with preliminary data from a NEON prototype site.http://www.biogeosciences.net/10/4957/2013/bg-10-4957-2013.pdf |
| spellingShingle | J. R. Taylor H. L. Loescher Automated quality control methods for sensor data: a novel observatory approach Biogeosciences |
| title | Automated quality control methods for sensor data: a novel observatory approach |
| title_full | Automated quality control methods for sensor data: a novel observatory approach |
| title_fullStr | Automated quality control methods for sensor data: a novel observatory approach |
| title_full_unstemmed | Automated quality control methods for sensor data: a novel observatory approach |
| title_short | Automated quality control methods for sensor data: a novel observatory approach |
| title_sort | automated quality control methods for sensor data a novel observatory approach |
| url | http://www.biogeosciences.net/10/4957/2013/bg-10-4957-2013.pdf |
| work_keys_str_mv | AT jrtaylor automatedqualitycontrolmethodsforsensordataanovelobservatoryapproach AT hlloescher automatedqualitycontrolmethodsforsensordataanovelobservatoryapproach |