Smarter Cities: Cleaning Electricity, Gas and Water Metered Consumption Data for Social and Urban Research
It is claimed that ‘Big Data’ could help cities become ‘smart’, utilise resources more efficiently, and improve inhabitants’ quality of life. Metered consumption data of electricity, gas and water are collected and stored for each household in Austra...
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
SDEWES Centre
2019-09-01
|
| Series: | Journal of Sustainable Development of Energy, Water and Environment Systems |
| Subjects: | |
| Online Access: |
http://www.sdewes.org/jsdewes/pid6.0242
|
| _version_ | 1819076545082294272 |
|---|---|
| author | Mishka Talent |
| author_facet | Mishka Talent |
| author_sort | Mishka Talent |
| collection | DOAJ |
| description | It is claimed that ‘Big Data’ could help cities become ‘smart’, utilise resources more efficiently, and improve inhabitants’ quality of life. Metered consumption data of electricity, gas and water are collected and stored for each household in Australia and provide a valuable source of information for researchers hoping to understand the patterns of consumption and improve resource utilisation. This work tests the assumption that these datasets are sufficiently ‘clean’ to allow interrogation and details the common errors encountered. An inspection of 1-7 years of meter readings of electricity, gas and water for ~60,000 households in Canberra, Australia as well as all datasets of the local government’s energy and water efficiency programs, found this not to be the case. Common errors found include: missing data, biases in erroneous data, errors generated by the data custodians, duplicate entries, the effect of different custodian objectives and of poor data constraints (free flowing text). |
| first_indexed | 2024-12-21T18:43:00Z |
| format | Article |
| id | doaj.art-4db4d8d3718c43e8b045299e624dc76a |
| institution | Directory Open Access Journal |
| issn | 1848-9257 |
| language | English |
| last_indexed | 2024-12-21T18:43:00Z |
| publishDate | 2019-09-01 |
| publisher | SDEWES Centre |
| record_format | Article |
| series | Journal of Sustainable Development of Energy, Water and Environment Systems |
| spelling | doaj.art-4db4d8d3718c43e8b045299e624dc76a2022-12-21T18:53:57ZengSDEWES CentreJournal of Sustainable Development of Energy, Water and Environment Systems1848-92572019-09-017346648110.13044/j.sdewes.d6.024200242Smarter Cities: Cleaning Electricity, Gas and Water Metered Consumption Data for Social and Urban ResearchMishka Talent0 Fenner School of Environment and Society, Australian National University, 46 Sullivans Creek Road, ANU, Canberra, ACT, 0200, Australia It is claimed that ‘Big Data’ could help cities become ‘smart’, utilise resources more efficiently, and improve inhabitants’ quality of life. Metered consumption data of electricity, gas and water are collected and stored for each household in Australia and provide a valuable source of information for researchers hoping to understand the patterns of consumption and improve resource utilisation. This work tests the assumption that these datasets are sufficiently ‘clean’ to allow interrogation and details the common errors encountered. An inspection of 1-7 years of meter readings of electricity, gas and water for ~60,000 households in Canberra, Australia as well as all datasets of the local government’s energy and water efficiency programs, found this not to be the case. Common errors found include: missing data, biases in erroneous data, errors generated by the data custodians, duplicate entries, the effect of different custodian objectives and of poor data constraints (free flowing text). http://www.sdewes.org/jsdewes/pid6.0242 Data cleaningData errorsDuplicate dataUnique dwellingsElectricityWater. |
| spellingShingle | Mishka Talent Smarter Cities: Cleaning Electricity, Gas and Water Metered Consumption Data for Social and Urban Research Journal of Sustainable Development of Energy, Water and Environment Systems Data cleaning Data errors Duplicate data Unique dwellings Electricity Water. |
| title | Smarter Cities: Cleaning Electricity, Gas and Water Metered Consumption Data for Social and Urban Research |
| title_full | Smarter Cities: Cleaning Electricity, Gas and Water Metered Consumption Data for Social and Urban Research |
| title_fullStr | Smarter Cities: Cleaning Electricity, Gas and Water Metered Consumption Data for Social and Urban Research |
| title_full_unstemmed | Smarter Cities: Cleaning Electricity, Gas and Water Metered Consumption Data for Social and Urban Research |
| title_short | Smarter Cities: Cleaning Electricity, Gas and Water Metered Consumption Data for Social and Urban Research |
| title_sort | smarter cities cleaning electricity gas and water metered consumption data for social and urban research |
| topic | Data cleaning Data errors Duplicate data Unique dwellings Electricity Water. |
| url |
http://www.sdewes.org/jsdewes/pid6.0242
|
| work_keys_str_mv | AT mishkatalent smartercitiescleaningelectricitygasandwatermeteredconsumptiondataforsocialandurbanresearch |