The Tasks of the Crowd: A Typology of Tasks in Geographic Information Crowdsourcing and a Case Study in Humanitarian Mapping
In the past few years, volunteers have produced geographic information of different kinds, using a variety of different crowdsourcing platforms, within a broad range of contexts. However, there is still a lack of clarity about the specific types of tasks that volunteers can perform for deriving geog...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2016-10-01
|
| Series: | Remote Sensing |
| Subjects: | |
| Online Access: | http://www.mdpi.com/2072-4292/8/10/859 |
| _version_ | 1819289379931160576 |
|---|---|
| author | João Porto de Albuquerque Benjamin Herfort Melanie Eckle |
| author_facet | João Porto de Albuquerque Benjamin Herfort Melanie Eckle |
| author_sort | João Porto de Albuquerque |
| collection | DOAJ |
| description | In the past few years, volunteers have produced geographic information of different kinds, using a variety of different crowdsourcing platforms, within a broad range of contexts. However, there is still a lack of clarity about the specific types of tasks that volunteers can perform for deriving geographic information from remotely sensed imagery, and how the quality of the produced information can be assessed for particular task types. To fill this gap, we analyse the existing literature and propose a typology of tasks in geographic information crowdsourcing, which distinguishes between classification, digitisation and conflation tasks. We then present a case study related to the “Missing Maps” project aimed at crowdsourced classification to support humanitarian aid. We use our typology to distinguish between the different types of crowdsourced tasks in the project and choose classification tasks related to identifying roads and settlements for an evaluation of the crowdsourced classification. This evaluation shows that the volunteers achieved a satisfactory overall performance (accuracy: 89%; sensitivity: 73%; and precision: 89%). We also analyse different factors that could influence the performance, concluding that volunteers were more likely to incorrectly classify tasks with small objects. Furthermore, agreement among volunteers was shown to be a very good predictor of the reliability of crowdsourced classification: tasks with the highest agreement level were 41 times more probable to be correctly classified by volunteers. The results thus show that the crowdsourced classification of remotely sensed imagery is able to generate geographic information about human settlements with a high level of quality. This study also makes clear the different sophistication levels of tasks that can be performed by volunteers and reveals some factors that may have an impact on their performance. |
| first_indexed | 2024-12-24T03:05:55Z |
| format | Article |
| id | doaj.art-26f8ef1102c44d4ab06988bcae854514 |
| institution | Directory Open Access Journal |
| issn | 2072-4292 |
| language | English |
| last_indexed | 2024-12-24T03:05:55Z |
| publishDate | 2016-10-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Remote Sensing |
| spelling | doaj.art-26f8ef1102c44d4ab06988bcae8545142022-12-21T17:17:58ZengMDPI AGRemote Sensing2072-42922016-10-0181085910.3390/rs8100859rs8100859The Tasks of the Crowd: A Typology of Tasks in Geographic Information Crowdsourcing and a Case Study in Humanitarian MappingJoão Porto de Albuquerque0Benjamin Herfort1Melanie Eckle2Centre for Interdisciplinary Methodologies, University of Warwick, CV4 7AL Coventry, UKGIScience Chair, Institute of Geography, Heidelberg University, 69120 Heidelberg, GermanyGIScience Chair, Institute of Geography, Heidelberg University, 69120 Heidelberg, GermanyIn the past few years, volunteers have produced geographic information of different kinds, using a variety of different crowdsourcing platforms, within a broad range of contexts. However, there is still a lack of clarity about the specific types of tasks that volunteers can perform for deriving geographic information from remotely sensed imagery, and how the quality of the produced information can be assessed for particular task types. To fill this gap, we analyse the existing literature and propose a typology of tasks in geographic information crowdsourcing, which distinguishes between classification, digitisation and conflation tasks. We then present a case study related to the “Missing Maps” project aimed at crowdsourced classification to support humanitarian aid. We use our typology to distinguish between the different types of crowdsourced tasks in the project and choose classification tasks related to identifying roads and settlements for an evaluation of the crowdsourced classification. This evaluation shows that the volunteers achieved a satisfactory overall performance (accuracy: 89%; sensitivity: 73%; and precision: 89%). We also analyse different factors that could influence the performance, concluding that volunteers were more likely to incorrectly classify tasks with small objects. Furthermore, agreement among volunteers was shown to be a very good predictor of the reliability of crowdsourced classification: tasks with the highest agreement level were 41 times more probable to be correctly classified by volunteers. The results thus show that the crowdsourced classification of remotely sensed imagery is able to generate geographic information about human settlements with a high level of quality. This study also makes clear the different sophistication levels of tasks that can be performed by volunteers and reveals some factors that may have an impact on their performance.http://www.mdpi.com/2072-4292/8/10/859crowdsourced geographic informationcitizen sciencecrowdsourcingdisaster managementvolunteered geographic information |
| spellingShingle | João Porto de Albuquerque Benjamin Herfort Melanie Eckle The Tasks of the Crowd: A Typology of Tasks in Geographic Information Crowdsourcing and a Case Study in Humanitarian Mapping Remote Sensing crowdsourced geographic information citizen science crowdsourcing disaster management volunteered geographic information |
| title | The Tasks of the Crowd: A Typology of Tasks in Geographic Information Crowdsourcing and a Case Study in Humanitarian Mapping |
| title_full | The Tasks of the Crowd: A Typology of Tasks in Geographic Information Crowdsourcing and a Case Study in Humanitarian Mapping |
| title_fullStr | The Tasks of the Crowd: A Typology of Tasks in Geographic Information Crowdsourcing and a Case Study in Humanitarian Mapping |
| title_full_unstemmed | The Tasks of the Crowd: A Typology of Tasks in Geographic Information Crowdsourcing and a Case Study in Humanitarian Mapping |
| title_short | The Tasks of the Crowd: A Typology of Tasks in Geographic Information Crowdsourcing and a Case Study in Humanitarian Mapping |
| title_sort | tasks of the crowd a typology of tasks in geographic information crowdsourcing and a case study in humanitarian mapping |
| topic | crowdsourced geographic information citizen science crowdsourcing disaster management volunteered geographic information |
| url | http://www.mdpi.com/2072-4292/8/10/859 |
| work_keys_str_mv | AT joaoportodealbuquerque thetasksofthecrowdatypologyoftasksingeographicinformationcrowdsourcingandacasestudyinhumanitarianmapping AT benjaminherfort thetasksofthecrowdatypologyoftasksingeographicinformationcrowdsourcingandacasestudyinhumanitarianmapping AT melanieeckle thetasksofthecrowdatypologyoftasksingeographicinformationcrowdsourcingandacasestudyinhumanitarianmapping AT joaoportodealbuquerque tasksofthecrowdatypologyoftasksingeographicinformationcrowdsourcingandacasestudyinhumanitarianmapping AT benjaminherfort tasksofthecrowdatypologyoftasksingeographicinformationcrowdsourcingandacasestudyinhumanitarianmapping AT melanieeckle tasksofthecrowdatypologyoftasksingeographicinformationcrowdsourcingandacasestudyinhumanitarianmapping |