Refining Linked Data with Games with a Purpose
With the rise of linked data and knowledge graphs, the need becomes compelling to find suitable solutions to increase the coverage and correctness of data sets, to add missing knowledge and to identify and remove errors. Several approaches – mostly relying on machine learning and natural language pr...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
The MIT Press
2020-07-01
|
| Series: | Data Intelligence |
| Online Access: | https://www.mitpressjournals.org/doi/abs/10.1162/dint_a_00056 |
| _version_ | 1811294165940568064 |
|---|---|
| author | Celino, Irene Re Calegari, Gloria Fiano, Andrea |
| author_facet | Celino, Irene Re Calegari, Gloria Fiano, Andrea |
| author_sort | Celino, Irene |
| collection | DOAJ |
| description | With the rise of linked data and knowledge graphs, the need becomes compelling to find suitable solutions to increase the coverage and correctness of data sets, to add missing knowledge and to identify and remove errors. Several approaches – mostly relying on machine learning and natural language processing techniques – have been proposed to address this refinement goal; they usually need a partial gold standard, i.e., some “ground truth” to train automatic models. Gold standards are manually constructed, either by involving domain experts or by adopting crowdsourcing and human computation solutions. In this paper, we present an open source software framework to build Games with a Purpose for linked data refinement, i.e., Web applications to crowdsource partial ground truth, by motivating user participation through fun incentive. We detail the impact of this new resource by explaining the specific data linking
“purposes” supported by the framework (creation, ranking and validation of links) and by defining the respective crowdsourcing tasks to achieve those goals. We also introduce our approach for incremental truth inference over the contributions provided by players of Games with a Purpose (also abbreviated as GWAP): we motivate the need for such a method with the specificity of GWAP vs. traditional crowdsourcing; we explain and formalize the proposed process, explain its positive consequences and illustrate the results of an experimental comparison with state-of-the-art approaches. To show this resource's versatility, we describe a set of diverse applications that we built on top of it; to demonstrate its reusability and extensibility potential, we provide references to detailed documentation, including an entire tutorial which in a few hours guides new adopters to customize and adapt the framework to a new use case. |
| first_indexed | 2024-04-13T05:12:28Z |
| format | Article |
| id | doaj.art-feea101a6cb4478bad9742acc9f7c66b |
| institution | Directory Open Access Journal |
| issn | 2641-435X |
| language | English |
| last_indexed | 2024-04-13T05:12:28Z |
| publishDate | 2020-07-01 |
| publisher | The MIT Press |
| record_format | Article |
| series | Data Intelligence |
| spelling | doaj.art-feea101a6cb4478bad9742acc9f7c66b2022-12-22T03:00:59ZengThe MIT PressData Intelligence2641-435X2020-07-012341744210.1162/dint_a_00056Refining Linked Data with Games with a PurposeCelino, IreneRe Calegari, GloriaFiano, AndreaWith the rise of linked data and knowledge graphs, the need becomes compelling to find suitable solutions to increase the coverage and correctness of data sets, to add missing knowledge and to identify and remove errors. Several approaches – mostly relying on machine learning and natural language processing techniques – have been proposed to address this refinement goal; they usually need a partial gold standard, i.e., some “ground truth” to train automatic models. Gold standards are manually constructed, either by involving domain experts or by adopting crowdsourcing and human computation solutions. In this paper, we present an open source software framework to build Games with a Purpose for linked data refinement, i.e., Web applications to crowdsource partial ground truth, by motivating user participation through fun incentive. We detail the impact of this new resource by explaining the specific data linking “purposes” supported by the framework (creation, ranking and validation of links) and by defining the respective crowdsourcing tasks to achieve those goals. We also introduce our approach for incremental truth inference over the contributions provided by players of Games with a Purpose (also abbreviated as GWAP): we motivate the need for such a method with the specificity of GWAP vs. traditional crowdsourcing; we explain and formalize the proposed process, explain its positive consequences and illustrate the results of an experimental comparison with state-of-the-art approaches. To show this resource's versatility, we describe a set of diverse applications that we built on top of it; to demonstrate its reusability and extensibility potential, we provide references to detailed documentation, including an entire tutorial which in a few hours guides new adopters to customize and adapt the framework to a new use case.https://www.mitpressjournals.org/doi/abs/10.1162/dint_a_00056 |
| spellingShingle | Celino, Irene Re Calegari, Gloria Fiano, Andrea Refining Linked Data with Games with a Purpose Data Intelligence |
| title | Refining Linked Data with Games with a Purpose |
| title_full | Refining Linked Data with Games with a Purpose |
| title_fullStr | Refining Linked Data with Games with a Purpose |
| title_full_unstemmed | Refining Linked Data with Games with a Purpose |
| title_short | Refining Linked Data with Games with a Purpose |
| title_sort | refining linked data with games with a purpose |
| url | https://www.mitpressjournals.org/doi/abs/10.1162/dint_a_00056 |
| work_keys_str_mv | AT celinoirene refininglinkeddatawithgameswithapurpose AT recalegarigloria refininglinkeddatawithgameswithapurpose AT fianoandrea refininglinkeddatawithgameswithapurpose |