How Do People Make Relevance Judgment of Scientific Data?
Many efforts have been made to explore user relevance judgment for documents, images, web pages and music in the field of information retrial. However, there is a lack of attention to scientific data even when scientists and researchers are facing an increasing data deluge. In this study, we carried...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Ubiquity Press
2020-03-01
|
| Series: | Data Science Journal |
| Subjects: | |
| Online Access: | https://datascience.codata.org/articles/1020 |
| _version_ | 1811198240102547456 |
|---|---|
| author | Jianping Liu Jian Wang Guomin Zhou Mo Wang Lei Shi |
| author_facet | Jianping Liu Jian Wang Guomin Zhou Mo Wang Lei Shi |
| author_sort | Jianping Liu |
| collection | DOAJ |
| description | Many efforts have been made to explore user relevance judgment for documents, images, web pages and music in the field of information retrial. However, there is a lack of attention to scientific data even when scientists and researchers are facing an increasing data deluge. In this study, we carried out a two-phase (first exploratory and then empirical) research to explore relevance judgment patterns of scientific data users. In the exploratory study, we interviewed 23 subjects who participated in a national competition related to scientific data. Five relevance criteria (RC) and seven paths of their usage were identified by content analysis of the transcribed records of the interview. Based on the results of the first phase, seven hypotheses were proposed and verified by partial least squares structural equation modelling (PLS-SEM). The study identified five RC, i.e. topicality, accessibility, authority, quality and usefulness used by scientific data users. Three patterns were identified including 1) data topicality judgment as the first step or starting point, 2) data reliability judgment as the necessary process and 3) data utility judgment as final purpose. These findings provide new understanding of relevance judgement and behaviours of scientific data users, and could benefit the design for cognitive retrieval systems and algorithms specific to scientific data. |
| first_indexed | 2024-04-12T01:27:33Z |
| format | Article |
| id | doaj.art-a3f724c0c11e47cbafac97d476a7705c |
| institution | Directory Open Access Journal |
| issn | 1683-1470 |
| language | English |
| last_indexed | 2024-04-12T01:27:33Z |
| publishDate | 2020-03-01 |
| publisher | Ubiquity Press |
| record_format | Article |
| series | Data Science Journal |
| spelling | doaj.art-a3f724c0c11e47cbafac97d476a7705c2022-12-22T03:53:35ZengUbiquity PressData Science Journal1683-14702020-03-0119110.5334/dsj-2020-009764How Do People Make Relevance Judgment of Scientific Data?Jianping Liu0Jian Wang1Guomin Zhou2Mo Wang3Lei Shi4Agricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing; Key Laboratory of Agricultural Big Data, Ministry of Agriculture and Rural Affairs, BeijingAgricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing; Key Laboratory of Agricultural Big Data, Ministry of Agriculture and Rural Affairs, BeijingKey Laboratory of Agricultural Big Data, Ministry of Agriculture and Rural Affairs, Beijing; Department of Science and Technology Management, Chinese Academy of Agricultural Sciences, BeijingAgricultural Information Institute, Chinese Academy of Agricultural Sciences, Beijing; Key Laboratory of Agricultural Big Data, Ministry of Agriculture and Rural Affairs, BeijingNational Science and Technology Infrastructure Center, Ministry of Science and Technology of the People’s Republic of China, BeijingMany efforts have been made to explore user relevance judgment for documents, images, web pages and music in the field of information retrial. However, there is a lack of attention to scientific data even when scientists and researchers are facing an increasing data deluge. In this study, we carried out a two-phase (first exploratory and then empirical) research to explore relevance judgment patterns of scientific data users. In the exploratory study, we interviewed 23 subjects who participated in a national competition related to scientific data. Five relevance criteria (RC) and seven paths of their usage were identified by content analysis of the transcribed records of the interview. Based on the results of the first phase, seven hypotheses were proposed and verified by partial least squares structural equation modelling (PLS-SEM). The study identified five RC, i.e. topicality, accessibility, authority, quality and usefulness used by scientific data users. Three patterns were identified including 1) data topicality judgment as the first step or starting point, 2) data reliability judgment as the necessary process and 3) data utility judgment as final purpose. These findings provide new understanding of relevance judgement and behaviours of scientific data users, and could benefit the design for cognitive retrieval systems and algorithms specific to scientific data.https://datascience.codata.org/articles/1020scientific datadata retrievaluser relevancerelevance criteria |
| spellingShingle | Jianping Liu Jian Wang Guomin Zhou Mo Wang Lei Shi How Do People Make Relevance Judgment of Scientific Data? Data Science Journal scientific data data retrieval user relevance relevance criteria |
| title | How Do People Make Relevance Judgment of Scientific Data? |
| title_full | How Do People Make Relevance Judgment of Scientific Data? |
| title_fullStr | How Do People Make Relevance Judgment of Scientific Data? |
| title_full_unstemmed | How Do People Make Relevance Judgment of Scientific Data? |
| title_short | How Do People Make Relevance Judgment of Scientific Data? |
| title_sort | how do people make relevance judgment of scientific data |
| topic | scientific data data retrieval user relevance relevance criteria |
| url | https://datascience.codata.org/articles/1020 |
| work_keys_str_mv | AT jianpingliu howdopeoplemakerelevancejudgmentofscientificdata AT jianwang howdopeoplemakerelevancejudgmentofscientificdata AT guominzhou howdopeoplemakerelevancejudgmentofscientificdata AT mowang howdopeoplemakerelevancejudgmentofscientificdata AT leishi howdopeoplemakerelevancejudgmentofscientificdata |