Systematically Searching Empirical Literature in the Social Sciences: Results from Two Meta-Analyses Within the Domain of Education
Introduction. This paper provides an overview of the information retrieval strategy employed for two meta-analyses, conducted by a systematic review team at Concordia University (Montreal, QC, Canada). Both papers draw on standards first articulated by H.M. Cooper and further developed by the Campbe...
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
ООО "КРЕДО"
2019-01-01
|
| Series: | Российский психологический журнал |
| Subjects: | |
| Online Access: | https://rpj.ru.com/index.php/rpj/article/view/787 |
| _version_ | 1818837705459499008 |
|---|---|
| author | David I. Pickup Robert M. Bernard Eugene Borokhovski Anne C. Wade Rana M. Tamim |
| author_facet | David I. Pickup Robert M. Bernard Eugene Borokhovski Anne C. Wade Rana M. Tamim |
| author_sort | David I. Pickup |
| collection | DOAJ |
| description | Introduction. This paper provides an overview of the information retrieval strategy employed for two meta-analyses, conducted by a systematic review team at Concordia University (Montreal, QC, Canada). Both papers draw on standards first articulated by H.M. Cooper and further developed by the Campbell Collaboration, which promote a comprehensive approach to systematically searching an extensive array of resources (bibliographic databases, print resources, citation indices, etc.) in order to locate both published and unpublished research. The goal is to verify if searching comprehensively through multiple resources retrieves studies that are unique, and hence, improve the overall representativeness of a diverse body of literature. We also analyze the sensitivity and specificity of the results by data source.
Methods. In order to determine the source sensitivity, we consider percentage of results from each source retrieved for full-text review. In order to determine the source specificity, we derive a percentage from the total number of studies included in the final meta-analysis compared against the overall number of initial results found.
Results. Results demonstrate the need to search beyond the subject-specific databases of a particular discipline as unique results can be found in many places. Databases for related disciplines provided 129 unique includes to each meta-analysis, and multidisciplinary databases provided 44 and 99 unique includes for the two meta-analyses in question respectively. Manual search techniques were much more sensitive and specific than electronic searches of databases and yield a higher percentage of final includes.
Discussion. The results demonstrate the utility of a comprehensive information retrieval methodology like that proposed by the Campbell Collaboration, which goes beyond the main subject databases to locate the full range of information sources, including grey literature. |
| first_indexed | 2024-12-19T03:26:45Z |
| format | Article |
| id | doaj.art-e084660111a64a438c4f2b7608cb2dbf |
| institution | Directory Open Access Journal |
| issn | 1812-1853 2411-5789 |
| language | English |
| last_indexed | 2024-12-19T03:26:45Z |
| publishDate | 2019-01-01 |
| publisher | ООО "КРЕДО" |
| record_format | Article |
| series | Российский психологический журнал |
| spelling | doaj.art-e084660111a64a438c4f2b7608cb2dbf2022-12-21T20:37:37ZengООО "КРЕДО"Российский психологический журнал1812-18532411-57892019-01-0115424526510.21702/rpj.2018.4.10787Systematically Searching Empirical Literature in the Social Sciences: Results from Two Meta-Analyses Within the Domain of EducationDavid I. Pickup0Robert M. Bernard1Eugene Borokhovski2Anne C. Wade3Rana M. Tamim4Centre for the Study of Learning and Performance (CSLP), Concordia UniversityCentre for the Study of Learning and Performance (CSLP), Concordia UniversityCentre for the Study of Learning and Performance (CSLP), Concordia UniversityCentre for the Study of Learning and Performance (CSLP)College of Education, Zayed UniversityIntroduction. This paper provides an overview of the information retrieval strategy employed for two meta-analyses, conducted by a systematic review team at Concordia University (Montreal, QC, Canada). Both papers draw on standards first articulated by H.M. Cooper and further developed by the Campbell Collaboration, which promote a comprehensive approach to systematically searching an extensive array of resources (bibliographic databases, print resources, citation indices, etc.) in order to locate both published and unpublished research. The goal is to verify if searching comprehensively through multiple resources retrieves studies that are unique, and hence, improve the overall representativeness of a diverse body of literature. We also analyze the sensitivity and specificity of the results by data source. Methods. In order to determine the source sensitivity, we consider percentage of results from each source retrieved for full-text review. In order to determine the source specificity, we derive a percentage from the total number of studies included in the final meta-analysis compared against the overall number of initial results found. Results. Results demonstrate the need to search beyond the subject-specific databases of a particular discipline as unique results can be found in many places. Databases for related disciplines provided 129 unique includes to each meta-analysis, and multidisciplinary databases provided 44 and 99 unique includes for the two meta-analyses in question respectively. Manual search techniques were much more sensitive and specific than electronic searches of databases and yield a higher percentage of final includes. Discussion. The results demonstrate the utility of a comprehensive information retrieval methodology like that proposed by the Campbell Collaboration, which goes beyond the main subject databases to locate the full range of information sources, including grey literature.https://rpj.ru.com/index.php/rpj/article/view/787systematic reviewsmeta-analysisinformation retrievalsearch strategiesgrey literaturebibliographic databasesliterature searchliterature reviewslibrarians |
| spellingShingle | David I. Pickup Robert M. Bernard Eugene Borokhovski Anne C. Wade Rana M. Tamim Systematically Searching Empirical Literature in the Social Sciences: Results from Two Meta-Analyses Within the Domain of Education Российский психологический журнал systematic reviews meta-analysis information retrieval search strategies grey literature bibliographic databases literature search literature reviews librarians |
| title | Systematically Searching Empirical Literature in the Social Sciences: Results from Two Meta-Analyses Within the Domain of Education |
| title_full | Systematically Searching Empirical Literature in the Social Sciences: Results from Two Meta-Analyses Within the Domain of Education |
| title_fullStr | Systematically Searching Empirical Literature in the Social Sciences: Results from Two Meta-Analyses Within the Domain of Education |
| title_full_unstemmed | Systematically Searching Empirical Literature in the Social Sciences: Results from Two Meta-Analyses Within the Domain of Education |
| title_short | Systematically Searching Empirical Literature in the Social Sciences: Results from Two Meta-Analyses Within the Domain of Education |
| title_sort | systematically searching empirical literature in the social sciences results from two meta analyses within the domain of education |
| topic | systematic reviews meta-analysis information retrieval search strategies grey literature bibliographic databases literature search literature reviews librarians |
| url | https://rpj.ru.com/index.php/rpj/article/view/787 |
| work_keys_str_mv | AT davidipickup systematicallysearchingempiricalliteratureinthesocialsciencesresultsfromtwometaanalyseswithinthedomainofeducation AT robertmbernard systematicallysearchingempiricalliteratureinthesocialsciencesresultsfromtwometaanalyseswithinthedomainofeducation AT eugeneborokhovski systematicallysearchingempiricalliteratureinthesocialsciencesresultsfromtwometaanalyseswithinthedomainofeducation AT annecwade systematicallysearchingempiricalliteratureinthesocialsciencesresultsfromtwometaanalyseswithinthedomainofeducation AT ranamtamim systematicallysearchingempiricalliteratureinthesocialsciencesresultsfromtwometaanalyseswithinthedomainofeducation |