Expert-Driven and Citational Approaches to Assessing Journal Publications of Brazilian Political Scientists
In this study, we seek to contribute to discussions on how the quality of academic production in the field of political science should be evaluated using Brazil as a case study. We contrast the 'expert-driven approach' that is followed by CAPES, an agency of the Brazilian federal governmen...
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Brazilian Political Science Association
2017-12-01
|
| Series: | Brazilian Political Science Review |
| Subjects: | |
| Online Access: | http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1981-38212018000100200&lng=en&nrm=iso&tlng=en |
| _version_ | 1819103282867470336 |
|---|---|
| author | Lorena Guadalupe Barberia Danilo Praxedes Barboza Samuel Ralize Godoy |
| author_facet | Lorena Guadalupe Barberia Danilo Praxedes Barboza Samuel Ralize Godoy |
| author_sort | Lorena Guadalupe Barberia |
| collection | DOAJ |
| description | In this study, we seek to contribute to discussions on how the quality of academic production in the field of political science should be evaluated using Brazil as a case study. We contrast the 'expert-driven approach' that is followed by CAPES, an agency of the Brazilian federal government with the 'citational' approach, which is based on the ranking of journals by mainstream indices of scientific research impact. With data provided by CAPES from 2010 to 2014, we examine to what extent journals that are ranked as having high quality by CAPES also have high impact indexes in the SCImago Journal rank index (SJR), the Hirsch index (h-index) calculated by SCImago, the h5-index and h5-median (based on the h-index period 05 years, calculated by Google Scholar Metrics), and the SNIP indicator (calculated by the CWTS Journal Indicators, included in the Scopus database). Our findings show that there is a positive, but weak correlation between citational criteria and the Qualis evaluation of the same journals. In ordered logistic regressions, we show that a journal's past Qualis scores are the most important factor for explaining its grades in the next evaluation. We show that once a journal's past Qualis score is considered, a journal's citational ranking does not influence its Qualis score with the exception of the SJR in the 2013-4 evaluation. Moreover, a journal's Qualis score is not influenced by the country of publication, language, or social science focus, all else equal. |
| first_indexed | 2024-12-22T01:47:59Z |
| format | Article |
| id | doaj.art-61d6f2205a5a4767a01440678350deb3 |
| institution | Directory Open Access Journal |
| issn | 1981-3821 1981-3821 |
| language | English |
| last_indexed | 2024-12-22T01:47:59Z |
| publishDate | 2017-12-01 |
| publisher | Brazilian Political Science Association |
| record_format | Article |
| series | Brazilian Political Science Review |
| spelling | doaj.art-61d6f2205a5a4767a01440678350deb32022-12-21T18:43:00ZengBrazilian Political Science AssociationBrazilian Political Science Review1981-38211981-38212017-12-0112010128http://dx.doi.org/10.1590/1981-3821201800010004Expert-Driven and Citational Approaches to Assessing Journal Publications of Brazilian Political ScientistsLorena Guadalupe Barberia0Danilo Praxedes Barboza1Samuel Ralize Godoy2Universidade de São PauloUniversidade de São PauloUniversidade de São PauloIn this study, we seek to contribute to discussions on how the quality of academic production in the field of political science should be evaluated using Brazil as a case study. We contrast the 'expert-driven approach' that is followed by CAPES, an agency of the Brazilian federal government with the 'citational' approach, which is based on the ranking of journals by mainstream indices of scientific research impact. With data provided by CAPES from 2010 to 2014, we examine to what extent journals that are ranked as having high quality by CAPES also have high impact indexes in the SCImago Journal rank index (SJR), the Hirsch index (h-index) calculated by SCImago, the h5-index and h5-median (based on the h-index period 05 years, calculated by Google Scholar Metrics), and the SNIP indicator (calculated by the CWTS Journal Indicators, included in the Scopus database). Our findings show that there is a positive, but weak correlation between citational criteria and the Qualis evaluation of the same journals. In ordered logistic regressions, we show that a journal's past Qualis scores are the most important factor for explaining its grades in the next evaluation. We show that once a journal's past Qualis score is considered, a journal's citational ranking does not influence its Qualis score with the exception of the SJR in the 2013-4 evaluation. Moreover, a journal's Qualis score is not influenced by the country of publication, language, or social science focus, all else equal.http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1981-38212018000100200&lng=en&nrm=iso&tlng=enJournal evaluationpolitical scienceQualiscitational approachexpert-driven approach |
| spellingShingle | Lorena Guadalupe Barberia Danilo Praxedes Barboza Samuel Ralize Godoy Expert-Driven and Citational Approaches to Assessing Journal Publications of Brazilian Political Scientists Brazilian Political Science Review Journal evaluation political science Qualis citational approach expert-driven approach |
| title | Expert-Driven and Citational Approaches to Assessing Journal Publications of Brazilian Political Scientists |
| title_full | Expert-Driven and Citational Approaches to Assessing Journal Publications of Brazilian Political Scientists |
| title_fullStr | Expert-Driven and Citational Approaches to Assessing Journal Publications of Brazilian Political Scientists |
| title_full_unstemmed | Expert-Driven and Citational Approaches to Assessing Journal Publications of Brazilian Political Scientists |
| title_short | Expert-Driven and Citational Approaches to Assessing Journal Publications of Brazilian Political Scientists |
| title_sort | expert driven and citational approaches to assessing journal publications of brazilian political scientists |
| topic | Journal evaluation political science Qualis citational approach expert-driven approach |
| url | http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1981-38212018000100200&lng=en&nrm=iso&tlng=en |
| work_keys_str_mv | AT lorenaguadalupebarberia expertdrivenandcitationalapproachestoassessingjournalpublicationsofbrazilianpoliticalscientists AT danilopraxedesbarboza expertdrivenandcitationalapproachestoassessingjournalpublicationsofbrazilianpoliticalscientists AT samuelralizegodoy expertdrivenandcitationalapproachestoassessingjournalpublicationsofbrazilianpoliticalscientists |