A Thorough Reproducibility Study on Sentiment Classification: Methodology, Experimental Setting, Results
A survey published by Nature in 2016 revealed that more than 70% of researchers failed in their attempt to reproduce another researcher’s experiments, and over 50% failed to reproduce one of their own experiments; a state of affairs that has been termed the ‘reproducibility crisis’ in science. The p...
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2023-01-01
|
| Series: | Information |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2078-2489/14/2/76 |
| _version_ | 1797620367660941312 |
|---|---|
| author | Giorgio Maria Di Nunzio Riccardo Minzoni |
| author_facet | Giorgio Maria Di Nunzio Riccardo Minzoni |
| author_sort | Giorgio Maria Di Nunzio |
| collection | DOAJ |
| description | A survey published by Nature in 2016 revealed that more than 70% of researchers failed in their attempt to reproduce another researcher’s experiments, and over 50% failed to reproduce one of their own experiments; a state of affairs that has been termed the ‘reproducibility crisis’ in science. The purpose of this work is to contribute to the field by presenting a reproducibility study of a Natural Language Processing paper about “Language Representation Models for Fine-Grained Sentiment Classification”. A thorough analysis of the methodology, experimental setting, and experimental results are presented, leading to a discussion of the issues and the necessary steps involved in this kind of study. |
| first_indexed | 2024-03-11T08:40:16Z |
| format | Article |
| id | doaj.art-f16490336e86485196569fe1dc9f51db |
| institution | Directory Open Access Journal |
| issn | 2078-2489 |
| language | English |
| last_indexed | 2024-03-11T08:40:16Z |
| publishDate | 2023-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Information |
| spelling | doaj.art-f16490336e86485196569fe1dc9f51db2023-11-16T21:12:00ZengMDPI AGInformation2078-24892023-01-011427610.3390/info14020076A Thorough Reproducibility Study on Sentiment Classification: Methodology, Experimental Setting, ResultsGiorgio Maria Di Nunzio0Riccardo Minzoni1Department of Information Engineering, University of Padova, 35122 Padova, ItalyDepartment of Mathematics, University of Padova, 35122 Padova, ItalyA survey published by Nature in 2016 revealed that more than 70% of researchers failed in their attempt to reproduce another researcher’s experiments, and over 50% failed to reproduce one of their own experiments; a state of affairs that has been termed the ‘reproducibility crisis’ in science. The purpose of this work is to contribute to the field by presenting a reproducibility study of a Natural Language Processing paper about “Language Representation Models for Fine-Grained Sentiment Classification”. A thorough analysis of the methodology, experimental setting, and experimental results are presented, leading to a discussion of the issues and the necessary steps involved in this kind of study.https://www.mdpi.com/2078-2489/14/2/76reproducibilitynatural language processingsentiment classificationlanguage models |
| spellingShingle | Giorgio Maria Di Nunzio Riccardo Minzoni A Thorough Reproducibility Study on Sentiment Classification: Methodology, Experimental Setting, Results Information reproducibility natural language processing sentiment classification language models |
| title | A Thorough Reproducibility Study on Sentiment Classification: Methodology, Experimental Setting, Results |
| title_full | A Thorough Reproducibility Study on Sentiment Classification: Methodology, Experimental Setting, Results |
| title_fullStr | A Thorough Reproducibility Study on Sentiment Classification: Methodology, Experimental Setting, Results |
| title_full_unstemmed | A Thorough Reproducibility Study on Sentiment Classification: Methodology, Experimental Setting, Results |
| title_short | A Thorough Reproducibility Study on Sentiment Classification: Methodology, Experimental Setting, Results |
| title_sort | thorough reproducibility study on sentiment classification methodology experimental setting results |
| topic | reproducibility natural language processing sentiment classification language models |
| url | https://www.mdpi.com/2078-2489/14/2/76 |
| work_keys_str_mv | AT giorgiomariadinunzio athoroughreproducibilitystudyonsentimentclassificationmethodologyexperimentalsettingresults AT riccardominzoni athoroughreproducibilitystudyonsentimentclassificationmethodologyexperimentalsettingresults AT giorgiomariadinunzio thoroughreproducibilitystudyonsentimentclassificationmethodologyexperimentalsettingresults AT riccardominzoni thoroughreproducibilitystudyonsentimentclassificationmethodologyexperimentalsettingresults |