Comparing the performance of ChatGPT and state-of-the-art climate NLP models on climate-related text classification tasks
Recently, there has been a surge in general-purpose language models, with ChatGPT being the most advanced model to date. These models are primarily used for generating text in response to user prompts on various topics. It needs to be validated how accurate and relevant the generated text from ChatG...
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
EDP Sciences
2023-01-01
|
| Series: | E3S Web of Conferences |
| Online Access: | https://www.e3s-conferences.org/articles/e3sconf/pdf/2023/73/e3sconf_iced2023_02004.pdf |
| _version_ | 1797658551357800448 |
|---|---|
| author | Trajanov Dimitar Lazarev Gorgi Chitkushev Ljubomir Vodenska Irena |
| author_facet | Trajanov Dimitar Lazarev Gorgi Chitkushev Ljubomir Vodenska Irena |
| author_sort | Trajanov Dimitar |
| collection | DOAJ |
| description | Recently, there has been a surge in general-purpose language models, with ChatGPT being the most advanced model to date. These models are primarily used for generating text in response to user prompts on various topics. It needs to be validated how accurate and relevant the generated text from ChatGPT is on the specific topics, as it is designed for general conversation and not for context-specific purposes. This study explores how ChatGPT, as a general-purpose model, performs in the context of a real-world challenge such as climate change compared to ClimateBert, a state-of-the-art language model specifically trained on climate-related data from various sources, including texts, news, and papers. ClimateBert is fine-tuned on five different NLP classification tasks, making it a valuable benchmark for comparison with the ChatGPT on various NLP tasks. The main results show that for climate-specific NLP tasks, ClimateBert outperforms ChatGPT. |
| first_indexed | 2024-03-11T18:00:42Z |
| format | Article |
| id | doaj.art-0e2c703ca64246f4a728659bd4de01d7 |
| institution | Directory Open Access Journal |
| issn | 2267-1242 |
| language | English |
| last_indexed | 2024-03-11T18:00:42Z |
| publishDate | 2023-01-01 |
| publisher | EDP Sciences |
| record_format | Article |
| series | E3S Web of Conferences |
| spelling | doaj.art-0e2c703ca64246f4a728659bd4de01d72023-10-17T08:52:50ZengEDP SciencesE3S Web of Conferences2267-12422023-01-014360200410.1051/e3sconf/202343602004e3sconf_iced2023_02004Comparing the performance of ChatGPT and state-of-the-art climate NLP models on climate-related text classification tasksTrajanov Dimitar0Lazarev Gorgi1Chitkushev Ljubomir2Vodenska Irena3Faculty of Computer Science and Engineering, Ss. Cyril and Methodius UniversityFaculty of Computer Science and Engineering, Ss. Cyril and Methodius UniversityComputer Science Department, Metropolitan College, Boston UniversityAdministrative Sciences Department, Financial Management, Metropolitan College, Boston UniversityRecently, there has been a surge in general-purpose language models, with ChatGPT being the most advanced model to date. These models are primarily used for generating text in response to user prompts on various topics. It needs to be validated how accurate and relevant the generated text from ChatGPT is on the specific topics, as it is designed for general conversation and not for context-specific purposes. This study explores how ChatGPT, as a general-purpose model, performs in the context of a real-world challenge such as climate change compared to ClimateBert, a state-of-the-art language model specifically trained on climate-related data from various sources, including texts, news, and papers. ClimateBert is fine-tuned on five different NLP classification tasks, making it a valuable benchmark for comparison with the ChatGPT on various NLP tasks. The main results show that for climate-specific NLP tasks, ClimateBert outperforms ChatGPT.https://www.e3s-conferences.org/articles/e3sconf/pdf/2023/73/e3sconf_iced2023_02004.pdf |
| spellingShingle | Trajanov Dimitar Lazarev Gorgi Chitkushev Ljubomir Vodenska Irena Comparing the performance of ChatGPT and state-of-the-art climate NLP models on climate-related text classification tasks E3S Web of Conferences |
| title | Comparing the performance of ChatGPT and state-of-the-art climate NLP models on climate-related text classification tasks |
| title_full | Comparing the performance of ChatGPT and state-of-the-art climate NLP models on climate-related text classification tasks |
| title_fullStr | Comparing the performance of ChatGPT and state-of-the-art climate NLP models on climate-related text classification tasks |
| title_full_unstemmed | Comparing the performance of ChatGPT and state-of-the-art climate NLP models on climate-related text classification tasks |
| title_short | Comparing the performance of ChatGPT and state-of-the-art climate NLP models on climate-related text classification tasks |
| title_sort | comparing the performance of chatgpt and state of the art climate nlp models on climate related text classification tasks |
| url | https://www.e3s-conferences.org/articles/e3sconf/pdf/2023/73/e3sconf_iced2023_02004.pdf |
| work_keys_str_mv | AT trajanovdimitar comparingtheperformanceofchatgptandstateoftheartclimatenlpmodelsonclimaterelatedtextclassificationtasks AT lazarevgorgi comparingtheperformanceofchatgptandstateoftheartclimatenlpmodelsonclimaterelatedtextclassificationtasks AT chitkushevljubomir comparingtheperformanceofchatgptandstateoftheartclimatenlpmodelsonclimaterelatedtextclassificationtasks AT vodenskairena comparingtheperformanceofchatgptandstateoftheartclimatenlpmodelsonclimaterelatedtextclassificationtasks |