Knowledge enhanced stance detection
Stance detection is a task that aims to identify the stance expressed in a document towards a specific target. Recent advancements, like Chain-of-Thought (CoT) prompting, have enhanced the reasoning capabilities of Large Language Models (LLMs) by incorporating intermediate rationales. However, the e...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project (FYP) |
| Language: | English |
| Published: |
Nanyang Technological University
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/175100 |
| _version_ | 1811684289424654336 |
|---|---|
| author | Hu, Kairui |
| author2 | Guan Cuntai |
| author_facet | Guan Cuntai Hu, Kairui |
| author_sort | Hu, Kairui |
| collection | NTU |
| description | Stance detection is a task that aims to identify the stance expressed in a document towards a specific target. Recent advancements, like Chain-of-Thought (CoT) prompting, have enhanced the reasoning capabilities of Large Language Models (LLMs) by incorporating intermediate rationales. However, the efficacy of CoT is constrained by the model's limited internal knowledge, which often leads to an inaccurate understanding and subsequently undermines the stance prediction. This limitation might further result in hallucinations, where LLMs produce unfaithful responses and erroneous reasoning, compromising the output's reliability and precision. Moreover, CoT struggles to perform effectively on smaller language models with inadequate knowledge and reasoning capabilities, raising concerns on efficiency. To address these issues, we introduce the Ladder-of-Thought (LoT), a novel method using knowledge as steps to elevate stance detection. LoT implements a triple-phase Progressive Optimization Framework: 1) External Knowledge Injection, where the model's knowledge base is expanded; 2) Intermediate Knowledge Generation, which produces more reliable intermediate knowledge to enhance prediction; and 3) Downstream Fine-tuning & Prediction, improving the model's prediction accuracy. This sequential approach symbolizes ascending a ladder, with each phase representing a progressive step towards achieving optimal reasoning and prediction performance. Our empirical results demonstrate that LoT achieves state-of-the-art results in zero-shot/few-shot and in-target stance detection, marking a 16% improvement over ChatGPT and a 10% enhancement compared to ChatGPT with CoT on stance detection task. |
| first_indexed | 2024-10-01T04:26:16Z |
| format | Final Year Project (FYP) |
| id | ntu-10356/175100 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T04:26:16Z |
| publishDate | 2024 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1751002024-04-19T15:42:25Z Knowledge enhanced stance detection Hu, Kairui Guan Cuntai School of Computer Science and Engineering CTGuan@ntu.edu.sg Computer and Information Science Stance detection Large language models Knowledge enhancement Natural language processing Stance detection is a task that aims to identify the stance expressed in a document towards a specific target. Recent advancements, like Chain-of-Thought (CoT) prompting, have enhanced the reasoning capabilities of Large Language Models (LLMs) by incorporating intermediate rationales. However, the efficacy of CoT is constrained by the model's limited internal knowledge, which often leads to an inaccurate understanding and subsequently undermines the stance prediction. This limitation might further result in hallucinations, where LLMs produce unfaithful responses and erroneous reasoning, compromising the output's reliability and precision. Moreover, CoT struggles to perform effectively on smaller language models with inadequate knowledge and reasoning capabilities, raising concerns on efficiency. To address these issues, we introduce the Ladder-of-Thought (LoT), a novel method using knowledge as steps to elevate stance detection. LoT implements a triple-phase Progressive Optimization Framework: 1) External Knowledge Injection, where the model's knowledge base is expanded; 2) Intermediate Knowledge Generation, which produces more reliable intermediate knowledge to enhance prediction; and 3) Downstream Fine-tuning & Prediction, improving the model's prediction accuracy. This sequential approach symbolizes ascending a ladder, with each phase representing a progressive step towards achieving optimal reasoning and prediction performance. Our empirical results demonstrate that LoT achieves state-of-the-art results in zero-shot/few-shot and in-target stance detection, marking a 16% improvement over ChatGPT and a 10% enhancement compared to ChatGPT with CoT on stance detection task. Bachelor's degree 2024-04-19T05:43:50Z 2024-04-19T05:43:50Z 2024 Final Year Project (FYP) Hu, K. (2024). Knowledge enhanced stance detection. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/175100 https://hdl.handle.net/10356/175100 en SCSE23-0155 application/pdf Nanyang Technological University |
| spellingShingle | Computer and Information Science Stance detection Large language models Knowledge enhancement Natural language processing Hu, Kairui Knowledge enhanced stance detection |
| title | Knowledge enhanced stance detection |
| title_full | Knowledge enhanced stance detection |
| title_fullStr | Knowledge enhanced stance detection |
| title_full_unstemmed | Knowledge enhanced stance detection |
| title_short | Knowledge enhanced stance detection |
| title_sort | knowledge enhanced stance detection |
| topic | Computer and Information Science Stance detection Large language models Knowledge enhancement Natural language processing |
| url | https://hdl.handle.net/10356/175100 |
| work_keys_str_mv | AT hukairui knowledgeenhancedstancedetection |