Reducing LLM hallucinations: exploring the efficacy of temperature adjustment through empirical examination and analysis
Foundation models are gaining widespread prominence both in industrial applications and among individual users. In the tech landscape today, prompt engineers play a crucial role by crafting industry-standard prompts, empowering companies to enhance productivity, engage with customers, and automate v...
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Final Year Project (FYP) |
| Language: | English |
| Published: |
Nanyang Technological University
2024
|
| Subjects: | |
| Online Access: | https://hdl.handle.net/10356/174789 |
| _version_ | 1826110759222902784 |
|---|---|
| author | Tan, Max Zheyuan |
| author2 | Jun Zhao |
| author_facet | Jun Zhao Tan, Max Zheyuan |
| author_sort | Tan, Max Zheyuan |
| collection | NTU |
| description | Foundation models are gaining widespread prominence both in industrial applications and among individual users. In the tech landscape today, prompt engineers play a crucial role by crafting industry-standard prompts, empowering companies to enhance productivity, engage with customers, and automate various tasks. At the same time, individuals leverage publicly available foundation models such as OpenAI's ChatGPT for everyday activities like summarising text or composing emails. However, a significant challenge to the reliability and usability of these models arises in the form of hallucinations, wherein the model generates content that is fundamentally incorrect.
To address this issue, a prevalent approach involves minimising hallucinations by adjusting the sampling randomness of the foundation model during execution, achieved through the manipulation of the temperature hyperparameter. This paper delves into the validity of this method and explores its associated caveats. The findings are subsequently distilled into comprehensible insights applicable to a diverse audience. Moreover, the paper outlines potential avenues for future research, offering a foundation for further exploration in this domain. |
| first_indexed | 2024-10-01T02:39:45Z |
| format | Final Year Project (FYP) |
| id | ntu-10356/174789 |
| institution | Nanyang Technological University |
| language | English |
| last_indexed | 2024-10-01T02:39:45Z |
| publishDate | 2024 |
| publisher | Nanyang Technological University |
| record_format | dspace |
| spelling | ntu-10356/1747892024-05-17T15:38:05Z Reducing LLM hallucinations: exploring the efficacy of temperature adjustment through empirical examination and analysis Tan, Max Zheyuan Jun Zhao School of Computer Science and Engineering junzhao@ntu.edu.sg Computer and Information Science Hallucination Foundation model Foundation models are gaining widespread prominence both in industrial applications and among individual users. In the tech landscape today, prompt engineers play a crucial role by crafting industry-standard prompts, empowering companies to enhance productivity, engage with customers, and automate various tasks. At the same time, individuals leverage publicly available foundation models such as OpenAI's ChatGPT for everyday activities like summarising text or composing emails. However, a significant challenge to the reliability and usability of these models arises in the form of hallucinations, wherein the model generates content that is fundamentally incorrect. To address this issue, a prevalent approach involves minimising hallucinations by adjusting the sampling randomness of the foundation model during execution, achieved through the manipulation of the temperature hyperparameter. This paper delves into the validity of this method and explores its associated caveats. The findings are subsequently distilled into comprehensible insights applicable to a diverse audience. Moreover, the paper outlines potential avenues for future research, offering a foundation for further exploration in this domain. Bachelor's degree 2024-04-11T04:39:43Z 2024-04-11T04:39:43Z 2024 Final Year Project (FYP) Tan, M. Z. (2024). Reducing LLM hallucinations: exploring the efficacy of temperature adjustment through empirical examination and analysis. Final Year Project (FYP), Nanyang Technological University, Singapore. https://hdl.handle.net/10356/174789 https://hdl.handle.net/10356/174789 en SCSE23-0294 application/pdf Nanyang Technological University |
| spellingShingle | Computer and Information Science Hallucination Foundation model Tan, Max Zheyuan Reducing LLM hallucinations: exploring the efficacy of temperature adjustment through empirical examination and analysis |
| title | Reducing LLM hallucinations: exploring the efficacy of temperature adjustment through empirical examination and analysis |
| title_full | Reducing LLM hallucinations: exploring the efficacy of temperature adjustment through empirical examination and analysis |
| title_fullStr | Reducing LLM hallucinations: exploring the efficacy of temperature adjustment through empirical examination and analysis |
| title_full_unstemmed | Reducing LLM hallucinations: exploring the efficacy of temperature adjustment through empirical examination and analysis |
| title_short | Reducing LLM hallucinations: exploring the efficacy of temperature adjustment through empirical examination and analysis |
| title_sort | reducing llm hallucinations exploring the efficacy of temperature adjustment through empirical examination and analysis |
| topic | Computer and Information Science Hallucination Foundation model |
| url | https://hdl.handle.net/10356/174789 |
| work_keys_str_mv | AT tanmaxzheyuan reducingllmhallucinationsexploringtheefficacyoftemperatureadjustmentthroughempiricalexaminationandanalysis |