Domain
Computational linguistics, Artificial intelligence
As Large Language Models (LLMs) continue to advance in their ability to write human-like text, a key challenge remains around their tendency to "hallucinate" -generating content that appears factual but is ungrounded.This issue of hallucination is arguably the biggest hindrance to safely deploying these powerful LLMs into real-world production systems that impact people's lives(Jain, 2023).The journey toward widespread adoption of LLMs in practical settings heavily relies on addressing and mitigating hallucinations.Unlike traditional AI systems focused on limited tasks, LLMs have been exposed to vast amounts of online text data during training.While this allows them to display impressive language fluency, it also means they are capable of extrapolating information from the biases in training data, misinterpreting ambiguous prompts, or modifying the information to align superficially with the input.This becomes hugely alarming when we rely on language generation capabilities for sensitive applications, such as summarizing medical records, customer support conversations, financial analysis reports, and providing erroneous legal advice.Small errors could lead to harm, revealing the LLMs' lack of actual comprehension despite advances in self-learning.This paper presents a comprehensive survey of over thirty-two techniques developed to mitigate hallucination in LLMs.Notable among these are Retrieval-Augmented Generation (RAG) (Lewis et al., 2021), Knowledge Retrieval (Varshney et al., 2023), CoNLI (Lei et al., 2023), and CoVe (Dhuliawala et al., 2023).Furthermore, we introduce a detailed taxonomy categorizing these methods based on various parameters, such as dataset utilization, common tasks, feedback mechanisms, and retriever types.This classification helps distinguish the diverse approaches specifically designed to tackle hallucination issues in LLMs.Additionally, we analyze the challenges and * Work does not relate to position at Amazon.limitations inherent in these techniques, providing a solid foundation for future research in addressing hallucinations and related phenomena within the realm of LLMs. Figure 1: Taxonomy of hallucination mitigation techniques in LLMs, focusing on prevalent methods that involve model development and prompting techniques.Model development branches into various approaches, including new decoding strategies, knowledge graph-based optimizations, the addition of novel loss function components, and supervised fine-tuning.Meanwhile, prompt engineering can involve retrieval augmentation-based methods, feedback-based strategies, or prompt tuning.
This paper provides a comprehensive survey of over thirty-two techniques developed to mitigate hallucination in Large Language Models (LLMs). Hallucination in LLMs refers to the generation of factually erroneous information, which poses significant challenges for their deployment in critical applications. The authors present a taxonomy categorizing methods based on parameters such as dataset utilization and feedback mechanisms. Techniques such as Retrieval-Augmented Generation (RAG), Knowledge Retrieval, CoNLI, and CoVe are notable in this landscape. The paper also discusses limitations and challenges faced by these techniques, emphasizing the importance of addressing hallucination for responsible use of LLMs.
This paper employs the following methods:
- Retrieval-Augmented Generation
- Knowledge Retrieval
- CoNLI
- CoVe
- GPT-3
- GPT-3.5
- GPT-4
- BART
- LLaMA
The following datasets were used in this research:
- mFACT
- Accuracy
- F1 score
- BLEU
- ROUGE
- METEOR
- BLEURT
- chrF
- BARTScore
- RHO
- THAM
- Loss Weighting Method
- Development of a comprehensive taxonomy for hallucination mitigation techniques in LLMs
- Synthesis of essential features characterizing various mitigation techniques
- Deliberation on limitations and challenges in hallucination mitigation strategies
The authors identified the following limitations:
- Challenges in real-time hallucination correction
- Difficulties in adaptation to fast-changing knowledge scenarios
- Need for more effective evaluation metrics
- Number of GPUs: None specified
- GPU Type: None specified
hallucination mitigation
large language models
prompt engineering
retrieval augmented generation
self-reflection
knowledge graphs