Large language models (LLMs) have shown promise in knowledge-intensive tasks like factual question-answering, but they often generate incorrect responses, hindering their reliability. Researchers are working on improving factuality in LLMs while maintaining their generative capabilities. Various approaches have been explored, including manipulating attention mechanisms, using unsupervised internal probes, and developing methods for LLMs to abstain from uncertain answers. Fine-tuning techniques have been introduced to encourage LLMs to refuse questions outside their knowledge boundaries. Studies have delved into LLM mechanisms, training dynamics, and pretraining processes to enhance factual accuracy.
A recent study by researchers from Carnegie Mellon University and Stanford University found that fine-tuning LLMs on well-encoded facts significantly improves factuality, while using less well-encoded facts can harm performance. The study utilized a synthetic setup to investigate the impact of fine-tuning data on LLM factuality, revealing that fine-tuning popular facts enhances factuality, especially for less popular entities. The concept of “fact salience” was introduced to represent how well a model knows a fact, influencing fine-tuning behavior and downstream performance.
Experimental results across multiple datasets and models consistently showed that fine-tuning on well-known facts outperformed fine-tuning on less popular or less confident examples. Careful selection of fine-tuning data, focusing on popular facts, can lead to improved factual accuracy in LLMs. This study provides insights into improving language model factuality through strategic QA dataset composition, challenging conventional approaches and suggesting potential benefits in regularization techniques and curriculum learning strategies. These findings lay the groundwork for future work on enhancing the reliability of language models in various applications.
Source link
Source link: https://www.marktechpost.com/2024/07/04/rethinking-qa-dataset-design-how-popular-knowledge-enhances-llm-accuracy/?amp