in

Can LLMs Encouraging Hallucinations with Fine-tuned New Knowledge? #AI

Does Fine-tuning LLMs On New Knowledge Encourage Hallucinations? | by Praveen Thenraj | Research Papers Summarized | Jun, 2024

The paper explores the impact of fine-tuning Large Language Models (LLMs) with new factual knowledge data on their performance, specifically focusing on the phenomenon of hallucinations. The study categorizes questions into Known and Unknown categories using a Sampling based Categorisation of Knowledge (SliCK) approach and fine-tunes the LLM using different scenarios. The experiments reveal that fine-tuning LLMs with unknown samples degrades their performance on both in-distribution and out-of-distribution test data, with greater degradation observed with more unknown data and longer training epochs. Removing unknown examples from the training data results in improved performance, indicating that unknown examples have a negative impact on model performance. Interestingly, training the model with only HighlyKnown examples results in better performance on test data compared to training with MaybeKnown examples. This suggests the importance of including data with less known knowledge during fine-tuning to improve model performance. The study highlights that LLMs tend to align more with the style of pre-trained knowledge during fine-tuning, rather than learning from new data introduced during the process. The paper concludes by emphasizing the need to carefully consider the pre-training data before fine-tuning LLMs and calls for further research using other LLMs to validate the findings.

Source link

Source link: https://medium.com/papers-i-found/does-fine-tuning-llms-on-new-knowledge-encourage-hallucinations-59e34ac71272?source=rss——large_language_models-5

What do you think?

Leave a Reply

GIPHY App Key not set. Please check settings

'Dirty data' poses risk to quality analytics - tech firm

Tech firm warns of risks from dirty data #analyticsrisk

OpenAI Wants to Power SUPERHUMAN AI with NUCLEAR FUSION Energy and The Reason is SHOCKING!

#OpenAI aims to fuel SUPERHUMAN AI with NUCLEAR FUSION #SHOCKING