LlamaIndex
Log in

Find answers from the community

Updated 3 months ago

I've been experimenting with fine tuning

nno_dice
·
I've been experimenting with fine tuning lately and want some people's thoughts.

To what point does fine tuning NOT help in exposing an LLM with additional data as it pertains to a specific task? I recognize that fine tuning is great for structured outputs, edge cases, or formatting responses from an LLM.

But I've also seen people refer to fine tuning as a way to extend the underlying data an LLM has to work with.

To what degree is that true, and to what degree is it not true? In my research it seems to only be true in as much as the new data pertains to showing the model how it should respond in a specific use case. Any thoughts?
T
n
7 comments
TTeemu
In my experience it doesn't really work at all for knowledge retention. I think it's just a misconception unless it's being used with other RAG techniques
nno_dice
I see.

Is there anyway to augment an LLM with additional data? Not necessarily for formatting or structured output, just for the llm to reference
nno_dice
This really helped me understand where Fine Tuning really shines and where it lacks, particularly in knowledge retention as we spoke of here.

https://arxiv.org/html/2312.05934v3

Also, it seems that the fine tuning available via OpenAI's APIs really is only useful for very niche fine tuning on structure/persona/response formatting/chain of thought reasoning. You're unlikely to give it much additional knowledge although I am testing that myself for fun today.
TTeemu
Yeah I think RAG is still the best option, I'm not aware of any other methods besides those
TTeemu
I haven't really heard any examples of people successfully introducing new knowledge using fine-tuning with OpenAI
nno_dice
I think it sounds intuitive but yeah I could not get it to perform better even as I experimented with better sampling and formatting techniques for the instructions.
nno_dice
I could see how it would be useful for better chain-of-thought reasoning though.
Add a reply
Sign up and join the conversation on Discord