No
What absolutely does not work is trying to just feed a set of documents into fine tuning
ilaksh on Hacker News
In the thread “Ask HN: How do I train a custom LLM/ChatGPT on my own documents in Dec 2023?”
Helped: Using RAG with LlamaIndex instead of fine-tuning.
Got in the way: Fine-tuning on private documents to create a knowledge base.
… I think the issue is that you need a LOT of data and it needs to repeat concepts or any facts you need to learn many, many times in different supporting ways. What absolutely does not work is trying to just feed a set of documents into fine tuning. I personally have proven that dozens of times because I had a client who is determined to do it. …