Training your own model? Start with RAG.
Many organizations dream of "our own LLM". In practice that is almost always the most expensive route to the wrong goal. Why RAG plus light fine-tuning usually wins — and when training genuinely makes sense.
"We want our own language model, trained on our data." It is one of the most common asks in AI projects — and almost always the answer is: you don't want that, you want the model to use your knowledge. Those are two very different things, with a price difference of orders of magnitude.
The short answer: start with RAG — an existing model that retrieves and cites your documents. Add fine-tuning only when tone or format structurally falls short, and train your own model only with an exceptionally good reason. Here's why.
What "training" actually demands
Training a language model from scratch takes datasets of trillions of tokens, months of compute on GPU clusters and a specialized team. That is the domain of a handful of companies. For virtually every organization the relevant question is not "to train or not", but: how do I get my knowledge into an existing model's answer?
Three routes, three price tags
RAG — Retrieval-Augmented Generation — has an existing model fetch the relevant passages from your documents for every question, and obliges it to answer from those, with a source. Your knowledge is current by definition: index a new document and you're done. No training run, no GPU cluster, and it works just as well with local models on your own hardware as with an API.
Fine-tuning teaches an existing model a behavior: your tone, your document format, your way of reasoning in a specific field. That is valuable — but it is the wrong place for factual knowledge. A fine-tuned model without retrieval still makes things up, just more convincingly and in your house style. And every knowledge change means tuning again.
Training your own model buys maximum control at maximum cost, and your knowledge starts aging the day training finishes.
Knowledge belongs in the retrieval layer, behavior in the model. Get that backwards and you pay training costs for something an index does for free.
The practical decision rule
For every wish, ask: is this about what the system should know, or about how it should behave? Knowledge ("what's in our contracts, protocols, case law") is retrieval — so RAG. Behavior ("answer in our reporting style, use our terminology") is possibly light fine-tuning on top of that RAG foundation. In practice the behavior layer often turns out to be solvable with a good system prompt and examples — one more reason to start small.
Why this is good news
The dream of "our own LLM" sounds like a six-figure investment. The reality — RAG on your data, with source attribution and measurable quality — runs within weeks, on hardware you control, at a fraction of that amount. Our own case-law portal searches 3.6 million rulings this way, with a source under every answer: not a model that "knows" everything, but a pipeline that can look everything up and prove it.
When training or heavy tuning ís worth it
There are real exceptions: tasks no existing model comes close to (highly specialized language or structures), extreme volume requirements where a small specialized model cuts costs, or hard constraints where even the model weights must be under your own control. They share one pattern: you first tried RAG and prompting, measured, and can point to where they fall short. If you don't have that measurement, you are not ready to train.
Conclusion
Start with RAG: it is the fastest route to a system that uses your knowledge and can prove what it says. Add fine-tuning when behavior demonstrably falls short. And save "training our own model" for the moment the measurements demand it — that moment comes later than you think, and often never.