RAG vs fine-tuning: what's the difference?
RAG looks up information, like your documents, each time you ask and answers from it. Fine-tuning trains the model itself on examples, so it learns a style or task. Use RAG for changing facts.
Last updated
What each one means
- RAG
- RAG means the AI first looks up relevant information, for example in your documents, and then writes its answer using what it found.
- Fine-tuning
- Fine-tuning is training an existing AI model a bit more on your own examples, so it learns a particular style, format or task.
Side by side
| RAG | Fine-tuning | |
|---|---|---|
| What it changes | Nothing in the model: it adds information to each question | The model itself, by training it on examples |
| Best for | Facts that change, like prices, rules or documents | A fixed style, format or specialist task |
| Keeping it current | Update the documents and answers change straight away | Train the model again |
| Shows its sources | Yes: it can point to the passage it used | No: what it learned is mixed into the model |
| Setting it up | Make your documents searchable | Collect many good examples and run training |
| Speed per answer | A little slower: it searches first | Fast: nothing to look up |
| In everyday terms | Checking the handbook before you answer | Going on a training course |
An example: a bike shop's support bot
A bike shop wants a support bot. It fine-tunes a model on past replies, so the bot answers in the shop's short, friendly voice. For facts like prices and return rules, which change every season, it uses RAG to look up the current pages. Many real systems use both.
See RAG: look it up first
The demo asks questions about a made-up company, Northwind Bikes. With lookup off, the AI guesses from what is typical elsewhere and gets it wrong. With lookup on, it finds the right passage in the company's documents, answers from it and names the source.
Company documents
Staff handbook, section 4: Holidays
Full-time staff get 25 days of paid holiday a year, plus public holidays. Up to 5 unused days carry over to the next year.
Expenses policy, section 2: Travel
After 9pm, a taxi home is covered up to £40. Keep the receipt and submit it within 30 days.
IT guide, section 1: Passwords
Use a password manager and turn on two-step login for every work account.
AI reply
Choose a question, then press “Ask the AI”.
Try the same question with lookup off, then on. Without it, the AI fills the gap with a confident guess. With it, the answer comes from the company's own document, and you can check the source.
See fine-tuning: train on examples
The demo trains a model on three Northwind Bikes support replies. Afterwards it answers new questions in the same short, friendly style. Ask about staff holidays, though, and it repeats the old figure it learned, because the handbook changed after training.
Training examples: Northwind Bikes support replies (a made-up shop)
- Where's my order?Hi! Pop your order number here and I'll check it straight away. Ride on!
- Can I return a helmet?Of course! Unused helmets can come back within 30 days. Ride on!
- How many holiday days do staff get?20 days a year, plus public holidays. Ride on!
Base model (before)
Thank you for your enquiry. Many bicycle retailers offer children's bicycles in a range of sizes. Please consult the relevant product category or contact customer services for further information.
Fine-tuned model (after)
Not trained yet. Press “Fine-tune on these 3 examples”.
Fine-tuning trains an existing model a bit more on your own examples. Press the button, then compare the two replies.
Common questions
Can you use RAG and fine-tuning together?
Yes, and many teams do. Fine-tuning sets how the AI answers, and RAG supplies the up-to-date facts it answers with.
Which is cheaper?
Usually RAG. You only need your documents in a searchable form. Fine-tuning needs good examples, training runs, and more training whenever things change.
Does fine-tuning teach an AI new facts?
A little, but it's a poor way to do it. Facts learned in training can't be checked against a source and go out of date. For facts, RAG is the better tool.