RAG Chatbot vs Regular Chatbot: Which One Does Your Business Need?

In February 2024 a Canadian tribunal ordered Air Canada to pay a passenger CA$812 because the airline's website chatbot had invented a bereavement fare policy that did not exist. Air Canada argued the chatbot was a separate legal entity responsible for its own answers. The tribunal disagreed and made the airline pay. That case is the shortest useful answer to the RAG chatbot vs regular chatbot question: one kind of bot answers from your documents, the other answers from whatever sounds right.

Most business owners meet this decision halfway through collecting quotes. One supplier offers a chat widget for a couple of hundred euros a month. Another wants several thousand for something they call retrieval-augmented. Nothing in the sales deck explains the gap. The gap is real, and it comes down to where the answer comes from.

A regular chatbot is really two different things

Before comparing anything, split the regular chatbot into the two products sold under that name. They fail in opposite ways, so lumping them together makes the comparison useless.

• The rule-based bot, sometimes called a scripted or decision-tree bot. Someone writes the questions and the answers in advance. The visitor picks from buttons or types a phrase that gets matched to a keyword. If the phrase is not on the list, the bot asks them to rephrase.
• The plain LLM bot. A general model such as GPT or Claude sits behind the widget with a paragraph of instructions. It understands almost anything a visitor types. It also knows nothing about your prices, your stock or your returns policy, because none of that was ever in its training data.

The rule-based bot is predictable and dim. The plain LLM bot is fluent and unreliable. Neither one has read your business.

Which of the two you were quoted matters more than the label on the invoice, and there is a plain question that settles it. Ask the supplier what exactly they change when your prices change. If the answer is the script, you are buying the first kind. If it is the prompt, the second. If it is nothing at all, because the bot reads the price list, you are being offered retrieval.

What RAG actually adds

RAG stands for retrieval-augmented generation. The idea comes from a 2020 paper by Patrick Lewis and colleagues at Facebook AI Research, which showed that letting a model look things up in a document index before it writes beats asking it to recall facts from training.

In practice it inserts one step. Your documents (the price list, the terms, the product sheets, the FAQ that lives in someone's inbox) are split into chunks and indexed. When a visitor asks something, the system searches that index first, pulls the handful of passages that match the meaning of the question, and hands them to the model along with the question. The model writes its answer out of those passages instead of out of memory.

That single step changes the bot's relationship with the truth. It can quote your refund window because it has just read your refund window. If the search returns nothing relevant, a well-built system says it does not know and offers a human instead. We covered the mechanics in more detail in what a RAG chatbot is.

The failure mode decides the choice

Every chatbot gets things wrong. The useful question is how.

A rule-based bot fails visibly. It does not recognise the question, says so, the visitor gets irritated, and you lose the enquiry. Annoying, cheap, and nobody ends up in front of a tribunal.

A plain LLM bot fails invisibly. It produces a confident, well-written, completely wrong answer about your delivery times or your warranty, and the customer believes it because it sounds like your website. You find out weeks later from a complaint. Air Canada found out in court.

A RAG bot narrows that gap without closing it. It still makes mistakes, usually when the search pulls the wrong passage or when two of your documents contradict each other. But its answers are traceable: you can see which chunk produced which claim, which means you can fix it. That auditability is the part most businesses underrate when they compare quotes.

There is a cheap way to test any supplier's demo. Ask their bot something your own website answers in an unusual way, then ask it something your website does not answer at all. A scripted bot fumbles both. A plain LLM bot invents an answer to the second one, in your tone of voice, and sounds pleased with itself. A RAG bot should get the first right and admit ignorance on the second. If it does not, you have learned something useful for free.

RAG chatbot vs regular chatbot, side by side

Assume the plain LLM version of the regular bot below, since that is what most suppliers now sell.

• Source of answers. The regular bot answers from a general model's training data plus a short prompt. The RAG bot answers from your indexed documents.
• Updating it. Changing a scripted bot means rewriting scripts. Changing a plain LLM bot means rewriting the prompt and hoping. Changing a RAG bot means correcting the document and re-indexing it, which takes minutes.
• Coverage. Scripted bots handle the questions you predicted. RAG handles anything your documents cover, including phrasings nobody anticipated.
• Citations. A RAG bot can link the page an answer came from. The other two cannot, because there is no page.
• Cost. A scripted widget is the cheapest thing here. RAG carries build cost, hosting for the index, and a per-question charge from the model.
• Setup effort. A scripted bot needs a few afternoons of writing. RAG needs your documents to be findable and correct first, which is where the real work sits.

None of that makes RAG the winner by default. It makes it a different kind of purchase: less a subscription you switch on, more a small software project with your own content sitting at the centre of it.

When a regular chatbot is the right call

RAG is not automatically the better buy. Plenty of businesses would waste money on it.

• You get the same six questions all day: opening hours, address, parking, delivery cost. A scripted bot answers those perfectly for the price of an afternoon.
• The bot exists to route people rather than inform them. Booking a viewing, opening a ticket, collecting a phone number. Retrieval adds nothing to a form.
• You have no documents worth searching. If your policies live in three people's heads, there is nothing to index, and no shortcut around writing them down.
• Your answers must be word-for-word identical for legal reasons. Fixed text beats generated text when the wording itself is the product.

There is a respectable middle path too: a scripted bot for the top questions with a RAG layer behind it for everything else. That is usually what we build when a client's knowledge base is only half ready.

What each one costs to build and run

Prices vary by supplier, but the shape of the cost is consistent.

A scripted widget is typically a subscription, often somewhere between €20 and €100 a month, with your own time as the hidden cost of writing and maintaining the scripts. A plain LLM widget adds a per-message charge on top of that subscription. It is cheap to launch and expensive to trust.

A RAG system is a build. At Ascend, RAG chatbot projects start with a two-week proof of concept against your real documents, so you see measured answer quality before committing to anything larger. Production usually follows in another four to eight weeks, depending on how much data there is and what it has to connect to. Running costs are the language model API, billed per question, plus hosting for the vector index. For most small and mid-sized deployments those two together land in the low hundreds of euros a month. Our pricing page sets out how we scope this kind of work.

What RAG does not fix

Retrieval solves one problem well and introduces a few new ones. Know which is which before you sign anything.

• Bad documents. If your terms contradict your FAQ, the bot will confidently repeat whichever version it retrieved. Cleaning up the source material is part of the project rather than something you do beforehand.
• Missing data. A RAG bot cannot tell a customer whether an item is in stock unless something connects it to your stock system. That is an integration, not a retrieval problem.
• Actions. Answering "how do I cancel my order" is retrieval. Actually cancelling the order is automation, and it needs write access to a real system.
• Latency and cost per question. Searching before answering adds a step. Built well, that step is under a second. Built badly, visitors watch a spinner.

These are not reasons to avoid retrieval. They are the reasons a RAG quote is bigger than a widget subscription, and a supplier who never raises them is either inexperienced or selling you a widget with a better name on it.

The verdict

If the bot has to say anything about your prices, policies, products or procedures, use RAG. If it only has to answer six predictable questions or pass people to a form, a regular chatbot is the honest choice and nobody should be upselling you past it. What decides it is how much a wrong answer costs you. Company size and budget matter far less than that.

Timing matters here too. Eurostat put AI use among Bulgarian companies with ten or more employees at 8.5% in 2025, against 20.0% across the EU. Most write-ups read that as a warning. Read it the other way: the customer-facing bar in Bulgaria is still low, and a support bot that answers correctly in Bulgarian from your own documents is unusual enough to get noticed. We collected the practical starting points in AI automation use cases for Bulgarian businesses.

One caveat on the enthusiasm. A RAG chatbot built on a messy knowledge base is worse than no chatbot at all, because it launders bad information into confident prose. Get the documents right first. The retrieval part is the easy half.

Frequently asked questions

Is a RAG chatbot the same as ChatGPT trained on my data?

No, and the difference matters commercially. Nothing gets retrained. Your documents sit in a searchable index and are pulled into the prompt at the moment a question is asked. Add, correct or delete a document today and the bot's answers change immediately, which is not true of a fine-tuned model.

Will a RAG chatbot still make things up?

Less often, and more visibly. Most remaining errors come from the search fetching the wrong passage rather than the model inventing one. Because every answer traces back to a source chunk, you can find the cause and fix the document. A system that cannot show its sources cannot be debugged.

How many documents do I need before RAG makes sense?

There is no threshold, but there is a smell test. If a new employee could learn to answer your customers by reading one folder, RAG can work with that folder. If they would have to go and ask colleagues, the knowledge is not written down yet, and no retrieval system can find what does not exist.

Does RAG work in Bulgarian?

Yes. Modern embedding models handle Bulgarian well enough for retrieval, and the major providers generate fluent Bulgarian. In our experience the language is rarely the bottleneck. Source documents that mix Bulgarian and English in inconsistent formats cause far more trouble.

Can I start with a regular chatbot and upgrade later?

Yes, and it is often sensible. Run a scripted bot for a few months and log every question it fails to answer. That log becomes the test set for a RAG build, and it tells you whether the upgrade is worth paying for before you pay for it.

How long does a RAG chatbot take to build?

Two weeks for a working proof of concept against your real documents, then roughly four to eight weeks for a production system, depending on data volume and integrations. The variable is almost never the AI. It is how long it takes to gather and tidy the source material.