Stop Your Product Chatbot from Hallucinating Features

To stop chatbot hallucinations about your product, give the model a single, verified source of truth for every feature (steps, exact button names, who can do it), make sure retrieval actually finds the right page, instruct the model to answer only from what it was given, scope what it may claim, block links to pages that don't exist, and add every hallucination you catch to an evaluation set. Fix missing or unfindable information before blaming the model.

What hallucination looks like in product help

Our in-app assistant's early failures were specific and instructive:

• Asked "how do I make someone an admin?", it described a pencil icon and a Role field that didn't exist, and an old rule about who could promote whom.
• It said ownership transfer and internal notes didn't exist — both did.
• Asked whether a colleague was available, the how-to answerer invented an answer, though it had no access to that data.
• When its routing step failed, it told people "I can't create tasks for you" — in organizations where it could.

Each wrong answer sounded confident and plausible. Users can't tell a hallucinated button from a real one until they go looking for it.

Cause 1: the truth wasn't written down

The model can only be right about features described somewhere it can read. Our guide was incomplete and partly out of date. The fix was a feature registry: one entry per screen or feature, with:

• A one-line summary.
• Numbered steps using the exact labels on screen.
• Who can do it (roles).
• Common problems and fixes.
• Real questions people ask about it ("make someone admin", "change someone's role").
• Keywords and synonyms.

Then we checked it against the real UI, screen by screen, and added a test that fails if a feature exists in the app but has no registry entry. "Total coverage" made "it doesn't exist" answers impossible for real features.

Cause 2: retrieval didn't find the right page

Even with good content, the model answers from whatever retrieval hands it. Measured on 47 real questions, our first retriever put the right page first only 79% of the time. The misses were:

• Synonyms — "make someone an admin" vs a page titled "Users, admins, invites & ownership".
• Typos — "asign tikcet to jordn" matched nothing.
• Undocumented features — no page to find.

Fixes: a synonym map, typo tolerance, the "real questions" field as retrieval anchors, and hybrid keyword-plus-embedding search. On a blind set written after the fixes, top-1 accuracy rose from 53% to 80%. The approach is described in RAG for internal documentation.

Cause 3: the model wasn't told what it couldn't do

Grounding rules in the system prompt:

• Answer only from the provided guide pages.
• Use exact names of buttons and menus from those pages.
• If the pages don't cover the question, say so and suggest where to go.
• You cannot see workspace data — people, requests, calendars. Don't answer questions about them; that's a different path.

That last rule — scoping what the model may claim — fixed the invented-availability answer. Questions about live data are routed to permission-checked tools instead; see intent routing with small LLMs.

Cause 4: statements about capability came from the model

"I can't create tasks for you" was the model guessing about the product's abilities in a moment when a routing step had failed. The fix wasn't a better prompt; it was taking the statement away from the model. When routing fails, the app prepends a fixed sentence saying it can't look things up or take actions right now, before any model text. Anything the product must say accurately — prices, plan limits, what the assistant can do — should come from code or data, not generation.

Cause 5: invented links

Models produce plausible URLs: /settings/admins, /help/roles. Our widget now renders a link only if its path is a real route listed in the feature registry, or an https:// URL. Anything else becomes plain text. That removes a whole class of dead ends — and blocks links injected via content the model read.

Keep it fixed: must-not-include evals

Every hallucination we catch becomes a test:

{ q: "how do I make someone an admin", mustInclude: ["Settings", "Users"], mustNotInclude: ["pencil"] }

The eval runs on every prompt, model or guide change. A "must not include" list is a memory of past failures that stops them returning silently. The method is in how to evaluate an LLM feature with a golden set.

OWASP's Top 10 for LLM applications lists misinformation — false or misleading output that appears credible — as a risk category. Its mitigations include retrieval from verified sources, automatic validation of key outputs, human oversight, transparent communication of limitations, and interfaces that discourage overreliance. The steps above are those recommendations made concrete for a help chatbot.

How users experience the fix

Grounding changes the texture of answers, and that's worth designing for:

• Shorter, more specific answers. "Open Settings → Users and, in the person's Position dropdown, pick a position with Admin access — it saves as soon as you pick. Only the owner can do this." instead of a friendly paragraph of plausible generalities.
• A visible source. Showing which guide page an answer came from ("From: Users, admins, invites & ownership") lets people check it and builds trust when it's right.
• Honest gaps. "The guide doesn't cover exporting to Excel — you can ask your admin or file a request" is a better experience than a confident invention, and it tells you what to document next.

Track the "doesn't cover" answers: they're a ranked list of documentation gaps, generated for free by real questions.

When the model is unavailable

A grounded design has a bonus: the retrieved guide page is already a good answer. When every model fails, LetRelay's assistant shows the best-matching page's steps and a link instead of a generated reply — which cannot hallucinate. See AI that degrades gracefully.

A checklist

• One verified entry per feature, with exact UI labels and roles
• A test that fails when a feature has no entry
• Retrieval measured on real questions (top-1 / top-3), with synonyms and typo tolerance
• Grounding rules: answer only from pages; say when not covered
• Scope rules: what the model can't see or claim
• Capability and pricing statements from code, not generation
• Links rendered only for real routes
• Every caught hallucination added as a must-not-include test

FAQ

Why does my chatbot invent product features?

Usually because the feature isn't documented where the model can read it, retrieval didn't find the right page, or the model wasn't told to answer only from what it was given. Fix the source and retrieval before tuning the model.

How do I ground a chatbot in my documentation?

Retrieve the relevant pages for each question, instruct the model to answer only from them and to say when they don't cover the question, and check answers against required and forbidden terms.

Can I stop a chatbot from making up links?

Yes. Only render links whose paths exist in a list of real routes, and show anything else as plain text.

How do I know hallucinations are fixed?

Add each one to an evaluation set with must-include and must-not-include checks, and run it on every change.

Sources

• OWASP — LLM09:2025 Misinformation
• Lewis et al. — Retrieval-Augmented Generation
• Liu et al. — Lost in the Middle
• Anthropic — Develop test cases