failure
The Support Bot Gave a Wrong Answer — When Must It Hand Off to a Human?
Treat the wrong answer as data, not as something to patch in place: capture the full exchange, work out whether the approved knowledge was missing, ambiguous, or ignored, and fix the boundary so this class of question routes to a person. The line is simple to state: anything involving money already spent, promises about delivery, disputes, or an upset customer belongs to a person. Status note, as everywhere on this site: you can open these pages and sign in to the workspace today, we have not connected external platform accounts for you, and the product has not formally launched.
What went wrong, mechanically
Confident-but-wrong answers come from three places. The knowledge base lacked the fact, so the bot filled the gap plausibly. The approved material contained two conflicting statements, and the bot picked one without noticing. Or the question was never an information question: a refund request, a complaint, a threat to dispute the charge, where the correct move was never an answer but a person.
Missing facts get added to approved material; conflicts get resolved by whoever owns the policy. But a refund request answered fluently means the handoff rules drew the circle too small, and no amount of extra knowledge makes answering it safe.
What to bring
The full conversation transcript, not just the offending reply, plus the exact approved documents that were connected when it happened. If this followed a bot update, record what changed; regressions arrive with edits, not spontaneously. Bring your current handoff rules in writing too; the fix will be an edit to them.
Step 1: Preserve the exchange and answer the customer yourself
Export or screenshot the whole conversation, then take over the chat personally: acknowledge the wrong answer, give the correct one, and say plainly that the assistant made a mistake. A customer who watches a merchant own the error stays; a customer who has to argue with a bot twice leaves.
Step 2: Classify the failure
Decide which of the three it was: the fact was missing from approved knowledge, the approved material contradicted itself, or the topic should never have been answered automatically. Check against the actual documents, not your memory of them. Misclassifying here produces the classic loop where knowledge gets added forever while refund requests keep getting answered.
Step 3: Redraw the handoff line around people and money
Write the permanent rule: payments and refunds, disputes and chargeback threats, delivery promises beyond what the approved material states, angry or distressed messages, and any request to override a stated policy all go to a person immediately. These are categories, not keywords; a customer asking 'can I just send cash instead' is a payments conversation even though the word refund never appeared.
Step 4: Add uncertainty-based triggers, not just topic triggers
Topic rules catch known dangers, but the next wrong answer will come from an unknown one. Add a second layer: whenever the bot cannot point to a specific approved source for its claim, or the question sits near the edge of the approved material, it hands off instead of guessing. An unanswered question costs minutes; a confident invention can cost the account.
Step 5: Replay the failure and a regression set
Re-run the exact failing conversation in the sandbox and confirm it now ends in a handoff. Then replay earlier scenarios to make sure the tightened rules did not start refusing ordinary questions; handoff policies fixed in one direction often break the other silently.
How you know it is actually fixed
Three checks, each observable: the original failing conversation now terminates in a handoff with the transcript intact; your normal-question script still passes at the same rate as before; and the new rules are written down somewhere a teammate could apply them without you. A fix that lives only as 'we told the bot to be careful' will resurface under a different subject line.
Limits worth stating plainly
Payments, refunds, and sensitive support cases must be handed to a person. Telegram transport and webhook access depend on a separately configured test bot. Neither line bends under pressure: a fluent apology generated by a bot does not make a refund safe, and a handoff rule cannot fire if the channel plumbing was never configured.
Handoff rules reduce exposure; they do not eliminate wrong answers inside the allowed scope. Approved material itself can be stale, so review the knowledge base on a schedule and read every escalation as a document that aged out.
Where Chimeward fits
Chimeward keeps the pieces of this investigation in one place: the conversation history shows where the sandbox or monitored chat lost context, the handoff board records which rules fired, and editing the flow updates the approved answers and refusal boundaries together. Drafting a support flow is the entry action; revisiting it after a failure should feel like editing a document, not redeploying software.
It does not decide which topics are sensitive; that judgment stays with the merchant. It removes the gap between deciding and enforcing: the rule change and the replay happen in the workspace where the failure is recorded.
FAQ
Questions this guide is for
The answer sounded right and cited plausible details. Is that still a knowledge problem?
Yes — plausible detail absent from approved material is invention. Check facts against documents and tighten the sourcing rule where they are missing.
Won't aggressive handoff rules frustrate customers?
Less than wrong answers do. A contextual handoff reads as service; a wrong refund policy reads as deception. Err toward handing off for money and emotion.
Who should own the decision about what the bot may never answer?
Whoever would handle the refund, dispute, or apology owns that answer's policy; rules should name them, not 'the team.'
Where do order drafts fit if the chat becomes a purchase?
Order drafts sit in the same supervised flow beside channels, knowledge, and handoffs; producing one is a tested check. The money conversation routes to a person.
Start in the workspace
Fix the boundary and replay the conversation
Sign in or create an account and you return to the Chimeward conversation on chimeward.kuca.app. Paste the failed exchange, adjust the handoff rules in the flow, and replay it in the sandbox until the chat ends with a person, not with another confident guess.