The 1-Second Support Reply: Anatomy of a Production Chatbot

Retrieval, escalation rules and tone control — the full stack behind a bot customers actually like.

A support bot that customers tolerate and a support bot that customers like are separated by about four engineering decisions, none of which are about the model.

The first is retrieval scope. Bots that search the entire knowledge base answer confidently about things they should have escalated. We scope retrieval by customer tier, product entitlement and account state, so the bot literally cannot see documentation that does not apply.

The second is the escalation rule set. Sentiment is a poor trigger; intent is a good one. Billing disputes, cancellation intent and anything touching a legal term escalate immediately regardless of how politely they were phrased.

The third is tone control, which is mostly about length. Support answers should be shorter than the question. We cap response length hard and let the customer ask for more, which reads as competence rather than evasion.

The fourth is latency budget. Under a second feels like a system; over three seconds feels like a queue. That constraint drives the caching and pre-embedding work that makes up most of the actual build.