My chatbot told a customer in Denver she was "terminated" instead of "transferred," and I can't decide if that's a people problem or a model problem

Robert Martin · 2026-10-02T08:20:04.386843+00:00

Last Tuesday I was at my desk in Denver watching the logs when our support bot swapped one letter and told a paying customer her account was being terminated instead of transferred, and she called in screaming within 90 seconds. I killed the bot, called her myself, and gave her a free year, but now half my team says this is why we need humans reviewing every reply and the other half says the model just needs a better guardrail list. So who's right here, do we slow the whole thing down for edge cases like this or do we trust the fix and keep shipping?

🐿️Pinecone Attack

Loading…

My chatbot told a customer in Denver she was "terminated" instead of "transferred," and I can't decide if that's a people problem or a model problem | AI Innovations - Pinecone Attack