What an AI support agent does that a chatbot can't, and how to keep one in check

What is an AI support agent?#

It's the step after a chatbot. A chatbot reads your help content and replies. An AI support agent does that too, but it's also connected to your store or helpdesk, so it can check the customer's actual order and change it.

The word agent is doing real work there. It means the software does things. A chatbot only writes text. Lots of vendors use the term loosely, so ask what it can actually change before you believe the label.

AI support agent vs chatbot#

Take a customer who writes "Can I change my delivery address? I've moved."

A chatbot finds your shipping policy and pastes the paragraph about address changes. The customer still has to email you, and you've got the ticket anyway.

An AI support agent looks up the order, sees it hasn't shipped, updates the address and tells the customer it's done. If the order has already shipped, it says so and offers what your rules allow. The chatbot answered the question. The agent fixed the problem. Our customer support bot page covers what the simpler kind is still good for.

What keeps an AI support agent in check#

Software that can refund orders needs limits, and they start with plain-language rules. You write down what it's allowed to do and when, like refund unshipped orders under €100, send anything bigger to a person, and it follows them.

On top of those rules, Hay.chat scores every reply before it goes out. At 80% or more it sends. Between 50 and 80% it pulls more of your docs and tries again, and under 50% it hands the conversation to your team. Both thresholds are yours to change.

And when you want a person in the loop, turn on message approval. Every reply then waits for someone on your team to approve, edit or block it. It's worth leaving on for the first few weeks, until you trust what you're reading. It holds replies, not actions, so the rules still matter.

How to measure an AI support agent#

Don't judge it by how many conversations it kept away from your team. That's containment, and it flatters anything that replies confidently. Count the conversations where the problem was solved and the customer didn't come back about it within a couple of days. Our containment rate page explains the gap.

Hay.chat as an AI support agent#

Hay.chat connects to Shopify and WooCommerce and works from your docs and playbooks. It looks orders up by number or email and updates addresses on orders that haven't shipped. When your playbook calls a case clear-cut, it cancels or refunds. The full list is on the actions page.

When it's not confident, or the customer asks for a person, the conversation lands in your queue with everything attached. After that, the dashboard's resolution rate and escalations show you which docs and playbooks need work.