AI in Customer Service: What Actually Works (and What Doesn't)
"Just hook ChatGPT up to customer service and you are done." That is how a lot of people picture AI in support. Reality is more complicated.
Where we started
Across nano., mate and MUSTAX we get somewhere around 150-200 customer enquiries a week. That is too much to answer on the side, and too little for a full-time hire.
Two years ago we started using AI. Here is our verdict. If you want to rebuild the complete setup step by step afterwards: our guide customer service automation walks you through all of it.
What actually works
1. Triage and categorisation
The AI reads every incoming email and categorises it: return, shipping question, product question, complaint, spam. That alone saves us 30 minutes a day. How this pre-sorting works technically is described in pre-sorting support emails with AI.
Success rate: the categorisation is reliable enough that we stopped checking it daily.
2. Standard answers to standard questions
"Where is my package?" That question makes up around 40 % of all our enquiries. The AI pulls the tracking info automatically and replies. We described the workflow for it in detail in automating WISMO enquiries.
Result: response time from 24h to 3 minutes. 24/7.
3. Drafted replies
For more complex enquiries the AI writes a draft reply that we only check and send. Writing goes considerably faster, because all that is left is the check.
What does NOT work
1. Emotional situations
Customer is furious because their birthday gift never arrived? That needs a human. The AI sounds hollow in moments like that and often makes it worse.
Our rule: negative sentiment → escalate to a human immediately.
2. Complex edge cases
"I bought the product 8 months ago, then I moved, now it does not work any more and I cannot find the invoice..." Too many variables. Too much context required.
3. In-depth product advice
"Which toothbrush is better for sensitive gums?" AI can help here, but the nuance is missing. We let humans handle this, because it builds trust.
Our setup (concretely)
- Gmail + n8n + Claude API for triage
- Template library with a collection of standard replies
- Escalation rules: negative sentiment, order value >200€, 2+ interactions → human
- Feedback loop: every AI reply is logged and reviewed monthly
One note: what we use here is not a chatbot in the classic sense. We explain the difference between chatbot, workflow and AI agent in this article.
The numbers
| Metric | Before | After |
|---|---|---|
| Response time (average) | 18h | a good 2 hours |
| Handling time per enquiry | 8 min | 3 min |
| Enquiries humans see | 100% | 35% |
Around 65 % of enquiries get answered completely without us. Answered correctly. Satisfaction has gone up, and not a single complaint about slow replies since.
How to start without getting stuck
The most common mistake: wanting everything at once. Our advice, from experience:
1. Triage only, at first. Let the AI categorise for 4 weeks and answer nothing. You see how well it understands your enquiries without a single customer ever getting a wrong answer.
2. Then automate one single question. Take the most frequent one, for us that was WISMO. One question, one data lookup (tracking), one clear stop condition.
3. Drafts before autopilot. Let the AI write drafts that you send. Only once you have gone weeks with barely any corrections should it send on its own.
4. Define escalation rules before you start. Not afterwards. Sentiment, order value, number of interactions: the limits have to be in place before the first automatic email goes out.
That is how we built trust step by step over 2 years. The 65 % were never the goal, they were the outcome.
My take
AI in customer service works as a filter. The filter holds back the repetitive enquiries so humans have time for what counts: solving real problems, building relationships.
Anyone selling AI as a "replacement for staff" has missed the point. It is a tool. A damn good one, used properly.
Want more detail?
Write to me on LinkedIn, at damian@flowhouse.ai or through the contact form. Happy to show you our exact setup.