Simulating clients who remember

Any waiter can carry a plate to a table. That is not what makes a restaurant good. What makes it good is what happens when you send something back.

We wanted to test whether our product actually works, end to end, without a human clicking through it. So we built fake customers. The first version was useless, and the reason why is the interesting part.

Why the first version was useless

It worked exactly as designed. It signed up, ordered work, received the result, approved it, and checked the invoice appeared. Everything passed.

But it approved because the script said approve. It complained because the script said complain. It never actually wanted anything, so it could never be let down.

It could prove the buttons worked. The buttons were never the risk.

The moment that decides whether you get paid again

Real service work is not "deliver, approve, done." Month two is almost entirely fixing things.

You send the first version. The client comes back with one specific complaint — the VAT figure is for the wrong quarter. You send a second version. And now the client looks at it while remembering the first one, and asks the only question that actually matters:

Did they fix the thing I said?

A beautiful second version that ignores your complaint is worse than a plain one that fixes it. The first one is not listening. And "they don't listen" is the reason people leave that never shows up in a complaint form.

To test that, your fake customer has to remember. That is the whole thing.

What our fake customers carry around

• every version they have been sent
• every complaint they made, and how many times it has been ignored
• how often they have been asked to go find something — people get tired of homework
• how much patience is left

And they can do the two things most tests forget about: go quiet and leave.

Real clients rarely send an angry final email. They just stop replying, and you find out at renewal. A test that only knows about explicit rejection never sees that coming — which is precisely the thing it should have been built to catch.

Let the AI have an opinion, not the verdict

This is the decision that makes the difference between a useful test and an expensive story generator.

"Did this version fix my complaint?" needs actual reading and judgement. So an AI answers it — and it gets to see what a real customer sees: the page, the document, the questions stacked above the work.

Everything else is arithmetic. How much patience being ignored costs versus being late. When somebody gives up. Whether this is a new complaint or the same one worded differently. That lives in plain rules that can be tested on their own.

If the AI decides both what it thinks and what that means, every run produces a believable story and no run produces an answer. That is how AI simulations end up beautiful and worthless.

Looking at the page, not the text

A customer does not receive a paragraph. They get an email, click a link, and look at a page. And the things that lose customers live in that gap.

A document that opens as a grey wall with the number they wanted on page three. An "Approve" button next to work that is obviously not finished. Three questions stacked above the deliverable, so the first thing they feel is homework rather than progress.

None of that is in the words. All of it is in the picture.

Small things that mattered

• One complaint, not a list. Real customers write one sentence and press send. Nobody writes a nine-point review, and a list gives you nothing to fix first.
• Put the old complaint front and center. Mention it in passing and you get an answer in passing — and that question was the entire point.
• A confused answer is not approval. The tempting shortcut is to treat a garbled response as "fine, carry on" so the test does not fail. That makes your test lie in the flattering direction, exactly when things were confusing — which is when you needed it most.
• Use several different customers. The same report can read perfectly to a marketing agency and be useless to a dentist. One fake customer never shows you that.

The number it gives us

Alongside all this we track something simple: a real engagement passes through fifteen stages, from first contact to renewal. How far has any real customer ever actually got?

Not "do the tests pass." How far has this thing ever taken somebody.

That number has been useful every single time we have looked at it, which is more than we can say for the test count.