Red-Team an AI Support Agent's Demo Before Launch

The demo answers politely. A real customer asks the same thing in another language, with a typo, a refund threat, and a policy edge case. That is where the agent earns trust or loses it.

A demo shows the best paraphrase; your job is to find the worst one, then keep the harness that found it.

• Keep roughly 25% of pre-launch cases adversarial or critical; a pre-launch suite with that share exposed every issue worth fixing before launch.
• Retest the same intent across paraphrases and languages.
• Repeat each safety case until the result is stable.
• Build a 50-to-100-case set from existing work and add each production incident as a permanent regression case.
• Report verified fixes separately from contained incidents.

Start with the worst paraphrase, not the demo

Build the opening batch from the cases you fear most: hostile paraphrases, multilingual retests, repeated runs, and policy edge cases. Answering the polished version is not enough. It must survive the version a tired customer sends late at night.

Write each case as an input, an expected behavior, and a failure mode. If you cannot say what the agent should do, the case is not testable.

A passing demo is a limited signal; a red-team set should surface the failures you would not want to discover in a customer conversation.

Build an eval set that behaves like real work

Pull cases from existing work, not from imagination. Include the refund request, the angry follow-up, the typo-heavy question, the non-English message, and the case that made your support team uncomfortable. Evaluate agents with realistic scenarios instead of idealized ones.

Group cases by intent, not by wording. Refund intents can arrive as a question, a demand, a complaint, or a half-formed sentence. The agent should recognize the intent and choose the safe action.

If you are testing Zendesk's Advanced AI agent product, test it where it will actually work: messaging and email channels, plus a Chat API for custom integrations.

A set is ready only when it includes the awkward cases your support team already knows.

Run the agent until it stops being lucky

Macha's Test Run feature lets you process a real Zendesk ticket, email, or document through the configured agent in a pre-launch sandbox.

Watch the full path, including tool calls, handoffs, and the final answer. Clean final answers can hide a broken step. You want to see the agent do the work, not just say the right thing.

Macha's Test Run simulation consumes credits for each AI action, with costs ranging from 0.5 to 9 by model and GPT-5.4 Mini set at 1 by default.

Run each case repeatedly. An agent can be lucky in one run and wrong in the next. A run earns its keep when the same case produces the same failure, or the same pass, across repeats.

Score the output against the expected behavior, not against tone. Polite wrong answers are still failures. Blunt correct answers may be acceptable.

Stopping after a pass is the trap. Rerun the case after a prompt change, a model change, or a new policy. Add the broken version to the regression set when it breaks.

Measure verified fixes, not contained damage

After launch, separate verified fixes from contained incidents. Contained incidents mean the agent did something bad and you stopped the damage. Verified fixes mean the same input no longer produces the failure. Those are different scores.

Report the difference. Containment protects the customer in the moment. Verification protects the next customer from the same mistake.

Keep the regression set alive. The next release cannot quietly reintroduce a fixed failure if that failure is still in the test set.

Track the cases that were fixed, the cases that are still failing, and the cases that only look fixed because the prompt was narrowed. Measurement earns trust when a fixed incident stays fixed and a new incident becomes a new test.

When the next incident lands, you already have the case, the expected behavior, and the regression test waiting for it.