Real-Ticket Checklist for AI Support Agents

You are choosing an AI support agent for a queue full of refunds, broken integrations, and customers who already tried the docs. A demo will make the work look simple; a live ticket will not. Run a 200-ticket shadow test before you buy: let the agent work tickets your team would normally handle, without sending the draft to the customer. The first useful number is how often the draft is close enough to send.

Real tickets expose the draft gap

The toolsradar evaluation covered eight support platforms over about three weeks of real support-queue testing. In that evaluation, Zendesk's agent copilot yielded usable draft replies about 60% to 70% of the time. That gap is the point: a draft that needs a full rewrite costs agent time, while a draft that needs a tone fix is a different state. Your test should separate those states before a vendor calls either a resolution.

Run the 200-ticket checklist before the contract

Keep the list below as a pass/fail test, not a ranking tool. Use tickets your team already knows, so a wrong answer is obvious. Record each result as send as is, light edit, rewrite, wrong, or cannot act. Aim for a failure that would hurt customers, not a tidy spreadsheet.

• Usable-draft rate: Send recent tickets through the copilot and count how many drafts need only light edits before a human would send them.
• Resolution-depth filter: Choose tickets that require account lookup, refund, or system change, then check whether the agent can complete the action or only explain it.
• Stale-KB failure check: Ask about a recently changed policy, a deprecated product, or a known edge case, and see if the answer matches current documentation.
• Setup-time verification: Time how long it takes to connect the help center, ticket source, and a live workflow.
• All-in cost per agent: Add base platform, AI add-on, seat changes, and integration fees before comparing vendor quotes.
• Per-resolution cost: Model the monthly ticket volume against per-resolution pricing and see where the bill rises as volume grows.

Depth and cost decide the winner

Draft quality is the first check, but resolution depth is where support teams gain or lose time. Score resolution depth as complete action, partial action, or explanation only. A bot that answers a password reset is not the same as a tool that can update a subscription, pull an order, and finish the ticket. Gartner's 2025 forecast, attributed to analyst Uma Challa, projects that about 40% of enterprise support tickets will need cross-system actions by late 2026.

If your queue is mostly simple questions, that gap may not matter. If it is full of billing, access, and account work, it becomes the deciding factor. Force the agent to do the work, not narrate it. If it can only say it would check the billing system, the resolution is still human. If it can update the record and leave a clean audit trail, the tool is doing something different.

Forrester's 2025 data associates Layer 3+ deployments, a deeper support tier, with a 63% reduction in average handle time compared with Layer 1–2 deployments. The warning is against buying only for chat: the value is in account changes, order lookups, permission fixes, and the work that keeps tickets open for days.

Pricing is where the comparison gets uncomfortable. Zendesk's Advanced AI add-on costs about $50 per agent per month above Suite Professional, resulting in an all-in price near $165 per agent per month. Intercom's Fin carries a per-resolution charge of $0.99.

The per-resolution model changes the math. A vendor can quote a low price and still produce a high bill if its definition of resolution is broad. Ask how a ticket is counted when the customer replies, when the agent escalates, or when the system closes automatically. Get the answer in writing before the pilot.

Per-agent pricing is easier to forecast, but it can hide add-ons, seats, and integrations. Build the model from the invoice you expect, not the price card.

Setup speed matters because it shortens the time between decision and evidence. Record the clock from first connection to first live ticket. If the vendor needs a long integration project before a live ticket, the test becomes a procurement exercise. If it can connect to your help center and ticket source quickly, you can judge output while the cost is still low. The toolsradar team had Intercom's Fin operating against a real help center in roughly 90 minutes.

Score the next 200 tickets with the same checklist and compare the results.