Score AI coding agents with a Three.js scene

A small scene exposes failure faster

You are deciding whether an AI coding agent can take a real frontend task. Give it a constrained Three.js scene that opens in a browser, shows required behavior, and can be merged after a short review. The reference prompt specified a one-page Three.js sci-fi hangar scene featuring hovering drones, warning lights, runway strips, and fog planes. A leaderboard rank shows how a model did on someone else's benchmark; a shipped file shows whether your team can merge it.

A Three.js sci-fi hangar prompt was run across varied model and harness combinations in /goal mode. It also called for a toggle to switch drone formations, a cinematic camera path, and a standalone HTML file with the JavaScript embedded inline. Keep the task small enough that a senior engineer can review it in one sitting.

The same prompt makes the comparison fair

Run the prompt through your shortlist in one harness mode. Use one input file, one browser, and one acceptance checklist across runs. In those tests, every GLM run carried the label GLM 5.3 Flash Max. Record that label exactly, and note the prompt version, date, browser version, and person who ran the test before scoring.

The DSH adapter does not provide a separate reasoning token count, so treat token cost as incomplete when that field is absent. The run table should carry model label, harness, output file, token count, wall-clock time, and dollar cost. If a run fails, save the output file and the console log before retrying. A repeatable test, not an isolated result, is the finish.

Do not let a pretty demo hide a missing toggle; the scorecard tracks shipped behavior, not screenshots. If the harness can fan out subagents in isolated worktrees, record whether your task actually needs that. Meta's Muse Code beta can run subagents in parallel isolated worktrees; Meta reported a test where it built six game features at once without collisions. For this scene, parallelism is usually overhead, not signal.

Score shipped behavior, not demo polish

Use the scorecard as the shipping gate. Keep the score compact: pass/fail for each gate, plus a short note on the breakage.

• Build pass: the file opens without console errors that block the scene.
• Visual pass: the required scene elements are present and readable.
• Runtime pass: the required interactions actually run.
• Required features: count only what a human can trigger.
• Edit count: the number of human changes needed before the file is mergeable.
• Cost: include prompt, retries, and local fixes.
• Time: include prompt, retries, and local fixes.
• Named failure modes: label each failure.

A Claude Code study of 567 pull requests across 157 open-source projects found that 83.8% eventually merged, but only 54.9% merged without extra changes, leaving 45.1% needing human revision. Expect human revision; the scorecard should measure how much.

Throughput can rise while review burden shifts. In the enterprise study, as AI-written pull requests grew, the share that received at least one human review dropped from 89% to 68%, automated AI review coverage rose from about 19% to 84%, and reviewer workload roughly doubled. After the enterprise company's 2x mandate, AI-written pull requests needed roughly 20% more merge time after the first human review, and 22% more total time.

Measure throughput separately from quality. A four-month Microsoft study of command-line AI coding agents found that developers who used the tools merged 24.0% more pull requests per engineer per day. By April 2026, the enterprise study's per-developer throughput was 2.09 times the pre-mandate baseline, moving from 21.2 to 44.3 merged pull requests for each active developer. Those numbers are useful for adoption, not for deciding whether an agent can ship your frontend task.

A failure ledger supports the next decision

Keep a failure ledger with short labels: missing toggle, broken camera, wrong file shape, unexplained token accounting, and merge-blocking errors. The ledger is ready when it says which model and harness shipped, what it cost, how many edits it needed, and which failure modes repeated.

Z.ai's GLM-5.2 was second on Code Arena in mid-June 2026, behind only Anthropic's Claude Fable 5. A public coding leaderboard position still does not remove the need for a shipping test. Promote an agent to real work only when it passes the scene repeatedly and its failure modes are boring. Boring means a missing label, not a missing required interaction. Use the ledger to compare vendors, not to rank models. If the same failure appears across different harnesses, treat it as a model weakness, not a one-off.