20-Task Test for a Cheap Router Like Fugu Max

Run the 20-task batch before you move traffic

Your agent pipeline is already paying for tokens, and a cheaper router can look like a free upgrade until the bill shows the router's internal work. On OpenRouter, Fugu Max's internal routing tokens are billed as ordinary input or output tokens. Keep this test: run the same 20 tasks through your current model and the router, measure output-token spend, wall time, and failure checks, then decide by async versus interactive use.

Fugu Max is the router to test. Sakana AI launched Fugu Max v1.0 and Fugu Ultra v2.0 on September 11, 2026. It lists at $2.00 per million input tokens and $6.00 per million output tokens, and Sakana says that puts it 40-60% below Sonnet 5 and GPT-5.6 Terra.

Architecture changes the bill. Fugu Max is a learned multi-agent orchestration system: it routes tasks across a fixed pool of open-weight and specialized models, including the NVIDIA Nemotron family under a collaboration with NVIDIA. When Fugu Max fans one request out to three or four models, the request may consume substantially more tokens than a direct single-model call, so the $6.00 per million output rate alone does not guarantee a low per-answer bill. For Fugu Max, the bill grows with total token usage, and orchestration increases total token usage, so tokens per answer, not the sticker rate, is the relevant budgeting measure.

Build the batch before the first API call

• Pick 20 tasks from recent real agent work. The finished state is a spreadsheet with task name, input prompt, expected output shape, pass/fail check, and the use case where it runs. Keep the prompts frozen. When a task needs a file, a tool call, or a second step, note that in the sheet so the second run cannot drift.
• Make the batch representative of the workload you would actually move. You are done when the mix includes short lookups, multi-step tool calls, long-form synthesis, and a task that usually fails. If your production traffic is mostly batch summaries, do not test a trivial set of trivia questions.
• Record the current model's baseline. Finish with output-token count, wall time, and pass/fail for each task. Keep the same timeout, retry limit, and temperature. When the baseline is noisy, run it twice and keep the median.
• Run the same 20 tasks through the router without changing prompts, retries, or timeout settings. The second spreadsheet should have the same columns. Do not clean up the prompts for the router. The test is the switch, so the prompt set stays fixed.
• Measure output-token spend and total token spend, then wall time. On OpenRouter, Fugu Max's median response time was about 5.4 seconds. Finally, run the same failure checks. The pass/fail column should match the baseline task by task.

Judge wall time by who is waiting

For async work, the pipeline sets the bar. Acceptable wall time is the time it can absorb without breaking a downstream deadline; an overnight batch can use a slower router if it finishes before the next job starts.

For interactive work, the person sets the bar: the router must stay within the time a person can wait without abandoning the task. When it makes the user wait noticeably longer than the current model, the cost saving may not be worth the delay.

Put a wall-time column next to the output-token column, so the decision is not a gut feeling about speed.

Make the call from the two spreadsheets

Treat the router as a candidate for a use case, not a replacement for every model in your stack. A nightly pipeline can qualify even when the router is slower than a direct model, because wall time is a scheduling problem, not an experience problem. Interactive work has a higher bar: a human is waiting, and a few extra seconds can turn a cost saving into a workflow tax.

Use the two spreadsheets to decide: if the router shows lower output-token spend, acceptable wall time, and no new failures, switch the async use case first. Keep the current model for interactive work when it fails any of those checks, and revisit only when the batch changes.