Tue 27 Oct 2026 EN ES
Tools

How to decide whether SWE-2 is worth switching to before you commit

Run a baseline before committing, because the benchmark row that matters depends on your ticket queue.

Illustration: How to decide whether SWE-2 is worth switching to before you commit

The free month is the trial period, not the verdict. SWE-2, the coding agent from the Devin maker Cognition, is worth it only if your work is mostly ticket-queue coding. If your day is long hostile terminal sessions, the frontier model still owns the deciding row.

Cognition released SWE-2 on September 10, 2026. The reported scores are 50.0% on FrontierCode 1.1 Main, 73.0% on DeepSWE 1.1, 92.8% on Terminal-Bench 2.1, and 27.3% on Terminal-Bench 4, the later terminal row. The Cognition SWE-2 cost claim is 64% cheaper than Fable 5.1, a rival model, at the FrontierCode 1.1 Main comparison point. The reported median first real code edit came after 18 steps on SWE-2's medium setting, down from 48 steps on SWE-1.7. No standalone API, model card, or context window had been published at launch.

The benchmark table is a map, not a receipt

Read the rows as work types, not as a single ranking. FrontierCode 1.1 Main and DeepSWE 1.1 are the rows that matter when your work is bounded tickets, pull requests, and small fixes. The terminal rows split: the earlier one looks strong on paper, but the later one punishes agents when the shell turns hostile. The last row is the number to weight if you spend your day in migrations, broken build systems, and long-running jobs.

The cost claim is useful, but it is tied to a comparison point. A cheap coding agent can win on a bounded ticket and still lose on a long terminal session. The median first-edit step count tells you how fast the agent gets to work, but it does not tell you whether the work survives review.

Before you trust any row, ask what it measures. A bounded-ticket benchmark rewards quick, clean edits. A terminal benchmark rewards persistence, error recovery, and the ability to keep a long session alive. Use your queue to decide which row gets the most weight.

The trial is where the switch gets tested

Do not let the trial drift into general usage. Set a narrow question before starting: which work type should this agent own? Then run the same kind of work you already do, not a polished demo. The baseline should be boring. Keep the same repo, ticket shape, and review bar. If the test set is too clean, the agent will look better than it will in production.

  1. Log your current ticket mix for a baseline period. Done means a simple count: bounded tickets, long terminal sessions, and tasks that need human review.
  2. Record the baseline before you switch. You are done when you have a sheet with task name, median first-edit steps, review burden, and cost per completed task.
  3. Compare the benchmark rows to your actual mix. Finish with a short verdict. For bounded tickets, the FrontierCode and DeepSWE rows carry the weight. For hostile terminal work, the Terminal-Bench 4 row carries the weight.
  4. Price a real task, not a demo. Done means a number you can defend: cost per real task after retries, failed runs, and human cleanup.
  5. Decide before the trial ends. You are done when you have a written note: keep, drop, or keep only for a narrow work type.

The trap is letting a single benchmark row do the deciding. A strong row can hide a weak work type. If your queue is mostly small tickets, the agent may feel fast and cheap. If your queue is mostly long terminal sessions, the same agent can feel slow, brittle, and expensive after cleanup.

The switch should survive your actual queue

Keep the frontier model for the row that matters if your work is hostile terminal sessions. Use SWE-2 for the work type where the reported speed and cost line up with your actual tasks. The missing documentation means you should not plan a custom pipeline around it until those details exist.

Write the decision in plain language. If you keep it, name the work type. If you drop it, save the baseline sheet. If the result is mixed, set a review date and a cost ceiling. Make the trial force a written decision before it ends.

Advertisement