Test an Open-Weight Model on Your In-Flight Coding Task

The in-flight task is the benchmark

You have a Claude Code or Codex task in flight, and you are weighing a subscription switch. The cleanest test is the task you are already running: a half-built feature, a failing test, or a refactor with a known end state. That task becomes the benchmark. It has context, edge cases, and a cost you can compare.

Cline Desktop is in beta on macOS and Windows, and Cline founder Saoud Rizwan launched the standalone desktop version on September 14. The useful part for this test is the import path, not the interface.

Do not start from a blank chat. A blank chat measures the model's enthusiasm, not its ability to inherit your in-flight state. Import the session, then change the engine underneath it.

The test, in order:

• Snapshot the original output, tests, plan and act steps, and cost.
• Import the same in-progress task into an import-capable agent.
• Rerun it on a local or open-weight model.
• Diff code, test results, plan and act behavior, and per-task cost.
• Log import failures and decide.

Import the session, then swap the model

Snapshot the original task before touching anything. Commit the working tree, save the test output, export the transcript, and note the cost so far. Keep a git tag, a test report, a session file, and a cost note that you can point to later.

Import the same in-progress task into an import-capable agent. Cline Desktop can import an existing conversation from Claude Code, Codex, or another supported agent, then resume work. The transcript should be visible, the file context should be present, and the next instruction should be accepted without re-explaining the whole problem.

Set the target model before rerunning. After an import, Cline Desktop continues the session with the provider and model selected in Cline rather than the original tool's. You can connect an existing API key to over 50 providers, use Cline's own provider, ClinePass, or run local models. Cline set the ClinePass subscription at $9.99 per month when it was introduced in June. The model name should be visible, the provider should be authenticated, and no hidden fallback to the old model should be possible.

The trap is assuming import means continuity. It does not. The transcript may arrive, but the plan, tool calls, file assumptions, and cost state may not. Log import failures, then check the imported history, the file list, and the provider authentication before you ask the model to continue. Cline's GitHub release page showed version 0.0.27 on September 14, following several September builds; its recent notes describe corrections spanning imported history, scheduled sessions, provider sign-in, Windows-related behavior, and web search.

Diff the work, not the vibes

Rerun the task with the same prompt, files, test suite, and definition of done. Keep any original plan, act step, or failing test; if a test is failing, keep it failing until the model fixes it. The second transcript should start from the imported state and end with a new set of changes.

Compare the code diff against the original diff. Look for missing edits, extra edits, and edits that pass tests but change behavior in a way the original did not. Run the same tests. If the original task had no tests, add the smallest test that proves the feature works, then run it on both outputs. Keep a side-by-side diff, a test result, and a note on which output is closer to the intended behavior.

Compare the plan and act behavior. Note whether the model asked for clarification or guessed, ran the right commands or invented them, and stopped when the task was done or kept editing. These are the signs that matter when you are deciding whether to trust an open-weight model with a live task.

Compare cost per task, not cost per token. A cheap model that needs extra retries is not cheap. A local model that finishes without retries may be the better deal, even if the hardware cost is real. Log the time, the number of steps, the number of failed tests, and the money spent. End with a verdict: cheaper, same, or more expensive, with the reason attached.

Use published benchmarks as a sanity check, not a substitute. Cline's Terminal-Bench 2.0 pass@1 scores show Kimi K3 at 82.02% under Cline's harness, 71.9% under Hermes, and 76.4% under OpenCode. Those numbers tell you the harness matters. They do not tell you whether the model can finish your half-built feature without breaking the build.

Make the switch only when the imported task passes your own diff. If it fails, keep the original agent for that task and test the open-weight model on a smaller slice. If it passes, keep the snapshot, the diff, and the cost note. That record lets you switch coding agents without losing the session, and it gives you a reason for the next task, not just a hope.