Run One Real Desktop Task on Holo4 27B and Measure What 61.7% Costs

You have a desktop task that a paid computer-use model already does, and you want to know whether a local model can do it for less. On OSWorld 2.0, Holo4 27B reached 61.7%, Opus 5.5 reached 81.8%, and Holo4 35B-A3B reached 30.9%. The public score is useful, but it is not your invoice.

H Company published the Holo4 weights on the Hugging Face blog on September 28, 2026. The family includes a 27B dense model and a 35B-A3B mixture-of-experts model. You can download BF16, FP8, NVFP4, or 4-bit GGUF weights. H Company rebuilt Holo4's harness to add reliable long-sequence memory and a shell on the desktop machine.

Test the 27B dense model first. The other model is a different decision.

The benchmark gap is the entry fee

The public result says Holo4 27B is competitive. It does not say your task will pass. H Company's Agentic Task Factory had generated roughly 10,000 tasks spanning web applications, MCP servers, and desktop environments. The breadth matters because a real desktop task is rarely a clean one-area job. If your task touches more than one of those areas, make it the local test.

Your own task is the real benchmark

Run the protocol before you change your automation stack. The output is a decision, not a demo. Your pass condition is the part of the test you control. Do not let the agent improvise the pass condition. A saved draft is not success if the task requires a final file.

• Pick a representative desktop task you would otherwise send to your paid baseline. Done is a clear start state, a required end state, and a deadline you would accept in production.
• Define pass/fail before the run. Write the rule: the task passes only when the file exists, the form submits, the window closes, or the expected value appears.
• Run Holo4 27B on your local machine. Choose a precision that fits your memory, and use the harness. Record the outcome as a completion, a failure, or a timeout, with the machine state saved.
• Log success, agent calls, tokens, the cost you would have paid, and the failure mode. Put the row in a table, not a paragraph of impressions.
• Compare the row to your current paid baseline. If your paid baseline is Opus 5.5, use the same task and the same pass condition. End with a clear answer: keep paying, self-host, or reject the task.

Keep the environment stable between attempts. Keep the account, files, and app versions unchanged. If the task needs a fresh state, say so in the pass/fail rule. If it does not, do not clean the machine just because the first attempt looked wrong.

The run should produce a decision

In a published Pac-Man-style Godot example, Holo4 27B used 68 agent calls and 2.4 million tokens, while Qwen3.8 27B used 197 calls and 11.4 million tokens. Use that as a sanity check for your own logging. If your task needs many calls, the token count will move the cost conversation faster than the success rate.

Treat that example as a calibration, not as your task. It shows that call count and token count can diverge from raw success. A model can finish a game and still burn tokens on a task you care about.

Then read the row like a lab note. Did Holo4 27B pass? How many calls did it take? How many tokens did it burn? What failure mode appeared? Did the shell help, or did the long-sequence memory fail?

Keep the failure mode separate from the score. A wrong click, a missed file, a stale window, and a bad tool call are different problems. They point to different fixes.

If the task passes and the logged cost is lower than your paid baseline, self-hosting has a case. If it fails, you now know the failure mode before you spent the month. If it passes but the token count is bad, keep the paid model for that task and test another task.