DarwinX: fix the agent harness, not the model

Salesforce AI Research published DarwinX, and the result is the one we keep arguing for in client calls: they took an agent from 43.5% to 93.0% on a browser-task benchmark without touching a single model weight. The paper describes evolving the harness — prompts, tools, skills, control flow — as population-based selection with the model frozen. If you are about to switch model vendors because your agent harness underperforms, read the numbers first. The scaffolding is probably where your points are.

What actually happened

DarwinX generates competing harness variants and promotes only the ones that survive three gates. Preserve-and-extend: a change advances only if it improves performance while keeping regressions inside a bounded tolerance. Separated exploration and confirmation: a promising edit clears cheap screening, then must survive high-fidelity revalidation before it influences later generations. Cross-lineage merging: complementary capabilities from different branches recombine, then face the preservation gates again.

The reported results, all with frozen weights:

• WebArena-Infinity (1,260 tasks): 43.5% to 93.0%, on GPT-5.5, evolved over 300 synthetic intents
• Terminal-Bench 2.1: 75.5% to 83.2% on GPT-5.5; 84.7% on a stronger base model
• TerminalWorld: 61% to 68.3% on Opus 4.8, trained on 94 tasks and tested on a held-out 41
• SWE-bench Verified: 80.8% to 84.2%, with the Terminal-Bench harness transferred unchanged

The agent is Salesforce's proprietary Monet, but the Beagle evaluation infrastructure is on GitHub under Apache 2.0.

Read the caveats, because the authors published them. The method needs reliable verifiers, and as VentureBeat notes, production tasks rarely arrive with one attached. The TerminalWorld margin is a one-task difference — suggestive, not decisive. And the headline 93% is the in-domain number on the benchmark it evolved against; the transfer results are the smaller, more honest ones.

Why it matters for your business

The 80.8% to 84.2% SWE-bench transfer is more useful to you than the 93%. It says harness improvements survive a change of task. That is the claim worth acting on.

Stop treating model choice as the lever. The cheapest experiment in your agent stack is not a vendor swap — it is rewriting the tool descriptions, trimming the context, and adding a verification step. Salesforce found gains from specialized skills like result-definition checks and file verification, not from turning up inference across the board.

Your verifier is the real investment. Everything here runs on being able to score an attempt automatically. Most small teams cannot, so they tune agents by vibes. Build the checker first — a test suite, a schema validator, a diff against expected output — and harness work becomes measurable instead of superstitious.

Keep the harness portable. If your prompts, tools and control flow live inside one vendor's agent builder, none of this compounds. Points you win are points you lose on migration. Keep the harness in your repo and let the model be the swappable part.

Budget evaluation compute deliberately. DarwinX costs more eval compute than hand-editing a prompt, and the authors say it is not worth it for simple, stable workflows. Apply it where a workflow is complex, repeated, and expensive to get wrong.

Sources: arXiv: DarwinX, VentureBeat.