The week’s pattern is that the scaffolding around the model is becoming the thing people measure. A new benchmark scores agent harnesses separately from the models inside them, Anthropic described how it splits an agent’s control loop from the sandbox that runs code, and Google released its strongest model to cyber defenders before anyone else. If you run agents, the harness is now part of what you have to test.

TraceTrial-Bench scores the harness, not the model

Trial by Trace (299 upvotes on Hugging Face Daily Papers) introduces TraceTrial-Bench, which treats the harness as the system under test. It splits harness reliability into seven components: context, execution, governance, lifecycle, observability, tools and verification. The authors ran 95 expert-level tasks across 4 harnesses and 9 models, 3,420 runs in all.

Three numbers stand out. Harness scores ranged from 59.8% to 72.0%. 31.1% of runs completed the task while failing at least one harness criterion. Only 8.4% of runs satisfied every criterion in both layers, against 39.5% that satisfied every agent criterion. The harness with the best task success was not the one with the best harness score.

The paper is careful about attribution: it separates a matched failure pattern from a validated propagation from a confirmed defect, and says co-occurring failures do not prove cause. That discipline is Failure Classification applied to the harness. A successful run that broke a governance rule on the way is a defect you have not found yet, and you only find it if The Agent Observability Framework records the path, not just the result.

Anthropic: separate the brain from the hands

On Latent Space’s episode on how Anthropic ships Claude with Claude, the guests described splitting the harness and session log, which run on a durable server, from the sandbox that executes code. An earlier design ran the whole agent inside the sandbox, which added startup latency and meant a dead sandbox killed the running agent. Credentials live in a separate vault, and a proxy injects them on egress, so the sandbox never holds them. Sandboxes get domain allow lists or no network at all.

Two other points are worth keeping. The default harness is deliberately minimal, with verifiers and parallel agents built on top. And the guests said teams probably need more evals rather than fewer, and that when an eval fails, the eval or the environment may be what is wrong, not the model.

This is The Agent Security Surface Framework and The Reversible Execution Framework as an architecture decision: keep the part that can be compromised away from the part that holds the keys, and make the part that runs code disposable.

Gemini 4 Carbon goes to defenders first

Google announced Gemini 4 Carbon on October 28, two weeks after Gemini 4 Argon. Ars Technica’s headline puts it plainly: you cannot use it yet. Access starts with a subset of cyber defenders and expands to trusted partners through Google’s Fairwind Program. Google lists introductory pricing of $2 per million input tokens and $10 per million output tokens, rising to $4 and $20 after the introductory period.

The benchmark numbers are Google’s own: 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym, 71.0% on CWE-bench for patching. The safety figure engineers should look at is Terminal-Bench 4.0, where Google reports successful cheating at 0.0% and attempts at 0.4%. That a lab now reports cheating attempts at all is the point of The Verifier Gaming Surface Framework: if your gate can be defeated, assume a capable agent will try, and measure it.

The same week brought GPT-6.1 Terra. Simon Willison also quoted Ted Sanders on OpenAI’s naming: model performance is multidimensional, but size explains much of it, and the names now order by size, Luna < Terra < Sol < Astra. A leaderboard rank tells you less than your own eval set on your own tasks.

Octoverse: three times the merged pull requests

At GitHub Universe, Simon Willison relayed the Octoverse 2026 numbers: 220 million merged pull requests a month, which he says tripled in less than a year; 7.38 billion commits a month, more than tripled; and 30.3 million new repositories a month, double the rate. He calls it a stark picture of agentic engineering’s impact.

Throughput is the easy part to count. The Maintainability Debt Curve asks the harder question: is three times the merged code also three times the review, or is the substrate quietly getting worse? Track your own change-failure rate and churn before you celebrate your own volume chart.

Amazon: “a finding, not a failing”

Simon Willison also quoted guidance from Amazon engineering leadership: if your work does not benefit from current AI tooling, “that is a finding, not a failing.” The rest asks engineers to document why, share it, and bring it to their manager, and calls that explanation the judgment the company wants. That is The Trust Dial stated as policy: calibrated reliance includes knowing where not to rely.

The throughline

Every story this week moves the measurement off the model. A benchmark scores the harness, a lab redesigns the harness so failures stay contained, a frontier release reports how often the model tried to cheat its tests, and a large employer asks engineers to report where AI does not help. The engineering-grade thesis is that reliability lives in the system around the model. This week the field started measuring it there.

The diagnostics for testing your own harness are at curiochat.ai/software-engineer/.