What did the SWE-Bench ProMax paper find about existing agent benchmarks?

It cites an audit finding that nearly 60% of unsolved SWE-bench Verified instances contain flawed tests — either too narrow, so they reject correct solutions, or too broad, so they check requirements nobody stated. It also notes frontier models can verbatim reproduce gold patches from training data. Its answer is to benchmark multi-file, behavior-preserving refactoring instead, which is harder to memorize and harder to fake.

Why does self-improvement research raise the stakes on evaluation?

Because every self-improvement method this week replaces human supervision with an automatic judgment: an external contract, a majority vote, a Bayesian belief over turns. Whatever that judgment measures is what the system optimizes toward. A loop that improves against a weak verifier does not get better — it gets better at the verifier, faster than before, and with less human attention on it.

This Week in AI: Self-Improving Agents, Grading Themselves

The week’s highest-ranked papers are dominated by systems that keep improving after they ship — continual learning, unsupervised self-distillation, agent co-evolution. One of the lower-ranked ones says the tests we grade them with are broken. Read together, that is the engineering story of the week: improvement is being moved inside the loop, which quietly transfers all the load onto the verifier — and the verifier is the part nobody has been auditing.

Continual learning, graded by an external contract

Macaron-V1: Towards Open Continual Learning with Self-Improvement and Mixture-of-LoRA (328 upvotes, the week’s second-most-noticed paper) is an open agent-model family built to keep learning after deployment. Two moving parts: recursive improvement of versioned model-harness pairs, where experience from one configuration is evaluated and used to build its successor, and a Mixture-of-LoRA architecture that freezes a base model, composes specialist adapters, and picks one per user turn. The flagship pairs a 744B GLM-5.2 base with four LoRAs for chat, agent, coding, and GenUI.

The detail worth stealing is not the adapters. It is that the harness is versioned alongside the model, and each successor is evaluated under an external contract. That is the difference between a system that learns and a system that drifts — and it is The Intelligence Loop drawn at model scale: failures only become permanent capability if something outside the loop decides what counts as a failure.

The benchmark audit nobody wanted

SWE-Bench ProMax: Benchmarking Agents on Large-Scale Multilingual Code Refactoring (126 upvotes) opens with an uncomfortable citation: an audit found nearly 60% of unsolved SWE-bench Verified instances contain flawed tests — overly narrow ones that reject correct solutions, or overly broad ones that check unstated requirements — and that frontier models can verbatim reproduce gold patches from training data. Its proposal is to move to coordinated, behavior-preserving refactoring across many files, which is both harder and harder to memorize.

This is The Verifier Gaming Surface in the clearest form the field has produced this year: the framework’s whole question is where your agent can defeat the gate instead of doing the work, and a benchmark whose tests are wrong 60% of the time in its hardest slice is a gate made of paper. Note the three distinct failure modes hiding in one finding — a gate can wrongly reject correct work, wrongly accept wrong work, or be bypassed entirely by memorization. Most private eval suites are audited for none of them. It is the same argument as gate erosion in code review, one level up: a gate does not have to be removed to stop working.

Supervision removed entirely

On-Policy Self-Distillation without Any Supervision (194 upvotes) drops external supervision altogether. U-OPSD samples multiple rollouts, builds a pseudo-solution by majority vote under a self-consistency threshold, and conditions the model on its own consensus — no ground truth, no environment feedback, no larger teacher model.

It is a real advance in post-training, and it replaces a human judgment with an automatic one. Self-consistency is a proxy for correctness, not a measurement of it — a model confidently wrong the same way across five rollouts votes itself right. Put that next to the ProMax finding and the shape is obvious: we are getting very good at optimizing against judgments we have not verified.

Red-teaming what the agent leaves behind

OpenART: Scaling Agent Red Teaming via Open-Ended Environment Evolution (159 upvotes, and climbing fast this week) starts from the property that separates agents from chat: they operate in persistent environments where an early state change influences decisions far into the future, so short static safety tests miss cumulative risk by construction. It ships over 10,000 validated stateful scenarios across 50 domains, drawn from a pool of more than 500,000 tools and skills.

If your agent testing is still single-turn prompt-response, this names the gap. The risk that matters is not what the agent says in turn one; it is what turn one wrote into the environment that turn forty acts on.

Your reasoning traces are replayable

Stealing Reasoning Traces from Proprietary LLM APIs (also the week’s HF paper, 94 upvotes, and a vanity domain) identifies an architectural flaw in how providers hide chain-of-thought. Anthropic, OpenAI and Google return encrypted reasoning blocks to the client — and those blocks replay across sessions, users, and models within a provider’s ecosystem. The attack takes a trace from a frontier model and replays it into a weaker sibling.

In the same week, terabytes of credentials were exposed in a supply-chain attack on LiteLLM — cloud keys, repository tokens and SSH secrets from Microsoft, Amazon, Cisco, Samsung and Salesforce among others, per CloudSEK and Hudson Rock. Neither is a model failure. Both are the layer around the model: an opaque token you hand back and forth, and a proxy every one of your agents authenticates through. That layer holds your secrets and almost none of your review attention.

Provenance arrives as regulation

Anthropic will watermark content processed — not just generated — by its models, rolling out machine-readable watermarks to comply with the EU AI Act’s requirement that providers mark AI-generated or manipulated audio, image, text and video. The law covers models released after August 2, with a grace period to December 2026.

Read the word processed. Text you wrote, passed through a model for editing, carries a machine-readable mark. That is a provenance signal arriving from the compliance direction rather than the engineering one, and it lands on a question this week’s other stories already raised: when a system improves itself, what is left that can prove what it did?

What the week is confirming

The field is moving improvement inside the loop — versioned harnesses, self-distillation, recursive credit assignment — and in the same seven days published evidence that the evaluation floor is weaker than assumed, that the reasoning traces meant to be opaque are replayable, and that provenance marking is now a legal requirement rather than a nice-to-have. Those facts multiply. A system that self-improves against a verifier you have not audited does not converge on quality; it converges on your verifier’s blind spots, faster than a supervised system would, and with less of you watching. The engineering work this year is not making agents improve. It is being able to prove what they improved toward.

If you want the framework version — the gates, the eval discipline, and the observability layer that make an improving agent safe to run — start at curiochat.ai/software-engineer.