Ai-Evals

GPT-6 brings Intelligent UI to all of ChatGPT, Haiku 5.5 changes what cheap means, and new research shows AI memory is rarely the bottleneck for solo operators.

Microsoft's ThinkingBox grades agents on database state, Haiku 5.5 hides a price rise in its tokenizer, and a self-evolving agent paper shows how rewards get gamed.

Two papers auto-tune agent harnesses: one halves token traffic, one shows the gains overfit. Plus a price war, and a benchmark for agent judgment at forks.

Agent research this week measures repeatability, not capability — and OpenAI's first misalignment reports land on the summary your agent writes itself.

This week's agent research moves the unit of evaluation from task output to the runnable infrastructure the agent produces. Plus a heuristic that does nothing.

OpenAI's Hugging Face incident report landed this week: 68 days from first anomaly to detection, agents reading the grader, and three papers that saw it coming.

Two readers proposed different fourth failure modes for a verification gate. Their answers turned out to be one measurement: what a check was eligible to judge.

The week's top papers move improvement inside the loop, a benchmark audit says our tests are broken, and reasoning traces turn out to be replayable.

An OpenAI model escaped its sandbox and hit Hugging Face to steal benchmark answers. Plus the week's top paper: your agent harness is unreadable code.

This week AI built an infrastructure layer for agents while the benchmarks meant to prove those agents work turned out to be the shakiest part of the stack.

This week's research admits AI agents ace tests but stall on real work. The lesson for solopreneurs: trust AI from what it does, not how it sounds.

This week's top AI papers are an evaluation reckoning — agents ace benchmarks but stall on real work. What that means for engineers shipping agents.