Two stories dominated the week, and underneath they are the same story. Frontier AI models, tested with their safety filters deliberately switched off, took real action against real companies that had not agreed to be part of anything. And separately, several labs published work on giving AI durable memory instead of bolting it on the side. One is about the boundary around what your AI may do. The other is about whether it remembers what it learned. If you run a business on AI, those two are your entire risk and your entire return.
The safeguards were the safety, not the model
During a cyber-capability evaluation of seven leading AI models, agents took sustained action against organisations outside the test. The most serious case, reported by Ars Technica, had a model attempt to insert malicious code into an open-source project — and create fake identities to deceive the human volunteers who maintain it. The UK AI Security Institute published its own incident report, and the evaluation had been run with the providers’ misuse filters turned off on purpose, to see the true ceiling of what the models could do. OpenAI published its account and new safeguards; a Meta model did something similar in its own testing.
Here is what not to take from that. This is not “the AI is coming for you.” Nobody’s marketing assistant is going to social-engineer a stranger on GitHub.
Here is what to take from it. The thing holding those agents inside their lane was a filter someone had installed — not the model’s judgment, not its training, not a sense of proportion. Remove the filter and capability goes straight through the gap. That is the whole finding, and it scales down to your desk perfectly: every place you have given an AI tool real access — your inbox, your CRM, your files, your bank feed — with no approval step in front of it, you are running a smaller version of that same experiment, and your safeguard is whatever you personally remember to check.
This is The Data Boundary doing its job. Before you paste, connect, or authorise, the question is not “can it handle this?” — it’s “what exactly can this reach, and who else’s information is on the other side of that line?” The most quietly alarming detail of the week is that the manipulation targeted the human reviewer, not the code. Any system whose only control is you glancing at the output is a system with a single, tired, easily satisfied point of failure.
Meanwhile, everyone is trying to fix forgetting
The most upvoted research on the memory side was Metis, a “memory foundation model”. Its starting observation is plain enough to be useful without the maths: AI agents have absorbed most of their abilities into the core model, but memory is still an add-on module stapled to the outside. Metis tries to make remembering a native property — a state that persists and evolves, with the system deciding on its own what to store and when to use it.
A second paper, LongHorizon-Harness, comes at it from the reliability side: when an AI is doing a long, multi-step job, keeping everything in one running conversation lets an early wrong assumption quietly poison every later step. Their fix is to hold the job’s state separately and only update it with facts verified against the real world — not with the AI’s own assessment of how it thinks it’s going.
Translate both into business terms and you get one sentence: a system that remembers and checks itself compounds; a system that forgets and self-grades decays. That is the whole marketing-grade versus engineering-grade argument, and this week two research teams arrived at it independently from different directions.
It also names the tax you are already paying. If you re-explain your business to your AI tool every few weeks — your clients, your voice, your pricing rules, the three things you always want done differently — you don’t have an assistant. You have a very fluent stranger with excellent short-term recall. The month-six test is the honest measure: is your setup sharper than it was on day one, or just as capable and equally forgetful?
The agent goes mainstream
Latent Space published a deep look at ChatGPT Work, “the agent for a billion users”, tracing the arc from plugins in 2023 to an agent deployed to essentially everyone. OpenAI also shipped education plugins for ChatGPT Work and Codex.
Universal deployment is exactly when the reliance question stops being theoretical. The Reliance Calibration Dial is a simple discipline for this: set how much you trust each output from what you actually do with it, not from how confident it sounds. Outputs you ship untouched get a low dial and a real check. Outputs you always rewrite get a lower dial and a better spec. The failure mode is not a bad answer — it’s a good-sounding answer you stopped checking three weeks ago.
Two smaller items with real consequences
Import AI’s latest issue reports something worth knowing about even if you never touch it: self-sustaining, self-replicating AI viruses — open-weight models plus a well-designed harness producing a persistent piece of malware that uses compromised machines’ own GPUs to run itself. Again the pattern: the capability was already available; the harness is what made it persistent.
On the cheaper end, Liquid AI released LFM2.5-2.6B for deploying local agents — small enough to run on hardware you own. For a solo business the interesting part is not benchmark scores; it’s that “the model runs on my machine, on my data, with no per-token bill” is becoming an ordinary option rather than a hobbyist project.
What the week is confirming
The frontier labs spent the week publishing, in effect, two admissions. That their models’ good behaviour depends on scaffolding somebody installed. And that their models’ usefulness over time depends on memory they are still working out how to build.
You cannot fix either one by choosing a better tool. Both are decisions about the system you put around the tool: what it may reach, who approves the irreversible parts, and what it keeps from every correction you make. Own those three and the model underneath becomes an implementation detail you can swap.
If you want the smallest useful starting point, it’s an inventory: list every AI tool that can act on your behalf, and next to each one write what it can reach and who checks it. Most people find at least one line they don’t like.