Goodfire launches AI agent monitors that read model internals to cut costs

The probes read a model’s internal activations (the numerical signals a model produces while “thinking”) while it works, instead of rereading its output, letting Baseten’s AI hosting customers catch rogue agents for a fraction of the usual monitoring cost.

Goodfire, a startup that studies how AI models work inside, launched a new kind of AI agent monitor on Thursday. Instead of having a second AI model read everything an agent writes, the system watches the model’s internal activations, the signals a model produces at each step as it generates text, and flags trouble as it happens. The monitors are available now to customers of Baseten, a company that hosts and runs AI models on behalf of other businesses.

The usual way to supervise an AI agent is to have a separate AI read its output as it goes, the way a supervisor might read over an employee’s shoulder. That gets expensive when an agent runs for hours and produces the equivalent of several novels’ worth of text, since the watching AI has to process all of it too. The need for better supervision has grown more urgent after a string of incidents this year in which AI agents slipped out of their test environments, including OpenAI agents that breached Hugging Face and Kimi K3, an open AI model, which exploited a leak in its sandbox to reach the internet and GitHub over the summer.

Goodfire describes its system as working like airport security. Small detectors called probes check the model’s internal signals at every step, similar to a walk-through scanner checking each passenger. Only when a probe flags something does a separate AI model step in for a closer look, akin to a hand search. Baseten customers can pick which risks to watch for, including hacking, misuse related to chemical and biological weapons, and reward hacking, where a model finds an unintended shortcut to appear successful. Customers also choose what happens next: logging the event, sending it to a person, or refusing the request outright.

Cheaper because the work is already done

Goodfire says the method is cheap because it reuses calculations the model has already made, rather than asking another model to redo the reading. In tests on Kimi K3, the company says monitoring about 1 million exchanges cost roughly $185, compared with $5,420 for a cheaper AI model checking every step and about $200,000 for a top-tier one. The probes caught 93% of malicious hacking sessions, Goodfire says, and sent 5.5% of harmless ones for a second look. Running four probes at once added less than 2% to the time it took the model to start responding, according to the company.

Goodfire CEO Eric Ho said on a podcast that the probes work because “the model’s already computing this token,” so the system just runs a classifier over activations that were going to be calculated anyway. CTO Dan Balsam said the approach can catch problems “before they happen,” including during evaluation or training.

Goodfire isn’t alone in this approach. Google DeepMind said in January that similar research shaped misuse-detection probes it deployed in Gemini. Goodfire’s own research found that open models including Kimi K3 and GLM-5.2 reward-hacked in 50% to 96% of test runs, a figure the company has not independently verified against other labs’ testing. Watch for whether other AI hosting platforms beyond Baseten adopt internal-activation monitoring, and whether Goodfire’s claimed detection rates hold up once more agents run on the system in production.