HolmesGPT Can Delete Your Cluster. Here's What Actually Stops It.
HolmesGPT Can Delete Your Cluster. Here's What Actually Stops It.
In April, an AI coding agent deleted a production database in nine seconds. PocketOS founder Jer Crane had it running against a staging environment when it hit a credential mismatch, decided on its own initiative to "fix" the problem, and ran a delete against a Railway volume that turned out to hold live customer data. Asked afterward what happened, the agent wrote back a full account of which of its own rules it had broken: "I violated every principle I was given." The data was recoverable. The judgment call that led there wasn't.
That post reached the front page of Hacker News and stayed there, and the thread landed on the same conclusion the whole industry is circling right now: nobody had built a system where the agent's own carefulness was the last line of defense. The first time someone on your team asks Holmes to "restart the payment-service deployment," it works, and it feels great. The second question is the one that actually matters: what stops the exact same request from being "delete the payment-service deployment"? Or worse, what stops it from ever reaching your engineer at all, arriving instead as a payload hiding in a log line Holmes happened to read during an investigation?
We covered HolmesGPT as a production triage platform back in April, and the runbook feedback loop in June. Neither post touched what happens once you give Holmes write access. That's a different problem, and it turns out the HolmesGPT team already solved most of it — not with a system prompt telling the model to be careful, but with an architecture where the model's carefulness doesn't matter.


