The AI that caught itself lying.
I built an incident-response system where AI agents detect problems, fix production infrastructure, and grade their own work. Then the dashboard reported a 58% false-positive rate — and the obvious fix would have been the wrong one.
What it does
Most "AI for operations" demos stop at suggesting a fix. A dashboard lights up, an LLM writes a paragraph about what might be wrong, and a human does the actual work. That's a chatbot wearing a hard hat.
This system takes real action. It watches a fleet of running containers, notices when one goes bad, reasons about what's actually causing it, and then issues real commands against real infrastructure — an actual restart on an actual machine. Then it checks whether that fixed anything, writes its own postmortem, and when it notices a pattern in its own mistakes, it proposes a change to its own detection logic.
Six agents, each with one job:
- Detector — reads the metrics, notices anomalies, opens an incident
- Triage — reasons over the dependency graph and past cases to find root cause
- Proposer — matches a runbook, decides what needs human approval
- Executor — runs the real remediation, records what happened
- Postmortem — grades the work against ground truth, writes to memory
- Injector — breaks things on purpose, so the loop can be tested honestly
The number that looked like failure
Once the system was running unattended, the dashboard surfaced an ugly statistic.
A 58% false-positive rate means the system was crying wolf more often than not. In a real operation, that number gets your alerting ignored within a week.
Here is the tempting move, and it's the one most people make: turn up the detection thresholds until the number looks good. Make the system less sensitive, false positives drop, dashboard turns green, ship it.
That fix would have worked. It also would have suppressed real incidents to make a metric look better — which is how you build a monitoring system that's quiet right up until the moment it matters.
What the data actually said
Instead of tuning to the number, I used the platform's own records to root-cause it — three queries joining the postmortem verdicts back against what had actually triggered each incident in the first place.
The false positives clustered on CPU. And the CPU values on those "false" alarms weren't noise drifting near idle. They were 97.0. That's a machine pinned at full saturation. That's not a false alarm — that's about as real as a fault gets.
The bug wasn't in the detector. The bug was in the grader.
The postmortem agent was asked to judge whether an incident had been real. But by the time it looked, the fix had already worked — the machine had recovered, and CPU was back to normal. So the agent saw a healthy machine and concluded nothing had ever been wrong.
It had collapsed two completely different questions into one: "was this a real problem?" and "is it better now?" A successfully-fixed emergency looked identical to a false alarm. The system was doing its job correctly and then grading itself as if it had failed.
Separate the two questions the prompt had merged. Ask was the original detection real — measured against the metrics at the moment it fired — and separately, did it recover after the fix. Two questions, two answers, no confusion between them.
False positives dropped from 27 to 1 on fresh data.
Why this is the whole point
There's a design decision underneath all of this, and it's the one I'd defend hardest.
The AI is never allowed to be the judge of whether it succeeded.
When the postmortem agent decides whether a fix worked, that verdict comes from measured metrics — hard numbers pulled from the database. The AI is used for what it's genuinely good at: categorizing, explaining, ranking hypotheses, writing the narrative. But the booleans — did this work, yes or no — come from data the model cannot talk its way around.
This matters more than any other choice in the system. An AI that grades its own homework will always eventually give itself an A. Once you let a self-improving loop score itself on vibes, it stops improving and starts congratulating. The grounding in real metrics is the only thing standing between "self-improving" and "self-deluding."
The system found a pattern in its own failures and proposed a fix to its own detection logic. A human still approves that change. Fully autonomous self-modification is how you get runaway feedback loops in production.
What broke along the way
Two other bugs are worth naming, because they're the kind of thing that only shows up when a system actually runs.
The detector cried wolf on idle machines
The first version flagged anomalies using a standard statistical test. It kept firing on nearly-idle machines — because when a machine's CPU drifts from 0.1% to 3.5%, that's statistically enormous relative to its own baseline, and operationally meaningless. Nobody cares. The rebuild added absolute floors (a value has to actually matter before the statistics matter), a baseline that refuses to be poisoned by the anomaly it's measuring, and a requirement that a problem persist across multiple readings before anyone gets woken up.
The faults were healing before the fix arrived
After fixing the grader, everything started grading as "flapping" — the fix ran, but the problem didn't stay fixed. The timing data explained it: the injected faults lasted 15–30 seconds, while the full detect→diagnose→repair pipeline took 80–270 seconds. The problems were curing themselves before the cure showed up. The system was heroically "fixing" things that had already recovered on their own, and then honestly reporting that its fix hadn't held.
Two independent processes racing on shared state — a classic distributed-systems problem, and one you cannot find in a demo. You find it by running the thing for real and reading the timestamps.
What this has to do with your business
You probably don't need an autonomous incident-response platform for a container fleet.
But if you're going to hand any part of your operation to AI — the phone, the invoices, the follow-ups — you should want it built by someone who has already learned, the hard way, that an AI system will confidently tell you it's working when it isn't.
An AI receptionist that says it booked the appointment. An invoice agent that reports it sent the reminder. A follow-up system that logs every lead as contacted. Every one of those can be wrong in exactly the way this system was wrong — reporting success against its own opinion instead of against reality.
The habit of grounding every claim in ground truth is the difference between an AI employee you can trust and one you have to check.
The short version
The system reported it was failing 58% of the time. It wasn't. It was succeeding and then misjudging its own success.
The obvious fix was the wrong fix. Tuning the thresholds would have made the dashboard green by hiding real emergencies.
The answer was in the data, not the model. Three queries against its own records found the actual bug — in the grader, not the detector.
Never let the AI decide whether it succeeded. Measure it. That single rule is what separates automation you can trust from automation you have to babysit.
What would an AI employee do for your business?
Thirty minutes, no cost, no pressure. You'll leave with a prioritized list of what to automate and what it's worth — whether or not you hire me.
Book a Free Audit