Last week, a security startup pointed an autonomous AI agent at McKinsey’s internal AI platform and walked away with 46.5 million confidential chat messages. No stolen passwords. No insider help. No human making decisions once the agent launched.
Two hours. That’s all it took.
If that doesn’t get your attention, here’s what should: the vulnerability that made it possible has existed since the 1990s.
What Actually Happened
McKinsey built an internal AI platform called Lilli in 2023. Over 40,000 employees use it daily — consultants running strategy work, M&A analysis, client research. It processes 500,000 prompts a month. It’s not a side project. It’s a core business tool.
Security firm CodeWall decided to test it. Their AI agent — no credentials, no insider knowledge, no human in the loop after launch — mapped Lilli’s publicly exposed API documentation, found 22 unauthenticated endpoints among 200+, and identified a SQL injection flaw that standard security scanners completely missed.
Then it ran 15 blind iterations, reading error messages, learning the query structure, and extracting production data until it had full read-and-write access to the entire database.
The haul: 46.5 million chat messages in plaintext. 728,000 confidential files. 57,000 user accounts. And 95 system prompts — the instructions controlling how Lilli behaves — all of them writable.
McKinsey patched within a day of being notified. A forensic investigation found no evidence that client data was actually exfiltrated. But that’s not the lesson here.
The Real Problem Nobody Is Talking About
Everyone is focused on the breach. We want to talk about what made it possible.
The attack didn’t exploit anything new. SQL injection is one of the most well-documented vulnerability classes in existence. It’s been on the OWASP Top 10 for decades. McKinsey’s security team — at a firm with billions in annual revenue — didn’t catch it in a production AI system that had been running for two years.
Why? Because they were securing the platform the way they’d secure any enterprise software. Servers locked down. Access controls in place. Standard scanners running. And none of that was designed for what AI agents actually expose.
Here’s the part that should keep operations leaders up at night: those 95 system prompts were writable.
System prompts are the instructions that govern how an AI behaves — what it says, what it won’t say, how it cites sources, what guardrails it follows. If an attacker rewrites them, they don’t need to touch a single line of code. No deployment. No process anomalies. No log trail. The AI just starts behaving differently, and nobody notices until the damage is done.
Poisoned financial models. Strategic recommendations quietly tilted. Confidential data embedded into responses that consultants copy into client-facing documents. All invisible. All plausible.
Organizations have spent decades securing their code, their servers, their supply chains. The prompt layer — the instructions that govern how AI systems actually behave — is the new high-value target. And almost nobody is treating it as one.
This Is the New Attack Surface
What CodeWall demonstrated isn’t just a story about McKinsey. It’s a preview of how AI systems will be targeted at scale.
The agent autonomously selected its target. It found the vulnerability without a checklist. It chained multiple weaknesses together — the SQL injection plus an IDOR flaw — to escalate access. It moved faster than any human attacker could.
This is what agentic attacks look like. And if you’re deploying AI agents across your operations without a security evaluation layer built into how those agents run, you are flying blind.
The question for every mid-market company deploying AI right now isn’t whether you have vulnerabilities. It’s whether you’ll find them before someone else does.
What Evaluation-First Actually Means for Security
At Magnetiz, we’ve built our entire approach around evaluation-first AI agents. That principle isn’t just about output quality — it’s about knowing what your agents are doing, what they’re exposing, and whether they’re behaving the way you think they are.
That means:
Continuous behavioral testing. Not a one-time security audit. Agents that run, test, and flag anomalies in how your AI systems respond over time.
Prompt layer governance. Treating system prompts like the critical infrastructure they are — versioned, audited, access-controlled.
Attack surface mapping before deployment. Understanding what your AI stack exposes before an adversarial agent does it for you.
The McKinsey breach was caught by a firm operating under a responsible disclosure policy. The next one might not be.
The Bottom Line
The rush to deploy AI has outpaced the security frameworks designed to protect it. That gap is now being actively exploited — not by humans manually probing for weaknesses, but by autonomous agents moving at machine speed.
McKinsey had the resources to patch fast and contain the damage. Most mid-market companies don’t have that cushion.
The time to build evaluation into your AI operations is before an agent finds the hole. Not after.