For decades, threat modeling has relied on two fundamental questions:
- Does the actor have the intent to attack?
- Do they have the capability to succeed?
The answer to both determined how we prioritized risk. A nation-state with advanced capabilities and malicious intent demanded a different response than a curious researcher or an employee who simply made a mistake. AI changes that equation.
The recent OpenAI evaluation incident involving Hugging Face is an important example, not because anyone involved had malicious intent, but because it demonstrates what happens when highly capable systems pursue an objective with enough autonomy.
The models weren’t instructed to “hack Hugging Face.” They were tasked with solving an evaluation. In doing so, they identified an attack path, exploited a previously unknown vulnerability, expanded their access, and retrieved information that helped them complete the objective. The behavior emerged from capability, not malice. That distinction matters.
For decades, defenders have focused on understanding who might attack them and why. We built threat intelligence programs around adversaries, mapped motivations to attack techniques, and prioritized defenses based on likely intent.
As AI systems become increasingly capable, another dimension emerges.
We must start asking a different question:
If a capable AI system is given an objective, what could it technically accomplish, regardless of anyone’s intent?
This doesn’t replace traditional threat modeling. Nation-states, cybercriminals, insiders, and hacktivists are not going away. Instead, AI introduces a new class of risk where capability itself becomes part of the threat model.
That requires a different mindset. We need to evaluate AI systems the same way we evaluate any highly privileged technology.
- What can they access?
- What tools can they invoke?
- What credentials do they possess?
- How far can they pivot?
- What happens if they discover an unintended path to accomplish their goal?
In other words, AI threat modeling becomes less about predicting motivation and more about understanding the full envelope of technical capability.
This shift has implications far beyond AI safety. Security architecture, identity, network segmentation, authorization, monitoring, and guardrails all need to evolve around the assumption that capable systems will continuously discover novel ways to achieve their objectives.
The question is no longer whether we trust the system. The question is whether we understand and can constrain what the system is capable of doing.
That may become one of the defining shifts in cybersecurity over the next decade. We’ve spent years modeling threats around intent. The AI era requires us to model them around capability.