The orchestration framework surrounding an AI model can alter agent compromise rates by 2.6 times, turning architecture selection into a security decision as critical as the model itself, according to new research published August 5, 2026 by AI security leader Julie Brunias. The study tested identical AI models wrapped in four different orchestration frameworks—LangChain, CrewAI, AutoGen, and SmolAgents—against thousands of adversarial payloads. The findings challenge the prevailing assumption that framework choice is primarily an engineering question centered on developer experience and ecosystem maturity.
Holding the underlying AI model constant across all tests, compromise rates ranged from 11.9% on the most resistant framework to 31.1% on the least resistant—a gap driven entirely by which orchestration layer was managing the agent's tool calls, memory, and multi-step reasoning. The evaluation harness ran the same set of adversarial attacks—including tool call hijacking, cross-tool injection, memory poisoning, and delegated authority abuse—against each framework configuration. Nothing about the model changed between the highest and lowest compromise figures. Nothing about the attack vectors changed. The only variable was the framework wrapping the model, yet the security outcome shifted by more than two and a half times.
The report finds that orchestration frameworks make architectural choices about tool call validation, context sharing between reasoning steps, memory persistence across tasks, and agent autonomy that directly determine how much room an attacker has to exploit vulnerabilities. "A framework that validates tool calls more strictly, or that segments memory more conservatively, closes off attack paths that a more permissive framework leaves wide open—regardless of which model is doing the reasoning," Brunias writes. A framework requiring every tool call to pass through an explicit schema check before execution gives attackers far less opportunity to inject malicious parameters than one letting the model call tools directly from its own generated text. That single design choice, made by framework authors before any deployment team touches the code, produces the kind of gap that separates acceptable security posture from one that should trigger serious pre-deployment conversations.
The research points to a blind spot in how teams currently evaluate orchestration frameworks. Public comparison guides from sites including Bestarion, Atlan, Moxo, Cordum, and Instinctools compare LangChain, CrewAI, and AutoGen on ecosystem maturity, memory handling, and human-in-the-loop support, but none run actual adversarial tests or report measured attack-success differences. Security typically appears as a single bullet point rather than a dimension weighted equally with developer experience. The report argues that model safety training doesn't carry over uniformly across frameworks—a well-aligned model wrapped in a framework offering attackers more room to maneuver can still land at a meaningfully worse real-world compromise rate than the same model in a tighter framework. Teams already deployed should test their current setup rather than waiting for migration plans, since retrofitting security after framework selection is possible but knowing where a specific configuration falls on the spectrum determines how urgently that work needs doing.
The report recommends treating framework security claims with the same skepticism applied to vendor marketing, noting that feature documentation reveals what a framework claims to do but not how it holds up against tool call hijacking or memory poisoning—data that only comes from running adversarial tests directly or finding someone who has. The conclusion frames the orchestration layer as performing real security-relevant work whether anyone designed it that way or not, and calls the model an incomplete picture of the attack surface—increasingly not even the most variable part. The bottom line: if your team is mid-decision on a framework or has already shipped one without adversarial testing, that's the conversation worth having this week, not after the next incident forces it. Procurement teams accustomed to evaluating vendors on compliance checklists now face a technical decision where documented features tell only half the story, and the other half requires adversarial red-teaming that few comparison guides currently provide.