Watermarking technology that European Union regulations now require for AI-generated content is changing how AI models handle tools and refuse harmful requests, according to a new report from Lasso Security published this week. The machine-readable codes meant to establish content provenance don't necessarily make AI agents perform worse, but they can alter behavior in unexpected ways, particularly when adversarial attacks are involved. With the EU AI Act now in force, providers like Anthropic and OpenAI have adopted watermarking methods such as Google DeepMind's SynthID-Text to comply with the law.
Lasso Security's testing found that watermarking reduced accuracy on six of seven models when evaluated for tool calling, using a benchmark called BFCL v4 single-turn AST. The affected models included phi-4, Llama-3.1-8B, Qwen3-32B, Qwen3-4B, gemma-3-12b, gemma-3-27b, and Granite-3.2-8B. When watermarks were present, these AI agents more frequently selected the wrong tool for a given task or used incorrect arguments for the right tool, leading to failures from malformed input or parsing errors. For refusals of harmful requests, watermarking had a small effect on obviously dangerous prompts tested with HarmBench and JailbreakBench, but the impact became much more significant in adversarial scenarios involving prompt injection, where attack success rates increased noticeably.
"Watermarking is designed for provenance, but SynthID-Text changes the process by which the model generates each next token," Lasso explained in its research. The company noted that at the model level, this shift can alter safety behavior, including whether a model refuses a harmful request and whether that refusal holds under prompt injection. The report also found that while the overall change in accuracy might appear minimal, that doesn't reveal whether the same individual calls succeed with and without the watermark—a call that becomes incorrect can be offset by another that becomes correct, leaving aggregate results nearly unchanged even though the model behaves differently on both items.
The altered behavior stems from how watermarking intervenes in the prediction process that produces specific words. Anthropic's implementation, for example, favors one statistically likely word over an alternative when Claude generates text—such as choosing "overcast" instead of "gray" when completing a sentence about cold weather. These low-stakes word choices occur repeatedly throughout a piece of generated text, creating a pattern that's invisible to human readers but detectable to anyone with the encoding key. While people won't notice the vocabulary bias, AI agents can be subtly sensitive to these differences in word selection, which explains the changes in tool handling and safety refusals. This effect extends beyond the watermarking organization's own agents—any agent based on an API client calling an Anthropic model would process whatever output variation follows from that company's watermarking.
According to Lasso, the findings don't argue against watermarking itself. Instead, the company recommends that security evaluations and red-teaming exercises include watermarked content when assessing agent deployment, ensuring that differences in agent behavior can be properly weighed. The need for this additional testing layer becomes especially important as watermarking spreads across the industry to meet regulatory requirements. Organizations deploying AI agents will need to balance compliance with content provenance laws against the subtle shifts in model performance that watermarking introduces, particularly in security-sensitive contexts where prompt injection attacks remain a real threat. The transparency mandate designed to combat manipulative AI content may require a more nuanced implementation strategy than regulators initially anticipated.