ToolNavs Find Useful AI Tools
Submit Sign in
Back to AI information
Open-Weight Models Proved 88% More Susceptible to Injection Attacks

Open-Weight Models Proved 88% More Susceptible to Injection Attacks

AI information • Admin • • 17 views

Open-weight models are easier to steer with instructions hidden inside data when they power agents. Security firm PromptArmor published evaluation results on October 16, 2026: across 9,800 trials using real-world style indirect prompt injection attack chains, the open-weight group showed a susceptibility rate of 26.7%, against 14.2% for the proprietary group, a gap of 88%. The open group covered DeepSeek, Kimi, GLM and Qwen; the proprietary group covered GPT, Claude and Gemini, with each provider's latest flagship and latest cost-effective model tested.

The widest gaps appear when agents act

Indirect prompt injection is dangerous because the attacker never talks to the model directly. Instructions are planted in a web page, document or email the agent reads, and the model treats data as commands. The three attack samples with the starkest gaps all involve real actions. In a scenario where an agent investigating an attacker website ended up running malicious code, susceptibility was 49% for the open group and 9% for the proprietary group. A malicious document that made the agent render a phishing login overlay scored 36% against 5%. An injected email that leaked mail content through a Markdown image scored 23% against 9%. The gap, in other words, is not about chat tone; it is about whether an agent holding tools and permissions mistakes foreign instructions for part of its task.

The report also offers a counter-intuitive comparison. Among proprietary providers, cost-effective models were far more susceptible than their own flagships, by 169%, while among open-weight providers the cost-effective models were slightly more robust than their flagships, by about 5%. In individual cases, the newer budget models from Claude and GPT lines were easier to compromise than their older flagships, while the newer budget models from Qwen and DeepSeek held up better than theirs. Asking only whether a model is the newest release is clearly not enough.

How to read these numbers

The methodology needs stating plainly. These results come from PromptArmor's own attack chains and its own evaluation environment, built from attack chains the company says it found in real agentic products; they are not an independent third-party replication. Model versions, system prompts and tool configurations all move the outcome, so 26.7% and 14.2% are best read as a comparison of relative risk, not a final safety verdict on any single model. The two figures also share one property: neither is zero. A 14.2% rate for the better-performing group still means roughly one in seven real-world style attack chains got through.

For teams deploying agents, three conclusions are actionable. First, treat injection susceptibility as a selection metric alongside benchmark scores and price, especially for agents that read external documents, pages and email. Second, do not assume a budget model inherits its flagship's safety behavior; retest with your own business data before launch. Third, do not rest the defense on the model's good judgment alone: least-privilege tool access, confirmation for outbound actions, and explicitly marking retrieved content as untrusted data are system-level measures both groups need equally. External evaluation is also moving inside the labs, as Anthropic bringing evaluators in-house shows; read together, the industry's way of checking agent safety is shifting from benchmark scores to real attack chains.

Recommended Tools

More