ToFu paper: a ‘Repeat-After-Me’ prompt makes six agent frameworks run tool calls the model refused, and fuzzing finds bugs in 118 of 273 MCP servers
Confirmed
The takeaway
A preprint posted Oct 9, 2026 (arXiv 2610.13651) studies the boundary between a language model, the orchestration framework and its tools.
Status
- Claim
Confirmed
- Our reporting
- High confidence
- Importance
- 2 of 5
- Last verified
- 12 October 2026
Your AI and this story
- GPT-6 Astra162 days after its cutoff
- Claude Opus 5.5101 days after its cutoff
- Gemini 3.8 Flash192 days after its cutoff
- Grok 4.7131 days after its cutoff
None of these four assistants can know about it. The closest, Claude Opus 5.5, stops 101 days before it.
Key facts
- Paper: ‘ToFu: Exploiting and Fuzzing the LLM-Orchestrator-Tool Trust Boundary’, arXiv 2610.13651, Oct 9, 2026; authors Md Nazmul Islam Ananto, Pierre Prevost-Avenel, Colin Smith, Kyungtae Kim, Taegyu Kim, Arslan Khan; not peer-reviewed
- ToFu derives security checks for each tool from its metadata and a CWE-guided sanitizer corpus, then generates fuzzing harnesses automatically (abstract)
- Result (abstract): 273 MCP servers with 3,405 tools tested; verified bugs in 343 tools across 118 servers
- CSO Online (Oct 12): 253 confirmed bugs after manual review, including 64 path-traversal, 7 server-side request forgery, 5 data-exfiltration and 3 OS command-injection flaws; most others were reflected-output or log-injection issues
- ‘Repeat-After-Me’ (CSO Online): the attacker asks the model to repeat text formatted as a tool call, then reframes the request as hypothetical; the model emits a structured tool call and the framework executes it, before the model’s refusal arrives
- Frameworks reported affected: LangGraph, Microsoft Agent Framework, AutoGen, Google Agent Development Kit 2.0, AWS Strands Agents and CrewAI
- The detailed demonstration used LangGraph 1.2.0 with Llama 3.1 8B through Ollama, dummy tools and only a system prompt as safeguard; CSO Online notes that this does not show every deployment behaves the same way
- The missing pre-execution check was reported to LangGraph’s maintainers; LangChain, Microsoft, Google, AWS and CrewAI had not responded to CSO Online
What happened
The paper looks at what happens after a model produces output in an agent system: the orchestration layer turns that output into a tool call, and the tool runs it. The authors show two weaknesses. First, frameworks execute any well-formed tool call in the model’s output, so a prompt that gets the model to reproduce a call as “quoted text” can trigger a real action even when the model declines the request. Second, many tools do not enforce the limits their own descriptions state and rely on the model to send safe inputs; automated fuzzing found path traversal, request forgery and command injection in open-source MCP servers.
The framework-bypass details come from CSO Online’s report; the abstract covers the fuzzing results only.
Why it matters
Agent safety is often discussed as a property of the model. This work locates failures in the software around the model: refusals do not help if the framework acts on the text anyway, and tool servers need their own input validation and authorization checks.
Sources
2 sources from 2 sites. Numbers match the chips in the text.
2 sources: 1 primary, 1 press
Primary
- arXiv 2610.13651: ToFu: Exploiting and Fuzzing the LLM-Orchestrator-Tool Trust Boundaryarxiv.org, paper
Press
Changes
- Filed (evening)