Recognition Without Enforcement: Configuration-Dependent Failures in LLM Agent Instruction Arbitration and External Control
[Kryptografia i bezpieczeństwo] Autorzy: Jun Wen Leong
LLM agents arbitrate among instructions from system prompts, users, memory, and tools, but this arbitration cannot be assumed to enforce trust boundaries. We identify a recognition-enforcement gap: source-format features (role-template position, channel metadata, formatting cues) are linearly decodable from model activations, and models can explicitly identify forged authority when prompted, yet some configurations still produce the conflicting tool call. We use "recognition" in this specific decodable-source-format-plus-verbalized-detection sense; crossed-probe controls show it is not a unified abstract trust representation. The gap is not an immutable property of model weights. Restrictive policies and diverse prompts can eliminate execution on the same models, while permissive configurations and particular prompt-model pairs yield deterministic failures. Across a fleet evaluation (au…
- Kategoria arxiv
- cs.CR
- Kategoria
- Kryptografia i bezpieczeństwo