SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignment
[Sztuczna inteligencja] Autorzy: Qinghua Mao, Wanying Qu, Dadi Guo i 8 więcej
The performance of LLM-based agents is jointly shaped by the base model and the harness used when interacting with the environment. This exposes them to safety risks in both harmful final responses and multi-step execution trajectories. Existing safety alignment mechanisms often rely on either external harness updates or policy optimization, yet applying either paradigm in isolation fails to bridge runtime control with intrinsic safety. We propose SafeEvolve, an experience-driven self-evolving framework for agent safety alignment. SafeEvolve leverages safety experience from completed on-policy trajectories to drive a continual loop of harness-policy co-evolution. On the harness side, SafeEvolve converts trajectory-level safety evidence into bounded, component-level updates across safety prompt and hierarchical skills, yielding auditable and reversible harness artifacts. On the policy si…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja