Vulnerability Localization Benchmark: Measuring Agentic Security Analysis at Repository Scale
[Kryptografia i bezpieczeństwo] Autorzy: Aman Priyanshu, Supriti Vijay, Kimia Majd i 8 więcej
Language-model agents increasingly operate over complete software repositories, yet cybersecurity evaluations primarily measure whether they can detect, reproduce, or repair vulnerabilities rather than whether they can locate the relevant code. We study vulnerability localization: given a weakness class and an unfamiliar repository, identify the implementation files associated with that weakness. We introduce the Vulnerability Localization Benchmark (VLoc Bench), comprising 500 real world vulnerabilities from 290 repositories across six package ecosystems and 147 CWE categories. Each task pairs repository snapshots immediately before and after a security fix. On the vulnerable snapshot, an agent receives only the CWE description and read-only terminal access and must return the affected files; on the patched snapshot, it must determine that the recorded vulnerability is no longer presen…
- Kategoria arxiv
- cs.CR
- Kategoria
- Kryptografia i bezpieczeństwo