Towards Robust Model Organisms for Eliciting Latent Knowledge
[Sztuczna inteligencja] Autorzy: Lennart Finke, Hardik Bhatnagar, Euodia Dodd i 4 więcej
A key problem in AI safety is eliciting latent knowledge (ELK), extracting information from a model that is misaligned and more knowledgeable than its overseer. For example, a misaligned model used to scan code for vulnerabilities may want to withhold its findings, i.e., sandbag, in order to exploit them later. To study ELK empirically, we need model organisms that mirror this setting. Existing model organisms fall short in several respects. In this work, we train model organisms to withhold solutions in a vulnerability detection task. We combine transcript distillation with counter-prompting and tamper-resistance training to instill three properties that we argue are critical for a useful ELK testbed: 1) Selectivity, the model sandbags when it thinks the overseer doesn't know the answer and complies otherwise, 2) Robustness, the model resists elicitation, and 3) Retention, the model re…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja