VERA: A Responsible AI Dashboard for Assessing LLM Deployments in Light of the EU AI Act
[Inżynieria oprogramowania] Autorzy: Robin Quériaux, Solveig Bachellery, Léa Déléris i 4 więcej
Institutions that deploy large language models in regulated settings often need technical evidence of responsible AI practices before approving a deployment. COMPL-AI provides a technical interpretation of the EU AI Act that organises this evidence under six principle groups and eighteen technical requirements, twelve of which current benchmarks can measure. The benchmark suites that produce this evidence are typically operated from a command line, while the staff who approve deployments are non-engineers. We present VERA, an open-source self-hosted dashboard deployed to facilitate responsible AI processes within a large European bank. A reviewer selects a model and runs public benchmarks against it. VERA scores the twelve requirements that current benchmarks can measure and instruments the energy consumed by each evaluation, adding run-level evidence for environmental impact. We run fo…
- Kategoria arxiv
- cs.SE
- Kategoria
- Inżynieria oprogramowania