Puls Dane publiczne, aktualizowane automatycznie
arXiv – nowe publikacje publikacja

Black-Box Uncertainty Quantification for Large Language Models via Ensemble-of-Ensembles

[Uczenie maszynowe] Autorzy: Wang Ma, Debarun Bhattacharjya, Junkyu Lee i 3 więcej

Reliable use of large language models (LLMs) requires uncertainty estimates that separate ambiguity intrinsic to a prompt from genuine knowledge gaps. Bayesian and deep-ensemble methods deliver this aleatoric/epistemic decomposition in principle, but are computationally prohibitive at LLM scale and require white-box access. Existing black-box methods, in contrast, summarize to a single scalar that cannot make the distinction. We introduce a two-level ensemble that bridges this gap. Stochastic decoding of a fixed prompt forms the inner ensemble; meaning-preserving semantic perturbations of the prompt form the outer ensemble. Variability is measured in a continuous embedding space, and the law of total covariance gives an exact within-/between-perturbation decomposition to estimate aleatoric and epistemic uncertainty (AU/EU); it requires only black-box sampling access. On top of this esti…

Kategoria arxiv
cs.LG
Kategoria
Uczenie maszynowe
Pdf
https://arxiv.org/pdf/2611.18453v1

Otwórz oryginał w źródle ↗

Więcej z tego źródła

Wszystkie