To Trust or Not to Trust: Retrieval-Augmented Fact Checking in Speech
[Uczenie maszynowe] Autorzy: Debajyoti Mazumder, Mamta, Abhirama Subramanyam Penamakuri
Online misinformation increasingly appears in spoken formats such as news clips, podcasts, interviews, political speeches, and social media videos, creating a need for fact-checking systems that can verify claims directly from speech. We introduce VeriSpeak, a probe benchmark for studying speech-based fact verification in Large Audio Language Models (LALMs). VeriSpeak contains 3,879 spoken claims spanning temporal, geographical, and relational facts, with balanced true and false labels. The benchmark is designed to examine whether factual verification ability transfers from text to speech, and whether retrieval-augmented LALMs can use textual evidence to correctly support or refute spoken claims. Our experiments reveal a consistent text-speech modality gap: LALMs that verify written claims reliably often fail on the same claims when spoken. Moreover, retrieval alone provides limited gai…
- Kategoria arxiv
- cs.LG
- Kategoria
- Uczenie maszynowe