Multi-Granularity Context-Enhanced RAG over Multimodal Knowledge Graphs
[Sztuczna inteligencja] Autorzy: Zongyu Wu, Yilong Wang, Xiaochen Wang i 5 więcej
Retrieval-augmented generation (RAG) is widely used to mitigate hallucination issues in large language models (LLMs) and multimodal large language models (MLLMs). In particular, knowledge graph (KG)-based RAG leverages structured knowledge to provide (M)LLMs with high-quality external information. Building on these works, recent studies have explored multimodal knowledge graphs (MMKGs) as knowledge bases for GraphRAG. This enables Graph RAG to integrate knowledge across multiple modalities, thereby further enhancing its performance. However, existing MMKG-based RAG methods generally follow a common pipeline in which different modalities are largely processed independently before being fusion. As a result, textual context is only used to a limited extent during visual information extraction and subsequent multimodal knowledge fusion. This brings a semantic gap between images and text whi…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja