HRIL: Learning Multimodal Synergy via Higher-Order Tensor Modeling
[Sztuczna inteligencja] Autorzy: Qun Dai, Liangjian Wen, Jiang Duan i 7 więcej
Self-supervised multimodal representation learning has achieved remarkable success across diverse domains, yet capturing synergistic information remains challenging due to the complexity of cross-modal interactions. Unlike the shared information across individual modalities, synergy arises when task-relevant signals emerge only from the joint configuration of multiple modalities and cannot be recovered from any modality in isolation. This work focuses on how to preserve the information capacity for such synergistic signals in multimodal representations. The key observation is that synergistic information is reflected in higher-order statistical dependence among modalities, which provides a principled target for explicitly modeling joint interactions. Motivated by this insight, we propose Higher-order Representation and Information Learning (HRIL), which constructs an empirical cross-mom…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja