MaxIM: Maximally Informative Incremental Summarization via Reinforcement Learning
[Sztuczna inteligencja] Autorzy: Jihwan Jeong, Guy Tennenholtz, Yinlam Chow i 2 więcej
AI systems must often maintain bounded memory of unbounded interaction histories. However, optimizing the memory representation, and its updates, for downstream task utility is difficult for standard reinforcement learning (RL) due to severe credit assignment bottlenecks induced by sparse, delayed rewards. We introduce MaxIM (Maximally Informative Incremental Memory), a framework that formulates incremental summarization as a capacity-constrained, information-theoretic reward-shaping problem. Grounded in value suboptimality bounds, we show that bounding sequential information loss motivates maximizing the predictive log-likelihood of future, task-relevant targets, a variational lower bound on predictive sufficiency. We introduce a recursive consistency condition to ensure summaries are grounded by maximizing mutual information (MI) with the observed history. We mitigate credit assignmen…
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja