Asymmetric On-Policy Distillation Learns In-Context Exploration
[Robotyka] Autorzy: Sriyash Poddar, Yanda Bao, Jacob Krantz i 5 więcej
Sequential decision-making under partial observability is difficult because agents must explore to infer hidden, task-relevant information before acting effectively. Learning such information-gathering behaviors tabula rasa via online reinforcement learning is sample-inefficient, whereas offline imitation averages expert actions across hidden states and fails to learn exploration. In this work, we propose ASTEROID, a simple and effective student--teacher supervised learning framework that can learn test-time exploration. Although a privileged expert never needs to explore, we demonstrate that a history-conditioned student can learn to do so when trained with on-policy contexts: the student first collects on-policy context under partial observability, and the privileged expert provides action labels conditioned on that context. By distilling expert actions directly conditioned on the stu…
- Kategoria arxiv
- cs.RO
- Kategoria
- Robotyka