Thinking with Looped Flows
[Uczenie maszynowe] Autorzy: Ayhan Suleymanzade, Chanhyuk Lee, Floor Eijkelboom i 3 więcej
Humans and machines often solve harder problems by spending more time on computation. In deep learning, looped models implement this idea during inference by recurrently updating a hidden state. In practice, however, their training backpropagates through only one or a few updates, making it hard to train early updates to support future ones. We propose looped flows, an approach that sidesteps this issue by training the recurrence with local denoising objectives. By imposing temporal association across denoising objectives through progressively decreasing noise levels and shared noise, the model is incentivized to learn recurrent states that transfer useful computation over time, even when gradients cover only a few updates. We then formulate inference as integrating the velocity of a probability flow parameterized by the learned denoiser, coupled with recurrent states. This allows solvi…
- Kategoria arxiv
- cs.LG
- Kategoria
- Uczenie maszynowe