One Loop, Two Gains: Can Active Learning win the Lottery for Free?
[Uczenie maszynowe] Autorzy: Benedikt Tscheschner, Eduardo Veas, Marc Masana
The lottery ticket hypothesis posits the existence of winning tickets: sparse subnetworks that, when trained in isolation from their original initialization, match the accuracy of the full dense network. The predominant method for discovering such tickets, iterative magnitude pruning, alternates pruning with full retraining from scratch until convergence over many cycles. Similarly, deep active learning also retrains a model from scratch after each acquisition round as new labels become available. Despite this shared reliance on iterative retraining with a substantial computational overhead, the two paradigms have been studied separately. We observe that the iterative training loop inherent to pool-based active learning already provides the exact computational structure that iterative magnitude pruning exploits, and propose Improve & Prune (I&P), a method that integrates magnitude pruni…
- Kategoria arxiv
- cs.LG
- Kategoria
- Uczenie maszynowe