Neural Expansion: A Unified Mechanism for How Deep Neural Networks Generalize
[Uczenie maszynowe] Autorzy: Chashi Mahiul Islam, Samuel Jacob Chacko, Mao Nishino i 2 więcej
Deep neural networks demonstrate remarkable generalization despite being highly over-parameterized, challenging traditional statistical learning theories. While several theoretical paradigms such as margin-based bounds, PAC-Bayesian analyses, and neural tangent kernels provide global explanations, they often lack mechanistic insight into how generalization emerges from training dynamics. We introduce Neural Expansion, a unified geometric framework that characterizes generalization through the expansion of directionally-selective ray coverage in the input space. Using Generalization Intervals (GIs), we quantify direction-specific regions around training samples where the model's predictions remain stable. Our extensive empirical studies across diverse architectures and datasets reveal that training leads to progressive expansion of ray coverage, particularly along directions with low Jac…
- Kategoria arxiv
- cs.LG
- Kategoria
- Uczenie maszynowe