Exactness at Inference: A Representational Criterion for Out-of-Distribution Generalization
[Uczenie maszynowe] Autorzy: Filipe Marinho Rocha, Inês Dutra, Vítor Santos Costa i 1 więcej
A model generalizes outside its training distribution only when it computes a representation structurally equivalent to the generating mechanism, not an approximation fitted to it. Such equivalence is necessary for exactness in and out of distribution, and extrapolation is governed by this exactness at inference, whatever its realization. Tensor Logic shows this: a zero-temperature contraction is equivalent to discrete logic, deducing in place with no artefact extracted, its tensors Boolean, its embeddings orthonormal, only its arithmetic continuous. Lacking infinite recursion it reaches Datalog, not Prolog, and though exact over closed domains it needs external memory to bind a novel entity. The criterion needs neither a discrete representation nor an extracted expression, and constrains inference, not training: an exact marginal in $[0,1]$ passes, a Neural Network thresholded to a har…
- Kategoria arxiv
- cs.LG
- Kategoria
- Uczenie maszynowe