Skip to yearly menu bar Skip to main content


Poster
in
Workshop: The Impact of Memorization on Trustworthy Foundation Models

Early-stopping Too Late? Traces of Memorization Before Overfitting in Generative Diffusion

Jerome Garnier-Brun · Luca Biggio · Marc Mezard · Luca Saglietti

[ ] [ Project Page ]
Sat 19 Jul 8:30 a.m. PDT — 9:30 a.m. PDT

Abstract:

In generative diffusion, early stopping is widely adopted as a criterion for minimizing the distance between the generated and target distributions. Yet, this benefit comes with no explicit guarantee against memorization. In this work, we study the distributional fidelity of denoising diffusion probabilistic models in a controlled setup, a hierarchical data model with tractable scores and marginal probabilities. Tracking the generative behavior through training, we identify a biased generalization phase preceding the minimum of the test loss, where the model increasingly favors samples with anomalously high overlap to training data, without yet reproducing them exactly.Our results highlight a subtle failure mode of diffusion training dynamics, suggesting that standard early stopping might be insufficient to prevent distorted generalization, well before the emergence of overt memorization.

Chat is not available.