The pursuit of robust and stable generative models continues to be a central challenge. A recent theoretical advance introduces a novel family of models, termed Gradient Flow Drifting generative models, offering a unifying mathematical framework.
Unifying Generative Dynamics: From Drifting to Gradient Flow
This work reveals a precise mathematical framework that equates the recently proposed Drifting Model with the Wasserstein gradient flow of the forward KL divergence, particularly when densities are approximated via kernel density estimation (KDE). The core insight is that the drifting field in these models directly corresponds to the particle velocity field of the Wasserstein-2 gradient flow of $KL(q|p)$ with KDE-approximated densities, differing only by a bandwidth-squared scaling factor. This equivalence broadens the scope of Gradient Flow Drifting generative models to include MMD-based generators as special cases, arising from Wasserstein gradient flows of various divergences under KDE approximation.
A Principled Approach to Mode Collapse and Blurring
Beyond establishing this equivalence, the researchers present a theoretically grounded mixed-divergence strategy. By combining reverse KL and $χ^2$ divergence gradient flows, these Gradient Flow Drifting generative models can simultaneously address the persistent issues of mode collapse and mode blurring. Furthermore, the framework is extended to Riemannian manifolds, relaxing constraints on kernel functions and enhancing applicability to complex semantic spaces. Preliminary experiments on synthetic benchmarks validate the theoretical underpinnings of this unified approach.
