Title: Variational Diffusion Autoencoders with Random Walk Sampling

Abstract: Variational inference methods and especially variational autoencoders (VAEs) specify scalable generative models that enjoy an intuitive connection to manifold learning, and can be viewed as an approximate homeomorphism between the data manifold and a latent Euclidean space. However, these approximations are well-documented to degenerate in training. Conversely, diffusion maps automatically infer the data topology and enjoy a rigorous connection to manifold learning, but do not scale easily or provide the inverse homeomorphism. We propose a) a principled measure for recognizing the mismatch between data and latent distributions and b) a method that combines the advantages of VAEs and diffusion maps to learn a homeomorphic generative model. The measure, the locally bi-Lipschitz property, is a sufficient condition for a homeomorphism and easy to compute and interpret. Our method, the variational diffusion autoencoder, is a novel generative algorithm that infers the topology of the data distribution, then models a random walk over the data. For efficient computation, we use stochastic versions of both variational inference and manifold learning optimization. We prove approximation theoretic results for the dimension dependence, and that locally isotropic sampling in the latent space results in a random walk over the reconstructed manifold. We also demonstrate our method on real and synthetic datasets, and show that it exhibits performance superior to other generative models.

Time: June 17, 2020, 1:00pm – 1:30pm (eastern)