Particle-based energetic variational inference

Yiwei Wang, Jiuhai Chen, Chun Liu, Lulu Kang

Research output: Contribution to journalArticlepeer-review

Abstract

We introduce a new variational inference (VI) framework, called energetic variational inference (EVI). It minimizes the VI objective function based on a prescribed energy-dissipation law. Using the EVI framework, we can derive many existing particle-based variational inference (ParVI) methods, including the popular Stein variational gradient descent (SVGD). More importantly, many new ParVI schemes can be created under this framework. For illustration, we propose a new particle-based EVI scheme, which performs the particle-based approximation of the density first and then uses the approximated density in the variational procedure, or “Approximation-then-Variation” for short. Thanks to this order of approximation and variation, the new scheme can maintain the variational structure at the particle level, and can significantly decrease the KL-divergence in each iteration. Numerical experiments show the proposed method outperforms some existing ParVI methods in terms of fidelity to the target distribution.

Original languageEnglish (US)
Article number34
JournalStatistics and Computing
Volume31
Issue number3
DOIs
StatePublished - May 2021

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Statistics and Probability
  • Statistics, Probability and Uncertainty
  • Computational Theory and Mathematics

Fingerprint

Dive into the research topics of 'Particle-based energetic variational inference'. Together they form a unique fingerprint.

Cite this