Rescent Advances in Deep Generative Model (4/4) - Score-Based Generative Modeling through Stochastic Differential Equations (SDE) (Connection to Diffusion Models)
10 Aug 2022< 목차 >
- Motivation
- Perturbing data with an SDE
- Reversing the SDE for sample generation
- Estimating the reverse SDE with score-based models and score matching
- Probability flow ODE
- Controllable generation for inverse problem solving
- Connection to Diffusion Models and Others
- Challenges of Score-based Models
- References
본 post 는 앞선 score matching post 와 마찬가지로 Score-Based Generative Modeling through Stochastic Differential Equations (SDE) 논문과 해당 논문의 저자 Yang Song 의 Blog Post 와 Seminar Video를 기반으로 해서 작성 했습니다.
Motivation
앞선 Post 들을 통해 우리는 다른 Generative Model 들과 Score Matching Models 을 비교해보며 어떤 차이가 있는지에 대해 알아봤습니다.
Perturbing data with an SDE
Fig.
Stochastic Process
Ordinary Differential Equation (ODE) and Stochastic Differential Equation (SDE)
Reversing the SDE for sample generation
Fig.
Fig.
Estimating the reverse SDE with score-based models and score matching
How to solve the reverse SDE
Probability flow ODE
Fig.
Controllable generation for inverse problem solving
Connection to Diffusion Models and Others
Denoising Diffusion Implicit Models (DDIM)
Challenges of Score-based Models
- First, the sampling speed is slow since it involves a large number of Langevin-type iterations.
- Second, it is inconvenient to work with discrete data distributions since scores are only defined on continuous distributions.
References
- Papers
- 2019 NIPS, Generative Modeling by Estimating Gradients of the Data Distribution
- 2020 NIPS, Improved Techniques for Training Score-Based Generative Models
- 2021 ICLR, Score-Based Generative Modeling through Stochastic Differential Equations
- 2021 NIPS, Maximum Likelihood Training of Score-Based Diffusion Models
- 2021 ICLR, Denoising Diffusion Implicit Models
- Blogs
- Videos