论文阅读笔记：Deep Unsupervised Learning using Nonequilibrium Thermodynamics

1、来源

论文连接1：http://ganguli-gang.stanford.edu/pdf/DeepUnsupDiffusion.pdf
论文连接2(带appendix)：https://arxiv.org/pdf/1503.03585v7
代码链接：https://github.com/Sohl-Dickstein/Diffusion-Probabilistic-Models
代码的环境配置(基于theano)参考：https://blog.csdn.net/u010948546/article/details/146217516?spm=1001.2014.3001.5501

2、论文推理过程

扩散模型的流程如下图所示，可以看出$q(x^{0,1,2\cdots ,T-1,T})$为正向加噪音过程，$p(x^{0,1,2\cdots ,T-1,T})$为逆向去噪音过程，具体过程参考https://blog.csdn.net/u010948546/article/details/144902864?spm=1001.2014.3001.5501。可以看出，逆向去噪的末端得到的图上还散布一些噪点。

2.1、名词解释

$q(x^0)$：$x^0$ 表示数据集的图像分布，例如在使用MNIST数据集时，$x^0$就表示MNIST数据集中的图像，而$q(x^0)$就表示数据集MNIST中数据集的分布情况。
$p(x^T)$：$x^T$表示$x^0$的加噪结果，$x^T$是逆向去噪的起点，因此$p(x^T)$是去噪起点的分布情况。与$\pi (x^T)$相同。
值得注意的是$p(x^t)$与$q(x^t)$是相同的。

2.2、推理过程

正向加噪过程满足马尔可夫性质，因此有公式1。

$$\begin{array}{c}\begin{array}{rl}q(x^{0,1,2\cdots ,T-1,T})& =q(x^0)\cdot \prod _{t=1}^{T}q(x^t|x^{t-1})=q(x^0)\cdot q(x^1|x^0)\cdot q(x^2|x^1)\dots q(x^T|x^{T-1}).\\ q(x^{1,2\cdots T}|x^0)& =q(x^1|x^0)\cdot q(x^2|x^1)\dots q(x^T|x^{T-1})).\end{array}\end{array}$$

逆向去噪过程如公式2。

$$\begin{array}{c}{p}_{\theta }(x^{0,1,2\cdots ,T-1,T})=p_{\theta }(x^T)\cdot \prod _{t=1}^T{p}_{\theta }(x^{t-1}|x^t)=p_{\theta }(x^T)\cdot p_{\theta }(x^{T-1}|x^T)\cdot p_{\theta }(x^{T-2}|x^{T-1})\dots p_{\theta }(x^0|x^1).\end{array}$$
公式2中的参数$\theta$就是深度学习模型中需要学习的参数。为了方便，省略公式2中的$\theta$，因此公式2被重写为公式3。
$$\begin{array}{c}p(x^{0,1,2\cdots ,T-1,T})=p(x^T)\cdot \prod _{t=1}^{T}p(x^{t-1}|x^t)=p(x^T)\cdot p(x^{T-1}|x^T)\cdot p(x^{T-2}|x^{T-1})\dots p(x^0|x^1).\end{array}$$

逆向去噪的目标是使得其终点与正向加噪的起点相同。也就是使得$p(x^0)$最大，即使得 逆向去噪过程为$x^0$的概率最大。

$$\begin{array}{c}\begin{array}{rl}p(x^0)& =\int p(x^0,x^1)d{x}^1(联合分布概率公式)\\ & =\int p(x^1)\cdot p(x^0|x^1)d{x}^1(贝叶斯概率公式)\\ & =\int \int p(x1,x2)d{x}^2\cdot p(x^0|x^1)d{x}^1(积分套积分)\\ & =\int \int p(x^2)\cdot p(x^1|x^2)\cdot p(x^0|x^1)d{x}^{1}d{x}^2(改写为二重积分)\\ & =\int \int p(x^2)\cdot p(x^1|x^2)\cdot p(x^0|x^1)d{x}^{1}d{x}^2\\ & =⋮\\ & =\int \int \cdots \int p(x^T)\cdot p(x^{T-1}|x^T)\cdot p(x^{T-2}|x^{-1})\cdots p(x^0|x^1)\cdot d{x}^{1}d{x}^2\cdots d{x}^T\\ & =\int p(x^{0,1,2\cdots T})d{x}^{1,2\cdots T}(T-1重积分)\\ & =\int d{x}^{1,2\cdots T}\cdot p(x^{0,1,2\cdots T})\cdot \frac{q(x^{1,2\cdots T}|x^0)}{q(x^{1,2\cdots T}|x^0)}\\ & =\int d{x}^{1,2\cdots T}\cdot q(x^{1,2\cdots T}|x^0)\cdot \frac{p(x^{0,1,2\cdots T})}{q(x^{1,2\cdots T}|x^0)}\\ & =\int d{x}^{1,2\cdots T}\cdot q(x^{1,2\cdots T}|x^0)\cdot \frac{p(x^T)\cdot p(x^{T-1}|x^T)\cdot p(x^{T-2}|x^{T-1})\dots p(x^0|x^1)}{q(x^1|x^0)\cdot q(x^2|x^1)\dots q(x^T|x^{T-1})}\\ & =\int d{x}^{1,2\cdots T}\cdot q(x^{1,2\cdots T}|x^0)\cdot p(x^T)\cdot \frac{p(x^{T-1}|x^T)\cdot p(x^{T-2}|x^{T-1})\dots p(x^0|x^1)}{q(x^1|x^0)\cdot q(x^2|x^1)\dots q(x^T|x^{T-1})}\\ & =\int d{x}^{1,2\cdots T}\cdot q(x^{1,2\cdots T}|x^0)\cdot p(x^T)\cdot \prod _{t=1}^T\frac{p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}\\ & =E_{x^{1,2,\cdots T}\sim q(x^{1,2\cdots T}|x^0)}p(x^T)\cdot \prod _{t=1}^T\frac{p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}(改写为期望的形式)\end{array}\end{array}$$
因此公式3中的参数$\theta$应满足
$$\begin{array}{c}\theta =arg\underset{\theta }{\max }p(x^0).\end{array}$$
公式4是对数据集中的一张图片进行求解，然而数据集中通常是有成千上万张图像的。假设数据集中有$N$张图像，因此有公式6，其目的是求得一组参数$\theta$，使得 $L$取得最大值。值得注意的是$q(x^0)$表示数据集中每张图片被采样出来的概率。
$$\begin{array}{c}\begin{array}{rl}L& =\sum _{n=0}^{N}q(x^0)\cdot log(p(x^0))\\ & =\int d{x}^0\cdot q(x^0)\cdot log(p(x^0))\\ & =\int d{x}^0\cdot q(x^0)\cdot log[E_{x^{1,2,\cdots T}\sim q(x^{1,2\cdots T}|x^0)}p(x^T)\cdot \prod _{t=1}^T\frac{p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}]\\ & \ge \int d{x}^0\cdot q(x^0)\cdot E_{x^{1,2,\cdots T}\sim q(x^{1,2\cdots T}|x^0)}log[p(x^T)\cdot \prod _{t=1}^T\frac{p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}]\\ & =\int d{x}^0\cdot q(x^0)\int q(x^{1,2\cdots T}|x^0)\cdot log[p(x^T)\cdot \prod _{t=1}^T\frac{p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}]\cdot d{x}^{1,2\cdots T}\\ & =\int d{x}^{0,1,2\cdots T}q(x^0)\cdot q(x^{1,2\cdots T}|x^0)\cdot log[p(x^T)\cdot \prod _{t=1}^T\frac{p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}]\\ & =\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log[p(x^T)\cdot \prod _{t=1}^T\frac{p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}]\\ & =\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log[\prod _{t=1}^T\frac{p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}]+\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log[p(x^T)]\\ & =K\end{array}\end{array}$$

因此有公式
$$\begin{array}{c}\begin{array}{rl}K& =\underset{K1}{\underbrace{\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log[\prod _{t=1}^T\frac{p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}]}}+\underset{K_2}{\underbrace{\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log[p(x^T)]}}\\ & =K_1+K_2\end{array}\end{array}$$

首先考虑 $K$中的第二项$K_2$。
$$\begin{array}{c}\begin{array}{rl}{K}_2& =\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log[p(x^T)]\\ & =\int q(x^0)\cdot q(x^1|x^0)\cdot q(x^2|x^1)\cdots q(x^T|x^{T-1})\cdot log[p(x^T)]\cdot d{x}^{0}d{x}^1\cdots d{x}^T\\ & =\int (\int q(x^1,x^0)\cdot d{x}^0)\cdot q(x^2|x^1)\cdots q(x^T|x^{T-1})\cdot log[p(x^T)]\cdot d{x}^1\cdots d{x}^T\\ & =\int q(x^1)\cdot q(x^2|x^1)\cdots q(x^T|x^{T-1})\cdot log[p(x^T)]\cdot d{x}^1\cdots d{x}^T\\ & =\int q(x^T)\cdot log[p(x^T)]\cdot d{x}^T\\ & =\int p(x^T)\cdot log[p(x^T)]\cdot d{x}^T\\ & =-H_p(x^T)\end{array}\end{array}$$
$p(x^T)$是一个均值为0，方差为1的高斯分布。参考【正态分布系列】正态分布的熵，可以计算出$K_2$如下所示。
$$\begin{array}{c}\begin{array}{rl}{K}_2& =-H_p(x^T)\\ & =-(\frac{1}{2}log[2\pi {\sigma }^2]+\frac{1}{2})\\ & =-(\frac{1}{2}log[2\pi ]+\frac{1}{2})\end{array}\end{array}$$
在代码中的计算过程如下图红框所示。

接下来考虑$K_1$。值得注意的是，论文中说明，为了避免边界效应，因此强迫 $p(x^0|x^1)=q(x^1|x^0)$。

$$\begin{array}{c}\begin{array}{rl}{K}_1& =\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log\left[\prod _{t=1}^T\frac{p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}\right]\\ & =\sum _{t=1}^T\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log\left[\frac{p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}\right]\\ & =\sum _{t=2}^T\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log\left[\frac{p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}\right]+\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log\left[\frac{p(x^0|x^1)}{q(x^1|x^0)}\right]\\ & =\sum _{t=2}^T\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log\left[\frac{p(x^{t-1}|x^t)}{q(x^t|x^{t-1})}\right]\\ & =\sum _{t=2}^T\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log[\frac{p(x^{t-1}|x^t)}{q(x^{t-1}|x^t)}\cdot \frac{q(x^{t-1})}{q(x^t)}]\\ & =\sum _{t=2}^T\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log[\frac{p(x^{t-1}|x^t)}{q(x^{t-1}|x^t,x^0)}\cdot \frac{q(x^{t-1}|x^0)}{q(x^t|x^0)}]\\ & =\sum _{t=2}^T\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log\left[\frac{p(x^{t-1}|x^t)}{q(x^{t-1}|x^t,x^0)}\right]+\sum _{t=2}^T\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log\left[\frac{q(x^{t-1}|x^0)}{q(x^t|x^0)}\right]\\ & =\sum _{t=2}^T\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log\left[\frac{p(x^{t-1}|x^t)}{q(x^{t-1}|x^t,x^0)}\right]+\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log\left[\prod _{t=2}^T\frac{q(x^{t-1}|x^0)}{q(x^t|x^0)}\right]\\ & =\sum _{t=2}^T\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log\left[\frac{p(x^{t-1}|x^t)}{q(x^{t-1}|x^t,x^0)}\right]+\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log\left[\frac{q(x^1|x^0)}{q(x^T|x^0)}\right]\\ & =\sum _{t=2}^T\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log\left[\frac{p(x^{t-1}|x^t)}{q(x^{t-1}|x^t,x^0)}\right]+\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log[q(x^1|x^0)]-\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log[q(x^T|x^0)]\\ & =\sum _{t=2}^T\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log\left[\frac{p(x^{t-1}|x^t)}{q(x^{t-1}|x^t,x^0)}\right]+\int d{x}^{0}d{x}^1\cdots d{x}^T\cdot q(x^0)\cdot q(x^1|x^0)\cdot q(x^2|x^1)\cdots q(x^T|x^{T-1})\cdot log[q(x^1|x^0)]-\int d{x}^{0,1,2\cdots T}\cdot q(x^{0,1,2\cdots T})\cdot log[q(x^T|x^0)]\end{array}\end{array}$$