Gumbel Softmax重参数和SF估计（Score Function Estimator，VAE/GAN/Policy Gradient中的重参数）

Gumbel Softmax

We derive the probability density function of the Gumbel-Softmax distribution with probabilities ${\pi }_1,\dots ,{\pi }_k$ and temperature $\tau$. We first define the logits $x_i=\log {\pi }_i$, and Gumbel samples $g_1,\dots ,g_k$, where $g_i\sim \text{Gumbel}(0,1)$ 【$\text{Gumbel}(0,1)$ stands for sampling from uniform distribution $\text{U}(0,1)$ to get $u_i$ first, then $g_i=-log(-log(u_{}$