Gumbel Softmax
We derive the probability density function of the Gumbel-Softmax distribution with probabilities π1,…,πk\pi_1, \ldots, \pi_kπ1,…,πk and temperature τ\tauτ. We first define the logits xi=logπix_i = \log \pi_ixi=logπi, and Gumbel samples g1,…,gkg_1, \ldots, g_kg1,…,gk, where gi∼Gumbel(0,1)g_i \sim \text{Gumbel}(0, 1)gi∼Gumbel(0,1) 【Gumbel(0,1)\text{Gumbel}(0, 1)Gumbel(0,1) stands for sampling from uniform distribution U(0,1)\text{U}(0,1)U(0,1) to get uiu_iui first, then gi=−log(−log(ui))g_i=-log(-log(u_i))gi=−log(−log(u
Gumbel Softmax重参数与SF估计解析

最低0.47元/天 解锁文章
599

被折叠的 条评论
为什么被折叠?



