Markdown草稿纸

该文探讨了实数集合R在信息处理中的作用,特别是涉及到了最小化问题的求解。文章引入了双线性函数和Bilinear矩阵表示,并详细阐述了变分推导和期望最大化算法(EM)在概率模型中的应用。通过拉格朗日乘子法处理正则化项,展示了如何优化目标函数。此外,还讨论了在数据建模和分析中,如概率图模型,如何利用EM算法进行参数估计和KL散度的使用。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

实数集合: R \mathbb R R

a r g    m i n ⏟ p i , q j    ∑ i , j ( m i j − μ − b i − b j − q j T p i – q j T ∣ N ( i ) ∣ − 1 / 2 ∑ s ∈ N ( i ) y s ) 2 + λ ( ∣ ∣ p i ∣ ∣ 2 2 + ∣ ∣ q j ∣ ∣ 2 2 + ∣ ∣ b i ∣ ∣ 2 2 + ∣ ∣ b j ∣ ∣ 2 2 + ∑ s ∈ N ( i ) ∣ ∣ y s ∣ ∣ 2 2 ) \underbrace{arg\;min}_{p_i,q_j}\;\sum\limits_{i,j}(m_{ij}-\mu-b_i-b_j-q_j^Tp_i – q_j^T|N(i)|^{-1/2}\sum\limits_{s \in N(i)}y_{s})^2+ \lambda(||p_i||_2^2 + ||q_j||_2^2 + ||b_i||_2^2 + ||b_j||_2^2 + \sum\limits_{s \in N(i)}||y_{s}||_2^2) pi,qj argmini,j(mijμbibjqjTpiqjTN(i)1/2sN(i)ys)2+λ(pi22+qj22+bi22+bj22+sN(i)ys22)

B i l i n e a r ( x , y ) = x ⊙ W × y + b Bilinear(x,y)=x⊙W\times y+b Bilinear(x,y)=xW×y+b

P ( z ∣ G , S ) log ⁡ P ( G ∣ S ) = log ⁡ ∫ P ( G ∣ z , S ) P ( z ∣ S ) d z = log ⁡ ∫ P ( G ∣ z , S ) P ( z ∣ S ) q ( z ) q ( z ) d z = log ⁡ E z ∼ q ( z ) [ P ( G ∣ z , S ) P ( z ∣ S ) q ( z ) ] ≥ E z ∼ q ( z ) [ log ⁡ P ( G ∣ z , S ) P ( z ∣ S ) q ( z ) ] = E z ∼ q ( z ) [ log ⁡ P ( G ∣ z , S ) ] + E z ∼ q ( z ) [ log ⁡ P ( z ∣ S ) q ( z ) ] = E z ∼ q ( z ) [ log ⁡ P ( G ∣ z , S ) ] − K L [ q ( z ) ∣ ∣ P ( z ∣ S ) ] \begin{aligned} P(z|G,S) \log P(G| S) &= \log \int P(G|z,S)P(z|S)dz \\ &=\log \int P(G|z,S)P(z|S)\frac {q(z)}{q(z)}dz\\ &=\log \mathbb{E}_{z \sim q(z) }\left [ \frac{P(G|z,S)P(z|S)}{q(z)} \right]\\ &\ge \mathbb{E}_{z \sim q(z) }\left [ \log\frac{P(G|z,S)P(z|S)}{q(z)} \right]\\ &=\mathbb{E}_{z \sim q(z) }\left [ \log P(G|z,S) \right]+\mathbb{E}_{z \sim q(z) }\left [ \log\frac{P(z|S)}{q(z)} \right]\\ &=\mathbb{E}_{z \sim q(z) }\left [ \log P(G|z,S) \right]-\rm{KL}\it \left [ q(z)||P(z|S) \right]\\ \end{aligned} P(zG,S)logP(GS)=logP(Gz,S)P(zS)dz=logP(Gz,S)P(zS)q(z)q(z)dz=logEzq(z)[q(z)P(Gz,S)P(zS)]Ezq(z)[logq(z)P(Gz,S)P(zS)]=Ezq(z)[logP(Gz,S)]+Ezq(z)[logq(z)P(zS)]=Ezq(z)[logP(Gz,S)]KL[q(z)P(zS)]

log ⁡ P ( G ∣ S ) ≥ E z ∼ q ( z ) [ log ⁡ P ( G ∣ z , S ) ] − K L [ q ( z ) ∣ ∣ P ( z ∣ S ) ] \begin{aligned} \log P(G| S) &\ge \mathbb{E}_{z \sim q(z) }\left [ \log P(G|z,S) \right]-\rm{KL}\it \left [ q(z)||P(z|S) \right]\\ \end{aligned} logP(GS)Ezq(z)[logP(Gz,S)]KL[q(z)P(zS)]

E z ∼ q ( z ) [ log ⁡ P ( G ∣ z , S ) ] = E ϵ ∼ N ( 0 , 1 ) [ log ⁡ P ( G ∣ μ + ϵ σ , S ) ] \begin{aligned} \mathbb{E}_{z \sim q(z) }\left [ \log P(G|z,S) \right]=\mathbb{E}_{\epsilon \sim N(0,1) }\left [ \log P(G|\mu+\epsilon\sigma,S) \right] \end{aligned} Ezq(z)[logP(Gz,S)]=EϵN(0,1)[logP(Gμ+ϵσ,S)]

∂ J ∂ θ = − 1 m ∑ i = 1 m ∂ J i ∂ θ = 1 m ∑ i = 1 m ( h ( i ) − y ( i ) ) x ( i ) (11) \frac{\partial J}{\partial \boldsymbol \theta}=-\frac1m\sum_{i=1}^m \frac{\partial J_i}{\partial \boldsymbol \theta}=\frac1m\sum_{i=1}^m (h^{(i)}-y^{(i)})\boldsymbol x^{(i)} \tag{11} θJ=m1i=1mθJi=m1i=1m(h(i)y(i))x(i)(11)

l o g ( P ( A ) ) × l e n 0.6 + l o g ( P ( B ) ) ( l e n + 1 ) 0.6 \frac{log(P(A)) \times len^{0.6}+log(P(B))}{(len+1)^{0.6}} (len+1)0.6log(P(A))×len0.6+log(P(B))

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值