pLSA图模型笔记

最新推荐文章于 2024-09-09 13:54:49 发布

原创最新推荐文章于 2024-09-09 13:54:49 发布 · 263 阅读

0 ·

CC 4.0 BY-SA版权

nlp 同时被 3 个专栏收录

5 篇文章

订阅专栏

主题模型

2 篇文章

订阅专栏

topicModel

2 篇文章

订阅专栏

pLSA图模型

求解目标p(W,D) ,整个文档集合的出现概率，使得p(W,D) 最大化，即为plsa的目标：

下面咱们开始求解p(W,D)：

$p(W,D) = \prod_{m}^{M} \prod_{n}^{N} p(w_{n},d_{m}) \\=\prod_{i} \prod_{j} p(w_{j},d_{i})^{n(w_{j},d_{i})}$

$n(w_{j},d_{i})$ 为词wj在文档di中出现的次数；

注释： $p(w_{n},d_{m})$ 为第m篇文档中，第n个词出现的概率， $p(w_{j},d_{i})$ 为第i篇文档中词表中第i个词出现的概率，由于di

$log(p(W,D)) =\sum_{i} \sum_{j} n(w_{j},d_{i}) log(p(w_{j},d_{i}) ) \\= \sum_{i} \sum_{j} n(w_{j},d_{i}) log(p(w_{j}|d_{i}) *p(d_{i})) \\\propto \sum_{i} \sum_{j} n(w_{j},d_{i}) log(p(w_{j}|d_{i})) \\= \sum_{i} \sum_{j} n(w_{j},d_{i}) log(\sum_{k=1}^{K} p(w_{j}|z_{k})p(z_{k}|d_{i}) ) \\= \sum_{i} \sum_{j} n(w_{j},d_{i}) \sum_{k=1}^{K} p(w_{j}|z_{k})p(z_{k}|d_{i})$

利用EM算法求 $Q(z_{k})$

$log(p(W,D)) = \sum_{i} \sum_{j} n(w_{j},d_{i}) \sum_{k=1}^{K} \frac{p(w_{j}|z_{k})p(z_{k}|d_{i})}{Q(z_{k})} Q(z_{k}) \\\propto \sum_{i} \sum_{j} n(w_{j},d_{i}) log( \sum_{k=1}^{K} p(w_{j}|z_{k})p(z_{k}|d_{i}) Q(z_{k})) \\\geqslant \sum_{i} \sum_{j} n(w_{j},d_{i}) \sum_{k=1}^{K} Q(z_{k}) log(p(w_{j}|z_{k})p(z_{k}|d_{i})) \\= \sum_{i} \sum_{j} n(w_{j},d_{i}) E_{z}(log(p(w_{j}|z_{k})p(z_{k}|d_{i})))$

$Q(z_{k}) = p(z_{k}|w_{j},d_{i}) \\= \frac{p(z_{k},w_{j},d_{i})}{\sum_{k=1}^{K} p(z_{k},w_{j},d_{i})} \\= \frac{p(z_{k},w_{j},d_{i})}{\sum_{k=1}^{K} p(z_{k},w_{j},d_{i})} \\= \frac{p(w_{j}|z_{k}) p(z_{k}|d_{i})p(d_{i})}{\sum_{k=1}^{K} p(w_{j}|z_{k}) p(z_{k}|d_{i})p(d_{i})} \\= \frac{p(w_{j}|z_{k}) p(z_{k}|d_{i})}{\sum_{k=1}^{K} p(w_{j}|z_{k}) p(z_{k}|d_{i})}$