NLP基础-CBOW&DEEP CBOW-影评分类

本文使用方法来自CMU课程《Neural Networks for NLP》

---

1. CBOW Continuous bag of words

2. 语言python

3. 框架dynet

ps. 英文注解写得不好，望见谅

---

CBOW图解

-CBOW
参照前一篇BOW，不同点在于
1. word embedding这里是定义了64维，丰富地表达词信息，BOW是直接以标签种类为维度的
2. 因为维度变了，所以计算方法有点变化，需要一个weight把word embedding之和转换为一个以标签种类为维度的结果。

# Define the model
EMB_SIZE = 64
W_emb = model.add_lookup_parameters((nwords, EMB_SIZE)) # Word embeddings
W_sm = model.add_parameters((ntags, EMB_SIZE))          # Softmax weights
b_sm = model.add_parameters((ntags))                      # Softmax bias

# A function to calculate scores for one value
def calc_scores(words):
  dy.renew_cg()
  cbow = dy.esum([dy.lookup(W_emb, x) for x in words])
  W_sm_exp = dy.parameter(W_sm)
  b_sm_exp = dy.parameter(b_sm)
  return W_sm_exp * cbow + b_sm_exp

DEEP CBOW图解

-DEEP CBOW
参照CBOW，不同点在于
1. 拥有中间层（hidden layer），这是deep的底线
2. 利用非线性方程计算，能够得到词组的含义

# Define the model
EMB_SIZE = 64
HID_SIZE = 64
# The deep model has two layers
HID_LAY = 2
W_emb = model.add_lookup_parameters((nwords, EMB_SIZE)) # Word embeddings
# The weight consists of two tuples
W_h = [model.add_parameters((HID_SIZE, EMB_SIZE if lay == 0 else HID_SIZE)) for lay in range(HID_LAY)]
b_h = [model.add_parameters((HID_SIZE)) for lay in range(HID_LAY)]
W_sm = model.add_parameters((ntags, HID_SIZE))   # Softmax weights
b_sm = model.add_parameters((ntags))       # Softmax bias

# A function to calculate scores for one value
def calc_scores(words):
  dy.renew_cg()
  h = dy.esum([dy.lookup(W_emb, x) for x in words])
  # The first weight with the first bias and ...
  for W_h_i, b_h_i in zip(W_h, b_h):
    h = dy.tanh( dy.parameter(W_h_i) * h + dy.parameter(b_h_i) )
  return dy.parameter(W_sm) * h + dy.parameter(b_sm)