论文笔记：Neural Architectures for Named Entity Recognition

Neural Architectures for Named Entity Recognition

2.3 CRF Tagging Models

• Input sentence: $\mathbf{X}=({\mathbf{X}}_1,{\mathbf{X}}_2,...,{\mathbf{X}}_{\mathbf{N}})$

• Matrix of scores (output by BiLSTM) : $\mathbf{P}\in {\mathbb{R}}^{n\times k}$

  -Score of the $j^{th}$ of $i^{th}$word in a sentence : $P_{i,j}$

• The numbers of distinct tags: $k$

• Sequence of predictions: $\mathbf{y}=(y_1,y_2,...,y_n)$

$\mathbf{y}$'s score: $s(\mathbf{X},\mathbf{y})=\sum _{i=0}^n{A}_{y_i,y_{i+1}}+\sum _{i=0}^n{P}_{i,y_i}$

• Matrix of transition scores: $A\in {\mathbb{R}}^{(k+2)\times (k+2)}$

  • score of a transition from the tag $i$ to tag $j$: $A_{i,j}$
  • start and end tag: $y_0,y_n$

Probability for the sequence $\mathbf{y}$:

$p(\mathbf{y}|\mathbf{X})=\frac{e^{s(\mathbf{X},\mathbf{y})}}{{\sum }_{\tilde{y}}\in {\mathbf{Y}}_{\mathbf{x}}{e}^{s(\mathbf{X},\tilde{\mathbf{y}})}}$

Maximize the log_probability of the correct tag sequence:

$log(p(\mathbf{y}|\mathbf{X}))=s(\mathbf{X},\mathbf{y})-log(\sum _{\tilde{y}\in {\mathbf{Y}}_{\mathbf{x}}}{e}^{s(\mathbf{X},\tilde{\mathbf{y}}})=s(\mathbf{X},\mathbf{y})-{logadd}_{\tilde{y}\in {\mathbf{Y}}_{\mathbf{x}}}s(\mathbf{X},\tilde{\mathbf{y}})$

• All possible tag sequences for a sentence $\mathbf{X}$: ${\mathbf{Y}}_{\mathbf{X}}$

Predict the output sequence by:

${\mathbf{y}}^{\ast }=argma{x}_{\tilde{y}\in {\mathbf{Y}}_{\mathbf{X}}}s(\mathbf{X},\tilde{\mathbf{y}})$