Week7-5Statistical POS tagging

POS tagging methods

• rule-based
• stochastic
  
  • HMM(generative)
  • Maximum Entropy(discriminative)
• transfer-based

HMM tagging

$$T = \arg \max P(T \mid W)\\ P(T \mid W ) = \frac{P(W \mid T) P(T)}{P(W)}$$

$P(W)$ is ignored, and $P(T)$ is called prior, and $P(W \mid T)$ is likelihood.

$$\begin{align} P(T) P(W \mid T) &= P(t_1, t_2, ..., t_n) P(w_1, w_2, ..., w_n \mid t_1, t_2, ... t_n)\\ &= \prod_{i}^n p(t_i \mid t_1, ..., t_{i-1}) \prod_i^n p(w_i \mid w_1, ..., w_{i-1}, t_1, t_2, ..., t_i) \end{align}$$

• Simplification 1
  
  • $P(W \mid T) = \prod P(w_i \mid t_i)$
    -Simplification 2
  • $P(T) = \prod P(t_i \mid t_{i-1})$
• Bigram approximation
  
  • $T = \arg \max P(T \mid W) = \prod P(w_i \mid t_i) P(t_i \mid t_{i-1})$

Evaluating taggers

• Data set
  
  • Training set
  • Development set
  • Test set
• Tagging accuracy

Transformation-based learning

Thoughts about POS taggers

• New domains
  
  • Lower performance
• Distributional Clustering