RegExp and classfier used in part-of-speech(POS) tagging

原创于 2016-01-12 13:02:17 发布 · 557 阅读

CC 4.0 BY-SA版权

文章标签：

18 篇文章

订阅专栏

本文介绍了如何使用正则表达式和分类器进行词性标注，通过模式匹配来判断单词的语法特性。示例中利用NLTK库处理布朗语料库中的新闻类别文本，实现基于后缀的词性标注。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

1. regular expression in pos
judge the characteristic of a certain word by suffix pattern matching

点击(此处)折叠或打开

>>> import nltk
>>> from nltk.corpus import brown
>>> brown_tagged_sents = brown.tagged_sents(categories='news')
>>> brown_sents = brown.sents(categories='news')
>>> patterns = [
... (r'.*ing$', 'VBG'),
... (r'.*ed$', 'VBD'),
... (r'.*es$', 'VBZ'),
... (r'.*ould$','MD'),
... (r'.*\'s$', 'NN$'),
... (r'.*s$', 'NNS'),
... (r'^-?[0-9]+(.[0-9]+)?$', 'CD'),
... (r'.*', 'NN')
... ]
>>> regexp_tagger = nltk.RegexpTagger(patterns)
>>> regexp_tagger.tag(brown_sents[3])[:10]
[(u'``', 'NN'), (u'Only', 'NN'), (u'a', 'NN'), (u'relative', 'NN'), (u'handful', 'NN'), (u'of', 'NN'), (u'such', 'NN'), (u'reports', 'NNS'), (u'was', 'NNS'), (u'received', 'VBD