Porting your code to NLTK 3.0

本文详细介绍了 NLTK 3.0 中的接口更改,包括文本处理方式、语法、解析器、概率计算、分词、命名实体识别等模块的变化。同时,提供了创建对象、操作列表、概率分布、分词、标签、解析等方法的更新说明,以及关于第三方软件环境变量的规范化。此外,文章还讨论了从 Python 2 到 Python 3 的过渡,并概述了 NLTK 3.0 的更多背景信息。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

Original link: https://github.com/nltk/nltk/wiki/Porting-your-code-to-NLTK-3.0

NLTK 3.0 contains a number of interface changes. These are being incorporated into a new version of the NLTK book, updated for Python 3 and NLTK 3.

The way NLTK works with unicode is changed: NLTK3 requires all text input to be unicode and always return text as unicode. Previously, some functions and classes worked on unicode and others required encoded bytestrings. Please make sure you're passing unicode to NLTK and expecting unicode output from NLTK - existing code that assumes bytestrings may start to fail.

Here are some changes you may need to make:

  • grammarContextFreeGrammar → CFGWeightedGrammar → PCFG,StatisticalDependencyGrammar → ProbabilisticDependencyGrammar,WeightedProduction → ProbabilisticProduction
  • draw.treeTreeSegmentWidget.node() → TreeSegmentWidget.label(),TreeSegmentWidget.set_node() → TreeSegmentWidget.set_label()
  • parsers: nbest_parse() → parse()
  • ccg.parse.chartEdgeI.next() → EdgeI.nextsym()
  • Chunk parser: top_node → root_labelchunk_node → chunk_label
  • WordNet properties are now access methods, e.g. Synset.definition →Synset.definition()
  • sem.relextractmk_pairs() → _tree2semi_rel()mk_reldicts() →semi_rel2reldict()show_clause() → clause()show_raw_rtuple() → rtuple()
  • corpusname.tagged_words(simplify_tags=True) →corpusname.tagged_words(tagset='universal')
  • util.clean_html() → BeautifulSoup.get_text()clean_html() is now dropped, install & use BeautifulSoup or some other html parser instead.
  • util.ibigrams() → util.bigrams()
  • util.ingrams() → util.ngrams()
  • util.itrigrams() → util.trigrams()
  • metrics.windowdiff → metrics.segmentation.windowdiff(),metrics.windowdiff.demo() was removed.
  • parse.generate2 was re-written and merged into parse.generate

Creating objects from strings:

  • Many objects now support a fromstring() method
  • tree.Tree.parse() → tree.Tree.fromstring()
  • tree.Tree() → tree.Tree.fromstring()
  • chunk.RegexpChunkRule.parse() → chunkRegexpChunkRule.fromstring()
  • grammar.parse_cfg() → CFG.fromstring() (same for other types of grammar)
  • sem.LogicParser.parse() → sem.Expression.fromstring()
  • sem.DrtParser.parse() → sem.DrtExpression.fromstring()
  • sem.parse_valuation() → sem.Valuation.fromstring()
  • sem.parse_type() → sem.Type.fromstring()

Operations on lists of sentences or other items:

  • tokenize.batch_tokenize() → tokenize.tokenize_sents()
  • tag.batch_tag() → tag.tag_sents()
  • parse.batch_parse() → parse.parse_sents()
  • classify.batch_classify() → classify.classify_many()
  • sem.batch_interpret() → sem.interpret_sents()
  • sem.batch_evaluate() → sem.evaluate_sents()
  • chunk.batch_ne_chunk() → chunk.ne_chunk_sents()

Changes in probability.FreqDist:

  • fdist.keys() → sorted(fdist)
  • fdist.inc(x) → fdist[x] += 1
  • fdist.samples() → sorted(fdist.keys())
  • fdist.Nr(r) → fdist.Nr()[r]
  • fdist.Nr_nonzero() → fdist.Nr().items()
  • cfdist.conditions() → sorted(cfdist.conditions())

Porter stemmer changes:

  • adjust_case()cons()cvc()doublec()m()step1ab()step1c()step2(),step3()step4()step5()vowelinstem() made private
  • ends()r()setto() removed

Removed modules, classes and functions:

Miscellaneous changes:

  • probability.ConditionalProbDist.default_factory now inherits from dict instead of defaultdict
  • probability.ConditionalProbDistI.default_factory now inherits from dict instead of defaultdict
  • probability.DictionaryConditionalProbDist.default_factory now inherits from dictinstead of defaultdict

Environment variables for third-party software:

More background on Python 3 and NLTK 3:

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值