example for document classify use nltk and python

1. get the movie comment and classify it into pos or neg
    code like below:

点击(此处)折叠或打开

1. >>> import nltki
   
2. >>> import random
   
3. >>> from nltk.corpus import movie_reviews
   
4. >>> documents = [(list(movie_reviews.words(fileid)), category)
   
5. ... for category in movie_reviews.categories()
   
6. ... for fileid in movie_reviews.fileids(category)]
   
7. >>> random.shuffle(documents)

2.get the features of the documents, that if the word in the selected document
    code like below:

点击(此处)折叠或打开

1. >>> all_words = nltk.FreqDist(w.lower() for w in movie_reviews.words())
   
2. >>> word_features = all_words.keys()[:2000]
   
3. >>> def document_features(document):
   
4. ... document_words = set(document)
   
5. ... features = {}
   
6. ... for word in word_features:
   
7. ... features['contains(%s)' % word] = (word in document_words)
   
8. ... return features

3.train and test the classifier for the document
    code like below:

点击(此处)折叠或打开

1. >>> featuresets = [(document_features(d), c) for (d,c) in documents]
   
2. >>> train_set, test_set = featuresets[100:], featuresets[:100]
   
3. >>> classifier = nltk.NaiveBayesClassifier.train(train_set)
   
4. >>> print nltk.classify.accuracy(classifier, test_set)
   
5. 0.73