K-Nearest Neighbors KNN

KNN is a kind of voting algorithm that calculating distance with every instance then choose the K nearest neighbors. Then according to the majority of these K examples, assign the unknown data with this major classification.

Euclidian Distance:

Coding by Machine Learning in Action
KNN.py

#-*-coding:utf-8-*-

from numpy import *
import operator 

def createDataSet():
    group = array([[1.0, 1.1], [1.0, 1.0], [0, 0], [0, 0.1]])
    labels = ['A', 'A', 'B', 'B']

    return group,labels

def classifyKNN0(inX, dataSet, labels, k):
    dataSetSize = dataSet.shape[0]
    print dataSet.shape
    diffMat = tile(inX, (dataSetSize,1)) - dataSet
    sqDiffMat = diffMat**2
    sqDistances = sqDiffMat.sum(axis=1)
    distances = sqDistances**0.5
    sortedDistIndicies = distances.argsort()
    classCount={}
    for i in range(k):
        voteIlabel = labels[sortedDistIndicies[i]]
        classCount[voteIlabel] = classCount.get(voteIlabel,0) + 1
    sortedClassCount = sorted(classCount.iteritems(), key=operator.itemgetter(1), reverse=True)

    return sortedClassCount[0][0]   

supervisedLearning.py

'''
Create on 2015/11/16 9:05:59
@author: Chen Yu in CMU-SYSU    
'''

import KNN

if __name__ == "__main__":
    group,labels = KNN.createDataSet()
    result = KNN.classifyKNN0([0,0], group, labels, 3)

    print result

KNN Realization

readlines：

www.oschina.net/question/558435_67609
http://www.cnblogs.com/qi09/archive/2012/02/10/2344964.html

int()：

http://www.iplaypython.com/jichu/int.html
http://www.cnblogs.com/dreamer-fish/p/3818341.html
http://www.2cto.com/kf/201212/180039.html
http://blog.sina.com.cn/s/blog_629b96210100pbre.html
http://zhidao.baidu.com/question/1638542204376319740.html?qbl=relate_question_1
http://zhidao.baidu.com/link?url=6tE7A7DCWT590TQlXC73nxyZNaZbWGOqHWCBn2RrPxcgHpNdyaB2c6JeOHwBQyxjYpkzHSDE84_YOO4pmpEdqQEghopRcmaDE0q3QXqb13a

zeros：

http://bbs.chinaunix.net/thread-1226344-1-1.html

tile：

http://jingyan.baidu.com/article/219f4bf7da4d8dde442d389e.html

Code For File Read

Plot

subplot(2, 3, 6)

http://www.codeweblog.com/matplotlib-pyplot%E4%B8%ADadd_subplot%E6%96%B9%E6%B3%95%E5%8F%82%E6%95%B0111%E7%9A%84%E5%90%AB%E4%B9%89/