谦芊珺

本文写在进行spark学习的过程中，学习过程主要参考 http://blog.youkuaiyun.com/u013719780/article/details/517687201、导包from pyspark import SparkContext2、提交脚本 spark-submit pythonapp.py3、导入数据user_data = sc.textFile('/路径/ml-100k/u.us

特征类型：数值型、类型型、文本型

1、归一化from pyspark.mllib.feature import Normalizernormlizer = Normalizer()vector = sc.parallelize([x])normalized_x_mllib = normlizer.transform(vector).first().toArray()

参考：http://www.cnblogs.com/pinard/p/6364932.html 参考：http://blog.youkuaiyun.com/u013719780/article/details/517750471、导入数据rawData = sc.textFile("/Users/youwei.tan/ml-100k/u.data")2、拆成字段rawRatings = rawData.ma