
spark
viziviuz
这个作者很懒,什么都没留下…
展开
-
pyspark dataframe 列的合并与拆分,单行转多行
test_score = spark.createDataFrame([{'name':'王强','score':'{"语文":119,"数学":125,"外语":120,"物理":96,"化学":70,"生物":40}'}, {'name':'刘明','score':'{"语文":105,"数学":121,"外语":110,...原创 2019-07-23 16:59:03 · 5856 阅读 · 1 评论 -
pyspark 操作
df = spark.createDataFrame([('Natalie', 'F', 16.0,160.0), ('Logan', None,18.0,182.0), ('Joseph', 'M',None,None), ...原创 2019-08-13 15:43:29 · 653 阅读 · 0 评论 -
spark 笔记 pyspark
pyspark.sql.functions 中函数应用 DataFrame 转变 from pyspark import SparkConf, SparkContext from pyspark.sql import SparkSession spark = SparkSession.builder\ .appName('test1') \ .master('loc...原创 2019-08-26 15:59:05 · 661 阅读 · 0 评论