Black&White的专栏

firstdef first(): Tfirst返回RDD中的第一个元素，不排序。scala> var rdd1 = sc.makeRDD(Array(("A","1"),("B","2"),("C","3")),2)rdd1: org.apache.spark.rdd.RDD[(String, String)] = ParallelCollectionRDD[33] at mak...

partitionBydef partitionBy(partitioner: Partitioner): RDD[(K, V)]该函数根据partitioner函数生成新的ShuffleRDD，将原RDD重新分区。scala> var rdd1 = sc.makeRDD(Array((1,"A"),(2,"B"),(3,"C"),(4,"D")),2)rdd1: org.apac...

mapmap是对RDD中的每个元素都执行一个指定的函数来产生一个新的RDD。 任何原RDD中的元素在新RDD中都有且只有一个元素与之对应。举例：scala> val a = sc.parallelize(1 to 9, 3)scala> val b = a.map(x => x*2)scala> a.collectres10: Array[Int] = Arra...

从spark官网看spark特性。spark是apache的顶级项目，所以官网地址spark加上apache.comhttp://spark.apache.org/1、spark是什么Apache Spark™ is a unified analytics engine for large-scale data processing.spark是针对于大规模数据处理的统一分析引擎。sp...