
spark
kong-kong
记录流水账
展开
-
spark 笔记1
#flatMap val favMovies = sc.parallelize(List("Pulp Fiction","Requiem for a dream","Aclockwork Orange")); favMovies.flatMap(movieTitle=>movieTitle.split(" ")).collect() #sample val data = s...原创 2019-02-23 15:00:36 · 275 阅读 · 0 评论 -
spark单机安装
spark下载地址:http://spark.apache.org/downloads.html #解压: tar -xzvf spark-2.4.0-bin-hadoop2.6.tgz cd spark-2.4.0-bin-hadoop2.6 #执行命名 进入控制台 ./bin/spark-shell var lines = sc.textFile("README...原创 2019-02-20 21:20:14 · 384 阅读 · 0 评论 -
java.net.BindException: 无法指定被请求的地址
单机版启动报错: 2019-02-21 09:46:24 WARN Utils:66 - Service 'sparkDriver' could not bind on a random free port. You may check whether configuring an appropriate binding address. 2019-02-21 09:46:24 WARN ...原创 2019-02-21 09:51:47 · 5474 阅读 · 0 评论 -
scala list
val names = List("a", b", "c") names.map(_.toUpperCase) names.foreach(println) names.foreach{ name => println(name) } # res10: List[Int] = List(1, 2, 1, 3, 4, 3, 5) 数据展开 # flatten可以把嵌套的展开 L...原创 2019-02-22 14:58:27 · 342 阅读 · 0 评论