
Spark
elite666
这个作者很懒,什么都没留下…
展开
-
Spark集群安装与启动
整个安装过程参考博文: https://blog.youkuaiyun.com/JavaMoo/article/details/77175579 遇到的问题JAVA_HOME is not set ,参考博文: https://blog.youkuaiyun.com/u014052851/article/details/76549451(一)启动Hadoop hadoop启动:只需要再hserver1(nam...原创 2018-08-19 08:51:41 · 368 阅读 · 0 评论 -
windows连接本地centos虚拟机的Pyspark的Jupyter开发环境
1首先安装Scala和spark,参考教程: https://blog.youkuaiyun.com/pucao_cug/article/details/72377219 2安装Python3和Jupyter,参考教程: https://blog.youkuaiyun.com/max_cola/article/details/78902597 3 centos下使Jupyter同时支持Python2和3 htt...原创 2018-08-20 15:22:59 · 600 阅读 · 0 评论 -
Spark_rdd_and_map_reduce基本操作
from __future__ import print_function, divisionfrom pyspark import SparkConf, SparkContextfrom pyspark.sql import SparkSessionspark = SparkSession.builder.master("local") \ .appName("test") \ ...原创 2018-08-21 21:07:15 · 985 阅读 · 0 评论