spark环境搭建

1. spark源码编译

（1）地址https://archive.apache.org/dist/spark/spark-2.4.0/spark-2.4.0-bin-sources.tgz

2. 编译

参考官网http://spark.apache.org/docs/latest/building-spark.html

3. 执行下面的命令进行编译

./dev/make-distribution.sh --name 2.6.0-cdh5.7.0 --tgz -Pyarn -Phadoop-2.6 -Phive -Phive-thriftserver -Dhadoop.version=2.6.0-cdh5.7.0

4. 对生成的tgz包进行安装

5. 测试

 ./bin/spark-shell

6. 上面搭建的是本地环境，后面再总结spark on yarn