
spark学习
会发paper的学渣
Computer knows how to connect zero with one,Mathmatic get how to skip from one to infinity
展开
-
sparkstreaming消费kafka
1、安装pip install pyspark==2.4.4, 具体spark环境搭建如下:https://blog.youkuaiyun.com/sslfk/article/details/123899383 2、将spark-streaming-kafka包放入python环境的Lib\site-packages\pyspark\jars中,关于spark-streaming-kafka包可以从https://download.youkuaiyun.com/download/sslfk/85115900 3、代码调用方式:原创 2022-04-12 15:04:07 · 578 阅读 · 1 评论 -
spark sql 和常用方法汇总
spark sql 学习: def main(args: Array[String]): Unit = { val sparksession = SparkSession.builder().appName("HBaseTest").master("local").getOrCreate() val sc = sparksession.sparkContext //设置日...原创 2021-12-07 14:06:35 · 276 阅读 · 1 评论 -
spark 学习 基础篇
http://blog.youkuaiyun.com/u013063153/article/details/52988241https://aiyanbo.gitbooks.io/spark-programming-guide-zh-cn/content/graphx-programming-guide/pregel-api.html转载 2018-03-02 14:33:21 · 151 阅读 · 0 评论