
spark
文章平均质量分 77
iteye_8118
这个作者很懒,什么都没留下…
展开
专栏收录文章
- 默认排序
- 最新发布
- 最早发布
- 最多阅读
- 最少阅读
-
编译Spark1.5.1源码
[root@cluster spark-1.5.1]# mvn -Dhadoop.version=2.6.0-cdh5.4.0 -Dscala-2.10.4 -Phive -Dmaven.test.skip=true clean package [INFO] Replacing original artifact with shaded artifact. [INFO] Replac...原创 2015-12-15 14:51:27 · 221 阅读 · 0 评论 -
【转】Scala的协变和逆变
转自:http://my.oschina.net/scipio/blog/282794 一、概念 协变和逆变是描述在集合中的多态关系与原来类型多态关系之间的关系。如果T1是T的子类,那么Container[T1]是不是Container[T]的子类呢?Variance注释允许你在类继承和多态类型之间表达下面的这些关系: * covariant(协变),[+T],C...原创 2016-01-11 20:56:58 · 178 阅读 · 0 评论 -
编译基于Hadoop2.6的Spark1.6源码
...... [INFO] Including org.spark-project.spark:unused:jar:1.0.0 in the shaded jar. [WARNING] hadoop-yarn-common-2.6.0.jar, hadoop-yarn-api-2.6.0.jar define 3 overlapping classes: [WARNING] - o...原创 2016-01-12 14:50:55 · 344 阅读 · 0 评论 -
Spark2.0版本神器Structured Streaming
Spark2.0.0神器原创 2016-07-28 17:35:41 · 261 阅读 · 0 评论 -
SparkStreaming定时器生成Job
/** * An interface to represent clocks, so that they can be mocked out in unit tests. */ private[spark] trait Clock { def getTimeMillis(): Long def waitTillTime(targetTime: Long): Long } ...原创 2016-11-15 18:47:11 · 884 阅读 · 0 评论