
spark
文章平均质量分 80
ZYJ_2012
这个作者很懒,什么都没留下…
展开
-
spark入门wordcount详解(JAVA)
maven依赖: org.apache.spark spark-core_2.10 1.6.0public class WordCount { public static void main(String[] args) { /** * 创建spark配置对象SparkConf,设置spark运行时配置信息,原创 2017-03-02 09:59:10 · 877 阅读 · 0 评论 -
Spark编写UDAF自定义函数(JAVA)
maven: org.apache.spark spark-core_2.10 1.6.0 org.apache.spark spark-sql_2.10 1.6.0 org.apache.spark spark-hive_2.10 1.6.0工具类 --> com.google.guava原创 2017-03-21 14:51:02 · 3372 阅读 · 0 评论 -
Spark编写UDF自定义函数(JAVA)
maven: org.apache.spark spark-core_2.10 1.6.0 org.apache.spark spark-sql_2.10 1.6.0 org.apache.spark spark-hive_2.10 1.6.0工具类 --> com.google.guava原创 2017-03-21 13:25:47 · 8780 阅读 · 0 评论 -
sparksql分组后topN(JAVA)
org.apache.spark spark-core_2.10 1.6.0 org.apache.spark spark-sql_2.10 1.6.0 org.apache.spark spark-hive_2.10 1.6.0public class RowNumberTopN { public static原创 2017-03-21 10:56:17 · 1519 阅读 · 0 评论 -
sparkSQL用jdbc访问mysql
maven: org.apache.spark spark-core_2.10 1.6.0 org.apache.spark spark-sql_2.10 1.6.0工具类 --> com.google.guava guava 18.0public class JDBCDataSource {原创 2017-03-17 17:51:18 · 3009 阅读 · 0 评论 -
spark对分组后value值进行排序(JAVA)
maven: org.apache.spark spark-core_2.10 1.6.0groupsort.txt:spark 100storm 90kafka 75hadoop 60zookeeper 100impala 80hbase 65hive 90flume 95elasticsearch 100sp原创 2017-03-03 13:24:58 · 1800 阅读 · 0 评论 -
spark二次排序简单例子(JAVA)
maven依赖: org.apache.spark spark-core_2.10 1.6.0public class SecondarySortBean implements Ordered,Serializable { private int first; private int second; public SecondarySo原创 2017-03-02 16:55:07 · 479 阅读 · 0 评论 -
spark入门cogroup简单例子(JAVA)
maven依赖: org.apache.spark spark-core_2.10 1.6.0public class CoGroup { public static void main(String[] args) { /** * 创建spark配置对象SparkConf,设置spark运行时配置信息,原创 2017-03-02 13:45:49 · 754 阅读 · 0 评论 -
spark自定义Accumulator高级应用(JAVA)
public class SessionAggrStatAccumulator implements AccumulatorParam { private static final long serialVersionUID = 6311074555136039130L; /** * zero方法,其实主要用于数据的初始化 * 那么,我们这里,就返回一个值,就是原创 2017-03-24 16:45:43 · 2743 阅读 · 3 评论