<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[H.king的博客]]></title><description><![CDATA[hadoop ->spark]]></description><link>https://blog.youkuaiyun.com/github_37643896</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; github_37643896]]></copyright><item><title><![CDATA[MongoDB 中文手册 | MongoDB 中文文档 | 官方手册中文版翻译]]></title><link>https://blog.youkuaiyun.com/github_37643896/article/details/108055261</link><guid>https://blog.youkuaiyun.com/github_37643896/article/details/108055261</guid><author>github_37643896</author><pubDate>Mon, 17 Aug 2020 14:35:36 +0800</pubDate><description><![CDATA[最近做了一些 MongoDB 官方文档的汉化工作 。


项目放在github 上 ，已经开源 。

我相信有很多同学都有汉化文档的经历，但是往往不能坚持到最后，或者因为工作量及时间安排的问题，不能坚持。在这里 可以合作翻译，一起分享知识点，做自己擅长的领域 。

稍后将放出连接 欢迎有兴趣的同学 一起参与 。

预览链接请点击Mongodb 中文手册


...]]></description><category></category></item><item><title><![CDATA[flink  streamWordCount  demo]]></title><link>https://blog.youkuaiyun.com/github_37643896/article/details/106119113</link><guid>https://blog.youkuaiyun.com/github_37643896/article/details/106119113</guid><author>github_37643896</author><pubDate>Thu, 14 May 2020 14:22:16 +0800</pubDate><description><![CDATA[flink 的 wordCount 流式计算的demo




package com.hbn.wc

import org.apache.flink.api.scala._
import org.apache.flink.streaming.api.scala.{DataStream, StreamExecutionEnvironment}

object StreamWordCount {

  def main(args: Array[String]): Unit = {


    val  e..]]></description><category></category></item><item><title><![CDATA[flink  wordcount  demo]]></title><link>https://blog.youkuaiyun.com/github_37643896/article/details/106117297</link><guid>https://blog.youkuaiyun.com/github_37643896/article/details/106117297</guid><author>github_37643896</author><pubDate>Thu, 14 May 2020 12:15:11 +0800</pubDate><description><![CDATA[demo


package com.hbn.wc

import org.apache.flink.api.scala.{DataSet, ExecutionEnvironment}

import org.apache.flink.api.scala._


object WordCount {
  def main(args: Array[String]): Unit = {

    //获取执行环境
    val env = ExecutionEnvironment.getExecutionE.]]></description><category></category></item><item><title><![CDATA[flink  Cannot find compatible factory for specified execution.target (=local)]]></title><link>https://blog.youkuaiyun.com/github_37643896/article/details/106117216</link><guid>https://blog.youkuaiyun.com/github_37643896/article/details/106117216</guid><author>github_37643896</author><pubDate>Thu, 14 May 2020 12:07:48 +0800</pubDate><description><![CDATA[初入 flink wordcount  报错如下




/Library/Java/JavaVirtualMachines/jdk1.8.0_201.jdk/Contents/Home/bin/java "-javaagent:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar=56188:/Applications/IntelliJ IDEA.app/Contents/bin" -Dfile.encoding=UTF-8 -clas...]]></description><category></category></item><item><title><![CDATA[flink  隐式转换的问题]]></title><link>https://blog.youkuaiyun.com/github_37643896/article/details/106116173</link><guid>https://blog.youkuaiyun.com/github_37643896/article/details/106116173</guid><author>github_37643896</author><pubDate>Thu, 14 May 2020 11:43:26 +0800</pubDate><description><![CDATA[初入 flink 报错如下 


Information:2020-05-14 11:18 - Build completed with 2 errors and 0 warnings in 3 s 77 ms
Error:(15, 60) could not find implicit value for evidence parameter of type org.apache.flink.api.common.typeinfo.TypeInformation[String]
    val re...]]></description><category></category></item><item><title><![CDATA[Java  操作 Spark  时  却不能设置泛型参数的问题 ？]]></title><link>https://blog.youkuaiyun.com/github_37643896/article/details/100711180</link><guid>https://blog.youkuaiyun.com/github_37643896/article/details/100711180</guid><author>github_37643896</author><pubDate>Tue, 10 Sep 2019 20:23:21 +0800</pubDate><description><![CDATA[最近在使用Java开发spark 过程中 遇到一个问题 即不能设置 返回对象的泛型参数

代码如下

jssc 为 JavaStreamingContext

使用 JavaSparkContext 也会遇到这种情况


List&lt;Tuple2&lt;String, Boolean&gt;&gt; blacklist = new ArrayList&lt;Tuple2&lt;Str...]]></description><category></category></item><item><title><![CDATA[Spark  sql  中 自定义实体类无法识别其中的 Date  字段]]></title><link>https://blog.youkuaiyun.com/github_37643896/article/details/100552691</link><guid>https://blog.youkuaiyun.com/github_37643896/article/details/100552691</guid><author>github_37643896</author><pubDate>Thu, 05 Sep 2019 10:19:35 +0800</pubDate><description><![CDATA[在使用过程中 创建了实体类

使用反射的方式将 rdd 转换为 df

但是在使用过程中 即使没有在 查询中使用该字段，依然会报错

主要提示为 `scala.MatchError: Thu Sep 05 10:16:25 CST 2019 (of class java.util.Date) `

具体内容如下


19/09/05 10:16:30 ERROR Executor: ...]]></description><category></category></item><item><title><![CDATA[spark  sql  使用过程中 无法识别输入字段]]></title><link>https://blog.youkuaiyun.com/github_37643896/article/details/100549490</link><guid>https://blog.youkuaiyun.com/github_37643896/article/details/100549490</guid><author>github_37643896</author><pubDate>Thu, 05 Sep 2019 10:12:52 +0800</pubDate><description><![CDATA[在使用spark sql 过程中 可能映射为实体类

但是该实体类的属性 ，并没有设置getter setter 方法 就会出现如下问题

提示 ： 无法识别某些字段


/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/bin/java "-javaagent:/Applications/Intelli...]]></description><category></category></item><item><title><![CDATA[spark  sqlcontext  读取json  文件]]></title><link>https://blog.youkuaiyun.com/github_37643896/article/details/100183474</link><guid>https://blog.youkuaiyun.com/github_37643896/article/details/100183474</guid><author>github_37643896</author><pubDate>Sun, 01 Sep 2019 15:29:54 +0800</pubDate><description><![CDATA[多行json 直接 使用 sqlcontext.read().json("path") 读取时候 报错如下 ：


Exception in thread "main" org.apache.spark.sql.AnalysisException: Since Spark 2.3, the queries from raw JSON/CSV files are disallowed when ...]]></description><category></category></item><item><title><![CDATA[Flume  自定义source   -- SQLSource  (转为 json 格式)]]></title><link>https://blog.youkuaiyun.com/github_37643896/article/details/87258584</link><guid>https://blog.youkuaiyun.com/github_37643896/article/details/87258584</guid><author>github_37643896</author><pubDate>Thu, 14 Feb 2019 11:40:22 +0800</pubDate><description><![CDATA[个人使用flume 相对较多 ，对他的采集任务比较喜欢 ,自己做了一些数据库方向的的拓展。

虽然 github 上 有很多 自定义的 flume sql-source 比如 大名鼎鼎 的https://github.com/keedio/flume-ng-sql-source

但是 我个人在使用的过程中遇到了一些问题 也一直没有得到解决 ，https://github.c...]]></description><category></category></item><item><title><![CDATA[Spark 创建RDD (集合，本地文件，hdfs文件)基础数据源]]></title><link>https://blog.youkuaiyun.com/github_37643896/article/details/82661453</link><guid>https://blog.youkuaiyun.com/github_37643896/article/details/82661453</guid><author>github_37643896</author><pubDate>Wed, 12 Sep 2018 14:54:20 +0800</pubDate><description><![CDATA[本文主要记录 一些基础 数据源 转换 成为 RDD  的案例  仅供参考

 

1.集合类转换为RDD



import java.util.Arrays;
import java.util.List;

import org.apache.spark.SparkConf;
import org.apache.spark.api.java.JavaRDD;
import org.apache....]]></description><category></category></item><item><title><![CDATA[Spark 启动报错时报错 java.lang.NoSuchMethodError: scala.collection.immutable.HashSet$.empty()Lscala/collect]]></title><link>https://blog.youkuaiyun.com/github_37643896/article/details/82661069</link><guid>https://blog.youkuaiyun.com/github_37643896/article/details/82661069</guid><author>github_37643896</author><pubDate>Wed, 12 Sep 2018 14:38:42 +0800</pubDate><description><![CDATA[今天在Spark测试时异常 报错 一直 提示 找不到 该方法

因为 本身代码量很小 而且  没有用到 错误提示的类  找了许久 才发现是 版本 对应问题


/Library/Java/JavaVirtualMachines/jdk1.8.0_181.jdk/Contents/Home/bin/java &quot;-javaagent:/Applications/IntelliJ IDEA.app/Co...]]></description><category></category></item><item><title><![CDATA[启动Hadoop时一直提示输入密码]]></title><link>https://blog.youkuaiyun.com/github_37643896/article/details/82463455</link><guid>https://blog.youkuaiyun.com/github_37643896/article/details/82463455</guid><author>github_37643896</author><pubDate>Thu, 06 Sep 2018 16:53:08 +0800</pubDate><description><![CDATA[
[root@wh bin]# sh $HADOOP_HOME/sbin/start-all.sh 
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [wh]
root@wh's password: 
wh: Authentication failed.
root...]]></description><category></category></item></channel></rss>