<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[bao_since的博客]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/bao_since</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; bao_since]]></copyright><item><title><![CDATA[PyFlink on K8s 部署实践]]></title><link>https://blog.youkuaiyun.com/bao_since/article/details/113845477</link><guid>https://blog.youkuaiyun.com/bao_since/article/details/113845477</guid><author>bao_since</author><pubDate>Mon, 01 Mar 2021 10:57:11 +0800</pubDate><description><![CDATA[前置介绍
1.1 K8s 简介

Pod: K8s 的原子调度单位，是一个或多个 Container 的组合，Container 共享同一个网络、存储。
Deployment: 对一组相同 Pod 的高级抽象，保障高可用。
Service: 定义服务的访问入口，通过 Label Selector 绑定后端 Pod 副本集。如果 K8s 内部有一个服务，需要在外部进行访问，此时可以通过 Service 用 LoadBalancer 或者 NodePort 的方式将其暴露出去。如果不希望或不需要对外暴露服务，可]]></description><category></category></item><item><title><![CDATA[启动Scala  [ERROR] Failed to construct terminal; falling back to unsupported]]></title><link>https://blog.youkuaiyun.com/bao_since/article/details/104640397</link><guid>https://blog.youkuaiyun.com/bao_since/article/details/104640397</guid><author>bao_since</author><pubDate>Tue, 03 Mar 2020 20:24:30 +0800</pubDate><description><![CDATA[参考stackoverflow答案：https://stackoverflow.com/questions/44317384/sbt-error-failed-to-construct-terminal-falling-back-to-unsupported


Welcome to Scala 2.11.12 (Java HotSpot(TM) 64-Bit Server VM, Java 1....]]></description><category></category></item><item><title><![CDATA[flink DataSet 连接 mongos集群使用mongo-hadoop]]></title><link>https://blog.youkuaiyun.com/bao_since/article/details/90645046</link><guid>https://blog.youkuaiyun.com/bao_since/article/details/90645046</guid><author>bao_since</author><pubDate>Mon, 10 Jun 2019 13:35:32 +0800</pubDate><description><![CDATA[flink 官方示例比较简单，也很久没更新过了。

mongos生产集群需要认证权限，按文档提示配置好规则，这里output到测试库，所以没配置太多

这里是单线程读取，还不清楚可不可以通过配置实现并行读取。


import com.mongodb.hadoop.MongoInputFormat;
import com.mongodb.hadoop.MongoOutputFormat;
impo...]]></description><category></category></item><item><title><![CDATA[Flink KeySelector示例，选择多个字段作为key]]></title><link>https://blog.youkuaiyun.com/bao_since/article/details/89207059</link><guid>https://blog.youkuaiyun.com/bao_since/article/details/89207059</guid><author>bao_since</author><pubDate>Wed, 22 May 2019 18:50:13 +0800</pubDate><description><![CDATA[public class RecordSeclectId
		implements KeySelector&lt;Record, String&gt; {
	private static final long serialVersionUID = 4780234853172462378L;

	@Override
	public String getKey(Record value) throw...]]></description><category></category></item><item><title><![CDATA[Flink standalone集群HA高可用搭建]]></title><link>https://blog.youkuaiyun.com/bao_since/article/details/90234181</link><guid>https://blog.youkuaiyun.com/bao_since/article/details/90234181</guid><author>bao_since</author><pubDate>Wed, 15 May 2019 13:35:12 +0800</pubDate><description><![CDATA[如果没有dfs，需要配置dfs路径的位置也可以用本地file路径替换,倒也能启动集群和作业，但不要这样用，在作业容错和重启时会出现问题，比如找不到存在其他机器磁盘上的状态。

HA 配置为 3masters 3slavers

java1.8-&gt;scala2.12-&gt;zookeeper3.4.14-&gt;flink1.8.0_2.12

1.配置集群间ssh免密登陆
 生成ss...]]></description><category></category></item><item><title><![CDATA[kafka-connect-distribute 模式，使用 debezium source 同步 MongoDB 集群到 Kafka]]></title><link>https://blog.youkuaiyun.com/bao_since/article/details/90055158</link><guid>https://blog.youkuaiyun.com/bao_since/article/details/90055158</guid><author>bao_since</author><pubDate>Fri, 10 May 2019 12:24:05 +0800</pubDate><description><![CDATA[从flink社区看到 MongoDB -&gt;Debezium-&gt; Kafka 作为 Flinkdatasource 的方案，试着搭了一套。

MongoDB 使用副本+分片

Debezium 作为 Kafka Connect 的插件，首先配置 Kafka Connect，选择的是 connect-distribute 模式，这里踩到了一个坑， 因为我的集群 java 版本还在 7...]]></description><category></category></item><item><title><![CDATA[服务器传不进文件？作业莫名其妙中断？]]></title><link>https://blog.youkuaiyun.com/bao_since/article/details/89969556</link><guid>https://blog.youkuaiyun.com/bao_since/article/details/89969556</guid><author>bao_since</author><pubDate>Wed, 08 May 2019 19:29:00 +0800</pubDate><description><![CDATA[可能是磁盘空间满了 ......

分析下磁盘占用吧


df -h




du -h --max-depth=1


]]></description><category></category></item><item><title><![CDATA[Kafka Manager与Kafka SASL+ACL配置]]></title><link>https://blog.youkuaiyun.com/bao_since/article/details/89101226</link><guid>https://blog.youkuaiyun.com/bao_since/article/details/89101226</guid><author>bao_since</author><pubDate>Wed, 10 Apr 2019 12:48:38 +0800</pubDate><description><![CDATA[根据需求，给 Kafka 集群配置不使用 Kerberos 的 SASL+ACL 。Kafka 配置部分参考资料比较充足，这里参考了胡夕老师的《Apache Kafka 实战》中相关章节，配置过程顺利。Kafka Manager监控配置了 SASL Kafka 集群部分的资料比较少，在认证部分出现较多问题，最终配置出一个“能用”版本。Kafka_2.11-1.0.0 kafka-manager-...]]></description><category></category></item><item><title><![CDATA[【备忘】Kafka-consumer-groups.sh查看重置offset]]></title><link>https://blog.youkuaiyun.com/bao_since/article/details/88538728</link><guid>https://blog.youkuaiyun.com/bao_since/article/details/88538728</guid><author>bao_since</author><pubDate>Wed, 13 Mar 2019 18:46:10 +0800</pubDate><description><![CDATA[使用kafka提供的脚本查看和修改消费者组offset

查看：


kafka-consumer-groups.sh --bootstrap-server ip:9092--group testConsumer --describe

修改：


kafka-consumer-groups.sh --bootstrap-server ip:9092 --group testConsumer -...]]></description><category></category></item><item><title><![CDATA[【Intellij idea】Exception in thread "main" java.lang.NoClassDefFoundError:]]></title><link>https://blog.youkuaiyun.com/bao_since/article/details/88537091</link><guid>https://blog.youkuaiyun.com/bao_since/article/details/88537091</guid><author>bao_since</author><pubDate>Wed, 13 Mar 2019 17:28:28 +0800</pubDate><description><![CDATA[参考@街灯下的小草 的这篇文章解决这个问题https://blog.youkuaiyun.com/yhl_jxy/article/details/78872641

使用IntelliJ IDEA启动Flink项目出现Exception in thread &quot;main&quot; java.lang.NoClassDefFoundError: org/apache/flink/streaming/api/functi...]]></description><category></category></item><item><title><![CDATA[SHELL脚本 移动mv指定目录中的所有文件到其他目录]]></title><link>https://blog.youkuaiyun.com/bao_since/article/details/82747396</link><guid>https://blog.youkuaiyun.com/bao_since/article/details/82747396</guid><author>bao_since</author><pubDate>Mon, 17 Sep 2018 19:28:28 +0800</pubDate><description><![CDATA[递归写的,不太好看，反正能用，多担待


#!/bin/bash

function func {
for name in $1*
do
if [ -d &quot;$name&quot; ]
then 
#	echo &quot;$name is a d&quot;
	func $name/ $2 $p
elif [ -f &quot;$name&quot; ]
then 
	echo &quot;$name is a file&quot;
	c=$2${1#*$p}
	i...]]></description><category></category></item><item><title><![CDATA[Mac傻瓜式git、go get代理教学]]></title><link>https://blog.youkuaiyun.com/bao_since/article/details/82259099</link><guid>https://blog.youkuaiyun.com/bao_since/article/details/82259099</guid><author>bao_since</author><pubDate>Fri, 31 Aug 2018 17:17:44 +0800</pubDate><description><![CDATA[git config --global https.proxy http://127.0.0.1:1086
git config --global https.proxy https://127.0.0.1:1086
git config --global http.proxy 'socks5://127.0.0.1:1086' 
git config --global https.proxy '...]]></description><category></category></item></channel></rss>