<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[不是一枚开发]]></title><description><![CDATA[借鉴了书籍、视频、他人博客，先致歉]]></description><link>https://blog.youkuaiyun.com/javahelpyou</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; javahelpyou]]></copyright><item><title><![CDATA[flink 自定义kudu connector中使用Metrics计数平均吞吐量，并推送到自定义kafkaReporter]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/143334327</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/143334327</guid><author>javahelpyou</author><pubDate>Tue, 29 Oct 2024 17:07:40 +0800</pubDate><description><![CDATA[ps：kudu写入实际指标包含2个，一个是每分钟流量，在try代码块中，一个是每分钟错误数量，在catch代码块中flink-conf.yaml配置。]]></description><category></category></item><item><title><![CDATA[k8s metrics-server服务监控pod 的 cpu、内存]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/139651335</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/139651335</guid><author>javahelpyou</author><pubDate>Thu, 13 Jun 2024 15:09:01 +0800</pubDate><description><![CDATA[修改components.yaml配置文件，在 containers 的 args 参数中增加。docker无法拉取镜像，先拉取到本地，yaml中配置的imagePullPolicy策略为。]]></description><category></category></item><item><title><![CDATA[HDFS 切换NameNode主备]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/139587150</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/139587150</guid><author>javahelpyou</author><pubDate>Tue, 11 Jun 2024 09:38:37 +0800</pubDate><description><![CDATA[(3) 将 namenode164 切换为 Active 备用节点。]]></description><category></category></item><item><title><![CDATA[以hive metastore报错举例，远程调试hadoop服务]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/139415099</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/139415099</guid><author>javahelpyou</author><pubDate>Mon, 03 Jun 2024 15:18:38 +0800</pubDate><description><![CDATA[在idea上配置remote debug即可远程调试程序。根据脚本，metastore服务是采用。]]></description><category></category></item><item><title><![CDATA[Waiting for table metadata lock引发java.sql.SQLException: No operations allowed after statement closed]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/139391748</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/139391748</guid><author>javahelpyou</author><pubDate>Mon, 03 Jun 2024 14:32:07 +0800</pubDate><description><![CDATA[mysql中如果一个表产生了Waiting for table metadata lock，针对表的任何操作都会被锁住，包括select，可以通过配置参数lock_wait_timeout减少等待时长。原因是存在未提交事物，阻塞DDL，继而阻塞所有同表的后续操作。DDL操作（表是一个分区表，通过自动化脚本定时新增分区）查找未提交事务id，kill。]]></description><category></category></item><item><title><![CDATA[gitlab pull拉取失败，报错Access denied. The provided password or token is incorrect]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/139230826</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/139230826</guid><author>javahelpyou</author><pubDate>Mon, 27 May 2024 10:22:01 +0800</pubDate><description><![CDATA[吐槽：报错显示密码错误加上不久前改了密码，我陷入深思，密码到底是什么？然并软，提示不友好。直至在stackoverflow上翻一翻，然后检查了一下，原来是Personal Access Tokens过期了。删除令牌，再拉取一下代码，弹出窗口填入令牌。]]></description><category></category></item><item><title><![CDATA[hive “Struct is not null“ gets evaluated as nullable always causing filter miss in the query]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/139195103</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/139195103</guid><author>javahelpyou</author><pubDate>Sat, 25 May 2024 12:21:15 +0800</pubDate><description><![CDATA[对比执行计划：在关闭cbo后，才会添加过滤。1.升级hive版本或者修改源码。2.在执行时sql时禁用CBO。通过查看执行计划，排查。]]></description><category></category></item><item><title><![CDATA[DBeaver连接含有Kerberos认证的集群的Hive表]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/139023935</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/139023935</guid><author>javahelpyou</author><pubDate>Sat, 18 May 2024 13:54:29 +0800</pubDate><description><![CDATA[以DBeaver为例，连接含有Kerberos认证的集群的Hive表。]]></description><category></category></item><item><title><![CDATA[flink 设置空闲等待推进水位线，避免子任务上游最小的水位线迟迟未达到触发时间]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/136429069</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/136429069</guid><author>javahelpyou</author><pubDate>Sun, 03 Mar 2024 12:16:03 +0800</pubDate><description><![CDATA[此时，如果有其中一个子任务没有数据，导致当前Task的水位线无法推进，窗口无法触发，需要等待上游最小的水位线达到触发时间。于是，flink添加了。多并行度的flink作业，watermark水位线的传递遵循。checkIfIdle()方法用于判断是否触发水位线推进。在设置WatermarkStrategy时，添加。]]></description><category></category></item><item><title><![CDATA[flink 报错ByteArraySerializer is not an instance of org.apache.kafka.common.serialization.Serializer]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/136421010</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/136421010</guid><author>javahelpyou</author><pubDate>Sat, 02 Mar 2024 19:48:05 +0800</pubDate><description><![CDATA[存在依赖冲突,把conf目录下flink-conf.yml中的classloader.resolve-order参数由默认的child-first改成parent-first。代码逻辑为从socket流中读取数据写入kafka。查看flink作业日志。提交到yarn集群后报错。]]></description><category></category></item><item><title><![CDATA[生产环境下，应用模式部署flink任务，通过hdfs提交]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/136177994</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/136177994</guid><author>javahelpyou</author><pubDate>Mon, 19 Feb 2024 21:03:24 +0800</pubDate><description><![CDATA[通过通过yarn.provided.lib.dirs配置选项指定位置，将flink的依赖上传到hdfs文件管理系统。]]></description><category></category></item><item><title><![CDATA[ktutil编写生成keytab文件的脚本、通过keytab文件认证用户]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/136160457</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/136160457</guid><author>javahelpyou</author><pubDate>Sun, 18 Feb 2024 23:23:53 +0800</pubDate><description><![CDATA[示例：为密码123456的hive用户，生成名称为hive.keytab的认证文件。生成keytab文件的脚本。]]></description><category></category></item><item><title><![CDATA[Nginx 正向代理、反向代理]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/136146945</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/136146945</guid><author>javahelpyou</author><pubDate>Sun, 18 Feb 2024 10:44:49 +0800</pubDate><description><![CDATA[正向代理主要是用来解决访问限制问题；反向代理则是提供负载均衡、安全防护等作用。]]></description><category></category></item><item><title><![CDATA[Nginx 缓存集成、清除、设置不缓存资源]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/136080712</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/136080712</guid><author>javahelpyou</author><pubDate>Thu, 08 Feb 2024 18:36:51 +0800</pubDate><description><![CDATA[使用缓存的优点在于减少数据传输，节省网络流量，加快响应速度；减轻服务器压力；提供服务端的高可用性；缺点在于数据的不一致问题；增加成本Nginx作为Web缓存服务器，介于客户端和应用服务器之间，当用户通过浏览器访问一个URL时，web缓存服务器会去应用服务器获取要展示给用户的内容，将内容缓存到自己的服务器上，当下一次请求到来时，如果访问的是同一个URL，web缓存服务器就会直接将之前缓存的内容返回给客户端。]]></description><category></category></item><item><title><![CDATA[NGINX upstream、stream、四/七层负载均衡以及案例示例]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/136076376</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/136076376</guid><author>javahelpyou</author><pubDate>Thu, 08 Feb 2024 11:30:19 +0800</pubDate><description><![CDATA[伴随单台服务器性能及单点故障问题的凸显，一方面需要增加系统的硬件处理能力，另一方面需要添加机器构建应用集群应用集群：将同一应用部署到多台机器上，组成处理集群，接收负载均衡设备分发的请求，进行处理并返回响应的数据负载均衡器:将用户访问的请求根据对应的负载均衡算法，分发到集群中的一台服务器进行处理。]]></description><category></category></item><item><title><![CDATA[Nginx 安全控制实践]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/136053085</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/136053085</guid><author>javahelpyou</author><pubDate>Tue, 06 Feb 2024 11:06:53 +0800</pubDate><description><![CDATA[把原来的http请求改造为https。]]></description><category></category></item><item><title><![CDATA[每隔一段时间重试，重试n次 java 工具类]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/131324460</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/131324460</guid><author>javahelpyou</author><pubDate>Wed, 21 Jun 2023 15:02:17 +0800</pubDate><description><![CDATA[若代码出现异常，则每隔一段时间重试一下，重试n次。]]></description><category></category></item><item><title><![CDATA[flink 从savepoint、checkpoint中恢复数据]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/129893328</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/129893328</guid><author>javahelpyou</author><pubDate>Sat, 01 Apr 2023 10:58:22 +0800</pubDate><description><![CDATA[flink作业因为故障导致restart strategy失败或升级flink版本重新发布任务，这时就需要从最近的checkpoint恢复。一般而言有两种方案，第一种方案是开启checkpoint且任务取消时不删除checkpoint（调整参数execution.checkpointing.externalized-checkpoint-retention），第二种方案是定时触发savepoint（编写代码调用flink rest api）。]]></description><category></category></item><item><title><![CDATA[yum install报错 cyrus-sasl-devel-2.1.26-23.el7.x86 64 was supposed to be installed but is not!]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/129810474</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/129810474</guid><author>javahelpyou</author><pubDate>Tue, 28 Mar 2023 10:24:32 +0800</pubDate><description><![CDATA[在正常的服务器上把rpm包下载到本地目录（注意：只有在当前服务器未安装时有效，已经安装可yum remove后重新安装）自制的yum源，在一台服务器上安装cdh客户端时，执行。在其他节点上安装正常，大概率是服务器环境的问题。scp命令分发到对应服务器上。]]></description><category></category></item><item><title><![CDATA[flink sql作业报错Timeout of 60000ms expired before the position for partition xxx could be determined]]></title><link>https://blog.youkuaiyun.com/javahelpyou/article/details/129776514</link><guid>https://blog.youkuaiyun.com/javahelpyou/article/details/129776514</guid><author>javahelpyou</author><pubDate>Sun, 26 Mar 2023 11:05:52 +0800</pubDate><description><![CDATA[新增流作业，将kafka集群的数据通过flink sql写入hdfs,所有的flink作业均出现订阅的topic某个分区超时；(3)查看页面，找到出现异常的taskmanager所对应的域名,通过ping和telnet命令测试是否网络正常，发现不通，生产环境存在2套网络,(一套管理ip、一套服务ip)，域名映射出错。（2）连接kafka对应的zookeeper客户端，找到kafka连接超时分区的leader对应的服务器ip。打通此节点和cdh集群之间的网络，配置正确的域名映射。]]></description><category></category></item></channel></rss>