<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[提拉米苏的博客]]></title><description><![CDATA[梦想必须热爱]]></description><link>https://blog.youkuaiyun.com/jklcl</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; jklcl]]></copyright><item><title><![CDATA[Flink的三种join]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/126976843</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/126976843</guid><author>jklcl</author><pubDate>Wed, 21 Sep 2022 17:38:10 +0800</pubDate><description><![CDATA[很多架构开始往流批一体进行过渡，其中flink面临最大的挑战之一就是做好流join。]]></description><category></category></item><item><title><![CDATA[Flink的Checkpoint]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/126962134</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/126962134</guid><author>jklcl</author><pubDate>Tue, 20 Sep 2022 22:03:45 +0800</pubDate><description><![CDATA[最近面试问的最多的就是Flink如何进行容错的，总结一下关于checkpoint的知识点]]></description><category></category></item><item><title><![CDATA[kafka使用经历总结]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/126837758</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/126837758</guid><author>jklcl</author><pubDate>Wed, 14 Sep 2022 11:34:37 +0800</pubDate><description><![CDATA[kafka使用经历]]></description><category></category></item><item><title><![CDATA[Redis使用经历总结]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/126828827</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/126828827</guid><author>jklcl</author><pubDate>Tue, 13 Sep 2022 11:44:18 +0800</pubDate><description><![CDATA[redis使用记录]]></description><category></category></item><item><title><![CDATA[基于jacoco和CI做代码覆盖率检测]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/125565240</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/125565240</guid><author>jklcl</author><pubDate>Fri, 01 Jul 2022 21:04:51 +0800</pubDate><description><![CDATA[念念不忘，必有回响]]></description><category></category></item><item><title><![CDATA[python的webpy]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/122235646</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/122235646</guid><author>jklcl</author><pubDate>Thu, 30 Dec 2021 14:12:16 +0800</pubDate><description><![CDATA[随便写写
import web

urls = (
    '/test', 'test',
    '/test1/(.+)', 'test1',
    '/test2.(.*)+', 'test2',
    '/test3/?', 'test3',
    '/test4/?', 'test4'
)

app = web.application(urls, globals())

class test:
    def GET(self):
        return 'Hello'

class]]></description><category></category></item><item><title><![CDATA[奇奇怪怪的ES搜索需求]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/122166589</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/122166589</guid><author>jklcl</author><pubDate>Mon, 27 Dec 2021 10:42:54 +0800</pubDate><description><![CDATA[需求：给定一个list，多个词语，快速返回每个词语所在doc的detail字段的个数，如果是查一个词语，可以通过模糊匹配的方式进行查询
GET index/_search?size=0
{
    "query":{
        "bool":{
            "should":[
                {
                    "wildcard":{
                        "detail":"*齐齐哈尔医学院附属第三医院*"
     ]]></description><category></category></item><item><title><![CDATA[FlinkCEP(02)--kafka数据源]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/122068497</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/122068497</guid><author>jklcl</author><pubDate>Tue, 21 Dec 2021 16:56:33 +0800</pubDate><description><![CDATA[需求：用户点击完立刻购买的信息
数据源kafka，这里比较重要的一点设置水位Watermarks

import com.alibaba.fastjson.JSON

import java.util.Properties
import org.apache.flink.api.common.serialization.SimpleStringSchema
import org.apache.flink.cep.pattern.conditions.SimpleCondition
import org.ap]]></description><category></category></item><item><title><![CDATA[FlinkCEP(01)--普通数据源]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/122067866</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/122067866</guid><author>jklcl</author><pubDate>Tue, 21 Dec 2021 16:51:43 +0800</pubDate><description><![CDATA[CEP(Complex Event Processing)就是在无界事件流中检测事件模式
步骤分为四步
1、构建数据源
2、构建规则
3、Pattern上进行过滤
4、结果处理
多种模式：
1、严格连续模式
2、松散连续
3、不确定的松散连续
        &lt;dependency&gt;
            &lt;groupId&gt;org.apache.flink&lt;/groupId&gt;
            &lt;artifactId&gt;flink-scala_2.11&]]></description><category></category></item><item><title><![CDATA[FlinkSQL自定义TableSink]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/120439412</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/120439412</guid><author>jklcl</author><pubDate>Tue, 23 Nov 2021 22:44:27 +0800</pubDate><description><![CDATA[从StreamTableSinkFactory开始

]]></description><category></category></item><item><title><![CDATA[TPC-H基准测试]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/121466501</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/121466501</guid><author>jklcl</author><pubDate>Tue, 23 Nov 2021 16:17:24 +0800</pubDate><description><![CDATA[TPC-H基准测试
cd /home/project/tpcd/dbgen
./dbgen -s 5 -f
mv ./*.tbl /home/project/file/data5g
Create external table lineitem (L_ORDERKEY INT, L_PARTKEY INT,L_SUPPKEY INT,L_LINENUMBER INT,L_QUANTITY DOUBLE,L_EXTENDEDPRICE DOUBLE,L_DISCOUNT DOUBLE,L_TAX DOUBLE,]]></description><category></category></item><item><title><![CDATA[Tensorflow问题]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/121167696</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/121167696</guid><author>jklcl</author><pubDate>Fri, 05 Nov 2021 17:45:04 +0800</pubDate><description><![CDATA[Tensorflow问题
1、Another metric with the same name already exists
File "/usr/local/python3/lib/python3.7/site-packages/keras/__init__.py", line 25, in &lt;module&gt;
    from keras import models
  File "/usr/local/python3/lib/python3.7/site-packages/keras/mo]]></description><category></category></item><item><title><![CDATA[Tensorflow On Spark (2021版)]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/120956969</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/120956969</guid><author>jklcl</author><pubDate>Mon, 25 Oct 2021 17:57:18 +0800</pubDate><description><![CDATA[之前网上的版本太老了，mnist_spark.py和mnist_data_setup.py测试文件的参数和位置，以及tensorflow-hadoop的版本(1.5)都更新了
一、运行环境准备
1、hadoop2.7：最好是HA
2、spark on yarn：必须是yarn环境，否则不能执行训练，要求必须多台机器的集群
配置好环境变量
export JAVA_HOME=/usr/local/jdk8
export HADOOP_HOME=/home/hadoop/service/hadoop2.7
exp]]></description><category></category></item><item><title><![CDATA[nginx做文件下载]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/119897999</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/119897999</guid><author>jklcl</author><pubDate>Tue, 24 Aug 2021 20:30:34 +0800</pubDate><description><![CDATA[安装：https://blog.youkuaiyun.com/jklcl/article/details/119360729?spm=1001.2014.3001.5501
conf/nginx.conf中配置
    server {
       listen       80;
       charset utf-8;
       server_name  服务器IP;
       root    /data/Downloads/;
       autoindex       on;
       aut]]></description><category></category></item><item><title><![CDATA[python采集信息到prometheus]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/119897809</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/119897809</guid><author>jklcl</author><pubDate>Tue, 24 Aug 2021 20:22:04 +0800</pubDate><description><![CDATA[采集脚本
# coding=utf-8

from prometheus_client import Gauge, start_http_server
import requests
import time

jvsn = Gauge('zhibiao1', '描述', ['app_name', 'app_user'])
tes = Gauge('zhibiao2', '描述', ['app_name', 'app_user'])

if __name__ == '__main__':
    start_]]></description><category></category></item><item><title><![CDATA[Flink基于State做千万用户的pv]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/119395449</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/119395449</guid><author>jklcl</author><pubDate>Wed, 04 Aug 2021 23:44:04 +0800</pubDate><description><![CDATA[需求：记录每天某一页面下所有用户的访问次数和第一次访问的时间
解法：

redis做缓存，每天一个map，设置ttl，用户访问次数做累积，过滤完先存到redis，sink的时候读redis，查出这个用户的总访问次数
用flink的keyby(user_id+date)，生成count和min，使用checkpoint进行容错

对于上边两种做法各有缺陷，第一种需要借助外部存储，任务出问题的时候重启无法保证累计不重复更新；第二种需要会占用大量的内存，无法清理过期的user_id+date，没几天就oom。
此]]></description><category></category></item><item><title><![CDATA[ghz做grpc服务压测]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/119360853</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/119360853</guid><author>jklcl</author><pubDate>Tue, 03 Aug 2021 22:52:25 +0800</pubDate><description><![CDATA[搭建的grpc服务是线下的服务，但是要求qps比较高，那么自己先进行一下压测吧
准备

ghz：https://github.com/bojand/ghz/releases直接下载对应的包，解压即可使用
proto：安装proto，并且准备好proto的协议文件

压测脚本
# 2个线程，100次请求，慢慢扩大c和n的值，防止把服务打挂，一般来说c的大小是grpc服务指定max-works数
./ghz -c 10 -n 10000 \
--insecure \
# 指定.proto协议的位置
--prot]]></description><category></category></item><item><title><![CDATA[nginx做grpc端口的负载均衡]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/119360729</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/119360729</guid><author>jklcl</author><pubDate>Tue, 03 Aug 2021 22:30:38 +0800</pubDate><description><![CDATA[最近需要部署一个grpc的服务，为了保证服务的qps达到标准，于是在一台机器上做多个server，这个时候得用到nginx做转发和负载
安装: 切记版本，试过1.12的，不可用
# 下载
wget http://nginx.org/download/nginx-1.18.0.tar.gz
 
 
# 解压
tar -zxf nginx-1.18.0.tar.gz
 
 
# 环境
yum install -y gcc-c++ pcre pcre-devel zlib zlib-devel openssl o]]></description><category></category></item><item><title><![CDATA[maven私仓之Nexus]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/113001185</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/113001185</guid><author>jklcl</author><pubDate>Mon, 26 Jul 2021 23:49:07 +0800</pubDate><description><![CDATA[maven私仓之Nexus
引言

]]></description><category></category></item><item><title><![CDATA[python上传maven私仓]]></title><link>https://blog.youkuaiyun.com/jklcl/article/details/119121131</link><guid>https://blog.youkuaiyun.com/jklcl/article/details/119121131</guid><author>jklcl</author><pubDate>Mon, 26 Jul 2021 23:42:53 +0800</pubDate><description><![CDATA[私仓地址：http://nexus.xxxprivate.com/#browse/search/pypi
1、绑定我们的私仓地址
pip install flask -i http://nexus.niceprivate.com/repository/pypi/simple
地址如下图，末尾加“/simple”


2、编写nexus的python配置
在用户目录下编辑.pypirc
vim ~/.pypirc
[distutils]
index-servers =
    pypi
    nexus
 ]]></description><category></category></item></channel></rss>