Openresty+Lua+Rdkafka使用

最新推荐文章于 2024-04-12 09:41:29 发布

原创最新推荐文章于 2024-04-12 09:41:29 发布 · 410 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#lua #开发语言 #nginx

Kafka 同时被 2 个专栏收录

2 篇文章

订阅专栏

lua

2 篇文章

订阅专栏

本文介绍了如何在CentOS 7.9环境下安装OpenResty，集成lua-resty-kafka库，并配置Nginx以使用Lua脚本生产Kafka消息。详细步骤包括安装依赖、下载编译OpenResty、配置Nginx及lua脚本，最后提供了测试和生产消息的代码示例。

Openresty+Lua+Rdkafka使用

安装openresty
安装环境：centos 7.9

cat /etc/redhat-release

安装依赖库：

yum -y install readline-devel pcre-devel openssl-devel gcc

下载openresty release包：

wget https://openresty.org/download/openresty-1.19.9.1.tar.gz

配置编译安装：

./configure --prefix=/opt/openresty --with-luajit
make && make install

安装lua-resty-kafka
```
wget https://github.com/doujiang24/lua-resty-kafka/archive/master.zip
```
将压缩包解压到/opt/openresty/lualib/resty/目录中。unzip lua-resty-kafka-master.zip

配置nginx.conf

user  test;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  logs/access.log  main;
    error_log   logs/error.log  notice;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;
    gzip  on;

    lua_package_path "/opt/openresty/lualib/resty/kafka/?.lua;;";
    lua_package_cpath "/opt/openresty/lualib/?.so;;";

    lua_shared_dict ngx_cache 128m;  # cache
    lua_shared_dict cache_lock 100k; # lock for cache

    server {
        listen       192.168.31.90:8080;
        server_name  localhost;

        #charset koi8-r;

        #access_log  logs/host.access.log  main;

        location / {
            root   html;
            index  index.html index.htm;
        }

        location = /message/produce {
            lua_code_cache on;
            charset utf-8;
            default_type 'application/json';
            content_by_lua_file "/opt/openresty/nginx/lua/message_produce.lua"; #引用的lua脚本
        }

        error_page  404              /404.html;
    }
}

编辑message_produce.lua脚本，参考下面生产消息的demo。

操作openresty
查看使用帮助：

./bin/openresty -h

启动：

./bin/openresty

重启：

./bin/openresty -s reload

结束：

./bin/openresty -s stop

测试
通过postman请求nginx接口，模拟生产消息，成功。

生产消息代码demo

local producer = require "resty.kafka.producer"

local broker_list = {
    {host = "192.168.31.90", port = 9092}
}

-- 接收post请求的body
ngx.req.read_body()
local topic_name = "test"
local key = nil
local message = ngx.req.get_body_data()

-- 使用异步发送消息
-- api_version: huawei cloud kafka version is 1.1.0 or 2.3.0, api_version should be 2.
local producer_config = {
    producer_type = "async",
    api_version = 2,
    required_acks = 1
}
local async_producer = producer:new(broker_list, producer_config)

local ok, err = async_producer:send(topic_name, key, message)
if not ok then
    ngx.say("send err:", err)
    return
else
    ngx.say("send success, ok:", ok)
end