基于K8S的Kafka全家桶安装(Kafka、Kafka Connect、 Schema Registry、KsqlDB)

最新推荐文章于 2025-06-19 15:04:09 发布

原创

最新推荐文章于 2025-06-19 15:04:09 发布 · 2.8k 阅读

3 ·

CC 4.0 BY-SA版权

文章标签：

#Kafka #Kafka Connect #Schema Registry #KsqlDB

本文详细介绍了如何在Kubernetes集群上安装Kafka、Kafka Connect、Schema Registry和KsqlDB。首先，通过Operator Lifecycle Manager和Strimzi Apache Kafka Operator安装基础组件。接着，逐步部署Kafka集群，配置Helm Charts以安装Schema Registry和Kafka Connect。最后，讲解了如何安装并设置KsqlDB使其能与外部Kafka Connect集群配合工作。

本文基于K8s集群安装了Kafka、Kafka Connect、 Schema Registry、KsqlDB

1. 安装Operator Lifecycle Manager

下载文件

https://github.com/operator-framework/operator-lifecycle-manager/releases/download/0.16.1/crds.yaml

https://github.com/operator-framework/operator-lifecycle-manager/releases/download/0.16.1/olm.yaml

安装

kubectl apply -f https://github.com/operator-framework/operator-lifecycle-manager/releases/download/0.16.1/crds.yaml
kubectl apply -f https://github.com/operator-framework/operator-lifecycle-manager/releases/download/0.16.1/olm.yaml

2. 安装Strimzi Apache Kafka Operator

下载文件：https://operatorhub.io/install/strimzi-kafka-operator.yaml

安装命令

cd kafka
kubectl create -f strimzi-kafka-operator.yaml

观察安装是否成功
```
kubectl get csv -n operators -w
```

结果显示如下表示安装成功

NAME                               DISPLAY   VERSION   REPLACES                           PHASE
strimzi-cluster-operator.v0.19.0   Strimzi   0.19.0    strimzi-cluster-operator.v0.18.0   Succeeded

3. 安装Kafka集群

下载https://raw.githubusercontent.com/strimzi/strimzi-kafka-operator/0.19.0/examples/kafka/kafka-persistent.yaml

编辑如下

apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
  name: my-cluster
spec:
  kafka:
    version: 2.5.0
    replicas: 3
    listeners:
      plain: {
     
     }
      tls: {
     
     }
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      delete.topic.enable: "true"
      transaction.sta

最低0.47元/天解锁文章