Kafka在Kubernetes中的集群部署

最新推荐文章于 2025-09-19 15:08:04 发布

原创

最新推荐文章于 2025-09-19 15:08:04 发布 · 1.8k 阅读

1 ·

CC 4.0 BY-SA版权

该博客介绍了如何在Kubernetes集群中使用Helm部署Kafka和Zookeeper。首先，通过Helm添加incubator仓库并创建StorageClass。接着，为Kafka和Zookeeper创建Local PV，确保它们在特定节点上运行。然后，定义Kafka的values.yaml文件并安装。最后，部署KafkaManager以实现Kafka集群的可视化管理。

Helm 安装Kafka

一.Helm部署Kafka

helm repo add incubator http://mirror.azure.cn/kubernetes/charts-incubator

[root@k8s-centos7-master-150 kafka]# helm repo list
NAME            URL
minio           https://helm.min.io/
bitnami         https://charts.bitnami.com/bitnami
incubator       http://mirror.azure.cn/kubernetes/charts-incubator

1 创建 StorageClass

首先先创建本地存储的 StorageClass

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: local-storage
provisioner: kubernetes.io/no-provisioner
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Retain

执行命令

kubectl apply -f local-storage.yaml 
storageclass.storage.k8s.io/local-storage created

[root@k8s-centos7-master-150 kafka]# kubectl get sc --all-namespaces -o wide
NAME            PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
local-storage   kubernetes.io/no-provisioner   Delete          WaitForFirstConsumer   false                  83m

2 创建 Kafka 的 pv

因为要在 k8s-centos7-node3-151 ，k8s-centos7-node4-152 这两个 k8s 节点上部署 2 个 Kafka 的 broker 节点，因此先在三个节点上创建这 2 个kafka broker节点的Local PV

kafka-local-pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: kafka-pv-0
spec:
  capacity:
    storage: 5Gi 
  accessModes:
  - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /data/kafka/data-0
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions:
        - key: kubernetes.io/hostname
          operator: In
          values:
          - k8s-centos7-node3-151
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name

最低0.47元/天解锁文章