Windows使用docker desktop 安装kafka、zookeeper集群

本文详细介绍了如何使用docker-compose在Windows系统上安装和配置Zookeeper集群,包括设置环境变量、数据卷以及创建yaml文件。随后,文章还提及了如何创建Kafka集群并解决连接zk的问题。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

docker-compose安装zookeeper集群

参考文章:http://t.csdn.cn/TtTYI

https://blog.youkuaiyun.com/u010416101/article/details/122803105?spm=1001.2014.3001.5501

准备工作:

​ 在开始新建集群之前,新建好文件夹,用来挂载kafka、zookeeper数据,如下:
zookeeper文件夹

D:\soft\docker\zookeeper\zoo1\data
D:\soft\docker\zookeeper\zoo1\datalog

D:\soft\docker\zookeeper\zoo2\data
D:\soft\docker\zookeeper\zoo2\datalog

D:\soft\docker\zookeeper\zoo3\data
D:\soft\docker\zookeeper\zoo3\datalog

kafka文件夹

D:\soft\docker\kafka\kafka1\data
D:\soft\docker\kafka\kafka2\data
D:\soft\docker\kafka\kafka3\data

image-20230806214058741 image-20230806214143307

image-20230806214255332 image-20230806214228248

1、下载zk镜像

docker pull zookeeper:3.6.3

image-20230806164624161

2、创建docker网络

docker network create zk-net

image-20230806165135181

3、创建zk集群

yml文件路径:D:\soft\docker\docker-compose-zookeeper-cluster.yml

yml文件内容:

version: '2'

networks:
  zk-net:
    external:
      name: zk-net

services:
  zoo1:
    image: 'zookeeper:3.6.3'
    container_name: zoo1
    hostname: zoo1
    environment:
      ZOO_MY_ID: 1
      ZOO_SERVERS: server.1=0.0.0.0:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=zoo3:2888:3888;2181
      ALLOW_ANONYMOUS_LOGIN: "yes"
    networks:
      - zk-net
    ports:
      - 2181:2181
      - 8081:8080
    volumes:
      - /D/soft/docker/zookeeper/zoo1/data:/data
      - /D/soft/docker/zookeeper/zoo1/datalog:/datalog
  zoo2:
    image: 'zookeeper:3.6.3'
    container_name: zoo2
    hostname: zoo2
    environment:
      ZOO_MY_ID: 2
      ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=0.0.0.0:2888:3888;2181 server.3=zoo3:2888:3888;2181
      ALLOW_ANONYMOUS_LOGIN: "yes"
    networks:
      - zk-net
    ports:
      - 2182:2181
      - 8082:8080
    volumes:
      - /D/soft/docker/zookeeper/zoo2/data:/data  
      - /D/soft/docker/zookeeper/zoo2/datalog:/datalog  
  zoo3:
    image: 'zookeeper:3.6.3'
    container_name: zoo3
    hostname: zoo3
    environment:
      ZOO_MY_ID: 3
      ZOO_SERVERS: server.1=zoo1:2888:3888;2181 server.2=zoo2:2888:3888;2181 server.3=0.0.0.0:2888:3888;2181
      ALLOW_ANONYMOUS_LOGIN: "yes"
    networks:
      - zk-net
    ports:
      - 2183:2181
      - 8083:8080
    volumes:
      - /D/soft/docker/zookeeper/zoo3/data:/data  
      - /D/soft/docker/zookeeper/zoo3/datalog:/datalog   

4、docker-compose启动、停止(指定配置文件)

docker-compose -f D:\soft\docker\docker-compose-zookeeper-cluster.yml up -d

启动成功:

image-20230806172731215

image-20230806174412849

指定配置文件停止

docker-compose -f D:\soft\docker\docker-compose-zookeeper-cluster.yml stop

docker-compose安装Kafka集群

参考文章:https://blog.youkuaiyun.com/u010416101/article/details/122916190

1、下载kafka镜像

docker pull bitnami/kafka:2.7.0

2、创建网络

因为上面zk创建过了,这里不需要再创建

3、创建kafka集群

文件路径:D:\soft\docker\docker-compose-kafka-cluster.yml

文件内容:

version: "3"

networks:
  zk-net:
    external:
      name: zk-net

services:
  kafka01:
    image: 'bitnami/kafka:2.7.0'
    restart: always    
    container_name: kafka01
    hostname: kafka01
    ports:
      - '9092:9092'
    environment:
      - KAFKA_BROKER_ID=1
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka01:9092
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zoo1:2181,zoo2:2181,zoo3:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
    volumes:
      - /D/soft/docker/kafka/kafka1:/bitnami/kafka     
    networks:
      - zk-net
  kafka02:
    image: 'bitnami/kafka:2.7.0'
    restart: always       
    container_name: kafka02
    hostname: kafka02
    ports:
      - '9093:9092'
    environment:
      - KAFKA_BROKER_ID=2
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka02:9092
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zoo1:2181,zoo2:2181,zoo3:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
    volumes:
      - /D/soft/docker/kafka/kafka2:/bitnami/kafka       
    networks:
      - zk-net
  kafka03:
    image: 'bitnami/kafka:2.7.0'
    restart: always       
    container_name: kafka03
    hostname: kafka03
    ports:
      - '9094:9092'
    environment:
      - KAFKA_BROKER_ID=3
      - KAFKA_CFG_LISTENERS=PLAINTEXT://:9092    
      - KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://kafka03:9092
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zoo1:2181,zoo2:2181,zoo3:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
    volumes:
      - /D/soft/docker/kafka/kafka3:/bitnami/kafka   
    networks:
      - zk-net

4、docker-compose启动、停止(指定配置文件)

docker-compose -f D:\soft\docker\docker-compose-kafka-cluster.yml up -d

kafka启动的时候,没有成功,后面参考了这篇文章:https://www.cnblogs.com/mihutao/p/16664588.html

如果kafka连接不上zookeeper,使用下面的命令加入zk-net网络:

docker network connect zk-net kafka01
docker network connect zk-net kafka02
docker network connect zk-net kafka03

重启:

docker-compose -f D:\soft\docker\docker-compose-zookeeper-cluster.yml restart
docker-compose -f D:\soft\docker\docker-compose-kafka-cluster.yml restart

由于kafka一直在自动重启,需要查看日志,docker查看容器日志:

docker logs a679f5a4a1d4

image-20230806214425867

### 如何在 Docker Desktop安装和配置 Kafka #### 安装 ZookeeperKafka 为了成功运行 Kafka,首先需要启动一个 Zookeeper 实例作为依赖服务。以下是通过 `docker-compose` 文件实现的完整过程。 #### 编写 docker-compose.yml 文件 创建名为 `docker-compose.yml` 的文件并添加以下内容: ```yaml version: '3' services: zookeeper: image: wurstmeister/zookeeper ports: - "2181:2181" kafka1: image: wurstmeister/kafka depends_on: - zookeeper ports: - "9092:9092" environment: KAFKA_ADVERTISED_HOST_NAME: localhost KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_BROKER_ID: 1 KAFKA_LISTENERS: PLAINTEXT://localhost:9092 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092 kafka2: image: wurstmeister/kafka depends_on: - zookeeper ports: - "9093:9093" environment: KAFKA_ADVERTISED_HOST_NAME: localhost KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_BROKER_ID: 2 KAFKA_LISTENERS: PLAINTEXT://localhost:9093 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9093 kafka3: image: wurstmeister/kafka depends_on: - zookeeper ports: - "9094:9094" environment: KAFKA_ADVERTISED_HOST_NAME: localhost KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 KAFKA_BROKER_ID: 3 KAFKA_LISTENERS: PLAINTEXT://localhost:9094 KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9094 ``` 上述配置定义了一个单节点的 Zookeeper 集群以及三个独立的 Kafka 节点[^1]。 #### 启动容器 执行以下命令以启动所有的服务: ```bash docker-compose up -d ``` 这将在后台启动所有必要的容器,并使它们保持运行状态。 #### 查看日志 如果需要监控各个组件的日志输出,可以使用以下命令分别查看每个 KafkaZookeeper 容器的日志流: ```bash docker logs -f kafka1 docker logs -f kafka2 docker logs -f kafka3 docker logs -f zookeeper ``` 这些命令可以帮助排查可能存在的问题或确认各组件正常工作[^2]。 #### 测试 Kafka 连接 可以通过指定 broker 地址的方式测试 Kafka 是否能够被客户端访问。例如,在本地机器上运行以下命令连接到第一个 Kafka Broker: ```bash kafka-console-producer.sh --broker-list localhost:9092 --topic test-topic ``` 此操作允许向主题发送消息。同样地,也可以设置消费者端接收来自该主题的消息: ```bash kafka-console-consumer.sh --bootstrap-server localhost:9092 --from-beginning --topic test-topic ``` 以上方法验证了整个系统的连通性和功能性[^4]。 ---
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值