阿里云使用docker急速部署kafka

最新推荐文章于 2024-05-29 09:48:29 发布

成都—大数据开发工程师—杨洋

最新推荐文章于 2024-05-29 09:48:29 发布

阅读量999

点赞数 1

CC 4.0 BY-SA版权

文章标签： kafka 阿里云 docker docker-compose zookeeper

本文链接：https://blog.youkuaiyun.com/weixin_43870646/article/details/125906836

Kafka部署

准备工作：阿里云服务器安装docker，docker-compose

下载docker-compose.yml

curl -sSL https://raw.githubusercontent.com/bitnami/bitnami-docker-kafka/master/docker-compose.yml > docker-compose.yml

修改 docker-compose.yml，配置内外网访问

version: "2"

services:
  zookeeper:
    image: docker.io/bitnami/zookeeper:3.8
    ports:
      - "2181:2181"
    volumes:
      - "zookeeper_data:/bitnami"
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
  kafka:
    image: docker.io/bitnami/kafka:3.2
    ports:
      - "9092:9092"
      - "9093:9093"
    volumes:
      - "kafka_data:/bitnami"
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=阿里云内网:2181
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
      - KAFKA_CFG_LISTENERS=CLIENT://0.0.0.0:9092,EXTERNAL://0.0.0.0:9093
      - KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://阿里云外网:9092,EXTERNAL://阿里云外网:9093
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=CLIENT
    depends_on:
      - zookeeper

volumes:
  zookeeper_data:
    driver: local
  kafka_data:
    driver: local

docker-compose部署命令

docker-compose up -d

执行后会自动部署zookeeper、kafka服务

docker 部署时报错

ERROR [KafkaServer id=1001] Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.apache.kafka.common.KafkaException: Socket server failed to bind to 阿里云外网:9092: Cannot assign requested address.

解决办法 ：KAFKA_CFG_LISTENERS=PLAINTEXT://0.0.0.0:9092

创建topic时报错

kafka-topics.sh  --bootstrap-server 阿里云内网:9092 --create --replication-factor 1 --partitions 3 --topic erpDataSync

Exception in thread "main" joptsimple.UnrecognizedOptionException: zookeeper is not a recognized option
	at joptsimple.OptionException.unrecognizedOption(OptionException.java:108)
	at joptsimple.OptionParser.handleLongOptionToken(OptionParser.java:510)
	at joptsimple.OptionParserState$2.handleArgument(OptionParserState.java:56)
	at joptsimple.OptionParser.parse(OptionParser.java:396)
	at kafka.admin.TopicCommand$TopicCommandOptions.<init>(TopicCommand.scala:567)
	at kafka.admin.TopicCommand$.main(TopicCommand.scala:47)
	at kafka.admin.TopicCommand.main(TopicCommand.scala)

解决办法：
用–bootstrap-server 阿里云外网:9092 替代原先的–zookeeper

验证是否创建成功：

kafka-topics.sh --bootstrap-server 阿里云外网:9092 --describe --topic erpDataSync

The valid options based on currently configured listeners are CLIENT,EXTERNAL