在GCP 上安装kafka

最新推荐文章于 2025-05-25 14:26:35 发布

原创最新推荐文章于 2025-05-25 14:26:35 发布 · 643 阅读

0 ·

CC 4.0 BY-SA版权

Cloud Native 专栏收录该内容

9 篇文章

订阅专栏

本文档详细介绍了如何在Google Cloud Platform (GCP)上安装和配置Kafka。首先，你需要在GCP上创建一个运行CentOS 7的虚拟机实例。接着，将进行Zookeeper和Kafka的安装，包括修改zookeeper.properties和server.properties配置文件。安装完成后，启动Zookeeper和Kafka。最后，通过验证步骤，包括在两个独立的终端上创建topic、producer和consumer，确保Kafka运行正常。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

安装kafka

准备

在GCP上创建Centos 7 的VM实例
安装步骤

# Installing dependencies
sudo su
yum install -y wget nano java

# Automatically creating the Zookeeper myid file
mkdir /tmp/zookeeper
touch /tmp/zookeeper/myid
echo "ZOOKEEPER-ID" > tmp/zookeeper/myid

# Downloading and extracting Kafka
wget http://mirror.ox.ac.uk/sites/rsync.apache.org/kafka/0.10.2.1/kafka_2.12-0.10.2.1.tgz
tar -xvzf kafka_2.12-0.10.2.1.tgz
cd kafka_2.12-0.10.2.1

修改zookeeper.properties

nano config/zookeeper.properties

# In milliseconds
tickTime=2000

# In ticks
initLimit=10
syncLimit=5
maxClientCnxns=30

# All Zookeeper servers need to be aware of other Zookeepers part of the cluster
server.1=localhost:2888:3888

修改server.properties配置

nano config/server.properties
# Broker id needs to be unique for each Kafka server
broker.id=0

# This will be the Zookeeper servers created before
zookeeper.connect=localhost:2181

# Examples of hostnames - kafka-1, kafka-2, kafka-3
host.name=localhost

启动zookeeper和kafka

bin/zookeeper-server-start.sh config/zookeeper.properties &
bin/kafka-server-start.sh config/server.properties &

验证
开启俩个终端，在终端1上创建topic 和producer


bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

在终端2 上创建consumer

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic test --from-beginning

这里写图片描述