kafka单机测试环境搭建

最新推荐文章于 2025-05-09 09:45:00 发布

原创

最新推荐文章于 2025-05-09 09:45:00 发布 · 352 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#kafka #搭建

本文档详细介绍了如何在单机上搭建kafka测试环境，包括下载带Scala的kafka安装包，安装JDK，配置环境变量，启动内置的Zookeeper和Kafka服务，创建并查看Topic，以及启动生产者和消费者进行消息传递测试。

kafka单服务器搭建

kafka搭建需要jdk， zookeeper

因此搭建前需先准备好JDK环境
由于kafka的安装包中已经内置一个zk，因此可以不用另外搭建，当然如果想独立处理，或者已经拥有zk也可以接入使用

下载地址：
http://kafka.apache.org/downloads

1.下载

下载带有scala的安装包

Binary downloads:
Scala 2.11  - kafka_2.11-2.3.0.tgz (asc, sha512)
Scala 2.12  - kafka_2.12-2.3.0.tgz (asc, sha512)

2.安装JDK

mkdir /app

tar -zxvf jdk-8u221-linux-x64.tar.gz

添加环境变量
vi ~/.bash_profile

export JAVA_HOME=/app/jdk1.8.0_221

export JRE_HOME=${JAVA_HOME}/jre

export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib

export PATH=.:${JAVA_HOME}/bin:$ZOOKEEPER_HOME/bin:$PATH

加载环境变量
source ~/.bash_profile

验证java环境

# java --version
Unrecognized option: --version
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
[root@kafka kafka_2.11-2.2.0]#

3.解压安装

 tar -zxf kafka_2.11-2.2.0.tgz 
 
 cd kafka_2.11-2.2.0/

4.启动zk

bin/zookeeper-server-start.sh config/zookeeper.properties

[root@kafka kafka_2.11-2.2.0]# bin/zookeeper-server-start.sh config/zookeeper.properties 
[2019-09-01 15:42:13,338] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2019-09-01 15:42:13,344] INFO autopurge.snapRetainCount set to 3 (org.apache.zookeeper.server.DatadirCleanupManager)
[2019-09-01 15:42:13,344] INFO autopurge.purgeInterval set to 0 (org.apache.zookeeper.server.DatadirCleanupManager)
[2019-09-01 15:42:13,344] INFO Purge task is not scheduled. (org.apache.zookeeper.server.DatadirCleanupManager)
[2019-09-01 15:42:13,344] WARN Either no config or no quorum defined in config, running  in standalone mode (org.apache.zookeeper.server.quorum.QuorumPeerMain)
[2019-09-01 15:42:13,361] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
[2019-09-01 15:42:13,361] INFO Starting server (org.apache.zookeeper.server.ZooKeeperServerMain)
 
 ....

[2019-09-01 15:42:33,437] INFO Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory)
[2019-09-01 15:42:33,444] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxnFactory)

5.启动kafka

bin/kafka-server-start.sh config/server.propertie


[2019-09-01 15:58:04,457] INFO [TransactionCoordinator id=0] Starting up. (kafka.coordinator.transaction.TransactionCoordinator)
[2019-09-01 15:58:04,501] INFO [TransactionCoordinator id=0] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)
[2019-09-01 15:58:04,504] INFO [Transaction Marker Channel Manager 0]: Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
[2019-09-01 15:58:04,621] INFO [/config/changes-event-process-thread]: Starting (kafka.common.ZkNodeChangeNotificationList