Kafka入门操作

最新推荐文章于 2025-07-02 11:43:06 发布

原创最新推荐文章于 2025-07-02 11:43:06 发布 · 2.8k 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#zookeeper #kafka

kafka 专栏收录该内容

16 篇文章

订阅专栏

本文介绍了Kafka的安装步骤，包括使用单独的Zookeeper而不是Kafka内置的，修改Kafka配置以连接外部Zookeeper，并提供了启动Kafka及创建、消费topic的简单测试。在启动Kafka时，需指定server.properties，并可以通过参数设置消费者从何处开始读取消息。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

问题：
Kafka的安装过程
kafka的启动
kafka的样例

http://apache.fayea.com/kafka/0.10.0.1/kafka_2.10-0.10.0.1.tgz 下载地址

1、解压Kafka
惊讶的是Kafka内置了Zookeeper的安装包以及启停Zookeeper的脚本，版本比较低，是3.3.4版本。理论上不应该使用Kafka的版本，因为Zookeeper是个通用分布式配置和协调系统.
实际上，也可以使用Kafka内置的Zookeeper，不过要注意，Zookeeper一般使用3台做集群，如果Kakfa的Broker多于3台，那么就取其中3台运行Zookeeper

[xulu@SZB-L0032015 ~]$ tar -xvf kafka_2.10-0.10.0.1.tgz -C /opt/cloud/kafka/

2、配置Zookeeper
本文使用单独安装的Zookeeper，而不是使用Kafka自带的Zookeeper，Kafka为了能够知道它要连接的Zookeeper地址，配置文件中提供了一系列和Zookeeper相关的配置参数
除了安装运行独立的Zookeeper，Kafka也可以使用安装包里的Zookeeper，如果Kafka要使用自己的Zookeeper，那么需要在 Kafka的bin目录下启动Zookeeper。因此，如果使用独立的Zookeeper的时候，就无需启动Kafka下面的Zookeeper了。在 Kafka启动过程中看到有关Zookeeper的日志，这是Kafka作为Zookeeper的客户端正在建立与Zookeeper服务器的通讯
具体的安装信息请看 zookeeper

3、配置Kafka
1. 修改配置文件config/server.properties
修改zookeeper为外部zookeeper

# Zookeeper connection string (see zookeeper docs for details).
# This is a comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002".
# You can also append an optional chroot string to the urls to specify the
# root directory for all kafka znodes.
zookeeper.connect=10.20.24.231:3181,10.20.25.199:3181,10.20.25.241:3181

3、1 config/producer.properties
无相关配置

3、2 修改 consumer.properties
配置外部zookeeper的地址

# Zookeeper connection string
# comma separated host:port pairs, each corresponding to a zk
# server. e.g. "127.0.0.1:3000,127.0.0.1:3001,127.0.0.1:3002"
zookeeper.connect=10.20.25.241:3181,10.20.25.199:3181,10.20.24.231:3181

4、启动Kafka
1.启动Kafka(需要指定server.properties）

[xulu@SZB-L0032015 bin]$ ./kafka-server-start.sh  ../config/server.properties

启动日志

[2016-12-19 09:47:46,803] INFO KafkaConfig values: 
        request.timeout.ms = 30000
        log.roll.hours = 168
        inter.broker.protocol.version = 0.10.0-IV1
        log.preallocate = false
        security.inter.broker.protocol = PLAINTEXT
        controller.socket.timeout.ms = 30000
        broker.id.generation.enable = true
        ssl.keymanager.algorithm = SunX509
        ssl.key.password = null
        log.cleaner.enable = true
        ssl.provider = null
        num.recovery.threads.per.data.dir = 1
        background.threads = 10
        unclean.leader.election.enable = true
        sasl.kerberos.kinit.cmd = /usr/bin/kinit
        replica.lag.time.max.ms = 10000
        ssl.endpoint.identification.algorithm = null
        sasl.mechanism.inter.broker.protocol = GSSAPI
        auto.create.topics.enable = true
        log.message.timestamp.difference.max.ms = 9223372036854775807
        zookeeper.sync.time.ms = 2000
        ssl.client.auth = none
        ssl.keystore.password = null
        offsets.topic.compression.codec = 0
        log.cleaner.io.buffer.load.factor = 0.9
        log.retention.hours = 168
        log.dirs = /tmp/kafka-logs
        ssl.protocol = TLS
        log.index.size.max.bytes = 10485760
        sasl.kerberos.min.time.before.relogin = 60000
        broker.rack = null
        log.retention.minutes = null
        connections.max.idle.ms = 600000
        ssl.trustmanager.algorithm = PKIX
        offsets.retention.minutes = 1440
        max.connections.per.ip = 2147483647
        replica.fetch.wait.max.ms = 500
        log.message.timestamp.type = CreateTime
        metrics.num.samples = 2
        port = 9092
        offsets.retention.check.interval.ms = 600000
        log.cleaner.dedupe.buffer.size = 134217728
        log.segment.bytes = 1073741824
        group.min.session.timeout.ms = 6000
        producer.purgatory.purge.interval.requests = 1000
        min.insync.replicas = 1
        ssl.truststore.password = null
        socket.receive.buffer.bytes = 102400
        log.flush.scheduler.interval.ms = 9223372036854775807
        num.io.threads = 8
        leader.imbalance.per.broker.percentage = 10
        zookeeper.connect = 10.20.24.231:3181,10.20.25.199:3181,10.20.25.241:3181
        queued.max.requests = 500
        offsets.topic.replication.factor = 3
        replica.socket.timeout.ms = 30000
        offsets.topic.segment.bytes = 104857600
        replica.high.watermark.checkpoint.interval.ms = 5000
        broker.id = 0
        ssl.keystore.location = null
        listeners = null
        log.flush.interval.messages = 9223372036854775807
        principal.builder.class = class org.apache.kafka.common.security.auth.DefaultPrincipalBuilder
        log.retention.ms = null
        offsets.commit.required.acks = -1
        sasl.kerberos.principal.to.local.rules = [DEFAULT]
        group.max.session.timeout.ms = 300000
        num.replica.fetchers = 1
        advertised.listeners = null
        replica.socket.receive.buffer.bytes = 65536
        delete.topic.enable = false
        log.index.interval.bytes = 4096
        metric.reporters = []
        compression.type = producer
        log.cleanup.policy = delete
        log.message.format.version = 0.10.0-IV1
        controlled.shutdown.max.retries = 3
        log.cleaner.threads = 1
        quota.window.size.seconds = 1
        zookeeper.connection.timeout.ms = 6000
        offsets.load.buffer.size = 5242880
        zookeeper.session.timeout.ms = 6000
        ssl.cipher.suites = null
        authorizer.class.name = 
        sasl.kerberos.ticket.renew.jitter = 0.05
        sasl.enabled.mechanisms = [GSSAPI]
        sasl.kerberos.service.name = null
        controlled.shutdown.enable = true
        offsets.topic.num.partitions = 50
        quota.window.num = 11
        message.max.bytes = 1000012
        log.cleaner.backoff.ms = 15000
        log.roll.jitter.hours = 0
        log.retention.check.interval.ms = 300000
        replica.fetch.max.bytes = 1048576
        log.cleaner.delete.retention.ms = 86400000
        fetch.purgatory.purge.interval.requests = 1000
        log.cleaner.min.cleanable.ratio = 0.5
        offsets.commit.timeout.ms = 5000
        zookeeper.set.acl = false
        log.retention.bytes = -1
        offset.metadata.max.bytes = 4096
        leader.imbalance.check.interval.seconds = 300
        quota.consumer.default = 9223372036854775807
        log.roll.jitter.ms = null
        reserved.broker.max.id = 1000
        replica.fetch.backoff.ms = 1000
        advertised.host.name = null
        quota.producer.default = 9223372036854775807
        log.cleaner.io.buffer.size = 524288
        controlled.shutdown.retry.backoff.ms = 5000
        log.dir = /tmp/kafka-logs
        log.flush.offset.checkpoint.interval.ms = 60000
        log.segment.delete.delay.ms = 60000
        num.partitions = 1
        num.network.threads = 3
        socket.request.max.bytes = 104857600
        sasl.kerberos.ticket.renew.window.factor = 0.8
        log.roll.ms = null
        ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
        socket.send.buffer.bytes = 102400
        log.flush.interval.ms = null
        ssl.truststore.location = null
        log.cleaner.io.max.bytes.per.second = 1.7976931348623157E308
        default.replication.factor = 1
        metrics.sample.window.ms = 30000
        auto.leader.rebalance.enable = true
        host.name = 
        ssl.truststore.type = JKS
        advertised.port = null
        max.connections.per.ip.overrides = 
        replica.fetch.min.bytes = 1
        ssl.keystore.type = JKS
 (kafka.server.KafkaConfig)
[2016-12-19 09:47:46,868] INFO starting (kafka.server.KafkaServer)
[2016-12-19 09:47:46,877] INFO Connecting to zookeeper on 10.20.24.231:3181,10.20.25.199:3181,10.20.25.241:3181 (kafka.server.KafkaServer)
[2016-12-19 09:47:46,893] INFO Starting ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2016-12-19 09:47:46,900] INFO Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:host.name=SZB-L0032015 (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:java.version=1.7.0_79 (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:java.vendor=Oracle Corporation (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:java.home=/opt/jdk1.7.0_79/jre (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:java.class.path=:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/aopalliance-repackaged-2.4.0-b34.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/argparse4j-0.5.0.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/connect-api-0.10.0.1.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/connect-file-0.10.0.1.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/connect-json-0.10.0.1.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/connect-runtime-0.10.0.1.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/guava-18.0.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/hk2-api-2.4.0-b34.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/hk2-locator-2.4.0-b34.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/hk2-utils-2.4.0-b34.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jackson-annotations-2.6.0.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jackson-core-2.6.3.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jackson-databind-2.6.3.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jackson-jaxrs-base-2.6.3.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jackson-jaxrs-json-provider-2.6.3.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jackson-module-jaxb-annotations-2.6.3.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/javassist-3.18.2-GA.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/javax.annotation-api-1.2.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/javax.inject-1.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/javax.inject-2.4.0-b34.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/javax.servlet-api-3.1.0.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/javax.ws.rs-api-2.0.1.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jersey-client-2.22.2.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jersey-common-2.22.2.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jersey-container-servlet-2.22.2.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jersey-container-servlet-core-2.22.2.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jersey-guava-2.22.2.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jersey-media-jaxb-2.22.2.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jersey-server-2.22.2.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jetty-continuation-9.2.15.v20160210.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jetty-http-9.2.15.v20160210.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jetty-io-9.2.15.v20160210.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jetty-security-9.2.15.v20160210.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jetty-server-9.2.15.v20160210.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jetty-servlet-9.2.15.v20160210.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jetty-servlets-9.2.15.v20160210.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jetty-util-9.2.15.v20160210.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/jopt-simple-4.9.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/kafka_2.10-0.10.0.1.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/kafka_2.10-0.10.0.1-sources.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/kafka_2.10-0.10.0.1-test-sources.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/kafka-clients-0.10.0.1.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/kafka-log4j-appender-0.10.0.1.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/kafka-streams-0.10.0.1.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/kafka-streams-examples-0.10.0.1.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/kafka-tools-0.10.0.1.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/log4j-1.2.17.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/lz4-1.3.0.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/metrics-core-2.2.0.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/osgi-resource-locator-1.0.1.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/reflections-0.9.10.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/rocksdbjni-4.8.0.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/scala-library-2.10.6.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/slf4j-api-1.7.21.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/slf4j-log4j12-1.7.21.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/snappy-java-1.1.2.6.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/validation-api-1.1.0.Final.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/zkclient-0.8.jar:/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin/../libs/zookeeper-3.4.6.jar (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:os.arch=amd64 (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:os.version=2.6.32-573.el6.x86_64 (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:user.name=xulu (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:user.home=/home/xulu (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,901] INFO Client environment:user.dir=/opt/cloud/kafka/kafka_2.10-0.10.0.1/bin (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,902] INFO Initiating client connection, connectString=10.20.24.231:3181,10.20.25.199:3181,10.20.25.241:3181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@40f8c9bf (org.apache.zookeeper.ZooKeeper)
[2016-12-19 09:47:46,917] INFO Waiting for keeper state SyncConnected (org.I0Itec.zkclient.ZkClient)
[2016-12-19 09:47:46,920] INFO Opening socket connection to server 10.20.25.199/10.20.25.199:3181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2016-12-19 09:47:46,925] WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.net.ConnectException: Connection refused
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[2016-12-19 09:47:47,031] INFO Opening socket connection to server 10.20.24.231/10.20.24.231:3181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2016-12-19 09:47:47,032] INFO Socket connection established to 10.20.24.231/10.20.24.231:3181, initiating session (org.apache.zookeeper.ClientCnxn)
[2016-12-19 09:47:47,196] INFO Session establishment complete on server 10.20.24.231/10.20.24.231:3181, sessionid = 0x1591492d7e70000, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2016-12-19 09:47:47,198] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
[2016-12-19 09:47:47,454] INFO Log directory '/tmp/kafka-logs' not found, creating it. (kafka.log.LogManager)
[2016-12-19 09:47:47,462] INFO Loading logs. (kafka.log.LogManager)
[2016-12-19 09:47:47,472] INFO Logs loading complete. (kafka.log.LogManager)
[2016-12-19 09:47:47,545] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
[2016-12-19 09:47:47,548] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
[2016-12-19 09:47:47,554] WARN No meta.properties file under dir /tmp/kafka-logs/meta.properties (kafka.server.BrokerMetadataCheckpoint)
[2016-12-19 09:47:47,609] INFO Awaiting socket connections on 0.0.0.0:9092. (kafka.network.Acceptor)
[2016-12-19 09:47:47,614] INFO [Socket Server on Broker 0], Started 1 acceptor threads (kafka.network.SocketServer)
[2016-12-19 09:47:47,639] INFO [ExpirationReaper-0], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2016-12-19 09:47:47,640] INFO [ExpirationReaper-0], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2016-12-19 09:47:47,765] INFO Creating /controller (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
[2016-12-19 09:47:47,784] INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
[2016-12-19 09:47:47,785] INFO 0 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
[2016-12-19 09:47:47,935] INFO [ExpirationReaper-0], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2016-12-19 09:47:47,936] INFO [ExpirationReaper-0], Starting  (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
[2016-12-19 09:47:47,954] INFO [GroupCoordinator 0]: Starting up. (kafka.coordinator.GroupCoordinator)
[2016-12-19 09:47:47,955] INFO [GroupCoordinator 0]: Startup complete. (kafka.coordinator.GroupCoordinator)
[2016-12-19 09:47:47,966] INFO [Group Metadata Manager on Broker 0]: Removed 0 expired offsets in 17 milliseconds. (kafka.coordinator.GroupMetadataManager)
[2016-12-19 09:47:47,982] INFO [ThrottledRequestReaper-Produce], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2016-12-19 09:47:47,983] INFO [ThrottledRequestReaper-Fetch], Starting  (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
[2016-12-19 09:47:48,000] INFO Will not load MX4J, mx4j-tools.jar is not in the classpath (kafka.utils.Mx4jLoader$)
[2016-12-19 09:47:48,044] INFO Creating /brokers/ids/0 (is it secure? false) (kafka.utils.ZKCheckedEphemeral)
[2016-12-19 09:47:48,057] INFO New leader is 0 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
[2016-12-19 09:47:48,064] INFO Result of znode creation is: OK (kafka.utils.ZKCheckedEphemeral)
[2016-12-19 09:47:48,067] INFO Registered broker 0 at path /brokers/ids/0 with addresses: PLAINTEXT -> EndPoint(SZB-L0032015,9092,PLAINTEXT) (kafka.utils.ZkUtils)
[2016-12-19 09:47:48,069] WARN No meta.properties file under dir /tmp/kafka-logs/meta.properties (kafka.server.BrokerMetadataCheckpoint)
[2016-12-19 09:47:48,119] INFO Kafka version : 0.10.0.1 (org.apache.kafka.common.utils.AppInfoParser)
[2016-12-19 09:47:48,119] INFO Kafka commitId : a7a17cdec9eaa6c5 (org.apache.kafka.common.utils.AppInfoParser)
[2016-12-19 09:47:48,120] INFO [Kafka Server 0], started (kafka.server.KafkaServer)、

Kafka简单测试
1.创建一个topic
参数信息：Zookeeper的信息，Topic的名字，Topic的Partition数，复制因子(复制因子必须小于等于Broker数目）

[xulu@SZB-L0032015 bin]$ ./kafka-topics.sh --create --zookeeper 10.20.24.231:3181 --replication-factor 1 --partitions 1 --topic test
Created topic "test".
[xulu@SZB-L0032015 bin]$ ./kafka-topics.sh --list --zookeeper 10.20.24.231:3181
test
[xulu@SZB-L0032015 bin]$

2.Producer创建消息
启动时，除了打印SLF4J之外，没有别的。下面可以直接输入生产的数据 (生产消息时不需要指定Partition，Kafka自动做Partition路由，每个Partition都是有Lead Partition和Follower Partitions组成，Lead Partition负责读写，而Follower Partitions只做复制，在Lead Partition挂了之后，自动做 Failover )

[xulu@SZB-L0032015 bin]$ ./kafka-console-producer.sh  --broker-list 10.20.25.199:9092 --topic test
this is message

3.Consumer消费消息
启动时，除了打印SLF4J之外，没有别的 –from-beginning实际上是指定offset的读取策略。
对于smallest和largest还是理解不到位，smallest和largest策略表示Zookeeper上的offset还没有初始化为正确值时，如何初始化offset的问题？试想，Producer生产了一批消息到Kafka中，但是Kafka尚未由任何Consumer读取，而Kafka的Offset是由Consumer进行初始化和赋值的，因此此时的Zookeeper上的offset并没有预期的0(0表示尚未读取过)，而是一个不正确的随机数，那么Consumer来读取消息时，是从头开始读还是从最大的位置等待Producer创建消息后再读取，此时就产生了两个选择，smallest表示从头读，largest表示从最大位置读
auto.offset.reset（默认是largest）:
What to do when there is no initial offset in ZooKeeper or if an offset is out of range:
smallest : automatically reset the offset to the smallest offset
largest : automatically reset the offset to the largest offset
anything else: throw exception to the consumer

[xulu@SZB-L0032015 bin]$ ./kafka-console-consumer.sh --zookeeper 10.20.25.241:3181 --topic test --from-beginning
this is message

4.此时在Producer终端输入内容，在Consumer终端可以立即收到