Step by Step Recipe for Securing Kafka with Kerberos-优快云博客

本文提供了一步一步的教程，详细介绍了如何在HDP2.5.0和Ambari2.4.1环境下，使用Kerberos安全地配置Kafka集群，包括设置监听器、创建主题、授权用户及验证过程。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Short Description:

Step by Step Recipe for Securing Kafka with Kerberos.

Article

I found it is a little tricky to get started with a Kerberos enabled Kafka cluster. I created this step by step recipe for securing Kafka with Kerberos, sending and receiving data on console. This is tested on HDP2.5.0 and Ambari 2.4.1.

Enabled Kerberos using the Ambari Kerberos setup wizard under Admin -- Kerberos menu.
On Ambari Kafka Config UI, change "listeners" property to "PLAINTEXTSASL://localhost:6667".
Restart Kafka as requested by Ambari.
Create a test topic in Kafka. Must use the kafka service user to do this.

$ cd /usr/hdp/current/kafka-broker/bin
$ sudo su kafka
$ kinit -k -t /etc/security/keytabs/kafka.service.keytab kafka/ip-10-0-1-130.ap-northeast-1.compute.internal
 
$ ./kafka-topics.sh --zookeeper ip-10-0-1-130.ap-northeast-1.compute.internal:2181 --create --topic foo --partitions 1 --replication-factor 1
 
Created topic "bar".

Grant permission to user. This can be done using Kafka native ACL mechanism or Apache Ranger. In the example, we use Kafka ACL. User bob needs to be existing in KDC.

# Grant user bob as producer on topic foo
./kafka-acls.sh --authorizer-properties zookeeper.connect=ip-10-0-1-130.ap-northeast-1.compute.internal:2181 \
  --add --allow-principal User:bob \
  --producer --topic foo
 
Adding ACLs for resource `Topic:foo`:
  User:bob has Allow permission for operations: Describe from hosts: *
  User:bob has Allow permission for operations: Write from hosts: *
 
Adding ACLs for resource `Cluster:kafka-cluster`:
  User:bob has Allow permission for operations: Create from hosts: *
 
Current ACLs for resource `Topic:foo`:
  User:bob has Allow permission for operations: Describe from hosts: *
  User:bob has Allow permission for operations: Write from hosts: *
 
# Grant user bob as consumer
./kafka-acls.sh --authorizer-properties zookeeper.connect=ip-10-0-1-130.ap-northeast-1.compute.internal:2181 \
  --add --allow-principal User:bob \
  --consumer --topic foo --group *

#--group后等跟等号，如--group=*。按照上面写法，只是赋给名称为connect-distributed.sh的groupid。因为kafka/bin下面第一个脚本是connect-distributed.sh



 
Adding ACLs for resource `Topic:foo`:
  User:bob has Allow permission for operations: Read from hosts: *
  User:bob has Allow permission for operations: Describe from hosts: *
 
Adding ACLs for resource `Group:connect-distributed.sh`:
  User:bob has Allow permission for operations: Read from hosts: *
 
Current ACLs for resource `Topic:foo`:
  User:bob has Allow permission for operations: Read from hosts: *
  User:bob has Allow permission for operations: Describe from hosts: *
  User:bob has Allow permission for operations: Write from hosts: *
 
Current ACLs for resource `Group:connect-distributed.sh`:
  User:bob has Allow permission for operations: Read from hosts: *

Confirm the above works using the kafka console producer and consumer scripts.

# Switch to bob user and log in to KDC.
$ kinit bob
 
# Start console producer 
$ ./kafka-console-producer.sh --broker-list ip-10-0-1-130.ap-northeast-1.compute.internal:6667 --topic foo --security-protocol PLAINTEXTSASL
 
# On another terminal, start console consumer
./kafka-console-consumer.sh --zookeeper ip-10-0-1-130.ap-northeast-1.compute.internal:2181 --topic foo --security-protocol PLAINTEXTSASL 
 
{metadata.broker.list=ip-10-0-1-130.ap-northeast-1.compute.internal:6667, request.timeout.ms=30000, client.id=console-consumer-57797, security.protocol=PLAINTEXTSASL}
 
# Type something on the producer terminal, it should appears on the console terminal immediately.