使用bitnami/kafka:4.1镜像搭建单机kafka同时开启SASL/PLAIN认证和TLS加密

原创已于 2025-10-22 17:23:48 修改 · 973 阅读

20 ·

CC 4.0 BY-SA版权

文章标签：

#kafka

于 2025-10-22 17:23:23 首次发布

KAFKA 专栏收录该内容

1 篇文章

订阅专栏

使用bitnami/kafka:4.1镜像搭建单机kafka同时开启SASL/PLAIN认证和TLS加密

镜像：bitnami/kafka
版本：4.1
依赖镜像: bitnami/minideb:bookworm
仓库地址：https://github.com/bitnami/containers/tree/main/bitnami/kafka/4.1

拉去镜像minideb镜像

编译kafka镜像4.1时依赖该镜像


docker pull bitnami/minideb:bookworm

将bitnami开源的containers仓库clone到本地

git clone https://github.com/bitnami/containers.git

或直接点击该连接下载https://github.com/bitnami/containers/archive/refs/heads/main.zip

进入containers-main\bitnami\kafka\4.1\debian-12目录，也就是Dockerfile所在目录

cd containers-main\bitnami\kafka\4.1\debian-12

编译kafka镜像4.1

docker build -t bitnami/kafka41:4.1 .

在这里插入图片描述

编译完成后，查看镜像

docker images | grep bitnami/kafka41

在这里插入图片描述

生成证书

以下脚本是官方提供的，用于生成证书，在执行过程中，需要注意一下几点：

当系统提示输入密码时，请对所有人使用相同的密码。
将 Common Name 或 FQDN 值设置为 Apache Kafka 容器主机名，例如容器hostname是kafka.example.com .输入此值后，当提示“你的名字和姓氏是什么”时，也输入kafka.example.com。或者，可以禁用主机名验证，将环境变量KAFKA_CFG_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM设置为空字符串。
设置 Apache Kafka 集群时（查看“设置 Apache Kafka 集群”了解更多信息），每个 Apache Kafka 代理和逻辑客户端都需要自己的密钥库。您必须对集群中的每个代理重复该过程。

生成证书脚本（可直接复制脚本执行，不作任何修改）：

#!/usr/bin/env bash

set -e

KEYSTORE_FILENAME="kafka.keystore.jks"
VALIDITY_IN_DAYS=3650
DEFAULT_TRUSTSTORE_FILENAME="kafka.truststore.jks"
TRUSTSTORE_WORKING_DIRECTORY="truststore"
KEYSTORE_WORKING_DIRECTORY="keystore"
CA_CERT_FILE="ca-cert"
KEYSTORE_SIGN_REQUEST="cert-file"
KEYSTORE_SIGN_REQUEST_SRL="ca-cert.srl"
KEYSTORE_SIGNED_CERT="cert-signed"

function file_exists_and_exit() {
  echo "'$1' cannot exist. Move or delete it before"
  echo "re-running this script."
  exit 1
}

if [ -e "$KEYSTORE_WORKING_DIRECTORY" ]; then
  file_exists_and_exit $KEYSTORE_WORKING_DIRECTORY
fi

if [ -e "$CA_CERT_FILE" ]; then
  file_exists_and_exit $CA_CERT_FILE
fi

if [ -e "$KEYSTORE_SIGN_REQUEST" ]; then
  file_exists_and_exit $KEYSTORE_SIGN_REQUEST
fi

if [ -e "$KEYSTORE_SIGN_REQUEST_SRL" ]; then
  file_exists_and_exit $KEYSTORE_SIGN_REQUEST_SRL
fi

if [ -e "$KEYSTORE_SIGNED_CERT" ]; then
  file_exists_and_exit $KEYSTORE_SIGNED_CERT
fi

echo
echo "Welcome to the Kafka SSL keystore and truststore generator script."

echo
echo "First, do you need to generate a trust store and associated private key,"
echo "or do you already have a trust store file and private key?"
echo
echo -n "Do you need to generate a trust store and associated private key? [yn] "
read generate_trust_store

trust_store_file=""
trust_store_private_key_file=""

if [ "$generate_trust_store" == "y" ]; then
  if [ -e "$TRUSTSTORE_WORKING_DIRECTORY" ]; then
    file_exists_and_exit $TRUSTSTORE_WORKING_DIRECTORY
  fi

  mkdir $TRUSTSTORE_WORKING_DIRECTORY
  echo
  echo "OK, we'll generate a trust store and associated private key."
  echo
  echo "First, the private key."
  echo
  echo "You will be prompted for:"
  echo " - A password for the private key. Remember this."
  echo " - Information about you and your company."
  echo " - NOTE that the Common Name (CN) is currently not important."

  openssl req -new -x509 -keyout $TRUSTSTORE_WORKING_DIRECTORY/ca-key \
    -out $TRUSTSTORE_WORKING_DIRECTORY/$CA_CERT_FILE -days $VALIDITY_IN_DAYS

  trust_store_private_key_file="$TRUSTSTORE_WORKING_DIRECTORY/ca-key"

  echo
  echo "Two files were created:"
  echo " - $TRUSTSTORE_WORKING_DIRECTORY/ca-key -- the private key used later to"
  echo "   sign certificates"
  echo " - $TRUSTSTORE_WORKING_DIRECTORY/$CA_CERT_FILE -- the certificate that will be"
  echo "   stored in the trust store in a moment and serve as the certificate"
  echo "   authority (CA). Once this certificate has been stored in the trust"
  echo "   store, it will be deleted. It can be retrieved from the trust store via:"
  echo "   $ keytool -keystore <trust-store-file> -export -alias CARoot -rfc"

  echo
  echo "Now the trust store will be generated from the certificate."
  echo
  echo "You will be prompted for:"
  echo " - the trust store's password (labeled 'keystore'). Remember this"
  echo " - a confirmation that you want to import the certificate"

  keytool -keystore $TRUSTSTORE_WORKING_DIRECTORY/$DEFAULT_TRUSTSTORE_FILENAME \
    -alias CARoot -import -file $TRUSTSTORE_WORKING_DIRECTORY/$CA_CERT_FILE

  trust_store_file="$TRUSTSTORE_WORKING_DIRECTORY/$DEFAULT_TRUSTSTORE_FILENAME"

  echo
  echo "$TRUSTSTORE_WORKING_DIRECTORY/$DEFAULT_TRUSTSTORE_FILENAME was created."

  # don't need the cert because it's in the trust store.
  rm $TRUSTSTORE_WORKING_DIRECTORY/$CA_CERT_FILE
else
  echo
  echo -n "Enter the path of the trust store file. "
  read -e trust_store_file

  if ! [ -f $trust_store_file ]; then
    echo "$trust_store_file isn't a file. Exiting."
    exit 1
  fi

  echo -n "Enter the path of the trust store's private key. "
  read -e trust_store_private_key_file

  if ! [ -f $trust_store_private_key_file ]; then
    echo "$trust_store_private_key_file isn't a file. Exiting."
    exit 1
  fi
fi

echo
echo "Continuing with:"
echo " - trust store file:        $trust_store_file"
echo " - trust store private key: $trust_store_private_key_file"

mkdir $KEYSTORE_WORKING_DIRECTORY

echo
echo "Now, a keystore will be generated. Each broker and logical client needs its own"
echo "keystore. This script will create only one keystore. Run this script multiple"
echo "times for multiple keystores."
echo
echo "You will be prompted for the following:"
echo " - A keystore password. Remember it."
echo " - Personal information, such as your name."
echo "     NOTE: currently in Kafka, the Common Name (CN) does not need to be the FQDN of"
echo "           this host. However, at some point, this may change. As such, make the CN"
echo "           the FQDN. Some operating systems call the CN prompt 'first / last name'"
echo " - A key password, for the key being generated within the keystore. Remember this."

# To learn more about CNs and FQDNs, read:
# https://docs.oracle.com/javase/7/docs/api/javax/net/ssl/X509ExtendedTrustManager.html

keytool -keystore $KEYSTORE_WORKING_DIRECTORY/$KEYSTORE_FILENAME \
  -alias localhost -validity $VALIDITY_IN_DAYS -genkey -keyalg RSA

echo
echo "'$KEYSTORE_WORKING_DIRECTORY/$KEYSTORE_FILENAME' now contains a key pair and a"
echo "self-signed certificate. Again, this keystore can only be used for one broker or"
echo "one logical client. Other brokers or clients need to generate their own keystores."

echo
echo "Fetching the certificate from the trust store and storing in $CA_CERT_FILE."
echo
echo "You will be prompted for the trust store's password (labeled 'keystore')"

keytool -keystore $trust_store_file -export -alias CARoot -rfc -file $CA_CERT_FILE

echo
echo "Now a certificate signing request will be made to the keystore."
echo
echo "You will be prompted for the keystore's password."
keytool -keystore $KEYSTORE_WORKING_DIRECTORY/$KEYSTORE_FILENAME -alias localhost \
  -certreq -file $KEYSTORE_SIGN_REQUEST

echo
echo "Now the trust store's private key (CA) will sign the keystore's certificate."
echo
echo "You will be prompted for the trust store's private key password."
openssl x509 -req -CA $CA_CERT_FILE -CAkey $trust_store_private_key_file \
  -in $KEYSTORE_SIGN_REQUEST -out $KEYSTORE_SIGNED_CERT \
  -days $VALIDITY_IN_DAYS -CAcreateserial
# creates $KEYSTORE_SIGN_REQUEST_SRL which is never used or needed.

echo
echo "Now the CA will be imported into the keystore."
echo
echo "You will be prompted for the keystore's password and a confirmation that you want to"
echo "import the certificate."
keytool -keystore $KEYSTORE_WORKING_DIRECTORY/$KEYSTORE_FILENAME -alias CARoot \
  -import -file $CA_CERT_FILE
rm $CA_CERT_FILE # delete the trust store cert because it's stored in the trust store.

echo
echo "Now the keystore's signed certificate will be imported back into the keystore."
echo
echo "You will be prompted for the keystore's password."
keytool -keystore $KEYSTORE_WORKING_DIRECTORY/$KEYSTORE_FILENAME -alias localhost -import \
  -file $KEYSTORE_SIGNED_CERT

echo
echo "All done!"
echo
echo "Delete intermediate files? They are:"
echo " - '$KEYSTORE_SIGN_REQUEST_SRL': CA serial number"
echo " - '$KEYSTORE_SIGN_REQUEST': the keystore's certificate signing request"
echo "   (that was fulfilled)"
echo " - '$KEYSTORE_SIGNED_CERT': the keystore's certificate, signed by the CA, and stored back"
echo "    into the keystore"
echo -n "Delete? [yn] "
read delete_intermediate_files

if [ "$delete_intermediate_files" == "y" ]; then
  rm $KEYSTORE_SIGN_REQUEST_SRL
  rm $KEYSTORE_SIGN_REQUEST
  rm $KEYSTORE_SIGNED_CERT
fi

脚本执行完成后会生成以下文件：需要关注生成文件的路径，默认是当前目录。在编辑docker-compose.yml文件时，需要将生成的文件挂载到容器中。

kafka.keystore.jks：密钥库文件，包含了 Kafka 服务器的证书和私钥。
kafka.truststore.jks：信任库文件，包含了 Kafka 服务器的证书。

在当前目录新建client-ssl.properties文件，内容如下。在docker-compose.yml文件中，需要将该文件挂载到容器中。

security.protocol=SASL_SSL
ssl.truststore.location=/opt/bitnami/kafka/config/certs/kafka.truststore.jks
ssl.truststore.password=12345678
ssl.endpoint.identification.algorithm=
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="123456";

开启TLS加密和SASL/PLAIN认证配置文件

在配置文件中，需要注意一下几点:

需要将KAFKA_CFG_ADVERTISED_LISTENERS配置为SASL_SSL://192.168.42.165:9092，其中192.168.42.165为Kafka 容器的宿主机 IP 地址。
hostname 为 Kafka 容器的主机名，与上文脚本中生成的证书中的CN、姓名等一致。
配置文件中提到的KAFKA_TLS_KEYSTORE_LOCATION、KAFKA_TLS_TRUSTSTORE_LOCATION需要与生成的证书文件路径一致。
配置文件中提到的KAFKA_CERTIFICATE_PASSWORD为生成的证书密码，需要与配置文件中一致。
volumes 中需要挂载生成的证书文件。
KAFKA_CERTIFICATE_PASSWORD是证书store密码，配置文件需要与实际设置的一致。

SASL认证中各个参数含义：

Controller认证

用于Kafka Controller节点与其他broker之间的认证，Controller负责分区leader选举、副本分配等管理任务，负责处理 Kafka 集群的元数据管理和协调。

KAFKA_CONTROLLER_USER=admin          # Controller节点的用户名
KAFKA_CONTROLLER_PASSWORD=123456     # Controller节点的密码

Broker间认证

用于Kafka集群中不同broker节点之间的内部通信，确保broker间的数据同步和通信安全。

KAFKA_INTER_BROKER_USER=admin        # Broker间通信的用户名
KAFKA_INTER_BROKER_PASSWORD=123456   # Broker间通信的密码

客户端认证

用于生产者和消费者客户端连接Kafka时的认证，确保客户端与Kafka集群的安全通信。支持多个用户，用逗号分隔

KAFKA_CLIENT_USERS=admin             # 客户端连接的用户名
KAFKA_CLIENT_PASSWORDS=123456        # 客户端连接的密码

version: '2'

services:
  kafka:
    image: bitnami/kafka41:latest
    hostname: kafka.example.com
    ports:
      - "9092:9092"
    environment:
      # KRaft
      - KAFKA_CFG_NODE_ID=0
      - KAFKA_CFG_PROCESS_ROLES=controller,broker
      - KAFKA_CFG_CONTROLLER_QUORUM_VOTERS=0@kafka:9093

      - BITNAMI_DEBUG=true  # 开启调试模式
      # Listeners
      - KAFKA_CFG_LISTENERS=SASL_SSL://:9092,CONTROLLER://:9093
      - KAFKA_CFG_LISTENER_NAME_SASL_SSL_SSL_CLIENT_AUTH=none  # 如果启用required，需要客户端上传证书。
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CONTROLLER:PLAINTEXT,SASL_SSL:SASL_SSL
      - KAFKA_CFG_ADVERTISED_LISTENERS=SASL_SSL://192.168.42.165:9092
      - KAFKA_CFG_CONTROLLER_LISTENER_NAMES=CONTROLLER
      - KAFKA_CFG_INTER_BROKER_LISTENER_NAME=SASL_SSL

      - KAFKA_TLS_KEYSTORE_LOCATION=/opt/bitnami/kafka/config/certs/kafka.keystore.jks
      - KAFKA_TLS_TRUSTSTORE_LOCATION=/opt/bitnami/kafka/config/certs/kafka.truststore.jks
      # - KAFKA_CLIENT_LISTENER_NAME=SASL_SSL # Remove this line if consumer/producer.properties are not required
      # SASL
      - KAFKA_CFG_SASL_MECHANISM_CONTROLLER_PROTOCOL=PLAIN
      - KAFKA_CFG_SASL_MECHANISM_INTER_BROKER_PROTOCOL=PLAIN
      - KAFKA_CONTROLLER_USER=admin
      - KAFKA_CONTROLLER_PASSWORD=123456
      - KAFKA_INTER_BROKER_USER=admin
      - KAFKA_INTER_BROKER_PASSWORD=123456
      - KAFKA_CLIENT_USERS=admin
      - KAFKA_CLIENT_PASSWORDS=123456
      # SSL
      - KAFKA_TLS_TYPE=JKS # or PEM
      - KAFKA_CERTIFICATE_PASSWORD=12345678
      - KAFKA_CFG_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM=none
    volumes:
      - /home/lwd/kafka/TLS/keystore/kafka.keystore.jks:/opt/bitnami/kafka/config/certs/kafka.keystore.jks
      - /home/lwd/kafka/TLS/truststore/kafka.truststore.jks:/opt/bitnami/kafka/config/certs/kafka.truststore.jks
      - ./client-ssl.properties:/client-ssl.properties

执行命令查看主题列表

kafka-topics.sh --bootstrap-server 192.168.42.165:9092 --list --command-config client-ssl.properties

如果没有设置上文的client-ssl.propertie文件，可进入容器后执行以下命令创建主题

kafka-topics.sh \
  --bootstrap-server 192.168.42.165:9092 \
  --create \
  --topic test-topic \
  --partitions 3 \
  --replication-factor 1 \
  --command-config <(cat << EOF
security.protocol=SASL_SSL
ssl.truststore.location=/opt/bitnami/kafka/config/certs/kafka.truststore.jks
ssl.truststore.password=12345678
ssl.endpoint.identification.algorithm=
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="123456";
EOF
)

执行以下命令查看主题列表

kafka-topics.sh \
  --bootstrap-server 192.168.42.165:9092 \
  --list \
  --command-config <(cat << EOF
security.protocol=SASL_SSL
ssl.truststore.location=/opt/bitnami/kafka/config/certs/kafka.truststore.jks
ssl.truststore.password=12345678
ssl.endpoint.identification.algorithm=
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="123456";
EOF
)