一、下载和启动kafka
启动kafka服务之前要先启动zookeeper,目前kafka的压缩包中是包含zookeeper的
1、从官网下载kafka的Binary压缩包:http://kafka.apache.org/downloads
2、解压下载的压缩包,windows上解压后的kafka文件夹放在了D:\java\kafka_2.12-2.1.1\bin\windows目录下
3、windows启动zookeeper
D:\java\kafka_2.12-2.1.1\bin\windows>zookeeper-server-start.bat ..\..\config\zookeeper.properties
4、windows启动kafka
D:\java\kafka_2.12-2.1.1\bin\windows>kafka-server-start.bat ..\..\config\server.properties
5、Ubuntu解压运行
解压
tar zxvf kafka_2.12-2.1.1.tgz
运行Zookeeper
kafka_2.12-2.1.1/bin/zookeeper-server-start.sh kafka_2.12-2.1.1/config/zookeeper.properties
运行kafka
kafka_2.12-2.1.1/bin/kafka-server-start.sh kafka_2.12-2.1.1/config/server.properties
运行时注意相对路径和在不同的命令行窗口执行命令
二、创建springboot项目进行整合
1、导入kafka的jar包
<dependency>
<groupId>org.springframework.kafka</groupId>
<artifactId>spring-kafka</artifactId>
</dependency>
2、配置application.properties
spring.kafka.bootstrap-servers=127.0.0.1:9092
spring.kafka.consumer.group-id=chat_group
3、编写收发消息的代码
package com.jf.netty.nettytest.chat;
import org.apache.kafka.clients.admin.AdminClient;
import org.apache.kafka.clients.admin.NewTopic;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.core.KafkaAdmin;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.support.SendResult;
import org.springframework.stereotype.Service;
import org.springframework.util.concurrent.ListenableFuture;
import org.springframework.util.concurrent.ListenableFutureCallback;
import javax.annotation.PostConstruct;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.TimeUnit;
@Service
public class KafkaServiceImpl extends AbsQueueService {
private static Logger logger = LoggerFactory.getLogger(KafkaServiceImpl.class);
private static final String CHAT_MESSAGE_TOPIC = "chat_message";
@Autowired
RoomService roomService;
@Autowired
private KafkaTemplate<String, String> kafkaTemplate;
@Autowired
private KafkaAdmin kafkaAdmin;
/**
* 初始化Kafka Topic
*/
@PostConstruct
public void init() {
//第一个是参数是topic名字,第二个参数是分区个数,第三个是topic的复制因子个数
NewTopic newTopic = new NewTopic(CHAT_MESSAGE_TOPIC, 1, (short) 1);
List<NewTopic> topics = Arrays.asList(newTopic);
AdminClient adminClient = AdminClient.create(kafkaAdmin.getConfig());
adminClient.createTopics(topics);
adminClient.close(5, TimeUnit.SECONDS);
}
/**
* 发送kafka消息
* @param value
*/
public void send(String value) {
ListenableFuture<SendResult<String, String>> future = kafkaTemplate.send(CHAT_MESSAGE_TOPIC, value);
future.addCallback(new ListenableFutureCallback<SendResult<String, String>>() {
@Override
public void onFailure(Throwable throwable) {
logger.info("发送kafka消息失败");
//发送失败相关处理
}
@Override
public void onSuccess(SendResult<String, String> stringStringSendResult) {
// logger.info("发送kafka消息成功");
}
});
}
/**
* 消费kafka消息
* @param record
*/
@KafkaListener(topics = CHAT_MESSAGE_TOPIC)
public void listen(ConsumerRecord<?, String> record) {
String[] value = splitMessage(record.value());
ChatEvent chatEvent = ChatEventPool.get();
chatEvent.setEvent(2);
chatEvent.setChannelId(value[0]);
chatEvent.setMessage(value[1]);
roomService.addEvent(chatEvent);
}
}
说明:
- 此处把发布消息和消费消息写在了一起,大多数情况下都会在不同类或不同工程中
- init方法在初始化时先新建topic,理论上来说使用kafkaTemplate.send时会自动新建topic,但是使用中会报错
- listen方法中包含一些业务代码,忽略就好
- 消费消息方法支持通配符@KafkaListener(topicPattern = "topic.*")
- springboot集成kafka还有大量的配置参数在application.properties,用于优化性能,详见下方链接