SpringBoot/SpringCloud初探八（消息队列）

最新推荐文章于 2024-06-12 18:28:19 发布

十维之眼

最新推荐文章于 2024-06-12 18:28:19 发布

阅读量1.1k

点赞数

分类专栏：工程化知识文章标签： kafka java spring boot

本文链接：https://blog.youkuaiyun.com/AutumnGingkgo/article/details/122165322

版权

工程化知识专栏收录该内容

9 篇文章

订阅专栏

本文介绍SpringBoot/SpringCloud如何与Kafka集成，重点讲解了使用supplier和consumer方式替代旧版的sink和source，通过示例展示了publisher发布消息到主题，subscriber订阅并打印消息的过程。同时，提到了subscriber A作为另一个服务，如何从队列获取并处理不同主题的消息。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

样例的消息队列采用kafka。相关软件（kafka软件和zookeeper软件）的安装见初探一（安装篇）。

注意：新的版本，kafka不采用sink和source（注解被弃用），而是改用supplier和consumer方式。

演示架构

左侧分支是publisher启动即发布一个主题关联的消息到stream-kafka，然后subscriber（服务本身）从队列获取该主题的消息并打印出来；

右侧分支是访问http://localhost:7888/request，则发布另外一个主题相关的消息到stream-kafka，然后subscriber A（另一个服务）从队列获取该主题的消息并打印出来。

一、Publisher

pom.xml

引入spring-cloud-stream和spring-cloud-stream-binder-kafka

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.6.1</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.abcd</groupId>
    <artifactId>MessageStreamPublisher</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>MessageStreamPublisher</name>
    <description>MessageStreamPublisher</description>
    <properties>
        <java.version>1.8</java.version>
        <spring-cloud.version>2021.0.0</spring-cloud.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream-binder-kafka</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream</artifactId>
            <scope>test</scope>
            <classifier>test-binder</classifier>
            <type>test-jar</type>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

application.yml

server:
  port: 7888

eureka:
  client:
    service-url:
      defaultZone: http://localhost:8761/eureka/

spring:
  application:
    name: msg_producer
  cloud:
    stream:
      function:
        definition: testSupplier;testConsumer;publishMsg
      bindings:
        testSupplier-out-0:
          destination: topic-hello
          content-type: text/plain
        testConsumer-in-0:
          destination: topic-hello
        publishMsg-out-0:
          destination: topic-user-request
          content-type: text/plain
      kafka:
        binder:
          brokers: localhost:9092
          #zk-nodes: localhost:2181
          auto-create-topics: true
          required-acks: 1

java

application实现supplier和consumer同体，即Publisher和Subscriber的URL和端口一样。

特别注意：方法返回采用Supplier和Consumer（java.util.function.Supplier/Consumer）

Application代码

@EnableEurekaClient
@SpringBootApplication
public class MessageStreamPublisherApplication {

    private static final Logger log = LoggerFactory.getLogger(MessageStreamPublisherApplication.class);

    public static void main(String[] args) {
        SpringApplication.run(MessageStreamPublisherApplication.class, args);
    }

    @Bean
    public Supplier<String> testSupplier(){
        return () -> {
            String message = "www.test.com";
            log.info("Sending value: " + message);
            return message;
        };
    }

    @Bean
    public Consumer<String> testConsumer(){
        return message -> {
            log.info("Received message: " + message);
        };
    }
}

以下service和controller实现supplier和consumer不同体（不同微服务实现）

Publisher Controller代码

@RestController
public class PublishController {

    @Autowired
    public PublishService publishService;

    @GetMapping("request")
    public void log(){
        publishService.log("userRequest");
    }
}

Publisher service代码

@Service
public class PublishService {

    private BlockingQueue<String> mRequestList = new LinkedBlockingDeque<>();

    public void log(String userRequest){
        mRequestList.offer(userRequest);
    }

    @Bean
    public Supplier<String> publishMsg(){
        return () -> {
            String request = mRequestList.poll();
            System.out.println("publish msg: " + request);
            return request;
        };
    }
}

二、Subscriber A

pom.xml

引入spring-cloud-stream和spring-cloud-stream-binder-kafka

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.6.1</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>com.abcd</groupId>
    <artifactId>MessageStreamSubscriberA</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>MessageStreamSubscriberA</name>
    <description>MessageStreamSubscriberA</description>
    <properties>
        <java.version>1.8</java.version>
        <spring-cloud.version>2021.0.0</spring-cloud.version>
    </properties>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream-binder-kafka</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-stream</artifactId>
            <scope>test</scope>
            <classifier>test-binder</classifier>
            <type>test-jar</type>
        </dependency>
    </dependencies>
    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-dependencies</artifactId>
                <version>${spring-cloud.version}</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

application.yml

server:
  port: 7889

eureka:
  client:
    service-url:
      defaultZone: http://localhost:8761/eureka/

spring:
  application:
    name: msg_consumer_a
  cloud:
    stream:
      kafka:
        binder:
          brokers: localhost:9092
          #zk-nodes: localhost:2181
          auto-create-topics: true
      bindings:
        receiveMsg-in-0:
          destination: topic-user-request
      function:
        definition: receiveMsg

java

Application代码

@SpringBootApplication
public class MessageStreamSubscriberAApplication {

    public static void main(String[] args) {
        SpringApplication.run(MessageStreamSubscriberAApplication.class, args);
    }

}

Service代码

@Service
public class SubscriberAService {

    @Bean
    public Consumer<String> receiveMsg(){
        return request -> {
            System.out.println("SubscriberA receive msg: " + request);
        };
    }

}