Pulsar源码解析-非分区Producer创建的底层实现

本章将详细介绍Producer创建底层的实现原理，介绍过程的代码是有删减的，只保留最核心的那部分，像参数校验这些会直接舍去，防止视觉干扰

一、客户端Producer创建入口

Producer创建是调用了PulsarClient#newProducer

public class PulsarClientImpl implements PulsarClient {
   
   

    public ProducerBuilder<byte[]> newProducer() {
   
   
        return new ProducerBuilderImpl<>(this, Schema.BYTES);
    }
    public <T> ProducerBuilder<T> newProducer(Schema<T> schema) {
   
   
        return new ProducerBuilderImpl<>(this, schema);
    }
    ....
}

又是建造者模式，使Api看起来更清晰、直观
还记得上一篇介绍PulsarClient创建时，有个配置对象，专门用来存所有参数的，我们看一下Producer是否也如此

public class ProducerBuilderImpl<T> implements ProducerBuilder<T> {
   
   

    private final PulsarClientImpl client;
    // 参数收集
    private ProducerConfigurationData conf;
    // 传输对象的约束
    private Schema<T> schema;
    // 发送前的拦截器
    private List<ProducerInterceptor> interceptorList;

	// 构造
    public ProducerBuilderImpl(PulsarClientImpl client, Schema<T> schema) {
   
   
        this(client, new ProducerConfigurationData(), schema);
    }
    // 参数配置Topic
    public ProducerBuilder<T> topic(String topicName) {
   
   
        conf.setTopicName(StringUtils.trim(topicName));
        return this;
    }
    // 参数配置发送超时时间
    public ProducerBuilder<T> sendTimeout(int sendTimeout, @NonNull TimeUnit unit) {
   
   
        conf.setSendTimeoutMs(sendTimeout, unit);
        return this;
    }
    public CompletableFuture<Producer<T>> createAsync() {
   
   
        return client.createProducerAsync(conf, schema, new ProducerInterceptors(interceptorList));
    }
    ...
 }

没错，路数一样，Consumer的创建不用猜也是这样。

可以看到，创建的动作是交给了PulsarClient#createProducerAsync()，而ProducerBuilderImpl只做了参数收集。那我们去看PulsarClient的实现createProducerAsync

二、客户端PulsarClient创建Producer分析

private <T> CompletableFuture<Producer<T>> createProducerAsync(String topic,
                                                                   ProducerConfigurationData conf,
                                                                   Schema<T> schema,
                                                                   ProducerInterceptors interceptors) {
   
                                            	
        CompletableFuture<Producer<T>> producerCreatedFuture = new CompletableFuture<>();
		
        getPartitionedTopicMetadata(topic).thenAccept(metadata -> {
   
   
            ProducerBase<T> producer;
            if (metadata.partitions > 0) {
   
   
                producer = newPartitionedProducerImpl(topic, conf, schema, interceptors, producerCreatedFuture,
                        metadata);
            } else {
   
   
                producer = newProducerImpl(topic, -1, conf, schema, interceptors, producerCreatedFuture);
            }

            producers.add(producer);
        }).exceptionally(ex -> {
   
   
            producerCreatedFuture.completeExceptionally(ex);
            return null;
        });

        return producerCreatedFuture;
    }

这里是先创建CompletableFuture，然后返回
提前返回，等到真正调用是get获取，异步编程将性能压缩到极致，Pulsar中使用了大量的异步编程，不熟悉CompletableFuture API的同学需要先熟悉一下。

上面代码getPartitionedTopicMetadata(topic)是获取topic的分区数，拿到分区数再决定创建分区Topic还是非分区Topic，创建完成放入producers，上一篇介绍过PulsarClient时初始化了两个集合将来收集生产消费者，你应该有印象。

本章仅介绍非分区TopicProducer实现，所以看else，通过构造的重载方法，最终是调用了ProducerImpl最全的那个构造

类图：

看构造之前先介绍一下类的继承关系

ProducerImpl：主要是发送消息的api
ProducerBase：定义发送消息api抽象方法以及消息体的构建入口
HandlerState：维护生产者的状态，例如 初始化、准备就绪、连接断开等

三、客户端Producer的构造分析

    public ProducerImpl(PulsarClientImpl client, String topic, ProducerConfigurationData conf,
                        CompletableFuture<Producer<T>> producerCreatedFuture, int partitionIndex, Schema<T> schema,
                        ProducerInterceptors interceptors) {
   
   
        super(client, topic, conf, producerCreatedFuture, schema, interceptors);
        // 一个链接下的生产者唯一标识，一个链接可以创建多个生产者
        this.producerId = client.newProducerId();
        this.producerName = conf.getProducerName();
        if (StringUtils.isNotBlank(producerName)) {
   
   
            this.userProvidedProducerName = true;
        }
        // 哪个分区，当前介绍的是非分区topic所以是-1
        this.partitionIndex = partitionIndex;
        // 发送中的消息队列，意思：调用服务端发送前放入队列，服务端响应成功从队列移除
        this.pendingMessages = createPendingMessagesQueue();
        // 如果服务端没有响应，数据全是在客户端，太多对内存有隐患。要限制最大数量
        if (conf.getMaxPendingMessages() > 0) {
   
   
            this.semaphore = Optional.of(new Semaphore(conf.getMaxPendingMessages(), true));
        } else {
   
   
            this.semaphore = Optional.empty();
        }
		// 消息压缩实现
        this.compressor = CompressionCodecProvider.getCompressionCodec(conf.getCompressionType());
		// 去重用的
        if (conf.getInitialSequenceId() != null) {
   
   
            long initialSequenceId = conf.getInitialSequenceId();
            this.lastSequenceIdPublished = initialSequenceId;
            this.lastSequenceIdPushed = initialSequenceId;
            this.msgIdGenerator = initialSequenceId + 1L;
        } else {
   
   
            this.lastSequenceIdPublished = -1L;
            this.lastSequenceIdPushed = -1L;
            this.msgIdGenerator = 0L;
        }
		// 为了数据传输安全，加密用的
        if (conf.isEncryptionEnabled()) {
   
   
            String logCtx = "[" + topic + "] [" + producerName + "] [" + producerId + "]";
            if (conf.getMessageCrypto() != null) {
   
   
                this.msgCrypto = conf.getMessageCrypto();
            } else {
   
   
                // default to use MessageCryptoBc;
                MessageCrypto msgCryptoBc;
                try {
   
   
                    msgCryptoBc = new MessageCryptoBc(logCtx, true);
                } catch (Exception e) {
   
   
                }
                this.msgCrypto = msgCryptoBc;
            }
        } else {
   
   
            this.msgCrypto = null;
        }
		// 固定间隔重新生成数据密钥密码
        if (this.msgCrypto != null) {
   
   
            // Regenerate data key cipher at fixed interval
            keyGeneratorTask = client.eventLoopGroup().scheduleWithFixedDelay(() -> {
   
   
                try {
   
   
                    msgCrypto.addPublicKeyCipher(conf.getEncryptionKeys(), conf.getCryptoKeyReader());
                } catch (CryptoException e) {
   
   
                }
            }, 0L, 4L, TimeUnit.HOURS);
        }
		// 发送超时控制
        if (conf.getSendTimeoutMs() > 0) {
   
   
            sendTimeout = client.timer().newTimeout(this, conf.getSendTimeoutMs(), TimeUnit.MILLISECONDS);
        }

        this.createProducerTimeout = System.currentTimeMillis() + client.getConfiguration().getOperationTimeoutMs();
        // 批量发送
        if (conf.isBatchingEnabled()) {
   
   
            BatcherBuilder containerBuilder = conf.getBatcherBuilder();
            if (containerBuilder == null) {
   
   
                containerBuilder = BatcherBuilder.DEFAULT;
            }
            this.batchMessageContainer = (BatchMessageContainerBase)containerBuilder.build();
            this.batchMessageContainer.setProducer(this);
        } else {
   
   
            this.batchMessageContainer = null;
        }
        // 数据打印频率
        if (client.getConfiguration().getStatsIntervalSeconds() > 0) {
   
   
            stats = new ProducerStatsRecorderImpl(client, conf, this);
        } else {
   
   
            stats = ProducerStatsDisabled.INSTANCE;
        }

        if (conf.getProperties().isEmpty()) {
   
   
            metadata = Collections.emptyMap();
        } else {
   
   
            metadata = Collections.unmodifiableMap(new HashMap<>(conf.getProperties()));
        }
		// 当前类引用传入ConnectionHandler
        this.connectionHandler = new ConnectionHandler(this,
        	new BackoffBuilder()
        	    .setInitialTime(client.getConfiguration().getInitialBackoffIntervalNanos(), TimeUnit.NANOSECONDS)
			    .setMax(client.getConfiguration().getMaxBackoffIntervalNanos(), TimeUnit.NANOSECONDS)<