RocketMq源码-Producer(三)

一草三心

已于 2024-12-08 12:05:33 修改

阅读量1.3k

点赞数 40

文章标签： java-rocketmq rocketmq 学习

于 2024-12-07 15:56:23 首次发布

本文链接：https://blog.youkuaiyun.com/qq_40228720/article/details/144309049

版权

一、Producer启动流程

DefaultMQProducer设置了NamesrvAddr地址，需要从nameserver获取broker信息

public static void main(String[] args) throws MQClientException, InterruptedException {

	System.setProperty("mqself.home","F:\\rocketmq");
	DefaultMQProducer producer = new DefaultMQProducer("please_rename_unique_group_name");

	producer.setNamesrvAddr("127.0.0.1:9876");

	producer.setInstanceName("dsk");
	producer.start();
	
	for (int i = 0; i < 2; i++) {
		try {
			Message msg = new Message("TopicTest" /* Topic */,
				"TagA" /* Tag */,
				("Hello RocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET) /* Message body */
			);

		   
			producer.setSendMsgTimeout(1000000000);
			SendResult sendResult = producer.send(msg);
			
			System.out.printf("%s%n", sendResult);
		} catch (Exception e) {
			e.printStackTrace();
			Thread.sleep(1000);
		}
	}
    producer.shutdown();
}

一、producer#start方法

一、注册producer

所有Producer的启动过程，最终都会调用DefaultMQProducerImpl#start方法。在start方法中的通过一个mQClientFactory对象，启动生产者的一大堆重要服务。这里其实就是一种设计模式，虽然有很多种不同的客户端，但是这些客户端的启动流程最终都是统一的，全是交由mQClientFactory对象来启动。

this.mQClientFactory = MQClientManager.getInstance().getOrCreateMQClientInstance(this.defaultMQProducer, rpcHook);

boolean registerOK = mQClientFactory.registerProducer(this.defaultMQProducer.getProducerGroup(), this);

而不同之处在于这些客户端在启动过程中，按照服务端的要求注册不同的信息。例如生产者注册到producerTable，消费者注册到consumerTable，管理控制端注册到adminExtTable。

public synchronized boolean registerProducer(final String group, final DefaultMQProducerImpl producer) {
	if (null == group || null == producer) {
		return false;
	}

	MQProducerInner prev = this.producerTable.putIfAbsent(group, producer);
	if (prev != null) {
		log.warn("the producer group[{}] exist already.", group);
		return false;
	}

	return true;
}

二、mQClientFactory#start

mQClientFactory#start启动生产者的一些重要服务

public void start() throws MQClientException {

	synchronized (this) {
		switch (this.serviceState) {
			case CREATE_JUST:
				this.serviceState = ServiceState.START_FAILED;
				// If not specified,looking address from name server
				if (null == this.clientConfig.getNamesrvAddr()) {
					this.mQClientAPIImpl.fetchNameServerAddr();
				}
				// Start request-response channel
				this.mQClientAPIImpl.start();
				// Start various schedule tasks
				this.startScheduledTask();
				// Start pull service
				this.pullMessageService.start();
				// Start rebalance service
				this.rebalanceService.start();
				// Start push service
				this.defaultMQProducer.getDefaultMQProducerImpl().start(false);
				log.info("the client factory [{}] start OK", this.clientId);
				this.serviceState = ServiceState.RUNNING;
				break;
			case START_FAILED:
				throw new MQClientException("The Factory object[" + this.getClientId() + "] has been created before, and failed.", null);
			default:
				break;
		}
	}
}

一、启动netty客户端

启动netty客户端，具备远程通信功能

// Start request-response channel
this.mQClientAPIImpl.start();

public void start() {
	this.remotingClient.start();
}

二、定时服务拉取nameserver信息

this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
	@Override
	public void run() {
		try {
			MQClientInstance.this.mQClientAPIImpl.fetchNameServerAddr();
		} catch (Exception e) {
			log.error("ScheduledTask fetchNameServerAddr exception", e);
		}
	}
}, 1000 * 10, 1000 * 60 * 2, TimeUnit.MILLISECONDS);

利用remotingClient向nameServer发送请求

public void updateNameServerAddressList(final String addrs) {
	String[] addrArray = addrs.split(";");
	List<String> list = Arrays.asList(addrArray);
	this.remotingClient.updateNameServerAddressList(list);
}

三、定时更新Topic信息

启动定时线程从nameServer更新topic信息

 this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
	@Override
	public void run() {
		try {
			MQClientInstance.this.updateTopicRouteInfoFromNameServer();
		} catch (Exception e) {
			log.error("ScheduledTask updateTopicRouteInfoFromNameServer exception", e);
		}
	}
}, 10, this.clientConfig.getPollNameServerInterval(), TimeUnit.MILLISECONDS);

// Producer
{
	Iterator<Entry<String, MQProducerInner>> it = this.producerTable.entrySet().iterator();
	while (it.hasNext()) {
		Entry<String, MQProducerInner> entry = it.next();
		MQProducerInner impl = entry.getValue();
		if (impl != null) {
			Set<String> lst = impl.getPublishTopicList();
			topicList.addAll(lst);
		}
	}
}

for (String topic : topicList) {
	this.updateTopicRouteInfoFromNameServer(topic);
}

producer端根据topic名称向nameserver发送请求，获取到的响应信息topicRouteData如下：

queueDatas属性维护了队列信息跟broker的信息，因为可能多个队列分别分散在不同的broker机器上。

brokerDatas属性维护了brokerName与brokerAddrs的信息，例如broker-a可能在多台机器上面。

所以当producer想要发送信息时就是根据topic名称获取到topicRouteData，然后再根据要发送的队列选择对应broker的所在机器地址，然后直接将请求发送给broker端。

四、启动定时器处理过期请求

请求超时处理，后面发送消息会详细解释。

private void startScheduledTask() {
	if (RequestFutureTable.getProducerNum().incrementAndGet() == 1) {
		this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
			@Override
			public void run() {
				try {
					RequestFutureTable.scanExpiredRequest();
				} catch (Throwable e) {
					log.error("scan RequestFutureTable exception", e);
				}
			}
		}, 1000 * 3, 1000, TimeUnit.MILLISECONDS);
	}
}

二、producer#send方法

一、Message信息

创建时需要指定topic信息，tag信息，字节数组。

/*
 * Create a message instance, specifying topic, tag and message body.
 */
Message msg = new Message("TopicTest" /* Topic */,
	"TagA" /* Tag */,
	("Hello RocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET) /* Message body */
);

二、消息发送

可靠同步发送方式：消息发送方发出数据后，同步等待，直到收到接收方发回响应之后才发下一个请求。

/*
 * Call send message to deliver message to one of brokers.
 */
producer.setSendMsgTimeout(1000000000);
SendResult sendResult = producer.send(msg);

for (int i = 0; i < 2; i++) {
	try {

		/*
		 * Create a message instance, specifying topic, tag and message body.
		 */
		Message msg = new Message("TopicTest" /* Topic */,
			"TagA" /* Tag */,
			("Hello RocketMQ " + i).getBytes(RemotingHelper.DEFAULT_CHARSET) /* Message body */
		);

		/*
		 * Call send message to deliver message to one of brokers.
		 */
		producer.setSendMsgTimeout(1000000000);
		SendResult sendResult = producer.send(msg);

		System.out.printf("%s%n", sendResult);
	} catch (Exception e) {
		e.printStackTrace();
		Thread.sleep(1000);
	}
}

三、关于producer端负载均衡

Producer在获取路由信息后，会选出一个MessageQueue去发送消息。这个选MessageQueue的方法就是一个索引自增然后取模的方式。然后根据MessageQueue再找所在的Broker，往Broker发送请求。

public MessageQueue selectOneMessageQueue(final TopicPublishInfo tpInfo, final String lastBrokerName) {
	if (this.sendLatencyFaultEnable) {
		try {
			int index = tpInfo.getSendWhichQueue().incrementAndGet();
			for (int i = 0; i < tpInfo.getMessageQueueList().size(); i++) {
				int pos = Math.abs(index++) % tpInfo.getMessageQueueList().size();
				if (pos < 0)
					pos = 0;
				MessageQueue mq = tpInfo.getMessageQueueList().get(pos);
				if (latencyFaultTolerance.isAvailable(mq.getBrokerName()))
					return mq;
			}

			final String notBestBroker = latencyFaultTolerance.pickOneAtLeast();
			int writeQueueNums = tpInfo.getQueueIdByBroker(notBestBroker);
			if (writeQueueNums > 0) {
				final MessageQueue mq = tpInfo.selectOneMessageQueue();
				if (notBestBroker != null) {
					mq.setBrokerName(notBestBroker);
					mq.setQueueId(tpInfo.getSendWhichQueue().incrementAndGet() % writeQueueNums);
				}
				return mq;
			} else {
				latencyFaultTolerance.remove(notBestBroker);
			}
		} catch (Exception e) {
			log.error("Error occurred when selecting message queue", e);
		}

		return tpInfo.selectOneMessageQueue();
	}

	return tpInfo.selectOneMessageQueue(lastBrokerName);
}

四、源码流程

DefaultMQProducer.send() -> DefaultMQProducerImpl.send() -> DefaultMQProducerImpl.sendDefaultImpl()->DefaultMQProducerImpl.sendKernelImpl->MQClientAPIImpl.sendMessageSync

case SYNC:
	long costTimeSync = System.currentTimeMillis() - beginStartTime;
	if (timeoutMillis < costTimeSync) {
		throw new RemotingTooMuchRequestException("sendMessage call timeout");
	}
	return this.sendMessageSync(addr, brokerName, msg, timeoutMillis - costTimeSync, request);

private SendResult sendMessageSync(
	final String addr,
	final String brokerName,
	final Message msg,
	final long timeoutMillis,
	final RemotingCommand request
) throws RemotingException, MQBrokerException, InterruptedException {
	RemotingCommand response = this.remotingClient.invokeSync(addr, request, timeoutMillis);
	assert response != null;
	return this.processSendResponse(brokerName, msg, response,addr);
}

流程总结：
核心分析sendDefaultImpl 方法：
1. 获取主题路由相关信息
2. for 循环发送(发送次数由retryTimesWhenSendFailed+1 来决定)
3. 调用sendKernelImpl 方法，获取路由表信息（如果没有则会从NameServer 中获取）；通过判断发送类型设置不同的入参，但是最终都调用了MQClientAPIImpl 类的sendMessage 方法。
4.sendMessageSync –>NettyRemotingClient.invokeSync()完成发送。NettyRemotingAbstract中的invokeSyncImpl 里面会大量使用Netty 进行调用。
5. 不同发送方式的sendResult 处理不同，如果是SYNC模式，如果发送失败那么就continue需要重新发送，如果是ONEWAY那么发送失败也不会做任何处理，所以这种方式是不可靠的；ASYNC模式下面讲解。

switch (communicationMode) {
	case ASYNC:
		return null;
	case ONEWAY:
		return null;
	case SYNC:
		if (sendResult.getSendStatus() != SendStatus.SEND_OK) {
			if (this.defaultMQProducer.isRetryAnotherBrokerWhenNotStoreOK()) {
				continue;
			}
		}

		return sendResult;
	default:
		break;
}

五、消息发送响应处理

每个request中都有一个属性Opaque，这个值是唯一的，表示当前请求编号；发送请求时会向responseTable中放入一个ResponseFuture，key为opaque；那么此时通过responseTable就可以将请求跟异步响应关联。

public RemotingCommand invokeSyncImpl(final Channel channel, final RemotingCommand request,
	final long timeoutMillis)
	throws InterruptedException, RemotingSendRequestException, RemotingTimeoutException {
	final int opaque = request.getOpaque();

	try {
		final ResponseFuture responseFuture = new ResponseFuture(channel, opaque, timeoutMillis, null, null);
		this.responseTable.put(opaque, responseFuture);
		final SocketAddress addr = channel.remoteAddress();
		channel.writeAndFlush(request).addListener(new ChannelFutureListener() {
			@Override
			public void operationComplete(ChannelFuture f) throws Exception {
				if (f.isSuccess()) {
					responseFuture.setSendRequestOK(true);
					return;
				} else {
					responseFuture.setSendRequestOK(false);
				}

				responseTable.remove(opaque);
				responseFuture.setCause(f.cause());
				responseFuture.putResponse(null);
				log.warn("send a request command to channel <" + addr + "> failed.");
			}
		});

		RemotingCommand responseCommand = responseFuture.waitResponse(timeoutMillis);
		if (null == responseCommand) {
			if (responseFuture.isSendRequestOK()) {
				throw new RemotingTimeoutException(RemotingHelper.parseSocketAddressAddr(addr), timeoutMillis,
					responseFuture.getCause());
			} else {
				throw new RemotingSendRequestException(RemotingHelper.parseSocketAddressAddr(addr), responseFuture.getCause());
			}
		}

		return responseCommand;
	} finally {
		this.responseTable.remove(opaque);
	}
}

当producer接收到远程的响应，就会根据opaque从responseTable中取出responseFuture，然后执行responseFuture中的invokeCallBack方法。

public void processResponseCommand(ChannelHandlerContext ctx, RemotingCommand cmd) {
	final int opaque = cmd.getOpaque();
	final ResponseFuture responseFuture = responseTable.get(opaque);
	if (responseFuture != null) {
		responseFuture.setResponseCommand(cmd);

		responseTable.remove(opaque);

		if (responseFuture.getInvokeCallback() != null) {
			executeInvokeCallback(responseFuture);
		} else {
			responseFuture.putResponse(cmd);
			responseFuture.release();
		}
	} else {
		log.warn("receive response, but not matched any request, " + RemotingHelper.parseChannelRemoteAddr(ctx.channel()));
		log.warn(cmd.toString());
	}
}

如果迟迟收不到响应，那么定时器就会将responseFuture从responseTable移除并执行invokeCallBack通知请求超时。

public void scanResponseTable() {
	final List<ResponseFuture> rfList = new LinkedList<ResponseFuture>();
	Iterator<Entry<Integer, ResponseFuture>> it = this.responseTable.entrySet().iterator();
	while (it.hasNext()) {
		Entry<Integer, ResponseFuture> next = it.next();
		ResponseFuture rep = next.getValue();

		if ((rep.getBeginTimestamp() + rep.getTimeoutMillis() + 1000) <= System.currentTimeMillis()) {
			rep.release();
			it.remove();
			rfList.add(rep);
			log.warn("remove timeout request, " + rep);
		}
	}

	for (ResponseFuture rf : rfList) {
		try {
			executeInvokeCallback(rf);
		} catch (Throwable e) {
			log.warn("scanResponseTable, operationComplete Exception", e);
		}
	}
}

六、异步发送消息

也很容易理解，将发送消息的任务丢到了线程池中执行，并准备了一个SendCallback回调方法。这是与同步发送的区别之一。

producer.send(msg, new SendCallback() {
   @Override
   public void onSuccess(SendResult sendResult) {
	   // do something
   }

   @Override
   public void onException(Throwable e) {
	   // do something
   }
});

public void send(final Message msg, final SendCallback sendCallback, final long timeout)
	throws MQClientException, RemotingException, InterruptedException {
	final long beginStartTime = System.currentTimeMillis();
	ExecutorService executor = this.getAsyncSenderExecutor();
	try {
		executor.submit(new Runnable() {
			@Override
			public void run() {
				long costTime = System.currentTimeMillis() - beginStartTime;
				if (timeout > costTime) {
					try {
						sendDefaultImpl(msg, CommunicationMode.ASYNC, sendCallback, timeout - costTime);
					} catch (Exception e) {
						sendCallback.onException(e);
					}
				} else {
					sendCallback.onException(
						new RemotingTooMuchRequestException("DEFAULT ASYNC send call timeout"));
				}
			}

		});
	} catch (RejectedExecutionException e) {
		throw new MQClientException("executor rejected ", e);
	}
}

区别二就是同步发送最终调用的方法是NettyRemotingClient#invokeSync，而异步发送最终调用的方法是NettyRemotingClient#invokeAsync。

异步方法里面定义了一个semaphoreAsync信号量来控制并发，同一时间不能有太多并发，否则broker服务器可能承受不住。

public void invokeAsyncImpl(final Channel channel, final RemotingCommand request, final long timeoutMillis,
	final InvokeCallback invokeCallback)
	throws InterruptedException, RemotingTooMuchRequestException, RemotingTimeoutException, RemotingSendRequestException {
	long beginStartTime = System.currentTimeMillis();
	final int opaque = request.getOpaque();
	boolean acquired = this.semaphoreAsync.tryAcquire(timeoutMillis, TimeUnit.MILLISECONDS);
	if (acquired) {
		final SemaphoreReleaseOnlyOnce once = new SemaphoreReleaseOnlyOnce(this.semaphoreAsync);
		long costTime = System.currentTimeMillis() - beginStartTime;
		if (timeoutMillis < costTime) {
			once.release();
			throw new RemotingTimeoutException("invokeAsyncImpl call timeout");
		}

		final ResponseFuture responseFuture = new ResponseFuture(channel, opaque, timeoutMillis - costTime, invokeCallback, once);
		this.responseTable.put(opaque, responseFuture);
		try {
			channel.writeAndFlush(request).addListener(new ChannelFutureListener() {
				@Override
				public void operationComplete(ChannelFuture f) throws Exception {
					if (f.isSuccess()) {
						responseFuture.setSendRequestOK(true);
						return;
					}
					requestFail(opaque);
					log.warn("send a request command to channel <{}> failed.", RemotingHelper.parseChannelRemoteAddr(channel));
				}
			});
		} catch (Exception e) {
			responseFuture.release();
			log.warn("send a request command to channel <" + RemotingHelper.parseChannelRemoteAddr(channel) + "> Exception", e);
			throw new RemotingSendRequestException(RemotingHelper.parseChannelRemoteAddr(channel), e);
		}
	} else {
		if (timeoutMillis <= 0) {
			throw new RemotingTooMuchRequestException("invokeAsyncImpl invoke too fast");
		} else {
			String info =
				String.format("invokeAsyncImpl tryAcquire semaphore timeout, %dms, waiting thread nums: %d semaphoreAsyncValue: %d",
					timeoutMillis,
					this.semaphoreAsync.getQueueLength(),
					this.semaphoreAsync.availablePermits()
				);
			log.warn(info);
			throw new RemotingTimeoutException(info);
		}
	}
}

如果异步消息发送失败会进行重试，然后又会调到NettyRemotingClient#invokeAsync中，直到重试次数使用完毕。