本文深入剖析了RabbitMQ客户端消费者的工作原理，包括连接建立、消息读取、消息处理等核心流程，并揭示了客户端消费消息时的单线程特性。
下面是RabbitMQ客户端消费者的基本示例。
public class Receive {
private final static String QUEUE_NAME = "hello";
public static void main(String[] args) throws IOException, TimeoutException {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("127.0.0.1");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
String queueName=channel.queueDeclare().getQueue();
channel.queueBind(queueName,"MY_EXCHANGE","");
DeliverCallback deliverCallback = (consumerTag, delivery) -> {
String message = new String(delivery.getBody(), "UTF-8");
System.out.println(" 线程"+Thread.currentThread().getName()+"收到消息：" + message);
};
channel.basicConsume(queueName, false, deliverCallback, consumerTag -> { });
}
}
启动程序接收消息，会发现线程id一直在改变，那么是不是说RabbitMQ客户端是多线程消费消息呢？从上面的客户端消费者示例代码我们很容易想到跟踪如下这句代码。
Connection connection = factory.newConnection();
跟踪代码最最终走到如下重载方法。
public Connection newConnection(ExecutorService executor, AddressResolver addressResolver, String clientProvidedName)
throws IOException, TimeoutException {
if(this.metricsCollector == null) {
this.metricsCollector = new NoOpMetricsCollector();
}
// make sure we respect the provided thread factory
FrameHandlerFactory fhFactory = createFrameHandlerFactory();
ConnectionParams params = params(executor);
// set client-provided via a client property
if (clientProvidedName != null) {
Map<String, Object> properties = new HashMap<String, Object>(params.getClientProperties());
properties.put("connection_name", clientProvidedName);
params.setClientProperties(properties);
}
if (isAutomaticRecoveryEnabled()) {
// see com.rabbitmq.client.impl.recovery.RecoveryAwareAMQConnectionFactory#newConnection
// No Sonar: no need to close this resource because we're the one that creates it
// and hands it over to the user
AutorecoveringConnection conn = new AutorecoveringConnection(params, fhFactory, addressResolver, metricsCollector); //NOSONAR
conn.init();
return conn;
} else {
List<Address> addrs = addressResolver.getAddresses();
Exception lastException = null;
for (Address addr : addrs) {
try {
FrameHandler handler = fhFactory.create(addr, clientProvidedName);
AMQConnection conn = createConnection(params, handler, metricsCollector);
conn.start();
this.metricsCollector.newConnection(conn);
return conn;
} catch (IOException e) {
lastException = e;
} catch (TimeoutException te) {
lastException = te;
}
}
if (lastException != null) {
if (lastException instanceof IOException) {
throw (IOException) lastException;
} else if (lastException instanceof TimeoutException) {
throw (TimeoutException) lastException;
}
}
throw new IOException("failed to connect");
}
}
上面代码首先重点关注下面这段。
FrameHandlerFactory fhFactory = createFrameHandlerFactory();
进入createFrameHandlerFactory（）方法代码如下。
protected synchronized FrameHandlerFactory createFrameHandlerFactory() throws IOException {
if(nio) {
if(this.frameHandlerFactory == null) {
if(this.nioParams.getNioExecutor() == null && this.nioParams.getThreadFactory() == null) {
this.nioParams.setThreadFactory(getThreadFactory());
}
this.frameHandlerFactory = new SocketChannelFrameHandlerFactory(connectionTimeout, nioParams, isSSL(), sslContextFactory);
}
return this.frameHandlerFactory;
} else {
return new SocketFrameHandlerFactory(connectionTimeout, socketFactory, socketConf, isSSL(), this.shutdownExecutor, sslContextFactory);
}
}
上面代码很显然是客户端是否使用nio，默认是不使用的，本文也通过非nio来讲述，所以这里返回的默认就是SocketFrameHandlerFactory，先记住后面还会用到。
回到newConnection（）方法然后关注如下代码片段，在没有做任何配置的情况下isAutomaticRecoveryEnabled()方法的返回值是true。
if (isAutomaticRecoveryEnabled()) {
// see com.rabbitmq.client.impl.recovery.RecoveryAwareAMQConnectionFactory#newConnection
// No Sonar: no need to close this resource because we're the one that creates it
// and hands it over to the user
AutorecoveringConnection conn = new AutorecoveringConnection(params, fhFactory, addressResolver, metricsCollector); //NOSONAR
conn.init();
return conn;
}
查看相关代码过后确认下面这行代码是重点。
conn.init();
进入init()方法查看到代码如下。
public void init() throws IOException, TimeoutException {
this.delegate = this.cf.newConnection();
this.addAutomaticRecoveryListener(delegate);
}
简单查看后确认如下代码为重点代码。
this.delegate = this.cf.newConnection();
进入newConnection()方法，代码如下。
public RecoveryAwareAMQConnection newConnection() throws IOException, TimeoutException {
Exception lastException = null;
List<Address> shuffled = shuffle(addressResolver.getAddresses());
for (Address addr : shuffled) {
try {
FrameHandler frameHandler = factory.create(addr, connectionName());
RecoveryAwareAMQConnection conn = createConnection(params, frameHandler, metricsCollector);
conn.start();
metricsCollector.newConnection(conn);
return conn;
} catch (IOException e) {
lastException = e;
} catch (TimeoutException te) {
lastException = te;
}
}
if (lastException != null) {
if (lastException instanceof IOException) {
throw (IOException) lastException;
} else if (lastException instanceof TimeoutException) {
throw (TimeoutException) lastException;
}
}
throw new IOException("failed to connect");
}
前面提到的SocketFrameHandlerFactory在这里就用到了，我们查看其对应的create方法。
public FrameHandler create(Address addr, String connectionName) throws IOException {
String hostName = addr.getHost();
int portNumber = ConnectionFactory.portOrDefault(addr.getPort(), ssl);
Socket socket = null;
try {
socket = createSocket(connectionName);
configurator.configure(socket);
socket.connect(new InetSocketAddress(hostName, portNumber),
connectionTimeout);
return create(socket);
} catch (IOException ioe) {
quietTrySocketClose(socket);
throw ioe;
}
}
这里如果熟悉Socket编程的话就不用多说了吧，就是创建Socket，然后返回到前一段代码跟踪如下代码行的代码。
conn.start();
进入start()方法，这里代码比较多，不再赘述，请查看如下代码行。
this._frameHandler.initialize(this);
一路跟踪到startMainLoop（）方法。
public void startMainLoop() {
MainLoop loop = new MainLoop();
final String name = "AMQP Connection " + getHostAddress() + ":" + getPort();
mainLoopThread = Environment.newThread(threadFactory, loop, name);
mainLoopThread.start();
}
很显然这里就是启动了一个线程，这里的MainLoop实现了Runnable接口，查看MainLoop的run()方法。
public void run() {
boolean shouldDoFinalShutdown = true;
try {
while (_running) {
Frame frame = _frameHandler.readFrame();
readFrame(frame);
}
} catch (Throwable ex) {
if (ex instanceof InterruptedException) {
// loop has been interrupted during shutdown,
// no need to do it again
shouldDoFinalShutdown = false;
} else {
handleFailure(ex);
}
} finally {
if (shouldDoFinalShutdown) {
doFinalShutdown();
}
}
}
这里就是就是一个循环一直在从Socket的流当中读取字节流，接下来的内容就涉及到了协议了，我们先跟踪如下代码行。
Frame frame = _frameHandler.readFrame();
看到如下代码。
public Frame readFrame() throws IOException {
synchronized (_inputStream) {
return Frame.readFrom(_inputStream);
}
}
上面代码的_inputStream就是Socket的输入流，继续跟踪。
public static Frame readFrom(DataInputStream is) throws IOException {
int type;
int channel;
try {
type = is.readUnsignedByte();
} catch (SocketTimeoutException ste) {
// System.err.println("Timed out waiting for a frame.");
return null; // failed
}
if (type == 'A') {
/*
* Probably an AMQP.... header indicating a version
* mismatch.
*/
/*
* Otherwise meaningless, so try to read the version,
* and throw an exception, whether we read the version
* okay or not.
*/
protocolVersionMismatch(is);
}
channel = is.readUnsignedShort();
int payloadSize = is.readInt();
byte[] payload = new byte[payloadSize];
is.readFully(payload);
int frameEndMarker = is.readUnsignedByte();
if (frameEndMarker != AMQP.FRAME_END) {
throw new MalformedFrameException("Bad frame end marker: " + frameEndMarker);
}
return new Frame(type, channel, payload);
}
上面代码是关键了，其实就是从流当中读取字节数据，然后封装成我们想要的数据对象交给下层处理，这就是TCP/IP协议分层的美妙之处，上层协议是不在乎下层的具体内容是什么的。这里客户端代码实现的便是应用层协议，和我们平时使用的HTTP协议在一个层，在Wireshark抓包当中，你可以看到这种类型的协议被称作AMQP。这里不再赘述其它内容，只讲述下面这一段。
if (frameEndMarker != AMQP.FRAME_END) {
throw new MalformedFrameException("Bad frame end marker: " + frameEndMarker);
}
这段代码你能想到网络协议相关的什么知识吗？答案是TCP是没有消息边界的，面向字节流的，需要应用层自行实现消息边界。
接下来我们分析客户端的线程问题，回到MainLoop的run（）方法分析如下代码行。
readFrame(frame);
进入上面代码行对应的方法。
private void readFrame(Frame frame) throws IOException {
if (frame != null) {
_missedHeartbeats = 0;
if (frame.type == AMQP.FRAME_HEARTBEAT) {
// Ignore it: we've already just reset the heartbeat counter.
} else {
if (frame.channel == 0) { // the special channel
_channel0.handleFrame(frame);
} else {
if (isOpen()) {
// If we're still _running, but not isOpen(), then we
// must be quiescing, which means any inbound frames
// for non-zero channels (and any inbound commands on
// channel zero that aren't Connection.CloseOk) must
// be discarded.
ChannelManager cm = _channelManager;
if (cm != null) {
ChannelN channel;
try {
channel = cm.getChannel(frame.channel);
} catch(UnknownChannelException e) {
// this can happen if channel has been closed,
// but there was e.g. an in-flight delivery.
// just ignoring the frame to avoid closing the whole connection
LOGGER.info("Received a frame on an unknown channel, ignoring it");
return;
}
channel.handleFrame(frame);
}
}
}
}
} else {
// Socket timeout waiting for a frame.
// Maybe missed heartbeat.
handleSocketTimeout();
}
}
这里代码很容易读懂，如果是心跳则什么都不做，否则将来自传输层字节流封装出来的数据对象交给对应的channel处理，这里主要跟踪如下代码行。
channel.handleFrame(frame);
进入上面代码行对应代码。
public void handleFrame(Frame frame) throws IOException {
AMQCommand command = _command;
if (command.handleFrame(frame)) { // a complete command has rolled off the assembly line
_command = new AMQCommand(); // prepare for the next one
handleCompleteInboundCommand(command);
}
}
这里说明一下上面这段代码，成功收取一条消息，会处理三个frame前两个frame不会进入if对应的代码，第三个会进入代码，在if里面new了一个AMQCommand为后续的消息做准备，我们这里只重点关注第三个frame的处理，即进入handleCompleteInboundCommand（）方法，查看代码如下。
public void handleCompleteInboundCommand(AMQCommand command) throws IOException {
// First, offer the command to the asynchronous-command
// handling mechanism, which gets to act as a filter on the
// incoming command stream. If processAsync() returns true,
// the command has been dealt with by the filter and so should
// not be processed further. It will return true for
// asynchronous commands (deliveries/returns/other events),
// and false for commands that should be passed on to some
// waiting RPC continuation.
this._trafficListener.read(command);
if (!processAsync(command)) {
// The filter decided not to handle/consume the command,
// so it must be a response to an earlier RPC.
if (_checkRpcResponseType) {
synchronized (_channelMutex) {
// check if this reply command is intended for the current waiting request before calling nextOutstandingRpc()
if (_activeRpc != null && !_activeRpc.canHandleReply(command)) {
// this reply command is not intended for the current waiting request
// most likely a previous request timed out and this command is the reply for that.
// Throw this reply command away so we don't stop the current request from waiting for its reply
return;
}
}
}
final RpcWrapper nextOutstandingRpc = nextOutstandingRpc();
// the outstanding RPC can be null when calling Channel#asyncRpc
if(nextOutstandingRpc != null) {
nextOutstandingRpc.complete(command);
markRpcFinished();
}
}
}
查看上面代码过后显然继续查看代码中涉及到的processAsync()方法即可，进入方法代码过后代码较多，查看如下代码行对应代码即可。
processDelivery(command, (Basic.Deliver) method);
方法中代码较多，具体处理不讲解了，查看如下代码片段。
this.dispatcher.handleDelivery(callback,
m.getConsumerTag(),
envelope,
(BasicProperties) command.getContentHeader(),
command.getContentBody());
进入相关方法过后查到下面代码片段。
public void handleDelivery(final Consumer delegate,
final String consumerTag,
final Envelope envelope,
final AMQP.BasicProperties properties,
final byte[] body) throws IOException {
executeUnlessShuttingDown(
new Runnable() {
@Override
public void run() {
try {
delegate.handleDelivery(consumerTag,
envelope,
properties,
body);
} catch (Throwable ex) {
connection.getExceptionHandler().handleConsumerException(
channel,
ex,
delegate,
consumerTag,
"handleDelivery");
}
}
});
}
注意上面代码的匿名内部类对象交给了executeUnlessShuttingDown（）方法进行处理，进入方法。
private void executeUnlessShuttingDown(Runnable r) {
if (!this.shuttingDown) execute(r);
}
继续跟踪方法。
private void execute(Runnable r) {
checkShutdown();
this.workService.addWork(this.channel, r);
}
继续跟踪代码。
public void addWork(Channel channel, Runnable runnable) {
if (this.workPool.addWorkItem(channel, runnable)) {
this.executor.execute(new WorkPoolRunnable());
}
}
上面代码很关键了，首先是将任务放到工作队列当中，然后提交了一个WorkPoolRunnable到线程池处理，那是不是以为这里客户端就是多线程的呢？我么先查看addWorkItem（）的代码。
public boolean addWorkItem(K key, W item) {
VariableLinkedBlockingQueue<W> queue;
synchronized (this) {
queue = this.pool.get(key);
}
// The put operation may block. We need to make sure we are not holding the lock while that happens.
if (queue != null) {
enqueueingCallback.accept(queue, item);
synchronized (this) {
if (isDormant(key)) {
dormantToReady(key);
return true;
}
}
}
return false;
}
上面代码很容易阅读逻辑，将任务放入队列过后，如果当前没有线程正在处理这里channel对应的任务则返回true，否则返回false，我们继续查看WorkPoolRunnable代码。
private final class WorkPoolRunnable implements Runnable {
@Override
public void run() {
int size = MAX_RUNNABLE_BLOCK_SIZE;
List<Runnable> block = new ArrayList<Runnable>(size);
try {
Channel key = ConsumerWorkService.this.workPool.nextWorkBlock(block, size);
if (key == null) return; // nothing ready to run
try {
for (Runnable runnable : block) {
runnable.run();
}
} finally {
if (ConsumerWorkService.this.workPool.finishWorkBlock(key)) {
ConsumerWorkService.this.executor.execute(new WorkPoolRunnable());
}
}
} catch (RuntimeException e) {
Thread.currentThread().interrupt();
}
}
再查看一下下面代码行对应的代码。
Channel key = ConsumerWorkService.this.workPool.nextWorkBlock(block, size);
进入nextWorkBlock（）方法查看代码。
public K nextWorkBlock(Collection<W> to, int size) {
synchronized (this) {
K nextKey = readyToInProgress();
if (nextKey != null) {
VariableLinkedBlockingQueue<W> queue = this.pool.get(nextKey);
drainTo(queue, to, size);
}
return nextKey;
}
}
再查看drainTo（）方法对应的代码。
private int drainTo(VariableLinkedBlockingQueue<W> deList, Collection<W> c, int maxElements) {
int n = 0;
while (n < maxElements) {
W first = deList.poll();
if (first == null)
break;
c.add(first);
++n;
}
return n;
}
通过前面几段代码分析，很明显了，客户端单个connection对应的单个channel实际上是单线程的，每次收到Socket消息都触发处理逻辑，从任务队列里面取出一定的任务进行依次处理，如果一个channel订阅了多个topic的话也是单线程依次处理的，所以在设计客户端业务代码的时候一定要注意这个问题，如果某一个消息处理耗时太久，会阻塞后续消息的处理，可以采取业务代码再起线程池的方式处理，如果要消息具有顺序性，可以给消息增加路由键，采用hash取模固定线程池中的某个线程处理某个路由键的消息。
扫一扫
3678
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
