kongkong的专栏

ConsumeMessageOrderlyServicelockOneMQ同步加锁 true:加锁成功public synchronized boolean lockOneMQ(final MessageQueue mq) { if (!this.stopped) { return this.defaultMQPushConsumerImpl.getRebalanceImpl().lock(mq); } return false;}tryLockLaterAndReconsume

MessageStoreConfig// TODO 延迟消息-时间间隔-定义 1s=1 5s=2 10s=3 ........String messageDelayLevel = "1s 5s 10s 30s 1m 2m 3m 4m 5m 6m 7m 8m 9m 10m 20m 30m 1h 2h"; CommitLog把原始的Topic和QueueId，转成Topic=SCHEDULE_TOPIC_XXXX，ququeId = delayLevel - 1然后发送到具体的定时队列publi

DefaultMQPushConsumerImplstartmQClientFactory = MQClientInstance调用MQClientInstance的start()方法this.mQClientFactory = MQClientManager.getInstance().getOrCreateMQClientInstance(this.defaultMQPushConsumer, this.rpcHook);// 调用MQClientInstance的start方法mQCl

RebalanceServicerunpublic void run() { log.info(this.getServiceName() + " service started"); while (!this.isStopped()) { this.waitForRunning(waitInterval); // 等待20s，然后超时自动释放锁执行doRebalance this.mqClientFactory.doRebalance(); // 具体逻辑 } log.info(t

MQClientInstancesendHeartbeatToAllBrokerWithLockpublic void sendHeartbeatToAllBrokerWithLock() { if (this.lockHeartbeat.tryLock()) { // 加锁 try { this.sendHeartbeatToAllBroker(); this.uploadFilterClassSource(); } catch (final Exception e) {

rocketmq-client本次演示api<dependency> <groupId>org.apache.rocketmq</groupId> <artifactId>rocketmq-spring-boot-starter</artifactId> <version>2.2.1</version></dependency>brokerMessageFilterpublic interfa

RocketMQUtilprivate static Message getAndWrapMessage(String destination, MessageHeaders headers, byte[] payloads) { if (destination == null || destination.length() < 1) { return null; } if (payloads == null || payloads.length < 1) { return nu

CommitLog这里关注FlushRealTimeServicepublic CommitLog(final DefaultMessageStore defaultMessageStore) { ................ if (FlushDiskType.SYNC_FLUSH == defaultMessageStore.getMessageStoreConfig().getFlushDiskType()) { // 同步 this.flushCommitLogService = n

CommitLog这里关注FlushRealTimeServicepublic CommitLog(final DefaultMessageStore defaultMessageStore) { ................ if (FlushDiskType.SYNC_FLUSH == defaultMessageStore.getMessageStoreConfig().getFlushDiskType()) { // 同步 this.flushCommitLogService = n

GroupCommitServicerunpublic void run() { CommitLog.log.info(this.getServiceName() + " service started"); while (!this.isStopped()) { // 未stop 一直循环 try { this.waitForRunning(10); // 每隔10ms允许1次 this.doCommit(); } catch (Exception e) { Comm

commitlog存储协议CommitLog.doAppend(单条)# ipv4: 8字节 ipv6:20字节int bornHostLength = (sysflag & MessageSysFlag.BORNHOST_V6_FLAG) == 0 ? 4 + 4 : 16 + 4;# ipv4: 8字节 ipv6:20字节int storeHostLength = (sysflag & MessageSysFlag.STOREHOSTADDRESS_V6_FLAG) ==

SendMessageProcessorasyncSendMessageprivate CompletableFuture<RemotingCommand> asyncSendMessage(ChannelHandlerContext ctx, RemotingCommand request, SendMessageContext mqtraceContext,

CommunicationModepublic enum CommunicationMode { SYNC, // 同步 ASYNC, // 异步 ONEWAY, // 发出去就不管了}重试次数# 同步默认 = 1+ 2 =3 ， # 非同步(异步、OneWay) = 1 (总共就1次，其实就是没有重试)int timesTotal = communicationMode == CommunicationMode.SYNC ? 1 + this.defaultMQPro

topics.json"TBW102":{ "order":false, "perm":7, "readQueueNums":8, "topicFilterType":"SINGLE_TAG", "topicName":"TBW102", "topicSysFlag":0, "writeQueueNums":8},"TopicOrder":{ "order":false, "perm":6, "readQueueNums":4, "topicFilter

SEND_MESSAGE:10sendSmartMsg = false/** sendSmartMsg = false情况下 单条消息 单条消息: 10 */public static final int SEND_MESSAGE = 10;SEND_MESSAGE_V2 : 310sendSmartMsg =true/** sendSmartMsg =true情况下 单条消息 单条消息: 310 */public static final int SEND_MESSAGE_V2 =

findConsumerIdList根据broker# topic:topic group:consumerGrouppublic List<String> findConsumerIdList(final String topic, final String group) { String brokerAddr = this.findBrokerAddrByTopic(topic); // 根据topic得到broker地址 if (null == brokerAddr) {

AllocateMessageQueueStrategyList<MessageQueue> allocate( final String consumerGroup, final String currentCID, final List<MessageQueue> mqAll, final List<String> cidAll);类图AllocateMessageQueueAveragely默认算法-平均public class Alloca

传输协议可见传输内容主要可以分为以下4部分：(1) 消息长度：总长度，四个字节存储，占用一个int类型；(2) 序列化类型&消息头长度：同样占用一个int类型，第一个字节表示序列化类型，后面三个字节表示消息头长度；(3) 消息头数据：经过序列化后的消息头数据；(4) 消息主体数据：消息主体的二进制字节数据内容NettyDecoder构造函数定义获取1个网络包的格式# FRAME_MAX_LENGTH=16777216public NettyDecoder() { su

mqbroker最后一句是sh ${ROCKETMQ_HOME}/bin/runbroker.sh org.apache.rocketmq.broker.BrokerStartup $@BrokerStartupmain 函数public static void main(String[] args) { start(createBrokerController(args));}createBrokerController函数public static BrokerControll

borker启动broker启动发现10911端口是docker的ip然后rocketmq-console也报错rocketmq-console连接x.x.0.1:10911，连接报错brokerIP1# 通过这个取到了x.x.0.1的ipString brokerIP1 = RemotingUtil.getLocalAddress()解决新建broker.propertiesbrokerIP1 = 192.168.5.1启动nohup sh bin/mqbroker -n

CommitLogpublic PutMessageResult putMessage(final MessageExtBrokerInner msg) { // Set the storage time 设置消息存储时间 msg.setStoreTimestamp(System.currentTimeMillis()); // Set the message body BODY CRC (consider the most appropriate sett

前言根据自己的算法，选择MessageQueue程序实现SendResult sendResult = producer.send(msg, new MessageQueueSelector() { @Override public MessageQueue select(List<MessageQueue> mqs, Message msg, Object arg) { Integer

DefaultMQProducerpublic SendResult send( Message msg) throws MQClientException, RemotingException, MQBrokerException, InterruptedException { Validators.checkMessage(msg, this); msg.setTopic(withNamespace(msg.getTopic())); r

相关文件需要distribution/conf.中的broker.conf 、 logback_broker.xml、logback_namesrv.xmlbroker.confbrokerClusterName = DefaultClusterbrokerName = broker-abrokerId = 0deleteWhen = 04fileReservedTime = 48brokerRole = ASYNC_MASTERflushDiskType = ASYNC_FLUSHn

git clone https://github.com/apache/rocketmq-externals.git修改配置文件并打包启动java -jar rocketmq-console-ng-2.0.0.jar >/jar/rockmq-console.log 2>&1 &访问http://localhost:8080...

command查看所有消费组及数量sh mqadmin consumerProgress -n 162.16.5.1:9876查看所有topic查看所有topic :sh mqadmin topicList -n localhost:9876查看topic信息统计sh mqadmin topicstatus -n localhost:9876 -t TopicTest查看集群信息sh mqadmin clusterList -n localhost:9876...

CommitLogCommtLog的存储其实是分多层的，CommitLog -> MappedFileQueue -> MappedFile，其中真正存储数据的是MappedFile。CommitLog存储目录，为用户目录下的/commitlog/${fileName}，每个文件大小为1G，不满则填充空白。文件命名规则:每个文件名字长度为20，具体命名为文件偏移量，左边补0。如第一个文件为00000000000000000000，第二个文件为00000000001073741824.

默认目录rootDir默认存储根目录storePathRootDir = System.getProperty("user.home") + File.separator + "store";commitlogstorePathCommitLog = System.getProperty("user.home") + File.separator + "store" + File.separator + "commitlog";commitlog默认文件大小// Commi

描述默认不配置brokerIP1和brokerIP2时，都会根据当前网卡选择一个IP使用，当你的机器有多块网卡或者装了docker后，很有可能会有问题，有可能brokerIP就是docker的ip，在其他机器是ping不通的broker.propertiesbrokerIP1=具体的IP启动brokernohup sh bin/mqbroker -n ${NAMESERVER_IP}:9876 -c broker.properties & ...

下载路径wget https://mirrors.tuna.tsinghua.edu.cn/apache/rocketmq/4.8.0/rocketmq-all-4.8.0-bin-release.zipStart Name Server > nohup sh bin/mqnamesrv & > tail -f ~/logs/rocketmqlogs/namesrv.log The Name Server boot success...注意：如果启动失败，修改ru

DefaultMessageStorepublic DefaultMessageStore(final MessageStoreConfig messageStoreConfig, final BrokerStatsManager brokerStatsManager, .................. if (messageStoreConfig.isEnableDLegerCommitLog()) { this.commitLog = new DLedg

git路径# 指定具体分支git clone -b rocketmq-all-4.8.0 https://github.com/apache/rocketmq.git相关路径设计路径Github路径

 1. 安装docker加速https://blog.youkuaiyun.com/kq1983/article/details/83041322 2. 镜像查询docker search rabbitmq:management3.  下载镜像docker pull macintoshplus/rabbitmq-management或docker pull rabb...

 MQ（事务消息）举个例子，Bob向Smith转账，那我们到底是先发送消息，还是先执行扣款操作？好像都可能会出问题。如果先发消息，扣款操作失败，那么Smith的账户里面会多出一笔钱。反过来，如果先执行扣款操作，后发送消息，那有可能扣款成功了但是消息没发出去，Smith收不到钱。除了上面介绍的通过异常捕获和回滚的方式外，还有没有其他的思路呢？下面以阿里巴巴的RocketMQ中间件为例...