rocketmq源码解析-nameserver和broker的启动

最新推荐文章于 2024-12-06 23:08:27 发布

cblstc

最新推荐文章于 2024-12-06 23:08:27 发布

阅读量709

点赞数 1

分类专栏： rocketmq

本文链接：https://blog.youkuaiyun.com/cblstc/article/details/118267431

版权

本文详细解析RocketMQ的NameServer和Broker的启动过程。首先介绍了创建工作目录、编译源码及启动NameServer和Broker的步骤。接着深入NameServer启动流程，包括创建NameServer Controller、初始化并开启NameServer，特别是定期移除不活跃broker的机制。再者，文章详述了Broker启动流程，涉及创建BrokerController、初始化及发送心跳到NameServer的步骤。最后，总结了NameServer维护Broker路由信息的方法和心跳机制。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

准备

创建rocketmq工作目录

基于rocketmq4.8.0源码解析。创建/Users/cbl/develop/rocketmq-4.8.0目录，在rocketmq-4.8.0下创建conf、logs和store文件夹，然后把源码包下的distribution/conf下的broker.conf、logback_broker.xml和logback_namesrv.xml文件拷贝到rocketmq-4.8.0/conf下。

编译

执行mvn install -Dmaven.test.skip=true编译源码包

运行NameServer

找到NamesrvStartup下的main方法，Run Configurations设置环境变量(Environment variables)：ROCKETMQ_HOME=/Users/cbl/develop/rocketmq-4.8.0，然后启动

运行broker

找到BrokerStartup下的main方法，同样也是配置环境变量，然后启动

NameServer启动流程

NamesrvStartup#main是NameServer的启动入口，跟踪找到main0方法。

主要分为两步：

创建NameServer Controller，主要是收集配置
开启NameServer

public static NamesrvController main0(String[] args) {
   

    try {
   
        // 创建namesrv controller，主要操作就是收集一些配置信息
        NamesrvController controller = createNamesrvController(args);
        // 开启NameServer，核心方法
        start(controller);
        String tip = "The Name Server boot success. serializeType=" + RemotingCommand.getSerializeTypeConfigInThisServer();
        log.info(tip);
        System.out.printf("%s%n", tip);
        return controller;
    } catch (Throwable e) {
   
        e.printStackTrace();
        System.exit(-1);
    }

    return null;
}

创建NameServer Controller

该操作主要进行配置的收集，如NamesrvConfig、NettyServerConfig

public static NamesrvController createNamesrvController(String[] args) throws IOException, JoranException {
   
    //...
    //创建NamesrvConfig
    final NamesrvConfig namesrvConfig = new NamesrvConfig();
    //创建NettyServerConfig，作为netty服务端的一些配置
    final NettyServerConfig nettyServerConfig = new NettyServerConfig();
    //设置启动端口号
    nettyServerConfig.setListenPort(9876);
    //解析启动-c参数
    if (commandLine.hasOption('c')) {
   
        String file = commandLine.getOptionValue('c');
        if (file != null) {
   
            InputStream in = new BufferedInputStream(new FileInputStream(file));
            properties = new Properties();
            properties.load(in);
            MixAll.properties2Object(properties, namesrvConfig);
            MixAll.properties2Object(properties, nettyServerConfig);

            namesrvConfig.setConfigStorePath(file);

            System.out.printf("load config properties file OK, %s%n", file);
            in.close();
        }
    }
    //解析启动-p参数
    if (commandLine.hasOption('p')) {
   
        InternalLogger console = InternalLoggerFactory.getLogger(LoggerName.NAMESRV_CONSOLE_NAME);
        MixAll.printObjectProperties(console, namesrvConfig);
        MixAll.printObjectProperties(console, nettyServerConfig);
        System.exit(0);
    }
    //将启动参数填充到namesrvConfig,nettyServerConfig
    MixAll.properties2Object(ServerUtil.commandLine2Properties(commandLine), namesrvConfig);

    if (null == namesrvConfig.getRocketmqHome()) {
   
        System.out.printf("Please set the %s variable in your environment to match the location of the RocketMQ installation%n", MixAll.ROCKETMQ_HOME_ENV);
        System.exit(-2);
    }

    // ...
    //todo 创建NameServerController
    final NamesrvController controller = new NamesrvController(namesrvConfig, nettyServerConfig);

    // remember all configs to prevent discard
    controller.getConfiguration().registerConfig(properties);

    return controller;
}

我们来看下NamesrvConfig主要有哪些配置

public class NamesrvConfig {
   
    private static final InternalLogger log = InternalLoggerFactory.getLogger(LoggerName.NAMESRV_LOGGER_NAME);
    //rocketmq主目录：/Users/cbl/develop/rocketmq-4.8.0
    private String rocketmqHome = System.getProperty(MixAll.ROCKETMQ_HOME_PROPERTY, System.getenv(MixAll.ROCKETMQ_HOME_ENV));
    //NameServer存储KV配置属性的持久化路径：/Users/cbl/namesrv/kvConfig.json
    private String kvConfigPath = System.getProperty("user.home") + File.separator + "namesrv" + File.separator + "kvConfig.json";
    //nameServer默认配置文件路径：/Users/cbl/namesrv/namesrv.properties
    private String configStorePath = System.getProperty("user.home") + File.separator + "namesrv" + File.separator + "namesrv.properties";
    private String productEnvName = "center";
    private boolean clusterTest = false;
    //是否支持顺序消息
    private boolean orderMessageEnable = false;

开启NameServer

接下来看下NameServer的启动方法，主要两个步骤。

初始化NamesrvController
调用NamesrvController#start

public static NamesrvController start(final NamesrvController controller) throws Exception {
   

    if (null == controller) {
   
        throw new IllegalArgumentException("NamesrvController is null");
    }

    boolean initResult = controller.initialize(); // 初始化操作
    if (!initResult) {
   
        controller.shutdown();
        System.exit(-3);
    }
    //钩子方法
    Runtime.getRuntime().addShutdownHook(new ShutdownHookThread(log, new Callable<Void>() {
   
        @Override
        public Void call() throws Exception {
   
            controller.shutdown();
            return null;
        }
    }));

    controller.start(); // 开始

    return controller;
}

初始化NamesrvController

该操作比较重要的一点是开启了扫描不活跃Broker的线程。

public boolean initialize() {
   
    //加载KV配置
    this.kvConfigManager.load();
    //创建NettyServer网络处理对象
    this.remotingServer = new NettyRemotingServer(this.nettyServerConfig, this.brokerHousekeepingService);
    this.remotingExecutor =
        Executors.newFixedThreadPool(nettyServerConfig.getServerWorkerThreads(), new ThreadFactoryImpl("RemotingExecutorThread_"));

    this.registerProcessor();
    //每隔10s扫描一次为活跃Broker
    this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
   

        @Override
        public void run() {
   
            //todo 每隔10s扫描一次为活跃Broker
            NamesrvController.this.routeInfoManager.scanNotActiveBroker();
        }
    }, 5, 10, TimeUnit.SECONDS);
    //开启定时任务:每隔10min打印一次KV配置
    this.scheduledExecutorService.scheduleAtFixedRate(new Runnable() {
   

        @Override
        public void run() {
   
            NamesrvController.this.kvConfigManager.printAllPeriodically();
        }
    }, 1, 10, TimeUnit.MINUTES);

    // ...

    return true;
}

定期移除不活跃的broker

RouteInfoManager.scanNotActiveBroker 定期移除不活跃的broker，所谓不活跃，即两次心跳间隔大于设定阈值（默认120）。

在讲解该操作前，先看下broker的路由信息存储在哪里。NameServer把路由信息存储到内存中，主要有以下table。

// RouteInfoManager.java
private final HashMap<String/* topic */, List<QueueData>> topicQueueTable; // topic表
// broker表，存broker的信息，如属于哪个集群、broker名称，有哪些地址
private final HashMap<String/* brokerName */, BrokerData> brokerAddrTable;
// broker集群表，维护broker集群
private final HashMap<String/* clusterName */, Set<String/* brokerName */>> clusterAddrTable; 
private final HashMap<String/* brokerAddr */, BrokerLiveInfo> brokerLiveTable; // broker心跳表，维护上一次心跳时间
private final HashMap<String/* brokerAddr */, List<String>/* Filter Server */> filterServerTable;

broker的心跳时间存储在brokerLiveTable中，接下来看下NameServer如何移除不活跃的broker

//超过120s，则认为Broker失效
public void scanNotActiveBroker() {
   
    //获得brokerLiveTable
    Iterator<Entry<String, BrokerLiveInfo>> it = this.brokerLiveTable.entrySet().iterator();
    //遍历brokerLiveTable
    while (it.hasNext()) {
   
        Entry<String, BrokerLiveInfo> next = it.next();
        long last = next.getValue().getLastUpdateTimestamp(); // 上次心跳时间
        //如果收到心跳包的时间距当时时间是否超过120s
        if ((last + BROKER_CHANNEL_EXPIRED_TIME) < System.currentTimeMillis()) {
   
            //关闭和该broker的连接
            RemotingUtil.closeChannel(next.getValue().getChannel());
            //移除broker
            it.remove();
            log.warn("The broker channel expired, {} {}ms", next.getKey(), BROKER_CHANNEL_EXPIRED_TIME);
            //维护路由表（更新路由表）
            this.onChannelDestroy(next.getKey(), next.getValue().getChannel());
        }
    }
}

重新维护路由表

当broker失活后，把跟该broker有关的记录删除。为了并发安全，需要对该操作加写锁。因为本身对路由表的操作是读多写少的。

//维护路由表（使用写锁）
public void onChannelDestroy(String remoteAddr, Channel channel) {
   
    String brokerAddrFound = null;
    if (channel != null) {
   
       // 首先加读锁，从BrokerLiveInfo找到需要被移除的broker
        try {
   
            try {
   
                this.lock.readLock().lockInterruptibly();
                Iterator<Entry<String, BrokerLiveInfo>> itBrokerLiveTable =
                    this.brokerLiveTable.entrySet().iterator();
                while (itBrokerLiveTable.hasNext()) {
   
                    Entry<String, BrokerLiveInfo> entry = itBrokerLiveTable.next();
                    if (entry.getValue().getChannel() == channel) {
   
                        brokerAddrFound = entry.getKey();
                        break;
                    }
                }
            } finally {
   
                this.lock.readLock().unlock();
            }
        } catch (Exception e) {
   
            log.error("onChannelDestroy Exception", e);
        }
    }

    if (null == brokerAddrFound) {
   
      //如果找不到broker，则brokerAddrFound=传递过来的broker
        brokerAddrFound = remoteAddr;
    } else {
   
        log.info("the broker's channel destroye" +
                "d, {}, clean it's data structure at once", brokerAddrFound);
    }

    if (brokerAddrFound != null && brokerAddrFound.length() > 0) {
   

        try {
   
            try {
   
                //申请写锁,根据brokerAddress从brokerLiveTable和filterServerTable移除
                this.lock.writeLock().lockInterruptibly();
                this.brokerLiveTable.remove(brokerAddrFound);
                this.filterServerTable.remove(brokerAddrFound);
                //维护brokerAddrTable
                String brokerNameFound = null;
                boolean removeBrokerName = false;
                Iterator<Entry<String, BrokerData>> itBrokerAddrTable =
                    this.brokerAddrTable.entrySet().iterator();
                //遍历brokerAddrTable，根据brokerAddr找到brokerName，然后移除找到的记录，由于篇幅过长，代码略
                // ... 
              
                //维护clusterAddrTable，删除集群中该brokerName相关的记录，代码略
                // ...
              
              	// topic对应多条队列，每条队列可能存在不同broker，根据brokerName找到指定broker的队列并删除
                // 删除后如果队列为空，则把topic记录删掉。
                if (removeBrokerName) {
   
                    Iterator<Entry<String, List<QueueData>>> itTopicQueueTable =
                        this.topicQueueTable.entrySet().iterator();
                    while (itTopicQueueTable.hasNext()) {
   
                        Entry<String, List<QueueData>> entry = itTopicQueueTable.next();
                        String topic = entry.getKey();
                        List<QueueData> queueDataList = entry.getValue();

                        Iterator<QueueData> itQueueData = queueDataList.iterator();
                        while (itQueueData.hasNext()) {
   
                            QueueData queueData = itQueueData.next();
                            if (queueData.getBrokerName().equals(brokerNameFound)) {
   
                                itQueueData.remove();
                                log.info("remove topic[{} {}], from topicQueueTable, because channel destroyed",
                                    topic, queueData);
                            }
                        }

                        if (queueDataList.isEmpty()) {
   
                            itTopicQueueTable.remove();
                            log.info("remove topic[{}] all queue, from topicQueueTable, because channel destroyed",
                                topic);
                        }
                    }
                }
            } finally {
   
                //释放写锁
                this.lock.writeLock().unlock();
            }
        } catch (Exception e) {
   
            log.error("onChannelDestroy Exception", e);
        }
    }
}