metadataUpdater

本文探讨了Kafka中Metadata类的功能及元数据更新机制。Metadata类负责管理Kafka集群的元数据信息,并确保这些信息的及时更新。当Producer尝试发送消息时,会检查元数据并触发更新流程。默认情况下,每5分钟会自动更新一次元数据。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

Metadata

Metadata 是封装了kafka元数据的类,是一个管理类
image

metadata的元数据第一次什么时候更新

image
每次调用producer.send方法,都会去metadata.cluster中去查找元数据,如果没有,就会标记需要更新,并且通过version等待更新.

public synchronized int requestUpdate() {
       //需要更新元数据
        this.needUpdate = true;
        return this.version;
    }

真正的更新是由sender线程去操作的,sender线程,依靠NetworkClient来完成网络的IO操作

        /**
         * Add a metadata request to the list of sends if we can make one
         */
        private void maybeUpdate(long now, Node node) {
            if (node == null) {
                log.debug("Give up sending metadata request since no node is available");
                // mark the timestamp for no node available to connect
                this.lastNoNodeAvailableMs = now;
                return;
            }
            String nodeConnectionId = node.idString();

            if (canSendRequest(nodeConnectionId)) {
                this.metadataFetchInProgress = true;
                MetadataRequest metadataRequest;
                if (metadata.needMetadataForAllTopics())
                    metadataRequest = MetadataRequest.allTopics();
                else
                    metadataRequest = new MetadataRequest(new ArrayList<>(metadata.topics()));
                ClientRequest clientRequest = request(now, nodeConnectionId, metadataRequest);
                log.debug("Sending metadata request {} to node {}", metadataRequest, node.id());
                doSend(clientRequest, now);
            } else if (connectionStates.canConnect(nodeConnectionId, now)) {
                // we don't have a connection to this node right now, make one
                log.debug("Initialize connection to node {} for sending metadata request", node.id());
                initiateConnect(node, now);
                // If initiateConnect failed immediately, this node will be put into blackout and we
                // should allow immediately retrying in case there is another candidate node. If it
                // is still connecting, the worst case is that we end up setting a longer timeout
                // on the next round and then wait for the response.
            } else { // connected, but can't send more OR connecting
                // In either case, we just need to wait for a network event to let us know the selected
                // connection might be usable again.
                this.lastNoNodeAvailableMs = now;
            }
        }

    }

这里可以看到请求元数据的时候,把topic的信息也发送过去了,如果在服务端配置了创建topic,就会自动的去创建topic了.

MetadataUpdater sender在循环发送数据的过程中,就会调用 mayBeupdate 来判断是否要更新元数据.

                                .define(METADATA_MAX_AGE_CONFIG,
                                        Type.LONG,
                                        5 * 60 * 1000,
                                        atLeast(0),
                                        Importance.LOW,
                                        CommonClientConfigs.METADATA_MAX_AGE_DOC)

默认情况下5分钟要更新一次元数据

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值