Metadata
Metadata 是封装了kafka元数据的类,是一个管理类
metadata的元数据第一次什么时候更新
每次调用producer.send方法,都会去metadata.cluster中去查找元数据,如果没有,就会标记需要更新,并且通过version等待更新.
public synchronized int requestUpdate() {
//需要更新元数据
this.needUpdate = true;
return this.version;
}
真正的更新是由sender线程去操作的,sender线程,依靠NetworkClient来完成网络的IO操作
/**
* Add a metadata request to the list of sends if we can make one
*/
private void maybeUpdate(long now, Node node) {
if (node == null) {
log.debug("Give up sending metadata request since no node is available");
// mark the timestamp for no node available to connect
this.lastNoNodeAvailableMs = now;
return;
}
String nodeConnectionId = node.idString();
if (canSendRequest(nodeConnectionId)) {
this.metadataFetchInProgress = true;
MetadataRequest metadataRequest;
if (metadata.needMetadataForAllTopics())
metadataRequest = MetadataRequest.allTopics();
else
metadataRequest = new MetadataRequest(new ArrayList<>(metadata.topics()));
ClientRequest clientRequest = request(now, nodeConnectionId, metadataRequest);
log.debug("Sending metadata request {} to node {}", metadataRequest, node.id());
doSend(clientRequest, now);
} else if (connectionStates.canConnect(nodeConnectionId, now)) {
// we don't have a connection to this node right now, make one
log.debug("Initialize connection to node {} for sending metadata request", node.id());
initiateConnect(node, now);
// If initiateConnect failed immediately, this node will be put into blackout and we
// should allow immediately retrying in case there is another candidate node. If it
// is still connecting, the worst case is that we end up setting a longer timeout
// on the next round and then wait for the response.
} else { // connected, but can't send more OR connecting
// In either case, we just need to wait for a network event to let us know the selected
// connection might be usable again.
this.lastNoNodeAvailableMs = now;
}
}
}
这里可以看到请求元数据的时候,把topic的信息也发送过去了,如果在服务端配置了创建topic,就会自动的去创建topic了.
MetadataUpdater sender在循环发送数据的过程中,就会调用 mayBeupdate 来判断是否要更新元数据.
.define(METADATA_MAX_AGE_CONFIG,
Type.LONG,
5 * 60 * 1000,
atLeast(0),
Importance.LOW,
CommonClientConfigs.METADATA_MAX_AGE_DOC)
默认情况下5分钟要更新一次元数据