Redis源码解析：22sentinel(三)客观下线以及故障转移之选举领导节点

最新推荐文章于 2024-12-14 00:00:00 发布

gqtcgq

最新推荐文章于 2024-12-14 00:00:00 发布

阅读量3.4k

点赞数 2

CC 4.0 BY-SA版权

分类专栏： redis

本文链接：https://blog.youkuaiyun.com/gqtcgq/article/details/51588259

本文深入解析Redis Sentinel系统，详细介绍了如何判断主节点是否客观下线，以及故障转移中选举领导节点的过程。通过哨兵之间的交互，包括发送`is-master-down-by-addr`命令，收集其他哨兵的反馈，实现主节点的故障检测与恢复。文中还探讨了选举纪元和Raft协议在哨兵系统中的应用。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

八：判断实例是否客观下线

当前哨兵一旦监测到某个主节点实例主观下线之后，就会向其他哨兵发送”is-master-down-by-addr”命令，询问其他哨兵是否也认为该主节点主观下线了。如果有超过quorum个哨兵（包括当前哨兵）反馈，都认为该主节点主观下线了，则当前哨兵就将该主节点实例标记为客观下线。

注意，客观下线的概念只针对主节点实例，而与从节点和哨兵实例无关。

1：发送”is-master-down-by-addr”命令

”is-master-down-by-addr”命令有两个作用：一是询问其他哨兵是否认为某个主节点已经主观下线；二是开始故障迁移时，当前哨兵向其他哨兵实例进行"拉票"，让其选自己为领导节点。

本节只关注该命令的第一个作用，此时，该命令的格式是：

"SENTINEL is-master-down-by-addr <masterip> <masterport> <sentinel.current_epoch> *";

在哨兵的“主函数”sentinelHandleRedisInstance中，通过调用函数sentinelAskMasterStateToOtherSentinels来向其他哨兵发送”is-master-down-by-addr”命令。该函数的代码如下：

void sentinelAskMasterStateToOtherSentinels(sentinelRedisInstance *master, int flags) {
    dictIterator *di;
    dictEntry *de;

    di = dictGetIterator(master->sentinels);
    while((de = dictNext(di)) != NULL) {
        sentinelRedisInstance *ri = dictGetVal(de);
        mstime_t elapsed = mstime() - ri->last_master_down_reply_time;
        char port[32];
        int retval;

        /* If the master state from other sentinel is too old, we clear it. */
        if (elapsed > SENTINEL_ASK_PERIOD*5) {
            ri->flags &= ~SRI_MASTER_DOWN;
            sdsfree(ri->leader);
            ri->leader = NULL;
        }

        /* Only ask if master is down to other sentinels if:
         *
         * 1) We believe it is down, or there is a failover in progress.
         * 2) Sentinel is connected.
         * 3) We did not received the info within SENTINEL_ASK_PERIOD ms. */
        if ((master->flags & SRI_S_DOWN) == 0) continue;
        if (ri->flags & SRI_DISCONNECTED) continue;
        if (!(flags & SENTINEL_ASK_FORCED) &&
            mstime() - ri->last_master_down_reply_time < SENTINEL_ASK_PERIOD)
            continue;

        /* Ask */
        ll2string(port,sizeof(port),master->addr->port);
        retval = redisAsyncCommand(ri->cc,
                    sentinelReceiveIsMasterDownReply, NULL,
                    "SENTINEL is-master-down-by-addr %s %s %llu %s",
                    master->addr->ip, port,
                    sentinel.current_epoch,
                    (master->failover_state > SENTINEL_FAILOVER_STATE_NONE) ?
                    server.runid : "*");
        if (retval == REDIS_OK) ri->pending_commands++;
    }
    dictReleaseIterator(di);
}

在函数中，轮训字典master->sentinels，针对其中的每一个哨兵实例ri：

属性ri->last_master_down_reply_time表示上次收到该哨兵实例ri对于"SENTINEL IS-MASTER-DOWN-BY-ADDR"命令回复的时间，如果该时间距离当前时间已经超过了5倍的SENTINEL_ASK_PERIOD，则清除其对于master的过时的状态记录：将SRI_MASTER_DOWN标记从实例标志位中清除；释放实例中的leader属性并置为NULL；

接下来开始向哨兵实例ri发送命令，但是在发送命令之前需要满足一定的条件，这些条件分别是：主节点master已经被标记为主观下线了；该哨兵实例处于连接状态；参数flags中设置了SENTINEL_ASK_FORCED标记，或者距离上次收到该哨兵实例的命令回复已超过SENTINEL_ASK_PERIOD；

满足以上所有条件之后，调用redisAsyncCommand向ri异步发送命令，命令的回调函数是sentinelReceiveIsMasterDownReply。

2：其他哨兵收到”is-master-down-by-addr”命令后的处理

当哨兵收到其他哨兵发来的”SENTINEL is-master-down-by-addr”命令后，调用函数sentinelCommand进行处理。该函数中处理”is-master-down-by-addr”的部分代码是：

void sentinelCommand(redisClient *c) {
    ...
    else if (!strcasecmp(c->argv[1]->ptr,"is-master-down-by-addr")) {
        /* SENTINEL IS-MASTER-DOWN-BY-ADDR <ip> <port> <current-epoch> <runid>*/
        sentinelRedisInstance *ri;
        long long req_epoch;
        uint64_t leader_epoch = 0;
        char *leader = NULL;
        long port;
        int isdown = 0;

        if (c->argc != 6) goto numargserr;
        if (getLongFromObjectOrReply(c,c->argv[3],&port,NULL) != REDIS_OK ||
            getLongLongFromObjectOrReply(c,c->argv[4],&req_epoch,NULL)
                                                              != REDIS_OK)
            return;
        ri = getSentinelRedisInstanceByAddrAndRunID(sentinel.masters,
            c->argv[2]->ptr,port,NULL);

        /* It exists? Is actually a master? Is subjectively down? It's down.
         * Note: if we are in tilt mode we always reply with "0". */
        if (!sentinel.tilt && ri && (ri->flags & SRI_S_DOWN) &&
                                    (ri->flags & SRI_MASTER))
            isdown = 1;

        /* Vote for the master (or fetch the previous vote) if the request
         * includes a runid, otherwise the sender is not seeking for a vote. */
        if (ri && ri->flags & SRI_MASTER && strcasecmp(c->argv[5]->ptr,"*")) {
            leader = sentinelVoteLeader(ri,(uint64_t)req_epoch,
                                            c->argv[5]->ptr,
                                            &leader_epoch);
        }

        /* Reply with a three-elements multi-bulk reply:
         * down state, leader, vote epoch. */
        addReplyMultiBulkLen(c,3);
        addReply(c, isdown ? shared.cone : shared.czero);
        addReplyBulkCString(c, leader ? leader : "*");
        addReplyLongLong(c, (long long)leader_epoch);
        if (leader) sdsfree(leader);
    } 
    ...
}