【Redis】Redis学习⑪ - redis高可用与集群-redis集群
概要
前一章的主从架构无法实现master和slave角色的自动切换，即当master出现redis服务异常、主机断电、磁盘损坏等问题导致master无法使用，而redis高可用无法实现自故障转移（将slave提升为master），需要手动改环境配置才能切换到slave redis 服务器，另外也无法横向扩展Redis服务的并行写入性能，当单台Redis服务器性能无法满足业务写入需求的时候就必须要一种方式解决以上的两个核心问题。 即：
1）master和slave角色的无缝切换，让业务无感知从而不影响业务使用
2）可以横向动态扩展Redis服务器，从而实现多台服务器并行写入以实现更高并发的目的
Redis集群实现方式：客户端分片 代理分片 Redis Cluster
Sentinel（哨兵）
Redis Cluster
Redis Cluster 集群节点维护
集群运行时间长久之后，难免由于硬件故障、网络规划、业务增长等原因对已有集群进行相应的调整，比如增加Redis node节点、减少节点、节点迁移、更换服务器等。
增加节点和删除节点会涉及到已有的槽位重新分配及数据迁移。
集群维护-动态添加节点
增加Redis node 节点，需要与之前的Redis node版本相同、配置一致，然后分别启动两台Redis node，因为一主一从。
同步之前Redis node的配置文件到 新机器
scp redis_cluster_63* 192.168.56.202:/usr/local/redis/etc/
启动redis服务
- 添加节点到集群
add-node new_host:new_port existing_host:existing_port
要添加的新redis节点IP和端口 添加到的集群中的mater IP:端口，加到集群之后默认是master节点但是没有slots数据，需要重新分配
- redis4添加方式：
redis-trib.rb add-node 172.18.200.106:6379 172.18.200.101:6379
- redis5添加方式：
redis-cli -a 123456 --cluster add-node 192.168.56.202:6379 192.168.56.199:6379
- 日志：
[root@gbase8c_private etc]# redis-cli -a 123456 --cluster add-node 192.168.56.202:6379 192.168.56.199:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Adding node 192.168.56.202:6379 to cluster 192.168.56.199:6379
>>> Performing Cluster Check (using node 192.168.56.199:6379)
M: ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: 858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380
slots: (0 slots) slave
replicates ff147ab31cb3984ec48d5354a184633c0594808e
M: 5b46c04d7fa5edba767392c51be13e0f0db522e9 192.168.56.201:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
M: 0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 596112ae9a792595f61dcb63d19b0911c7b20797 192.168.56.201:6380
slots: (0 slots) slave
replicates 0809910b37930a25aa32776bba8272c4493ecc62
S: 282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380
slots: (0 slots) slave
replicates 5b46c04d7fa5edba767392c51be13e0f0db522e9
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.56.202:6379 to make it join the cluster.
[OK] New node added correctly.
- 分配槽位
添加主机之后需要对添加至集群中的新主机重新分片否则其没有分片
- redis3/4:
redis-trib.rb reshard 172.18.200.107:6379
redis-trib.rb check 172.18.200.101:6379
- redis5:
redis-cli -a 123456 --cluster check 192.168.56.201:6379
- 日志：
[root@gbase8c_private etc]# redis-cli -a 123456 --cluster check 192.168.56.199:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.199:6379 (ff147ab3...) -> 0 keys | 5461 slots | 1 slaves.
192.168.56.201:6379 (5b46c04d...) -> 0 keys | 5461 slots | 1 slaves.
192.168.56.200:6379 (0809910b...) -> 1 keys | 5462 slots | 1 slaves.
192.168.56.202:6379 (7fec9dbf...) -> 0 keys | 0 slots | 0 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.56.199:6379)
M: ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
S: 858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380
slots: (0 slots) slave
replicates ff147ab31cb3984ec48d5354a184633c0594808e
M: 5b46c04d7fa5edba767392c51be13e0f0db522e9 192.168.56.201:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
M: 0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
M: 7fec9dbf3ff41ac54784936e6db497fb2dfba93e 192.168.56.202:6379 #新添加的master没有槽位
slots: (0 slots) master
S: 596112ae9a792595f61dcb63d19b0911c7b20797 192.168.56.201:6380
slots: (0 slots) slave
replicates 0809910b37930a25aa32776bba8272c4493ecc62
S: 282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380
slots: (0 slots) slave
replicates 5b46c04d7fa5edba767392c51be13e0f0db522e9
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
- 使用命令对新加的主机重新分配槽位：
redis-cli -a 123456 --cluster reshard 192.168.56.202:6379
- 日志：
[root@gbase8c_private etc]# redis-cli -a 123456 --cluster reshard 192.168.56.202:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing Cluster Check (using node 192.168.56.202:6379)
M: 7fec9dbf3ff41ac54784936e6db497fb2dfba93e 192.168.56.202:6379
slots: (0 slots) master
S: 858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380
slots: (0 slots) slave
replicates ff147ab31cb3984ec48d5354a184633c0594808e
S: 596112ae9a792595f61dcb63d19b0911c7b20797 192.168.56.201:6380
slots: (0 slots) slave
replicates 0809910b37930a25aa32776bba8272c4493ecc62
M: 5b46c04d7fa5edba767392c51be13e0f0db522e9 192.168.56.201:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
M: 0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: 282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380
slots: (0 slots) slave
replicates 5b46c04d7fa5edba767392c51be13e0f0db522e9
M: ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 4096 #分配多少个槽位
What is the receiving node ID? 7fec9dbf3ff41ac54784936e6db497fb2dfba93e #接收slot的服务器ID,手动输入202的ID
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1: all #将哪些源主机的槽位分配给202，all是自动在所有的redis node 选择划分，如果是从redis cluster 删除主机可以使用此方式将主机上的槽位全部移动到别的redis主机
......
Moving slot 2717 from ff147ab31cb3984ec48d5354a184633c0594808e
Moving slot 2718 from ff147ab31cb3984ec48d5354a184633c0594808e
Moving slot 2719 from ff147ab31cb3984ec48d5354a184633c0594808e
Moving slot 2720 from ff147ab31cb3984ec48d5354a184633c0594808e
......
Do you want to proceed with the proposed reshard plan (yes/no)?yes #确认分配
Moving slot 2720 from 192.168.56.199:6379 to 192.168.56.202:6379:
......
- 验证重新分配槽位之后的集群状态
重新分配槽位是自动从每个Redis node上移动一些槽位到新的master上
redis-cli -a 123456 --cluster check 192.168.56.199:6379
- 日志：
[root@gbase8c_private logs]# redis-cli -a 123456 --cluster check 192.168.56.199:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.199:6379 (ff147ab3...) -> 0 keys | 4096 slots | 1 slaves.
192.168.56.201:6379 (5b46c04d...) -> 0 keys | 4096 slots | 1 slaves.
192.168.56.200:6379 (0809910b...) -> 1 keys | 4096 slots | 1 slaves.
192.168.56.202:6379 (7fec9dbf...) -> 0 keys | 4096 slots | 0 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.56.199:6379)
M: ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379
slots:[1365-5460] (4096 slots) master
1 additional replica(s)
S: 858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380
slots: (0 slots) slave
replicates ff147ab31cb3984ec48d5354a184633c0594808e
M: 5b46c04d7fa5edba767392c51be13e0f0db522e9 192.168.56.201:6379
slots:[12288-16383] (4096 slots) master
1 additional replica(s)
M: 0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379
slots:[6827-10922] (4096 slots) master
1 additional replica(s)
M: 7fec9dbf3ff41ac54784936e6db497fb2dfba93e 192.168.56.202:6379
slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
S: 596112ae9a792595f61dcb63d19b0911c7b20797 192.168.56.201:6380
slots: (0 slots) slave
replicates 0809910b37930a25aa32776bba8272c4493ecc62
S: 282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380
slots: (0 slots) slave
replicates 5b46c04d7fa5edba767392c51be13e0f0db522e9
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
- 为新的master添加slave节点
redis-cli -a 123456 --cluster add-node 192.168.56.202:6380 192.168.56.202:6379
- 日志：
[root@gbase8c_private logs]# redis-cli -a 123456 --cluster add-node 192.168.56.202:6380 192.168.56.202:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Adding node 192.168.56.202:6380 to cluster 192.168.56.202:6379
>>> Performing Cluster Check (using node 192.168.56.202:6379)
M: 7fec9dbf3ff41ac54784936e6db497fb2dfba93e 192.168.56.202:6379
slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
S: 858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380
slots: (0 slots) slave
replicates ff147ab31cb3984ec48d5354a184633c0594808e
S: 596112ae9a792595f61dcb63d19b0911c7b20797 192.168.56.201:6380
slots: (0 slots) slave
replicates 0809910b37930a25aa32776bba8272c4493ecc62
M: 5b46c04d7fa5edba767392c51be13e0f0db522e9 192.168.56.201:6379
slots:[12288-16383] (4096 slots) master
1 additional replica(s)
M: 0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379
slots:[6827-10922] (4096 slots) master
1 additional replica(s)
S: 282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380
slots: (0 slots) slave
replicates 5b46c04d7fa5edba767392c51be13e0f0db522e9
M: ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379
slots:[1365-5460] (4096 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
>>> Send CLUSTER MEET to node 192.168.56.202:6380 to make it join the cluster.
[OK] New node added correctly.
- 更改新节点状态为slave
需要手动将其指定为某个master的slave，否则其默认角色为master
redis-cli -h 192.168.56.202 -p 6380 -a 123456
cluster nodes #查看当前集群节点，找到目标master的ID
cluster replicate xxxxx #将其设置为slave，命令格式为 cluster replicate MASTERID
cluster nodes #验证改为指定master的slave
- 日志:
[root@gbase8c_private logs]# redis-cli -h 192.168.56.202 -p 6380 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.202:6380> cluster nodes
858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380@16380 slave ff147ab31cb3984ec48d5354a184633c0594808e 0 1698248832613 1 connected
0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379@16379 master - 0 1698248833000 3 connected 6827-10922
7fec9dbf3ff41ac54784936e6db497fb2dfba93e 192.168.56.202:6379@16379 master - 0 1698248833000 7 connected 0-1364 5461-6826 10923-12287
ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379@16379 master - 0 1698248834762 1 connected 1365-5460
596112ae9a792595f61dcb63d19b0911c7b20797 192.168.56.201:6380@16380 slave 0809910b37930a25aa32776bba8272c4493ecc62 0 1698248832000 3 connected
282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380@16380 slave 5b46c04d7fa5edba767392c51be13e0f0db522e9 0 1698248831000 5 connected
5b46c04d7fa5edba767392c51be13e0f0db522e9 192.168.56.201:6379@16379 master - 0 1698248833645 5 connected 12288-16383
d7a45981a0ea916581aca08922d759b07225a2c7 192.168.56.202:6380@16380 myself,master - 0 1698248831000 0 connected
192.168.56.202:6380> cluster replicate 7fec9dbf3ff41ac54784936e6db497fb2dfba93e
OK
192.168.56.202:6380> cluster nodes
858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380@16380 slave ff147ab31cb3984ec48d5354a184633c0594808e 0 1698248871000 1 connected
0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379@16379 master - 0 1698248872125 3 connected 6827-10922
7fec9dbf3ff41ac54784936e6db497fb2dfba93e 192.168.56.202:6379@16379 master - 0 1698248869839 7 connected 0-1364 5461-6826 10923-12287
ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379@16379 master - 0 1698248870000 1 connected 1365-5460
596112ae9a792595f61dcb63d19b0911c7b20797 192.168.56.201:6380@16380 slave 0809910b37930a25aa32776bba8272c4493ecc62 0 1698248870000 3 connected
282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380@16380 slave 5b46c04d7fa5edba767392c51be13e0f0db522e9 0 1698248873144 5 connected
5b46c04d7fa5edba767392c51be13e0f0db522e9 192.168.56.201:6379@16379 master - 0 1698248871121 5 connected 12288-16383
d7a45981a0ea916581aca08922d759b07225a2c7 192.168.56.202:6380@16380 myself,slave 7fec9dbf3ff41ac54784936e6db497fb2dfba93e 0 1698248869000 0 connected
192.168.56.202:6380> config set masterauth 123456
OK
192.168.56.202:6380> info replication
# Replication
role:slave
master_host:192.168.56.202
master_port:6379
master_link_status:up
master_last_io_seconds_ago:5
master_sync_in_progress:0
slave_repl_offset:0
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:533a9f4b87c8d53f96a2015d4049e5e345a132b3
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:0
- 验证当前集群状态
redis-cli -a 123456 --cluster check 192.168.56.199:6379
- 日志：
[root@gbase8c_private logs]# redis-cli -a 123456 --cluster check 192.168.56.199:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.199:6379 (ff147ab3...) -> 0 keys | 4096 slots | 1 slaves.
192.168.56.201:6379 (5b46c04d...) -> 0 keys | 4096 slots | 1 slaves.
192.168.56.200:6379 (0809910b...) -> 1 keys | 4096 slots | 1 slaves.
192.168.56.202:6379 (7fec9dbf...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.56.199:6379)
M: ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379
slots:[1365-5460] (4096 slots) master
1 additional replica(s)
S: d7a45981a0ea916581aca08922d759b07225a2c7 192.168.56.202:6380
slots: (0 slots) slave
replicates 7fec9dbf3ff41ac54784936e6db497fb2dfba93e
S: 858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380
slots: (0 slots) slave
replicates ff147ab31cb3984ec48d5354a184633c0594808e
M: 5b46c04d7fa5edba767392c51be13e0f0db522e9 192.168.56.201:6379
slots:[12288-16383] (4096 slots) master
1 additional replica(s)
M: 0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379
slots:[6827-10922] (4096 slots) master
1 additional replica(s)
M: 7fec9dbf3ff41ac54784936e6db497fb2dfba93e 192.168.56.202:6379
slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
1 additional replica(s)
S: 596112ae9a792595f61dcb63d19b0911c7b20797 192.168.56.201:6380
slots: (0 slots) slave
replicates 0809910b37930a25aa32776bba8272c4493ecc62
S: 282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380
slots: (0 slots) slave
replicates 5b46c04d7fa5edba767392c51be13e0f0db522e9
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
集群维护-动态删除节点
添加节点的时候是先添加node节点到集群，然后分配槽位，删除节点的操作与添加节点的操作正好相反，是先将被删除的redis node上的槽位迁移到集群中的其他redis node节点上，然后再将其删除。
如果一个redis node节点上的槽位没有被完全迁移，删除该node的时候会提示有数据且无法删除。
- 迁移master的槽位之其他master
-
被迁移的master服务器必须保证没有数据
-
redis4删除方式：
redis-trib.rb reshard 172.18.200.101:6379
redis-trib.rb fix 172.18.200.101:6379 #迁移失败需要修复集群
- redis5删除方式：
redis-cli -a 123456 --cluster reshard 192.168.56.199:6379
- 日志：
[root@gbase8c_private etc]# redis-cli -a 123456 --cluster reshard 192.168.56.199:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing Cluster Check (using node 192.168.56.199:6379)
M: ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379
slots:[1365-5460] (4096 slots) master
1 additional replica(s)
M: 0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379
slots:[6827-10922] (4096 slots) master
1 additional replica(s)
M: 5b46c04d7fa5edba767392c51be13e0f0db522e9 192.168.56.201:6379
slots:[12288-16383] (4096 slots) master
1 additional replica(s)
M: 7fec9dbf3ff41ac54784936e6db497fb2dfba93e 192.168.56.202:6379
slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
1 additional replica(s)
S: 858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380
slots: (0 slots) slave
replicates ff147ab31cb3984ec48d5354a184633c0594808e
S: 282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380
slots: (0 slots) slave
replicates 5b46c04d7fa5edba767392c51be13e0f0db522e9
S: d7a45981a0ea916581aca08922d759b07225a2c7 192.168.56.202:6380
slots: (0 slots) slave
replicates 7fec9dbf3ff41ac54784936e6db497fb2dfba93e
S: 596112ae9a792595f61dcb63d19b0911c7b20797 192.168.56.201:6380
slots: (0 slots) slave
replicates 0809910b37930a25aa32776bba8272c4493ecc62
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
How many slots do you want to move (from 1 to 16384)? 4096 #迁移master上的多少个槽位
What is the receiving node ID? ff147ab31cb3984ec48d5354a184633c0594808e #接收槽位的服务器ID
Please enter all the source node IDs.
Type 'all' to use all the nodes as source nodes for the hash slots.
Type 'done' once you entered all the source nodes IDs.
Source node #1: 5b46c04d7fa5edba767392c51be13e0f0db522e9 #从哪个服务器迁移4096个槽位
Source node #2: done #done表示没有其他master了
......
Moving slot 16379 from 5b46c04d7fa5edba767392c51be13e0f0db522e9
Moving slot 16380 from 5b46c04d7fa5edba767392c51be13e0f0db522e9
Moving slot 16381 from 5b46c04d7fa5edba767392c51be13e0f0db522e9
Moving slot 16382 from 5b46c04d7fa5edba767392c51be13e0f0db522e9
Moving slot 16383 from 5b46c04d7fa5edba767392c51be13e0f0db522e9
Do you want to proceed with the proposed reshard plan (yes/no)? yes #是否继续
......
Moving slot 16380 from 192.168.56.201:6379 to 192.168.56.199:6379:
Moving slot 16381 from 192.168.56.201:6379 to 192.168.56.199:6379:
Moving slot 16382 from 192.168.56.201:6379 to 192.168.56.199:6379:
Moving slot 16383 from 192.168.56.201:6379 to 192.168.56.199:6379:
- 验证槽位迁移完成
redis-cli -a 123456 --cluster check 192.168.56.199:6379
- 日志：
[root@gbase8c_private etc]# redis-cli -a 123456 --cluster check 192.168.56.199:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.199:6379 (ff147ab3...) -> 0 keys | 8192 slots | 2 slaves.
192.168.56.200:6379 (0809910b...) -> 1 keys | 4096 slots | 1 slaves.
192.168.56.201:6379 (5b46c04d...) -> 0 keys | 0 slots | 0 slaves.
192.168.56.202:6379 (7fec9dbf...) -> 0 keys | 4096 slots | 1 slaves.
[OK] 1 keys in 4 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.56.199:6379)
M: ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379 #已迁入
slots:[1365-5460],[12288-16383] (8192 slots) master
2 additional replica(s)
M: 0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379
slots:[6827-10922] (4096 slots) master
1 additional replica(s)
M: 5b46c04d7fa5edba767392c51be13e0f0db522e9 192.168.56.201:6379 #虽然是master 但已经没有了槽位
slots: (0 slots) master
M: 7fec9dbf3ff41ac54784936e6db497fb2dfba93e 192.168.56.202:6379
slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
1 additional replica(s)
S: 858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380
slots: (0 slots) slave
replicates ff147ab31cb3984ec48d5354a184633c0594808e
S: 282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380
slots: (0 slots) slave
replicates ff147ab31cb3984ec48d5354a184633c0594808e
S: d7a45981a0ea916581aca08922d759b07225a2c7 192.168.56.202:6380
slots: (0 slots) slave
replicates 7fec9dbf3ff41ac54784936e6db497fb2dfba93e
S: 596112ae9a792595f61dcb63d19b0911c7b20797 192.168.56.201:6380
slots: (0 slots) slave
replicates 0809910b37930a25aa32776bba8272c4493ecc62
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
- 从集群删除服务器
虽然槽位已经迁移完成，但是服务器IP信息还在集群当中，因此还需要将IP信息从集群删除。
redis-cli -a 123456 --cluster del-node IP:Port ID
- redis4删除方式：
redis-trib.rb del-node 172.18.200.102:6379 master的IP
- redis5删除方式：
redis-cli -a 123456 --cluster del-node 192.168.56.201:6379 master的IP
- 日志：
[root@gbase8c_private logs]# redis-cli -a 123456 --cluster del-node 192.168.56.201:6379 5b46c04d7fa5edba767392c51be13e0f0db522e9
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Removing node 5b46c04d7fa5edba767392c51be13e0f0db522e9 from cluster 192.168.56.201:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
[root@gbase8c_private logs]# redis-cli -a 123456 --cluster del-node 192.168.56.200:6379 596112ae9a792595f61dcb63d19b0911c7b20797
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Removing node 596112ae9a792595f61dcb63d19b0911c7b20797 from cluster 192.168.56.200:6379
>>> Sending CLUSTER FORGET messages to the cluster...
>>> SHUTDOWN the node.
- 验证node是否删除
发现node已经被删除，但是由于201:6380之前是200:6379的slave，所以删除后会导致相应的master缺少slave，需要重新为没有slave的master分配slave。
可以发现202有两个slave，因此需要将其中一个slave转移为200的slave。
redis-cli -a 123456 --cluster check 192.168.56.199:6379
- 日志：
[root@gbase8c_private logs]# redis-cli -a 123456 --cluster check 192.168.56.199:6379
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.199:6379 (ff147ab3...) -> 0 keys | 8192 slots | 1 slaves.
192.168.56.200:6379 (0809910b...) -> 1 keys | 4096 slots | 0 slaves.
192.168.56.202:6379 (7fec9dbf...) -> 0 keys | 4096 slots | 2 slaves.
[OK] 1 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 192.168.56.199:6379)
M: ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379
slots:[1365-5460],[12288-16383] (8192 slots) master
1 additional replica(s)
M: 0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379
slots:[6827-10922] (4096 slots) master
M: 7fec9dbf3ff41ac54784936e6db497fb2dfba93e 192.168.56.202:6379
slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
2 additional replica(s)
S: 858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380
slots: (0 slots) slave
replicates ff147ab31cb3984ec48d5354a184633c0594808e
S: 282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380
slots: (0 slots) slave
replicates 7fec9dbf3ff41ac54784936e6db497fb2dfba93e
S: d7a45981a0ea916581aca08922d759b07225a2c7 192.168.56.202:6380
slots: (0 slots) slave
replicates 7fec9dbf3ff41ac54784936e6db497fb2dfba93e
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
- 重新分配slave
将 202:6380转移为200的slave。
redis-cli -h 192.168.56.199 -p 6379 -a 123456
cluster nodes
cluster replicate #master的ID
- 日志：
[root@gbase8c_private logs]# redis-cli -h 192.168.56.199 -p 6379 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.199:6379> cluster nodes
0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379@16379 master - 0 1698334771000 3 connected 6827-10922
7fec9dbf3ff41ac54784936e6db497fb2dfba93e 192.168.56.202:6379@16379 master - 0 1698334771091 7 connected 0-1364 5461-6826 10923-12287
858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380@16380 slave ff147ab31cb3984ec48d5354a184633c0594808e 0 1698334772000 8 connected
282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380@16380 slave 7fec9dbf3ff41ac54784936e6db497fb2dfba93e 0 1698334773542 8 connected
d7a45981a0ea916581aca08922d759b07225a2c7 192.168.56.202:6380@16380 slave 7fec9dbf3ff41ac54784936e6db497fb2dfba93e 0 1698334772461 7 connected
ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379@16379 myself,master - 0 1698334771000 8 connected 1365-5460 12288-16383
[root@gbase8c_private logs]# redis-cli -h 192.168.56.202 -p 6380 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.202:6380> cluster replicate 0809910b37930a25aa32776bba8272c4493ecc62
OK
- 验证集群master与slave对应关系
redis slave节点一定不能和master在一个服务器，必须为开主机交叉备份模式，避免主机故障后主备全部挂掉，如果出现master和slave在同一台node的情况，则需要按照以上步骤重新进行slave分配，直到相互交叉备份位置。
集群维护-模拟master宕机
目前的架构为三主三从，互为跨主机master slave模式，测试master宕机之后是否会自动切换至slave。
- 测试数据写入
测试在master写入数据，并在其对应的slave验证数据
日志：
[root@gbase8c_private logs]# redis-cli -h 192.168.56.200 -p 6379 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.200:6379> set key1 value1
OK
192.168.56.200:6379> get key1
"value1"
192.168.56.200:6379> quit
[root@gbase8c_private logs]# redis-cli -h 192.168.56.202 -p 6380 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.202:6380> keys *
1) "key1"
192.168.56.202:6380> get key1
(error) MOVED 9189 192.168.56.200:6379 #slave 不提供读写，只提供数据备份即master的选举
- 停止master并验证故障转移
redis master停止服务后，其对应的slave会被选举为master继续处理数据的读写操作
[root@gbase8c_private logs]# redis-cli -h 192.168.56.202 -p 6379 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.202:6379> shutdown
not connected> quit
- 验证slave日志
需要相应的故障转移时间
4392:S 26 Oct 2023 23:52:51.375 # Error condition on socket for SYNC: Connection refused
4392:S 26 Oct 2023 23:52:52.938 * Connecting to MASTER 192.168.56.202:6379
4392:S 26 Oct 2023 23:52:52.939 * MASTER <-> REPLICA sync started
4392:S 26 Oct 2023 23:52:52.939 # Error condition on socket for SYNC: Connection refused
4392:S 26 Oct 2023 23:52:54.091 * Connecting to MASTER 192.168.56.202:6379
4392:S 26 Oct 2023 23:52:54.091 * MASTER <-> REPLICA sync started
4392:S 26 Oct 2023 23:52:54.092 # Error condition on socket for SYNC: Connection refused
4392:S 26 Oct 2023 23:52:54.907 * FAIL message received from 0809910b37930a25aa32776bba8272c4493ecc62 about 7fec9dbf3ff41ac54784936e6db497fb2dfba93e
4392:S 26 Oct 2023 23:52:54.908 # Cluster state changed: fail
4392:S 26 Oct 2023 23:52:55.025 # Start of election delayed for 872 milliseconds (rank #0, offset 3598).
4392:S 26 Oct 2023 23:52:55.127 * Connecting to MASTER 192.168.56.202:6379
4392:S 26 Oct 2023 23:52:55.128 * MASTER <-> REPLICA sync started
4392:S 26 Oct 2023 23:52:55.129 # Error condition on socket for SYNC: Connection refused
4392:S 26 Oct 2023 23:52:56.138 # Starting a failover election for epoch 9.
4392:S 26 Oct 2023 23:52:56.153 # Failover election won: I'm the new master. #成为新的master
4392:S 26 Oct 2023 23:52:56.153 # configEpoch set to 9 after successful failover
4392:M 26 Oct 2023 23:52:56.153 # Setting secondary replication ID to 42fcbcd45e54c3c56fbca8605b342cf56a8d8fd5, valid up to offset: 3599. New replication ID is 58ddd3c3b0299cd29e3adb75fe2359aa9ce65f30
4392:M 26 Oct 2023 23:52:56.153 * Discarding previously cached master state.
4392:M 26 Oct 2023 23:52:56.153 # Cluster state changed: ok #状态切换成功
- 验证slave状态
[root@gbase8c_private logs]# redis-cli -h 192.168.56.199 -p 6380 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.199:6380> info replication
# Replication
role:master #角色以切换为master
connected_slaves:0
master_replid:58ddd3c3b0299cd29e3adb75fe2359aa9ce65f30
master_replid2:42fcbcd45e54c3c56fbca8605b342cf56a8d8fd5
master_repl_offset:3598
second_repl_offset:3599
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1779
repl_backlog_histlen:1820
- 验证数据读写
确认slave 199:6380切换为master后可以继续为业务提供读写且数据没有丢失
集群维护-导入现有redis数据
导入数据需要redis cluster不能与被导入的数据有重复的key名称，否则导入不成功或中断。
- 基础环境准备
导入数据之前需要关闭各redis服务器的密码，包括集群中的各node和源redis server ，避免认证带来的环境不一致从而无法导入，可以加参数 --cluster-replace 强制替换redis cluster已有的key
全部修改：
[root@gbase8c_private logs]# redis-cli -h 192.168.56.199 -p 6379 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.199:6379> config set requirepass ""
OK
192.168.56.199:6379> quit
[root@gbase8c_private logs]# redis-cli -h 192.168.56.199 -p 6380 -a 123456
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
192.168.56.199:6380> config set requirepass ""
OK
......
- 执行数据导入
将源redis server的数据直接导入到redis cluster
- redis 3/4:
redis-trib.rb import --from 172.18.200.107:6382 --replace 172.18.200.107:6379
- redis 5:
redis-cli --cluster import 集群服务器IP:PORT --cluster-from 外部Redisnode-IP:PORT --cluster-copy --cluster-replace
- 日志：
redis-cli --cluster import 192.168.56.199:6379 --cluster-from 192.168.56.201:6379 --cluster-copy
[root@gbase8c_private logs]# redis-cli --cluster import 192.168.56.199:6379 --cluster-from 192.168.56.201:6379 --cluster-copy
>>> Importing data from 192.168.56.201:6379 to cluster 192.168.56.199:6379
>>> Performing Cluster Check (using node 192.168.56.199:6379)
M: ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379
slots:[1365-5460],[12288-16383] (8192 slots) master
1 additional replica(s)
M: 0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379
slots:[6827-10922] (4096 slots) master
1 additional replica(s)
S: 7fec9dbf3ff41ac54784936e6db497fb2dfba93e 192.168.56.202:6379
slots: (0 slots) slave
replicates 282761264f247f7cec59b5d8f0dbe922050865aa
S: 858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380
slots: (0 slots) slave
replicates ff147ab31cb3984ec48d5354a184633c0594808e
M: 282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380
slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
1 additional replica(s)
S: d7a45981a0ea916581aca08922d759b07225a2c7 192.168.56.202:6380
slots: (0 slots) slave
replicates 0809910b37930a25aa32776bba8272c4493ecc62
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
*** Importing 5 keys from DB 0
Migrating key3 to 192.168.56.199:6380: OK
Migrating aaa to 192.168.56.200:6379: OK
Migrating key1 to 192.168.56.200:6379: Source 192.168.56.201:6379 replied with error:
ERR Target instance replied with error: BUSYKEY Target key name already exists. #有重复数据 中断
清掉后重新执行:
[root@gbase8c_private logs]# redis-cli --cluster import 192.168.56.199:6379 --cluster-from 192.168.56.201:6379 --cluster-copy
>>> Importing data from 192.168.56.201:6379 to cluster 192.168.56.199:6379
>>> Performing Cluster Check (using node 192.168.56.199:6379)
M: ff147ab31cb3984ec48d5354a184633c0594808e 192.168.56.199:6379
slots:[1365-5460],[12288-16383] (8192 slots) master
1 additional replica(s)
M: 0809910b37930a25aa32776bba8272c4493ecc62 192.168.56.200:6379
slots:[6827-10922] (4096 slots) master
1 additional replica(s)
S: 7fec9dbf3ff41ac54784936e6db497fb2dfba93e 192.168.56.202:6379
slots: (0 slots) slave
replicates 282761264f247f7cec59b5d8f0dbe922050865aa
S: 858b0507025e970ffcc2855b85e676fa3a0db34f 192.168.56.200:6380
slots: (0 slots) slave
replicates ff147ab31cb3984ec48d5354a184633c0594808e
M: 282761264f247f7cec59b5d8f0dbe922050865aa 192.168.56.199:6380
slots:[0-1364],[5461-6826],[10923-12287] (4096 slots) master
1 additional replica(s)
S: d7a45981a0ea916581aca08922d759b07225a2c7 192.168.56.202:6380
slots: (0 slots) slave
replicates 0809910b37930a25aa32776bba8272c4493ecc62
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
*** Importing 4 keys from DB 0
Migrating key3 to 192.168.56.199:6380: OK
Migrating aaa to 192.168.56.200:6379: OK
Migrating ccc to 192.168.56.199:6380: OK
Migrating key2 to 192.168.56.199:6379: OK
redis扩展集群方案
除了redis官方自带的redis cluster集群之外，还有一些开源的集群解决方案可供参考使用。
-
codis
Codis是一个分布式Redis解决方案，对于上层的应用来说，连接到Codis Proxy和连接原生的Redis Server没有显著区别。
上层应用可以像使用单机的redis一样使用，Codis 底层会处理请求的转发，不停机的数据迁移等工作，所有后边的一切事情，对于前面的客户端来说都是透明的，可以简单的认为后边连接的是一个内存无限大的redis服务.
codis-proxy 相当于redis，即连接codis-proxy和连接redis是没有任何区别的，codis-proxy无状态，不负责记录是否在哪保存，数据在zookeeper 记录，即codis proxy向zookeeper查询key的记录位置，proxy将请求转发到一个组进行处理，一个组里面有一个master和一个或多个slave组成，默认有1024个槽位，redis cluster默认有16384个槽位，其把不同的槽位的内容放在不同的group。
Github地址：https://github.com/CodisLabs/codis/blob/release3.2/doc/tutorial_zh.md -
twemproxy
由twemproxy双向代理客户端实现分片，即代替用户将数据分片到不同的后端服务器进行读写，其还支持memcached，可以为proxy配置算法，缺点为twemproxy是瓶颈，不支持数据迁移
Github地址：https://github.com/twitter/twemproxy
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
