MySQL Mgr集群安装部署

MGR简介

基于传统异步复制和半同步复制的缺陷——数据的一致性问题无法保证,MySQL官方在5.7.17版本正式推出组复制(MySQL Group Replication,简称MGR),以插件形式提供,实现了分布式下数据的最终一致性。

MGR特点如下: 1)高一致性,基于原生复制及paxos协议的组复制技术,并以插件的方式提供,提供一致数据安全保证; 2)高容错性,只要不是大多数节点坏掉就可以继续工作,有自动检测机制,当不同节点产生资源争用冲突时,不会出现错误,按照先到者优先原则进行处 理,并且内置了自动化脑裂防护机制; 3)高扩展性,节点的新增和移除都是自动的,新节点加入后,会自动从其他节点上同步状态,直到新节点和其他节点保持一致,如果某节点被移除了,其他节点自动更新组信息,自动维护新的组信息; 4)高灵活性,有单主模式和多主模式,在同一个group内,不允许两种模式同时存在。在单主模式下,主节点宕机时,会自动重新选择新的master,当旧的master恢复加入后,新master不会发生改变。所有更新操作都在主节点上进行;在多主模式下,所有的server成员都可以同时接受更新。 5) 由若干个节点共同组成一个复制组,一个事务的提交,必须经过组内大多数节点(N/2+1)决议并通过,才能得以提交。

wget https://downloads.mysql.com/archives/get/p/23/file/mysql-8.0.28-1.el7.x86_64.rpm-bundle.tar

MGR主要限制条件

  • 仅支持InnoDB表,并且每张表一定要有一个主键,或者非Null+唯一键

  • 必须打开GTID特性,二进制日志格式必须设置为ROW

  • 不支持复制过滤,如果有节点设置了复制过滤,将影响节点间决议的达成。

  • 不支持超大事务。

  • MGR忽略表锁和命名锁,在MGR中lock tables. unlock tables. get_lock. release_lock等这些表锁和命名锁将被忽略。

  • 多主模式中,默认不支持Serializable隔离级别。

  • 多主模式下,对同一个对象进行并发的有冲突的ddl和dml操作导致这种冲突在部分成员节点中无法检测到,最终可能导致数据不一致。

  • 多主模式下可能导致死锁,比如select…for update在不同节点执行,由于多节点锁无法共享,很容易导致死锁。

其他等限制参考官方文档:

MySQL :: MySQL 8.0 Reference Manual :: 18.3.1 Group Replication Requirements

一、创建3台MySQL环境

1.1 申请3台linux主机

主机ip地址操作系统版本
mg1192.168.2.112redhat 7
mg2192.168.2.34redhat 7
mg3192.168.2.170redhat 7

1.2. 安装mysql

services:
  mysql:
    image: mysql:8.0.28
    container_name: mysql
    environment:
      MYSQL_ROOT_PASSWORD: Hbis@123
    command: --default-authentication-plugin=mysql_native_password
    ports:
      - "3308:3306"
      - "33061:33061"
    network_mode: host
    volumes:
      - ./mysql_data:/var/lib/mysql
      - ./my.cnf:/etc/mysql/conf.d/mysql.cnf
    healthcheck:
      test: [ "CMD", "mysqladmin" ,"ping", "-h", "localhost" ]
      interval: 5s
      timeout: 10s
      retries: 20
​
  proxysql:
    image: proxysql/proxysql:latest
    container_name: proxysql
    ports:
      - "6032:6032"   # 管理端口
      - "6033:6033"   # 应用连接端口
    volumes:
      - ./proxysql.cnf:/etc/proxysql.cnf

二、修改MySQL参数

--mgr1

cat > /etc/my.cnf <<"EOF"
[mysqld]
user=mysql
port=3308
character_set_server=utf8mb4
secure_file_priv=
server-id = 1
default-time-zone = '+8:00'
log_timestamps = SYSTEM
log-bin =
binlog_format=row
binlog_checksum=NONE
skip-name-resolve
log_slave_updates = 1
gtid-mode=ON
enforce-gtid-consistency=on
default_authentication_plugin=mysql_native_password
max_allowed_packet = 500M
lower_case_table_names = 1
sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION
expire_logs_days= 60
max_connections = 20000
innodb_buffer_pool_size=1024M
slow_query_log = ON
#slow_query_log_file=/usr/local/mysql/data/mysql-slow.log
​
#安装mysql_replication引擎前提
master_info_repository=TABLE
relay_log_info_repository=TABLE
relay_log=lhrmgr31-relay-bin
​
#同步算法,官网建议设置该参数使用 XXHASH64 算法
transaction_write_set_extraction=XXHASH64
#集群的uuid,组名必须是一个uuid
loose-group_replication_group_name="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
#是否随着服务启动集群,这里设置否
loose-group_replication_start_on_boot=OFF
#集群本机端口,和服务端口不同,以本机端口 33061 接受来自组中成员的传入连接
loose-group_replication_local_address= "192.168.2.112:33061"
#集群包含的所有节点
loose-group_replication_group_seeds= "192.168.2.112:33061,192.168.2.34:33061,192.168.2.170:33061"
#是否设置为主节点,当创建集群时其他加入的节点都以该节点为目标加入集群
loose-group_replication_bootstrap_group=OFF
#设置白名单
loose-group_replication_ip_whitelist="192.168.2.112,192.168.2.34,192.168.2.170"
​
report_host=192.168.2.112
report_port=3308
EOF
​
​
-- mgr2
cat >  /etc/my.cnf <<"EOF"
[mysqld]
user=mysql
port=3308
character_set_server=utf8mb4
secure_file_priv=
server-id = 2
default-time-zone = '+8:00'
log_timestamps = SYSTEM
log-bin =
binlog_format=row
binlog_checksum=NONE
skip-name-resolve
log_slave_updates = 1
gtid-mode=ON
enforce-gtid-consistency=on
default_authentication_plugin=mysql_native_password
max_allowed_packet = 500M
lower_case_table_names = 1
sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION
expire_logs_days= 60
max_connections = 20000
innodb_buffer_pool_size=1024M
slow_query_log = ON
#slow_query_log_file=/usr/local/mysql/data/mysql-slow.log
​
#安装mysql_replication引擎前提
master_info_repository=TABLE
relay_log_info_repository=TABLE
relay_log=lhrmgr31-relay-bin
​
#同步算法,官网建议设置该参数使用 XXHASH64 算法
transaction_write_set_extraction=XXHASH64
#集群的uuid,组名必须是一个uuid
loose-group_replication_group_name="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
#是否随着服务启动集群,这里设置否
loose-group_replication_start_on_boot=OFF
#集群本机端口,和服务端口不同,以本机端口 33061 接受来自组中成员的传入连接
loose-group_replication_local_address= "192.168.2.34:33061"
#集群包含的所有节点
loose-group_replication_group_seeds= "192.168.2.112:33061,192.168.2.34:33061,192.168.2.170:33061"
#是否设置为主节点,当创建集群时其他加入的节点都以该节点为目标加入集群
loose-group_replication_bootstrap_group=OFF
#设置白名单
loose-group_replication_ip_whitelist="192.168.2.112,192.168.2.34,192.168.2.170"
​
report_host=192.168.2.34
report_port=3308
EOF
​
-- mgr3
cat > /etc/my.cnf <<"EOF"
[mysqld]
user=mysql
port=3308
character_set_server=utf8mb4
secure_file_priv=
server-id = 3
default-time-zone = '+8:00'
log_timestamps = SYSTEM
log-bin =
binlog_format=row
binlog_checksum=NONE
skip-name-resolve
log_slave_updates = 1
gtid-mode=ON
enforce-gtid-consistency=on
default_authentication_plugin=mysql_native_password
max_allowed_packet = 500M
lower_case_table_names = 1
sql_mode=STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_ENGINE_SUBSTITUTION
expire_logs_days= 60
max_connections = 20000
innodb_buffer_pool_size=1024M
slow_query_log = ON
#slow_query_log_file=/usr/local/mysql/data/mysql-slow.log
​
#安装mysql_replication引擎前提
master_info_repository=TABLE
relay_log_info_repository=TABLE
relay_log=lhrmgr31-relay-bin
​
#同步算法,官网建议设置该参数使用 XXHASH64 算法
transaction_write_set_extraction=XXHASH64
#集群的uuid,组名必须是一个uuid
loose-group_replication_group_name="aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"
#是否随着服务启动集群,这里设置否
loose-group_replication_start_on_boot=OFF
#集群本机端口,和服务端口不同,以本机端口 33061 接受来自组中成员的传入连接
loose-group_replication_local_address= "192.168.2.170:33061"
#集群包含的所有节点
loose-group_replication_group_seeds= "192.168.2.112:33061,192.168.2.34:33061,192.168.2.170:33061"
#是否设置为主节点,当创建集群时其他加入的节点都以该节点为目标加入集群
loose-group_replication_bootstrap_group=OFF
#设置白名单
loose-group_replication_ip_whitelist="192.168.2.112,192.168.2.34,192.168.2.170"
​
report_host=192.168.2.170
report_port=3308
EOF

三、重启MySQL环境

查看MySQL的主机名、server_id和server_uuid
[root@mgr1 ~]# mysql -uroot -proot123 -e "select @@hostname,@@server_id,@@server_uuid"
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+-------------+--------------------------------------+
| @@hostname | @@server_id | @@server_uuid                        |
+------------+-------------+--------------------------------------+
| mgr1       |       1 | 63f351cc-cdf1-11ed-a386-0242ac48000b |
+------------+-------------+--------------------------------------+
​
[root@mgr2 /]# mysql -uroot -proot123 -e "select @@hostname,@@server_id,@@server_uuid"
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+-------------+--------------------------------------+
| @@hostname | @@server_id | @@server_uuid                        |
+------------+-------------+--------------------------------------+
| mgr2       |       2 | 3401b339-cdf1-11ed-9b3f-0242ac48000c |
+------------+-------------+--------------------------------------+
​
[root@mgr3 /]# mysql -uroot -proot123 -e "select @@hostname,@@server_id,@@server_uuid"
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------+-------------+--------------------------------------+
| @@hostname | @@server_id | @@server_uuid                        |
+------------+-------------+--------------------------------------+
| mgr3       |       3 | 32b2fe4a-cdf1-11ed-989c-0242ac48000d |
+------------+-------------+--------------------------------------+

四、安装MGR插件(所有节点执行)

mysql> install plugin group_replication soname 'group_replication.so';
Query OK, 0 rows affected (0.08 sec
​
MySQL [(none)]> show plugins;
+---------------------------------+----------+--------------------+----------------------+---------+
| Name                            | Status   | Type               | Library              | License |
+---------------------------------+----------+--------------------+----------------------+---------+
...................................                           
| group_replication               | ACTIVE   | GROUP REPLICATION  | group_replication.so | GPL     |
+---------------------------------+----------+--------------------+----------------------+---------+
46 rows in set (0.01 sec)

五、设置复制账号(所有节点执行)

-- 创建账号不记录binlog
mysql> SET SQL_LOG_BIN=0;
Query OK, 0 rows affected (0.00 sec)
​
mysql> CREATE USER repl@'%' IDENTIFIED BY 'repl';
Query OK, 0 rows affected (0.02 sec)
​
mysql> GRANT REPLICATION SLAVE ON *.* TO repl@'%';
Query OK, 0 rows affected (0.00 sec)
​
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.01 sec)
​
mysql> SET SQL_LOG_BIN=1;
Query OK, 0 rows affected (0.01 sec)
​
mysql> change replication source to source_user='repl', source_password='repl' for channel 'group_replication_recovery';
Query OK, 0 rows affected, 2 warnings (0.10 sec)

六、启动MGR单主模式

6.1 启动MGR

-- 只在主库(192.168.2.112)上执行
mysql> set global group_replication_bootstrap_group=on;
Query OK, 0 rows affected (0.00 sec)
​
mysql> start group_replication;
Query OK, 0 rows affected, 1 warning (1.16 sec)
​
mysql> set global group_replication_bootstrap_group=off;
Query OK, 0 rows affected (0.02 sec)
​
-- 查看mgr组信息 
select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST   | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | b6876345-8c95-11f0-8f09-fa163ee831ba | 192.168.2.112 |        3308 | ONLINE       | PRIMARY     | 8.0.28         | XCom                       |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
3 rows in set (0.00 sec)

6.2 其他节点加入MGR

-- 在从库(192.168.2.170,192.168.2.34)上执行(必须先清理事务,否则加入组失败)
mysql> start group_replication;
ERROR 3092 (HY000): The server is not configured properly to be an active member of the group. Please see more details on error log.
​
mysql> reset master;
Query OK, 0 rows affected (0.15 sec)
mysql> start group_replication;
Query OK, 0 rows affected, 1 warning (4.01 sec)
​
-- 再次查看MGR组信息
mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST   | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | b6876345-8c95-11f0-8f09-fa163ee831ba | 192.168.2.112 |        3308 | ONLINE       | PRIMARY     | 8.0.28         | XCom                       |
| group_replication_applier | bb7db288-8c95-11f0-b3c8-fa163eb1e28d | 192.168.2.170 |        3308 | ONLINE       | SECONDARY   | 8.0.28         | XCom                       |
| group_replication_applier | be7b73a2-8c95-11f0-85a9-fa163ea70aac | 192.168.2.34  |        3308 | ONLINE       | SECONDARY   | 8.0.28         | XCom                       |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
3 rows in set (0.00 sec)

结论:可以看到,3个节点状态为online,192.168.2.112为PRIMARY主节点,只有主节点可以写入,其他节点只读,MGR单主模式搭建成功,单主也是默认模式。

– 其他SECONDARY节点,执行写操作报错:

mysql> create database test;
ERROR 1290 (HY000): The MySQL server is running with the --super-read-only option so it cannot execute this statement

七、测试同步

在主节点上创建测试数据,然后在其它节点查询:

-- 创建测试库
mysql> create database testdb;
Query OK, 1 row affected (0.09 sec)
-- 创建测试表,并插入测试数据
mysql> CREATE TABLE testdb.`t1` ( `id` int(11) NOT NULL AUTO_INCREMENT, `name` varchar(100) DEFAULT NULL, PRIMARY KEY (`id`) ) ENGINE=InnoDB;
Query OK, 0 rows affected, 1 warning (0.12 sec)

mysql> insert into testdb.t1 values (1,'tony');
Query OK, 1 row affected (0.09 sec)

mysql> insert into testdb.t1 values (2,'lion');
Query OK, 1 row affected (0.12 sec)

mysql> insert into testdb.t1 values (3,'jack');
Query OK, 1 row affected (0.05 sec)

mysql> select * from testdb.t1;
+----+------+
| id | name |
+----+------+
|  1 | tony |
|  2 | lion |
|  3 | jack |
+----+------+
3 rows in set (0.01 sec)

-- 其他2个节点查询出来的结果一样
mysql> select * from testdb.t1;
+----+------+
| id | name |
+----+------+
|  1 | tony |
|  2 | lion |
|  3 | jack |
+----+------+
3 rows in set (0.04 sec)

八、多主和单主模式切换

7.1 查询当前模式

mysql> show variables like '%group_replication_single_primary_mode%';
+---------------------------------------+-------+
| Variable_name                         | Value |
+---------------------------------------+-------+
| group_replication_single_primary_mode | ON    |
+---------------------------------------+-------+
1 row in set (0.08 sec)

mysql> SELECT @@group_replication_single_primary_mode;
+-----------------------------------------+
| @@group_replication_single_primary_mode |
+-----------------------------------------+
|                                       1 |
+-----------------------------------------+
1 row in set (0.00 sec)

结论:参数group_replication_single_primary_mode为ON,表示单主模式。

7.2 函数实现多主和单主在线切换

函数切换:从MySQL 8.0.13开始,可以使用函数进行在线修改MGR模式,不需要重启组复制。

-- 单主切多主
select group_replication_switch_to_multi_primary_mode(); 

-- 多主切单主,入参需要传入主库的server_uuid
select group_replication_switch_to_single_primary_mode('@@server_uuid') ;

7.2.1 单主切多主模式

mysql> select @@group_replication_single_primary_mode;
+-----------------------------------------+
| @@group_replication_single_primary_mode |
+-----------------------------------------+
|                                       1 |
+-----------------------------------------+
1 row in set (0.01 sec)

mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST   | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | b6876345-8c95-11f0-8f09-fa163ee831ba | 192.168.2.112 |        3308 | ONLINE       | PRIMARY     | 8.0.28         | XCom                       |
| group_replication_applier | bb7db288-8c95-11f0-b3c8-fa163eb1e28d | 192.168.2.170 |        3308 | ONLINE       | SECONDARY   | 8.0.28         | XCom                       |
| group_replication_applier | be7b73a2-8c95-11f0-85a9-fa163ea70aac | 192.168.2.34  |        3308 | ONLINE       | SECONDARY   | 8.0.28         | XCom                       |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
3 rows in set (0.00 sec)

-- 单主切多主
mysql> select group_replication_switch_to_multi_primary_mode();
+--------------------------------------------------+
| group_replication_switch_to_multi_primary_mode() |
+--------------------------------------------------+
| Mode switched to multi-primary successfully.     |
+--------------------------------------------------+
1 row in set (1.38 sec)

mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST   | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | b6876345-8c95-11f0-8f09-fa163ee831ba | 192.168.2.112 |        3308 | ONLINE       | PRIMARY     | 8.0.28         | XCom                       |
| group_replication_applier | bb7db288-8c95-11f0-b3c8-fa163eb1e28d | 192.168.2.170 |        3308 | ONLINE       | PRIMARY     | 8.0.28         | XCom                       |
| group_replication_applier | be7b73a2-8c95-11f0-85a9-fa163ea70aac | 192.168.2.34  |        3308 | ONLINE       | PRIMARY     | 8.0.28         | XCom                       |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
3 rows in set (0.00 sec)


mysql> select @@group_replication_single_primary_mode;
+-----------------------------------------+
| @@group_replication_single_primary_mode |
+-----------------------------------------+
|                                       0 |
+-----------------------------------------+
1 row in set (0.02 sec)

结论:所有成员角色都变为PRIMARY主节点,成功切换成多主模式。 

7.2.2 多主切单主模式

-- 重新切换成单主模式,主节点选择mgr1
mysql> select @@group_replication_single_primary_mode;
+-----------------------------------------+
| @@group_replication_single_primary_mode |
+-----------------------------------------+
|                                       0 |
+-----------------------------------------+
1 row in set (0.02 sec)

mysql> select @@server_uuid;
+--------------------------------------+
| @@server_uuid                        |
+--------------------------------------+
| 63f351cc-cdf1-11ed-a386-0242ac48000b |
+--------------------------------------+
1 row in set (0.00 sec)

mysql> select group_replication_switch_to_single_primary_mode('63f351cc-cdf1-11ed-a386-0242ac48000b') ;
+-----------------------------------------------------------------------------------------+
| group_replication_switch_to_single_primary_mode('63f351cc-cdf1-11ed-a386-0242ac48000b') |
+-----------------------------------------------------------------------------------------+
| Mode switched to single-primary successfully.                                           |
+-----------------------------------------------------------------------------------------+
1 row in set (0.59 sec)

mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST   | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | b6876345-8c95-11f0-8f09-fa163ee831ba | 192.168.2.112 |        3308 | ONLINE       | PRIMARY     | 8.0.28         | XCom                       |
| group_replication_applier | bb7db288-8c95-11f0-b3c8-fa163eb1e28d | 192.168.2.170 |        3308 | ONLINE       | SECONDARY   | 8.0.28         | XCom                       |
| group_replication_applier | be7b73a2-8c95-11f0-85a9-fa163ea70aac | 192.168.2.34  |        3308 | ONLINE       | SECONDARY   | 8.0.28         | XCom                       |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
3 rows in set (0.00 sec)

mysql> select @@group_replication_single_primary_mode;
+-----------------------------------------+
| @@group_replication_single_primary_mode |
+-----------------------------------------+
|                                       1 |
+-----------------------------------------+
1 row in set (0.00 sec)

结论:成功切换成单主模式,主节点选择mgr1。

九 模拟主节点宕机

– 如果是其中一个SECONDARY宕机,则不影响集群运行,这里模拟主节点宕机。

9.1 查询当前MGR成员状态

mysql> select @@group_replication_single_primary_mode;
+-----------------------------------------+
| @@group_replication_single_primary_mode |
+-----------------------------------------+
|                                       1 |
+-----------------------------------------+
1 row in set (0.00 sec)

mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST   | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | b6876345-8c95-11f0-8f09-fa163ee831ba | 192.168.2.112 |        3308 | ONLINE       | PRIMARY     | 8.0.28         | XCom                       |
| group_replication_applier | bb7db288-8c95-11f0-b3c8-fa163eb1e28d | 192.168.2.170 |        3308 | ONLINE       | SECONDARY   | 8.0.28         | XCom                       |
| group_replication_applier | be7b73a2-8c95-11f0-85a9-fa163ea70aac | 192.168.2.34  |        3308 | ONLINE       | SECONDARY   | 8.0.28         | XCom                       |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
3 rows in set (0.00 sec)

结论:当前为单主模式,所有成员都是online,主节点为mgr1。

9.2 模拟主节点宕机

[root@monitor mysql-cluster]# docker-compose down
[+] Running 3/3
 ✔ Container mysql                Removed                                                                                                                                                                7.7s 
 ✔ Container proxysql             Removed                                                                                                                                                                0.8s 
 ✔ Network mysql-cluster_default  Removed       

9.3 再次查看集群状态

mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST   | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | b6876345-8c95-11f0-8f09-fa163ee831ba | 192.168.2.112 |        3308 | ONLINE       | PRIMARY     | 8.0.28         | XCom                       |
| group_replication_applier | bb7db288-8c95-11f0-b3c8-fa163eb1e28d | 192.168.2.170 |        3308 | ONLINE       | SECONDARY   | 8.0.28         | XCom                       |
| group_replication_applier | be7b73a2-8c95-11f0-85a9-fa163ea70aac | 192.168.2.34  |        3308 | ONLINE       | SECONDARY   | 8.0.28         | XCom                       |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
3 rows in set (0.00 sec)

mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST   | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | bb7db288-8c95-11f0-b3c8-fa163eb1e28d | 192.168.2.170 |        3308 | ONLINE       | PRIMARY     | 8.0.28         | XCom                       |
| group_replication_applier | be7b73a2-8c95-11f0-85a9-fa163ea70aac | 192.168.2.34  |        3308 | ONLINE       | SECONDARY   | 8.0.28         | XCom                       |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
2 rows in set (0.00 sec)

结论:原主节点为mgr1状态变为UNREACHABLE,随后被剔除,主节点变为mgr3。

当进行自动主库选举时会按照下列优先级进行:

1.低版本优先,当组复制中同时存在高版本和低版本MySQL时,低版本会被选举为主库(向下兼容)。
2.如果有多个低版本,则会参考group_replication_member_weight(0-100)变量的值,默认为50,数字越大优先级越高。
3.如果版本,group_replication_member_weight都一样,最后考虑的因素是UUID的排序,UUID最小的选举为主库。

mysql> show variables like 'group_replication_member_weight';
+---------------------------------+-------+
| Variable_name                   | Value |
+---------------------------------+-------+
| group_replication_member_weight | 50    |
+---------------------------------+-------+
1 row in set (0.00 sec)

9.4 新的主节点上操作

mysql> insert into testdb.t1 values (4,'leo');
Query OK, 1 row affected (0.00 sec)

mysql> insert into testdb.t1 values (5,'Aaron');
Query OK, 1 row affected (0.00 sec)

mysql> select * from testdb.t1;
+----+-------+
| id | name  |
+----+-------+
|  1 | tony  |
|  2 | lion  |
|  3 | jack  |
|  4 | leo   |
|  5 | Aaron |
+----+-------+
5 rows in set (0.00 sec)


-- 其他节点查询结果一致
mysql> select * from testdb.t1;
+----+-------+
| id | name  |
+----+-------+
|  1 | tony  |
|  2 | lion  |
|  3 | jack  |
|  4 | leo   |
|  5 | Aaron |
+----+-------+
5 rows in set (0.00 sec)

9.5 原主节点重新启动

[root@monitor mysql-cluster]# docker exec -it mysql mysql -uroot -pHbis@123
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 405
Server version: 8.0.28 MySQL Community Server - GPL

Copyright (c) 2000, 2022, Oracle and/or its affiliates.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> SHOW VARIABLES LIKE 'bind_address';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| bind_address  | *     |

mysql> select * from performance_schema.replication_group_members;
+---------------------------+-----------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID | MEMBER_HOST | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+-----------+-------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier |           |             |        NULL | OFFLINE      |             |                |                            |
+---------------------------+-----------+-------------+-------------+--------------+-------------+----------------+-----------------
1 row in set (0.03 sec)
-- 以备库角色加入原有的MGR群
mysql> start group_replication;
Query OK, 0 rows affected, 1 warning (20.74 sec)

mysql> select * from performance_schema.replication_group_members;
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| CHANNEL_NAME              | MEMBER_ID                            | MEMBER_HOST   | MEMBER_PORT | MEMBER_STATE | MEMBER_ROLE | MEMBER_VERSION | MEMBER_COMMUNICATION_STACK |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
| group_replication_applier | b6876345-8c95-11f0-8f09-fa163ee831ba | 192.168.2.112 |        3308 | ONLINE       | SECONDARY   | 8.0.28         | XCom                       |
| group_replication_applier | bb7db288-8c95-11f0-b3c8-fa163eb1e28d | 192.168.2.170 |        3308 | ONLINE       | PRIMARY     | 8.0.28         | XCom                       |
| group_replication_applier | be7b73a2-8c95-11f0-85a9-fa163ea70aac | 192.168.2.34  |        3308 | ONLINE       | SECONDARY   | 8.0.28         | XCom                       |
+---------------------------+--------------------------------------+---------------+-------------+--------------+-------------+----------------+----------------------------+
3 rows in set (0.00 sec)


mysql> select * from testdb.t1;
+----+-------+
| id | name  |
+----+-------+
|  1 | tony  |
|  2 | lion  |
|  3 | jack  |
|  4 | leo   |
|  5 | Aaron |
+----+-------+
5 rows in set (0.02 sec)

结论:原主节点为mgr1 以备库角色加入到集群,并且新主节点的操作也同步到同步到mgr1中。

注意: 原主节点为mgr1 启动后,不要以主节点的形式,加入到MGR集群中,无法成功加入到mgr集群,甚至会因为一些错误操作,需要重新初始化该节点,才能正常加入集群。所以,当原来的主节点宕机后,注意检查参数loose-group_replication_start_on_boot、loose-group_replication_bootstrap_group,建议参数文件中都设置成OFF。

十、docker-compose快速安装脚本部署

install.sh

#!/bin/bash
this_script_dir=$(cd $(dirname ${BASH_SOURCE[0]}) && pwd)
cd $this_script_dir
source ./.env

set -e
function showMsg(){ echo -e "\033[32m$1\033[0m"; }
function showErr(){ echo -e "\033[31m$1\033[0m"; }


# 2. 节点与对应变量
declare -A NODES=(
  [node1]="$MYSQL_NODE1_IP"
  [node2]="$MYSQL_NODE2_IP"
  [node3]="$MYSQL_NODE3_IP"
)
declare -A STATES=(
  [node1]="MASTER"
  [node2]="BACKUP"
  [node3]="BACKUP"
)
declare -A IDs=(
  [node1]="150"
  [node2]="120"
  [node3]="100"
)

# 端口检查函数
check_port() {
    local PORT=$1
    for NODE in "${!NODES[@]}"; do
      IP=${NODES[$NODE]}
        ssh -o StrictHostKeyChecking=no "$SSH_USER@$IP" "ss -lnt | awk '{print \$4}' | grep -q ':${PORT}\$'" \
            && { showErr ">>> $NODE 上端口 $PORT 已被占用"; exit 1; } \
            || showMsg ">>> $NODE 上端口 $PORT 可用"
    done
}

# 拷贝镜像并加载到所有节点
load_image() {
    for NODE in "${!NODES[@]}"; do
      IP=${NODES[$NODE]}
        showMsg ">>> 在 $NODE 创建目录 /opt/mysql-install/"
        ssh -o StrictHostKeyChecking=no "$SSH_USER@$IP" "mkdir -p /opt/mysql-install"

        showMsg ">>> 拷贝 mysql.tar.gz keepalived.tar.gz 到 $NODE"
        scp -o StrictHostKeyChecking=no mysql.tar.gz "$SSH_USER@$IP":/opt/mysql-install//mysql.tar.gz
        scp -o StrictHostKeyChecking=no keepalived.tar.gz "$SSH_USER@$IP":/opt/mysql-install//keepalived.tar.gz

        showMsg ">>> 在 $NODE 加载 mysql 镜像"
        ssh -o StrictHostKeyChecking=no "$SSH_USER@$IP" "docker load -i /opt/mysql-install/mysql.tar.gz"

        showMsg ">>> 在 $NODE 加载 keepalived 镜像"
        ssh -o StrictHostKeyChecking=no "$SSH_USER@$IP" "docker load -i /opt/mysql-install/keepalived.tar.gz"
    done
}

# 渲染配置文件
render_config() {
    # 生成临时目录
    TMP_DIR=$(mktemp -d)
    trap "rm -rf $TMP_DIR" EXIT

    # 修改配置文件
    for NODE in "${!NODES[@]}"; do
      IP=${NODES[$NODE]}
      ID=${IDs[$NODE]}
      STATE=${STATES[$NODE]}

      showMsg ">>> 正在处理 $NODE ($IP) ..."
      sed \
        -e "s/MYSQL_PORT/$MYSQL_PORT/g" \
        -e "s/STATE/$STATE/g" \
        -e "s/MYSQL_NODE1_IP/$MYSQL_NODE1_IP/g" \
        -e "s/MYSQL_NODE2_IP/$MYSQL_NODE2_IP/g" \
        -e "s/MYSQL_NODE3_IP/$MYSQL_NODE3_IP/g" \
        "conf/mysql-${NODE}.cnf" > "$TMP_DIR/my.cnf"
      # 渲染 docker-compose.yml
      cat > "$TMP_DIR/docker-compose.yml" <<EOF
services:
  mysql:
    image: mysql:8.0.28
    container_name: mysql
    environment:
      MYSQL_ROOT_PASSWORD: $ROOT_PWD
    volumes:
      - ./mysql_data:/var/lib/mysql
      - ./my.cnf:/etc/mysql/conf.d/mysql.cnf
    network_mode: host
    healthcheck:
      test: [ "CMD", "mysqladmin" ,"ping", "-h", "localhost" ]
      interval: 5s
      timeout: 10s
      retries: 20
  keepalived:
    image: keepalived
    container_name: keepalived
    restart: always
    network_mode: host
    cap_add:
      - NET_ADMIN
      - NET_BROADCAST
      - NET_RAW
    environment:
      KEEPALIVED_INTERFACE: "$NIC"
      KEEPALIVED_PASSWORD: "123456"
      KEEPALIVED_STATE: "$STATE"
      KEEPALIVED_PRIORITY: "$ID"
      KEEPALIVED_UNICAST_PEERS: "#PYTHON2BASH:['$MYSQL_NODE1_IP','$MYSQL_NODE2_IP','$MYSQL_NODE3_IP']"
      KEEPALIVED_VIRTUAL_IPS: "$VIP"
EOF

      # 远程创建目录
      ssh -o StrictHostKeyChecking=no "$SSH_USER@$IP" "mkdir -p /opt/mysql-install/"
      # 拷贝文件
      scp "$TMP_DIR/my.cnf"       "$SSH_USER@$IP:/opt/mysql-install/"
      scp "$TMP_DIR/docker-compose.yml" "$SSH_USER@$IP:/opt/mysql-install/"
      #scp "./.env" "$SSH_USER@$IP:/opt/mysql-install/"

    done
}
start_service(){
    for NODE in "${!NODES[@]}"; do
    IP=${NODES[$NODE]}
      showMsg ">>> 在 $NODE ($IP) 启动服务"
      ssh -o StrictHostKeyChecking=no "${SSH_USER}@${IP}" "docker-compose -f /opt/mysql-install/docker-compose.yml up -d"
    done
}
init_mgr() {
    # 1. 所有节点:创建 repl 用户
    for NODE in "${!NODES[@]}"; do
      IP=${NODES[$NODE]}
      showMsg ">>> 在 $NODE ($IP) 上创建 MGR 用户 repl ..."
      ssh -o StrictHostKeyChecking=no "$SSH_USER@$IP" "
        docker exec mysql mysql -uroot -p$ROOT_PWD -e \"
          set sql_log_bin=0;
          create user if not exists 'repl'@'%' identified by 'repl';
          grant replication slave on *.* to 'repl'@'%';
          flush privileges;
          set sql_log_bin=1;
          change replication source to source_user='repl', source_password='repl' for channel 'group_replication_recovery';
      \" "
    done

    # 2. 主库初始化
    MASTER_IP=${NODES[node1]}
    showMsg ">>> 在主库 ($MASTER_IP) 上启动 group_replication "
    ssh -o StrictHostKeyChecking=no "$SSH_USER@$MASTER_IP" "
      docker exec mysql mysql -uroot -p$ROOT_PWD -e \"
        set global group_replication_bootstrap_group=on;
        start group_replication;
        set global group_replication_bootstrap_group=off;
    \" "

    # 3. 从库加入集群
    for NODE in node2 node3; do
      IP=${NODES[$NODE]}
      showMsg ">>> 在从库 ($IP) 加入 group_replication ..."
      ssh -o StrictHostKeyChecking=no "$SSH_USER@$IP" "
        docker exec mysql mysql -uroot -p$ROOT_PWD -e \"
          reset master;
          start group_replication;
      \" "
    done
    sleep 5
    # 4. 在主库查看集群成员
    showMsg ">>> 查看集群状态 ..."
    ssh -o StrictHostKeyChecking=no "$SSH_USER@$MASTER_IP" "
      docker exec mysql mysql -uroot -p$ROOT_PWD -e \"
        select group_replication_switch_to_multi_primary_mode();
        select * from performance_schema.replication_group_members;
    \" "
}

wait_mysql_healthy() {
    for NODE in "${!NODES[@]}"; do
        IP=${NODES[$NODE]}
        showMsg ">>> 等待 $NODE ($IP) MySQL 服务变为 healthy ..."

        # 最多等待 300 秒
        for i in {1..60}; do
            STATUS=$(ssh -o StrictHostKeyChecking=no "$SSH_USER@$IP" \
                "docker inspect --format='{{json .State.Health.Status}}' mysql 2>/dev/null" | tr -d '"')

            if [ "$STATUS" == "healthy" ]; then
                showMsg ">>> $NODE ($IP) MySQL 服务 healthy"
                break
            else
                showMsg ">>> $NODE ($IP) MySQL 状态: $STATUS (第 $i 次检查)"
                sleep 5
            fi

            if [ $i -eq 60 ]; then
                showErr ">>> $NODE ($IP) MySQL 在 300 秒内未变为 healthy,退出!"
                exit 1
            fi
        done
    done
}


main() {
    check_port $MYSQL_PORT
    load_image
    render_config
    start_service
    wait_mysql_healthy
    sleep 3
    init_mgr
    showMsg "MySQL 集群安装完成"
}
main "$@"
#!/bin/bash
this_script_dir=$(cd $(dirname ${BASH_SOURCE[0]}) && pwd)
cd $this_script_dir
source ./.env

set -e
function showMsg(){ echo -e "\033[32m$1\033[0m"; }
function showErr(){ echo -e "\033[31m$1\033[0m"; }

# 节点列表(跟部署脚本保持一致)
declare -A NODES=(
  [node1]="$MYSQL_NODE1_IP"
  [node2]="$MYSQL_NODE2_IP"
  [node3]="$MYSQL_NODE3_IP"
)

uninstall_node(){
    local NODE=$1
    local IP=$2

    showMsg ">>> [$NODE] 停止并删除容器"
    ssh -o StrictHostKeyChecking=no "$SSH_USER@$IP" "cd /opt/mysql-install && docker-compose down || true"

    showMsg ">>> [$NODE] 删除容器残留"
    ssh -o StrictHostKeyChecking=no "$SSH_USER@$IP" "docker rm -f mysql keepalived >/dev/null 2>&1 || true"

    showMsg ">>> [$NODE] 删除挂载目录 /opt/mysql-install"
    ssh -o StrictHostKeyChecking=no "$SSH_USER@$IP" "rm -rf /opt/mysql-install"

    showMsg ">>> [$NODE] 删除 mysql/keepalived 镜像"
    ssh -o StrictHostKeyChecking=no "$SSH_USER@$IP" "docker rmi -f mysql:8.0.28 keepalived >/dev/null 2>&1 || true"
}

main(){
    for NODE in "${!NODES[@]}"; do
        IP=${NODES[$NODE]}
        if [ -z "$IP" ]; then
            showErr "[$NODE] 没有配置 IP,跳过"
            continue
        fi
        uninstall_node "$NODE" "$IP"
    done

    showMsg "MySQL 集群卸载完成"
}

main "$@"
MYSQL_NODE1_IP=192.168.2.112
MYSQL_NODE2_IP=192.168.2.129
MYSQL_NODE3_IP=192.168.2.159
VIP=192.168.2.164               # keepalived的vip,需根据实际规划填写
NIC=eth0                        # 安装机器的内网ip地址所属网卡名称
MYSQL_PORT=3308
SSH_USER=root
REPL_USER=repl
REPL_PWD=repl123
ROOT_PWD=Hbis@123

评论
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值