1、MySQL读写分离概述
MySQL读写分离是一种常见的数据库架构优化方案,旨在提高数据库的整体性能和处理能力。在MySQL主从复制环境中,主服务器(Master)主要负责处理写操作(如插入、更新、删除),而从服务器(Slave)负责处理读操作(如查询)。通过这种方式,读写操作被分配到不同的服务器上运行,从而减少主服务器的负载,并提高系统的并发处理能力。
读写分离能够有效地解决数据库在高并发读写场景下的性能瓶颈,特别适用于读操作远多于写操作的应用场景,如电商网站、社交平台等。同时,它还能够增强系统的容错性和可扩展性,增加从服务器的数量以分散读请求,既提升了系统的响应速度,又减轻了单一数据库的压力。然而,读写分离在实现时需要保证数据的实时性与一致性,尤其是处理写操作后从服务器的同步延迟问题,这需要通过合适的调度策略来解决。
2、MySQL读写分离的功能
提高性能:通过将读操作和写操作分离,读写分离可以有效减轻主数据库的负担,提高系统的整体性能。
负载均衡:通过在多个从数据库上分发读请求,可以实现负载均衡,避免单点故障,提高系统的并发处理能力。
提高可用性:读写分离架构可以提高系统的可用性。如果主数据库发生故障,可以快速切换到从数据库,确保系统的正常运行
扩展性:读写分离可以方便地扩展数据库系统。通过添加更多的从数据库,可以横向扩展系统的读取能力,应对不断增长的读取需求。
故障恢复:当主数据库发生故障时,从数据库可以顶替主数据库的角色,保证系统的持续运行,降低系统者机的风险。
3、MySQL读写分离应用场景
高并发读取:适用于读操作频繁、、写操作相对较少的场景,如新闻网站、电子商务平台等能够显著提升系统的读取性能。
数据报表查询:对于需要大量数据分析和报表生成的应用,读写分离可以加快数据查询速度,提高报表生成的效率。
全文搜索:对于需要进行全文搜索的应用,读写分离可以将搜索操作分担到从数据库上,减轻主数据库的压力。
分布式部署:在分布式系统中,通过读写分离可以简化数据访问模式,提高系统的整体性能和稳走性。
大型社交平台:适用于大型社交平台等需要处理大量用户读取请求的场景,通过读写分离可以更好地应对高并发读取的挑战。
4、实现Mysql读写分离的方法
为常见的MySQL读写分离分为以下两种:
【基于程序代码内部实现】
在代码中根据select和insert进行路由分类,这种方法是生产环境中应用最广泛的,优点:性能较好,因为直接在程序代码中实现,不需要额外的硬件设备,减少开支。
缺点:需要开发人员实现,运维人员难以参与。如果是大型复杂的Java应用,在代码中实现读写分离可能会导致较大的代码改动。
【基于中间代理层实现】
代理位于客户端和服务器之间,接收客户端请求后,通过判断将请求转发到后端数据库。常见的代理程序有以下几种:
1.MySQL-Proxy:MySQL的开源项目,通过Lua脚本进行SQL判断和路由。需要编写大量Lua脚本,较为复杂。
2.Atlas:由奇虎360开发,基于MySQL-Proxy进行优化,增加了更多功能特性,支持事务和存储过程。在360内部,每天承载的读写请求数达数十亿。
3.Amoeba:由陈思儒开发,基于Java语言,阿里巴巴曾在生产环境中使用。优点是易用、移植性强,但不支持事务和存储过程。
4.Mycat:基于阿里开源的Cobar产品而研发,国内最活跃的、性能最好的开源数据库中间件之一。优点是性能可靠稳定、强大的技术团队、体系完善、社区活跃。
5、读写分离工作原理
客户端所有的写操作(插入、更新、删除)可以通过Amoeba服务器将请求调度到主服务器上进行。
客户端所有的读操作(查询)可以通过Amoeba服务器将请求分发到多个从数据库上进行,以实现负载均衡。
主服务器和从服务器的数据通过主从复制机制保持数据的一致性。
这种架构模式可以提高系统的性能、可用性和扩展性,是许多高负载应用和网站常用的数据库架构设计模式。
1.集群说明
| 主机 | 应用软件 | ip |
| HAProxy01 | haproxy-*tar.gz + keepalived_* tar.gz | |
| HAProxy02 | haproxy-*tar.gz + keepalived_* tar.gz | |
| MySQL01 | mysql-*-e17-x86 64.targz | 192.168.216.3(CentOS-3) |
| MySQL02 | mysql-*-e17-x86 64.targz | 192.168.216.5(CentOS-5) |
2、HAProxy01和HAProxy02源码安装haproxy
# yum -y install openssl-devel zlib-devel pcre-devel systemd-devel
# yum -y install haproxy
# rpm -ql haproxy
# cp /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg-bak
# vim /etc/haproxy/haproxy.cfg /最少连接(主备)
global
log 127.0.0.1 local2
pidfile /var/run/haproxy.pid
user haproxy
group haproxy
daemon
nbproc 1defaults
mode tcp
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10m
timeout client 10m
timeout server 10m
timeout http-keep-alive 10s
timeout check 10slisten status
bind *:9999
mode http
stats uri /status
stats auth admin:class@qm.com (用户admin密码class@qm.com)
stats hide-versionfrontend db_read
bind *:3306
default_backend mysql_readbackend mysql_read
balance leastconn
server mysql02 192.168.216.5:3306 check
server mysql01 192.168.216.3:3306 check backupfrontend db_write
bind *:3307
default_backend mysql_writebackend mysql_write
balance leastconn
server mysql01 192.168.216.3:3306 check
server mysql02 192.168.216.5:3306 check backup
# /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -c
# vim /usr/lib/systemd/system/haproxy.service
[Unit]
Description=HAProxy Load Balancer
After=syslog.target network.target[Service]
EnvironmentFile=/etc/sysconfig/haproxy
ExecStartPre=/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -c
ExecStart=/usr/sbin/haproxy-systemd-wrapper -f /etc/haproxy/haproxy.cfg -p /run/haproxy.cfg
ExecReload=/bin/kill -USR2 $MAINPID
KillMode=mixed
[Install]
WantedBy=multi-user.target
# systemctl daemon-reload
# systemctl enable --now haproxy.service
# systemctl status haproxy.service
# ps -ef|grep haproxy
# ss -Input|grep haproxy
# sed -i 's/#$ModLoad imudp/$ModLoad imudp/g' /etc/rsyslog.conf
# sed -i 's/#$UDPServerRun 514/$UDPServerRun 514/g' /etc/rsyslog.conf
# echo "local2.* /var/log/haproxy.log" >> /etc/rsyslog.conf
# systemctl restart rsyslog.service
# ps -eflgrep rsyslog
# ss -nput|grep rsyslog
# systemctl reload haproxy.service
# systemctl status haproxy.service
# ps -efgrep haproxy
# ss -Inputgrep haproxy
# ll /var/log/haproxy.log
# tail -f /var/log/haproxy.log
3、HAProxy01源码安装Keepalived
# yum -y install openssl-devel libnl-devel
# cd /root/tools/
# rz -E /上传keepalived-*.tar.gz
# tar -xvf keepalived-*.tar.gz
# cd keepalived.*
# ./configure --help
# ./configure -prefix=/usr/local/keepalived-2.2.8 (配置)
# make (编译)
# make install (安装)
# In -s /usr/local/keepalived-* /usr/local/keepalived
# vim /usr/local/keepalived/etc/keepalived/keepalived.conf
global_defs {
router_id QM_WEB
script_user root
enable_script_security
}
vrrp_script HAPROXY_CHECK_STATUS {
script "/usr/local/keepalived/web_status_check.sh"
interval 3
weight -3
}
vrrp_instance QM_NGINX {
state MASTER
interface ens33
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass qm.com
}
virtual_ipaddress {
192.168.116.252 [写一个没用过的](虚拟ip)
}
track_script {
HAPROXY_CHECK_STATUS
}
}
# vim /usr/local/keepalived/web_status_check.sh
HAPROXY_PROCESS_STATUS=$(ps -ef|grep haproxy|grep -v grep|wc -l)
HAPROXY_PORT_STATUS=$(ss -lnput|grep haproxy|wc -l)
((HAPROXY_PROCESS_STATUS==0 || HAPROXY_PORT_STATUS==0)) && pkill keepalived
# chmod +x /usr/local/keepalived/web_status_check.sh
# systemctl daemon-reload
# systemctl enable --now keepalived.service
# systemctl status keepalived.service
# ps -efgrep keepalived
# ip address
4、HAProxy02源码安装Keepalived
# yum -y install openssl-devel libnl-devel
# cd /root/tools/
# rz -E //上传keepalived-*.tar.gz
# tar -xvf keepalived-*.tar.gz
# cd keepalived-*
# ./configure --help
# ./configure -prefix=/usr/local/keepalived-2.2.8
# make
# make install
# In -s /usr/local/keepalived-* /usr/local/keepalived
# vim /usr/local/keepalived/etc/keepalived/keepalived.conf
global_defs {
router_id QM_WEB
script_user root
enable_script_security
}
vrrp_script HAPROXY_CHECK_STATUS {
script "/usr/local/keepalived/web_status_check.sh"
interval 3
weight -3
}
vrrp_instance QM_NGINX {
state BACKUP
interface ens33
virtual_router_id 51
priority 99
advert_int 1
authentication {
auth_type PASS
auth_pass qm.com
}
virtual_ipaddress {
192.168.116.252
}
track_script {
HAPROXY_CHECK_STATUS
}
}
# vim /usr/local/keepalived/web_status_check.sh
HAPROXY_PROCESS_STATUS=$(ps -ef|grep haproxy|grep -v grep|wc -l)
HAPROXY_PORT_STATUS=$(ss -lnput|grep haproxy|wc -l)
((HAPROXY_PROCESS_STATUS==0 || HAPROXY_PORT_STATUS==0)) && pkill keepalived
# chmod 755 /usr/local/keepalived/web_status_check.sh
# systemctl daemon-reload
# systemctl enable --now keepalived.service
# systemctl status keepalived.service
# ps -ef|grep keepalived
# ip address
5、MySQL01和MySQL02创建程序用户并授权
# mysql -hlocalhost -P3306 -uroot -p //MySQL01执行
mysql> SELECT user,host FROM mysql.user;
mysql> CREATE USER `qm`@'%' IDENTIFIED by 'class@qm.com';
mysql> SELECT user,host FROM mysql.user;
mysql> GRANT ALL ON `qm`.* to 'qm'@'%';
mysql> SHOW GRANTS FOR 'qm'@'%';
mysql> exit
# mysql -hlocalhost -P3306 -uroot -p //MySQL02执行
mysql>SELECT user,host FROM mysql.user;
mysql> exit
6、通过HAProxy登录MySQL并验证主从同步
# mysql -h192.168.116.252 -P3306 -uqm -p //只读用户
mysql>SELECT @@hostname;
@@hostname CentOS-5 1 row in set(0.00 sec)
mysql>SELECT name,sex FROM qm.stu_info;
mysql> exit
# mysql -h192.168.116.252 -P3307 -uqm -p //读写用户添加数据
mysql>SELECT @@hostname;
@@hostname CentOS-3 1 row in set(0.00 sec)
mysql>INSERT INTO qm.stu_info(name,sex)VALUES('wangwu','male' );
mysql>SELECT name,sex FROM qm.stu_info;
mysql> exit
# mysql -h192.168.116.252 -P3306 -uqm -p //只读用户验证数据同步
mysql> SELECT @@hostname;
@@hostname CentOs-5
1 row in set(0.01 sec)
mysql> SELECT name,sex FROM qm.stu_info;
mysql> exit
7、模拟MySQL01故障
# systemctl stop mysqld.service //MySQL01执行
# mysql -h192.168.116.253 -P3306 -uqm -p //只读用户
mysql> SELECT @@hostname;
@@hostname CentOs-5
1 row in set(0.01 sec)
mysql> exit
# mysql -h192.168.116.253 -P3307 -uqm -p //读写用户
mysql> SELECT @@hostname;
@@hostname CentOs-5
1 row in set(0.01 sec)
mysql> exit
# mysql -hlocalhost -P3306 -uroot -p //MySQL02执行
mysql>SHOW SLAVE STATUS\G
//Slave_I0 Running:Connecting,Slave_SQL Running:Yes
mysql>STOP SLAVE;
//防止MySQL01恢复导致的数据不一致问题
mysql>INSERT INTO qm.stu_info(name,sex) VALUES('zhaoliu','male');
mysql>SELECT name,sex FROM qm.stu_info;
mysql> exit
8、HAProxy01和HAProxy02剔除MySQL01
# vim /etc/haproxy/haproxy.cfg
global
log 127.0.0.1 local2
pidfile /var/run/haproxy.pid
user haproxy
group haproxy
daemon
nbproc 1defaults
mode tcp
log global
option httplog
option dontlognull
option http-server-close
option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10m
timeout client 10m
timeout server 10m
timeout http-keep-alive 10s
timeout check 10slisten status
bind *:9999
mode http
stats uri /status
stats auth admin:class@qm.com
stats hide-versionfrontend db_read
bind *:3306
default_backend mysql_readbackend mysql_read
balance leastconn
server mysql02 192.168.216.5:3306 check
#server mysql01 192.168.216.3:3306 check backupfrontend db_write
bind *:3307
default_backend mysql_writebackend mysql_write
balance leastconn
#server mysql01 192.168.216.3:3306 check
server mysql02 192.168.216.5:3306 check backup
# /usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -c
# systemctl reload haproxy.service
# systemctl status haproxy.service
# ps -eflgrep haproxy
# ss -Inputlgrep haproxy
9、MySQL02主创建同步用户并查看状态
# mkdir -pv /data/backup/mysql
# mysqldump -hlocalhost -P3306 -uroot -p --master-data=2 --single-transaction -B qm >/data/backup/mysql/mysql02-qm.sql
# cat /data/backup/mysql/mysql02-qm.sql
# sz -E /data/backup/mysql/mysql02-qm.sql //下载备份文件mysql02-qm.sql
# mysql -hlocalhost -P3306 -uroot -p
mysql> SHOW GRANTS FOR 'slave'@'192.168.216.%'; //ERROR 1411 (42000)
mysql> CREATE USER `slave`@'192.168.216.%' IDENTIFIED by 'class@qm.com';
mysql> SELECT user,host FROM mysql.user;
mysql> GRANT REPLICATION SLAVE ON *.* to 'slave'@'192.168.216.%';
mysql> SHOW GRANTS FOR 'slave'@'192.168.216.%';
mysql> SHOW MASTER STATUS;
File Position Binlog _Do_DB Binlog _Ignore _DB Executed _Gtid_Set mysql-bin.000004 865 1 row in set(0.00 sec)
mysql> exit
10、MySQL01从配置异步复制
# mysql -h192.168.216.5 -P3306 -uslave -p //验证与MySQL02连接状态# cd /root/tools/
# rz -E //上传mysql02-qm.sql
# mysql -hlocalhost -P3306 -uroot -p < mysql02-qm.sql
# grep MASTER mysql02-qm.sql
# mysql -hlocalhost -P3306 -uroot -p
mysql> SHOW DATABASES;
mysql> CHANGE MASTER TO master_host='192.168.216.5',master_user='slave',master_password='class@qm.com',master_log_file='mysql-bin.000004',master_log_pos=865;
//修改为自己的主host,user,password,log_file,log_pos
mysql> START SLAVE;
mysql> SHOW SLAVE STATUS;
mysql> SHOW SLAVE STATUS\G
//Slave_IO_Running:No,Slave SOL Running:Yes
mysql> STOP SLAVE;
mysql> RESET SLAVE;
mysql> CHANGE MASTER TO master_log_file='mysql-bin.000007',master_log_pos=154;
mysql> START SLAVE;
mysql> SHOW SLAVE STATUS\G
//Slave IO Running:Yes,Slave SQL Running:Yes
mysql> EXIT
11、HAProxy01和HAProxy02修改配置还原集群
# vim /etc/haproxy/haproxy.cfg
global
log 127.0.0.1 local2
pidfile /var/run/haproxy.pid
user haproxy
group haproxy
daemon
#nbproc 1defaults
mode tcp
log global
#option httplog
option dontlognull
option http-server-close
#option forwardfor except 127.0.0.0/8
option redispatch
retries 3
timeout http-request 10s
timeout queue 1m
timeout connect 10m
timeout client 10m
timeout server 10m
timeout http-keep-alive 10s
timeout check 10slisten status
bind *:9999
mode http
stats uri /status
stats auth admin:class@qm.com
stats hide-versionfrontend db_read
bind *:3306
default_backend mysql_readbackend mysql_read
balance leastconn
server mysql02 192.168.216.5:3306 check
server mysql01 192.168.216.3:3306 check backupfrontend db_write
bind *:3307
default_backend mysql_writebackend mysql_write
balance leastconn
server mysql01 192.168.216.3:3306 check
server mysql02 192.168.216.5:3306 check backup
# /usr/local/haproxy/sbin/haproxy-f /usr/local/haproxy/etc/haproxy.cfg -c
# systemctl reload haproxy.service
# systemctl status haproxy.service
# ps -eflgrep haproxy
# ss -Input|grep haproxy
829

被折叠的 条评论
为什么被折叠?



