（高可用）一、LVM

akswyh

已于 2023-01-31 12:40:09 修改

阅读量527

点赞数

分类专栏：运维文章标签：运维

于 2023-01-29 20:56:44 首次发布

本文链接：https://blog.youkuaiyun.com/akswyh/article/details/128793951

版权

运维专栏收录该内容

10 篇文章

订阅专栏

文章目录

一、LVM

一、LVM

使用了内核模块：ip_vs
实现了负载均衡

LVM实验一

LVM三种策略

NAT: client->VS->RS->VS->client（数据原路返回）
TUNNEL: client->VS->RS->client（不需保证VS和RS在同一网段）
NAT: client->VS->RS->client（要保证VS和RS在同一网段，VS负责将目标MAC替换为相应的RS的MAC，不做ARP解析）

配置rhel8.6源主机

关闭图形化界面
配置虚拟网卡的NAT，DHCP
配置本地yum源，使用rhel8.6的iso
开机自动挂载镜像

vim /etc/fstab
/root/rhel.iso /iso

记得关闭防火墙和selinux
克隆3台主机 192.168.147.129/130/131

实验环境搭建

配置3台主机
修改主机名hostnamectl
修改hosts，互相解析
安装相应的软件

# 129作为调度器，安装ipvsadm
yum install -y ipvsadm

# 130、131安装httpd设置开机自启
yum install -y httpd
systemctl enable --now httpd

配置负载均衡

设置VIP地址：192.168.147.100
129上:100/24
130/131上:100/32，表示并非真实存在，防止数据包来了被直接丢弃
129上配置ipvsadm策略

ipvsadm -A -t 192.168.147.100:80 -s rr
ipvsadm -a -t 192.168.147.100:80 -r 192.168.147.130 -g
ipvsadm -a -t 192.168.147.100:80 -r 192.168.147.131 -g

此时因为130/131上都有IP地址192.168.147.100，在同一网段冲突，此时调度器并没有起作用，需要在130/131上设置不对192.168.147.100地址的arp广播进行回应

# 删除server1上的arp缓存
arp -an
arp -d 192.168.147.100

# 在130/131上配置arptables
arptables -A INPUT -d 192.168.147.100 -j DROP
arptables -A OUTPUT -s 192.168.147.100 -j mangle --mangle-ip-s 192.168.147.130

# 保存arptables策略，并设置开机自启
arptables-save > /etc/sysconfig/arptables
systemctl restart arptables.service
arptables -F
arptables -nL
systemctl enable arptables.service

LVM实验二

当RS挂掉的时候，LVM调度器需要检测到并处理故障
当LVM本身挂掉（双机热备）

Keepalived-LVS管理软件

健康检测：支持4/7检测
主备冗余：采用VRRP协议的HeartBeat
配置文件：keepalived -f /etc/keepalived/keepalived.conf
使用了keepalived后，不需要再用ipvsadm配置策略

实验环境搭建

配置主从两个调度器，两台服务器

# 129作为主调度器、132作为备份调度器，安装ipvsadm、keepalived
yum install -y ipvsadm
yum install -y keepalived

# 130、131安装httpd设置开机自启
yum install -y httpd
systemctl enable --now httpd

修改keepalived配置文件

/etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
   notification_email {
     root@localhost  # 出现故障时往哪个邮箱发通知
   }
   notification_email_from Alexandre.Cassen@firewall.loc
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
   vrrp_skip_check_adv_addr
   # vrrp_strict  # 这个要注释掉
   vrrp_garp_interval 0
   vrrp_gna_interval 0
}

vrrp_instance VI_1 {
    state MASTER  # 当前服务器的身份
    interface ens160  # 指明网卡
    virtual_router_id 51 # 这个一般不用修改，除非同一网段有多个采用双机热备的集群
    priority 100  # 当前服务器的优先级
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.147.100  # vs的地址
    }
}

virtual_server 192.168.147.100 80 { # 配置vs
    delay_loop 6
    lb_algo rr  # 调度算法
    lb_kind DR  # 连接方式DR、NAT、TUNNEL
    # persistence_timeout 50  # 用于需要有保持连接需求的情形
    protocol TCP

    real_server 192.168.147.130 80 {  # 配置rs
        weight 1
        TCP_CHECK {
            connect_timeout 3
            retry 3
            delay_before_retry 3
        }
    }
    real_server 192.168.147.131 80 {
        weight 1
        TCP_CHECK{
            connect_timeout 3
            retry 3
            delay_before_retry 3
        }
    }
}

设置开机keepalived自启

systemctl enabled --now keepalived

出现报错（网卡名修改错了）

-- Unit keepalived.service has begun starting up.
Jan 09 03:23:54 server1 Keepalived[2445]: Starting Keepalived v2.1.5 (07/13,2020)
Jan 09 03:23:54 server1 Keepalived[2445]: Running on Linux 4.18.0-372.9.1.el8.x86_64 #1 SMP Fri Apr 15 22:12:19 EDT 2022 (built for Linux 4.18.0)
Jan 09 03:23:54 server1 Keepalived[2445]: Command line: '/usr/sbin/keepalived' '-D'
Jan 09 03:23:54 server1 Keepalived[2445]: Opening file '/etc/keepalived/keepalived.conf'.
Jan 09 03:23:54 server1 Keepalived[2445]: (/etc/keepalived/keepalived.conf: Line 13) number '0' outside range [1e-06, 4294]
Jan 09 03:23:54 server1 Keepalived[2445]: (/etc/keepalived/keepalived.conf: Line 13) vrrp_garp_interval '0' is invalid
Jan 09 03:23:54 server1 Keepalived[2445]: (/etc/keepalived/keepalived.conf: Line 14) number '0' outside range [1e-06, 4294]
Jan 09 03:23:54 server1 Keepalived[2445]: (/etc/keepalived/keepalived.conf: Line 14) vrrp_gna_interval '0' is invalid
Jan 09 03:23:54 server1 systemd[1]: keepalived.service: Can't open PID file /var/run/keepalived.pid (yet?) after start: No such file or directory
Jan 09 03:23:54 server1 Keepalived[2446]: NOTICE: setting config option max_auto_priority should result in better keepalived performance
Jan 09 03:23:54 server1 Keepalived[2446]: Starting Healthcheck child process, pid=2447
Jan 09 03:23:54 server1 Keepalived[2446]: Starting VRRP child process, pid=2448
Jan 09 03:23:54 server1 Keepalived_healthcheckers[2447]: Opening file '/etc/keepalived/keepalived.conf'.
Jan 09 03:23:54 server1 Keepalived_healthcheckers[2447]: Initializing ipvs
Jan 09 03:23:54 server1 Keepalived_healthcheckers[2447]: Gained quorum 1+0=1 <= 2 for VS [192.168.147.100]:tcp:80
Jan 09 03:23:54 server1 Keepalived_healthcheckers[2447]: Activating healthchecker for service [192.168.147.130]:tcp:80 for VS [192.168.147.100]:tcp:80
Jan 09 03:23:54 server1 Keepalived_healthcheckers[2447]: Activating healthchecker for service [192.168.147.131]:tcp:80 for VS [192.168.147.100]:tcp:80
Jan 09 03:23:54 server1 Keepalived_vrrp[2448]: Registering Kernel netlink reflector
Jan 09 03:23:54 server1 Keepalived_vrrp[2448]: Registering Kernel netlink command channel
Jan 09 03:23:54 server1 Keepalived_vrrp[2448]: Opening file '/etc/keepalived/keepalived.conf'.
Jan 09 03:23:54 server1 Keepalived_vrrp[2448]: (/etc/keepalived/keepalived.conf: Line 19) WARNING - interface eth160 for vrrp_instance VI_1 doesn't exist
Jan 09 03:23:54 server1 Keepalived_vrrp[2448]: Non-existent interface specified in configuration
Jan 09 03:23:54 server1 Keepalived_vrrp[2448]: Stopped - used 0.000000 user time, 0.001017 system time
Jan 09 03:23:54 server1 Keepalived[2446]: pid 2448 exited with permanent error CONFIG. Terminating
Jan 09 03:23:54 server1 Keepalived[2446]: CPU usage (self/children) user: 0.000000/0.000000 system: 0.001513/0.001988
Jan 09 03:23:54 server1 Keepalived[2446]: Stopped Keepalived v2.1.5 (07/13,2020)
Jan 09 03:23:54 server1 Keepalived_healthcheckers[2447]: Shutting down service [192.168.147.130]:tcp:80 from VS [192.168.147.100]:tcp:80
Jan 09 03:23:54 server1 Keepalived_healthcheckers[2447]: Shutting down service [192.168.147.131]:tcp:80 from VS [192.168.147.100]:tcp:80
Jan 09 03:23:54 server1 Keepalived_healthcheckers[2447]: Stopped - used 0.000000 user time, 0.001128 system time
Jan 09 03:25:24 server1 systemd[1]: keepalived.service: start operation timed out. Terminating.
Jan 09 03:25:24 server1 systemd[1]: keepalived.service: Failed with result 'timeout'.
-- Subject: Unit failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- The unit keepalived.service has entered the 'failed' state with result 'timeout'.
Jan 09 03:25:24 server1 systemd[1]: Failed to start LVS and VRRP High Availability Monitor.
-- Subject: Unit keepalived.service has failed
-- Defined-By: systemd
-- Support: https://access.redhat.com/support
--
-- Unit keepalived.service has failed.

安装mailx用于接收报警邮件

yum install -y mailx

正常启动keepalived之后，又出现问题

访问192.168.147.100时得不到响应，但是访问192.168.147.130和192.168.147.131都可以得到响应
发现是因为重启了虚拟机，原来130和131网卡ens160上配置的地址192.168.147.100都没了

测试

[root@rhel86 ~]# for i in {1..5};do curl 192.168.147.100;done
server2
server3
server2
server3
server2

再分别挂掉RS和LVM进行测试

iptables的优先级比LVM要高

# 在LVM主机上的iptables（防火墙）添加了策略后，比如drop掉所有进来的数据包
iptables -A INPUT -j DROP

# 此时访问192.168.147.100就没有响应了

LVS缺点

LVS在大规模网络中应用存在不足
缺少TCP标志位和对DDOS攻击防御
性能无法线性扩展

DR工作模式的缺点

RS和VS必须工作在同一网段
RS上也有VIP，客户端可以跳过VS直接访问

NAT工作模式的缺点

RS和路由器的网络配置复杂

TUNNEL工作模式的缺点

RS的配置复杂（要支持IPIP模块）
RS上也有VIP，客户端可以跳过VS直接访问

新转发模式FULLNAT

阿里开发
主要思想：引入local address（lip），将原来的cip->rip转换为cip->lip->rip
实现LVS-RS的跨VLAN通讯，并且INPUT
和OUTPUT数据流都通过LVS
SYSPROXY：synflood攻击防御模块
TCP标志位和DDOS攻击防御策略
Cluster部署模式
ipvsadm默认不支持，需要重新编译内核

高可用存在问题

LVM之间的session同步问题
- 共享一个全局session
- 路由器上的等价路由（VIP在路由器上）