LVS调试算法总结
ipvs scheduler:根据其调度时是否考虑各RS当前的负载状态
分为两种:静态方法和动态方法
静态方法:
仅根据算法本身进行调度
1.RR:roundrobin,轮询,较常用
2、WRR:Weighted RR,加权轮询,较常用(如:根据各个后端RS配置不同进行不同权重的分配后调度)
3.SH:Source Hashing,实现session sticky,源IP地址hash;将来自于同一个IP地址的请求始终发往第一次挑中的RS,从而实现会话绑定(取IP地址的前三段Hash后取模可以如果hash值一致则表明为同一IP)
4.DH:Destination Hashing:目的地址Hash,第一次轮询调度至RS,后续将发往同一个目的地址的请求始终发送至第一次挑中的RS,典型使用场景时正向代理缓存场景中的负载均衡,如:web缓存
动态方法:
主要根据每个RS当前的负载状态以及调度算法进行调度Overhead=value较小的RS将被调度
1.LC:least connections 适用于长连接应用
overhead=activeconns*256+inactivecoons
2.WLC:weighted LC,默认调度方法,较常用
Overhead=(activeconns*256+inactiveconns)/weight
3.SED:Shortest Expection Delay,初始连接高权重优先,只检查活动连接,而不考虑非活动连接
Overhead=(activeconns+1)*256/weight
4.NQ:Never Queue,第一轮均匀分配,后续SED
5.LBLC:Locality-Based LC,动态的DH算法,使用场景:根据负载状态实现正向代理,实现web cache等
6.LBLCR:LBLC with Replication,带复制功能的LBLC,解决LBLC负载不均衡问题,从负载重的复制到负载轻的RS,实现web cache等
内核版本4.15版本后新增调度算法:FO和OVF
FO(weighted Fail over)调度算法,在此FO算法中,遍历虚拟服务所关联的真实服务链表,找到还未过载(未设置IP_VS_DEST_F_OVERLOAD标志)的且权重最高的真实服务器,进行调度
OVF(Overflow)调度算法,基于真实服务器的活动连接数量和权重值实现。将新连接调度到权重值最高的真实服务器,直到其活动连接数量超过权重值,之后调度到下一个权重值最高的真实服务器,在此OVF算法中,遍历虚拟服务相关联的真实服务器链表,找到权重值最高的可用真实服务器。一个可用的真实服务器需要同时满足以下条件:
-
未过载(未设置IP_VS_DEST_F_OVERLOAD标志)
-
真实服务器当前的活动连接数量小于其权重值
-
其权重值不为零
LVS的NAT、DR模型实现
lvs的nat模型实现
环境准备
#共四台机器
一台:internet client:192.168.10.6/24 GW:无 #仅主机模式
一台:lvs
ens33 仅主机 192.168.10.100/24
ens37 NAT 10.0.0.8/24
两台RS:
RS1:10.0.0.7/24 GW:10.0.0.8 NAT
RS1:10.0.0.17/24 GW:10.0.0.8 NAT
配置过程:
[root@internet ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
NAME=ens33
UUID=d185405d-15dc-46c6-b68a-f159101fd39d
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.10.6
NETMASK=255.255.255.0
[root@lvs ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
NAME=ens33
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.10.100
NETMASK=255.255.255.0
[root@lvs ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens37
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
NAME=ens37
DEVICE=ens37
ONBOOT=yes
IPADDR=10.0.0.8
NETMASK=255.255.255.0
[root@RS1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=dhcp
DEFROUTE=yes
NAME=ens33
DEVICE=ens33
ONBOOT=yes
IPADDR=10.0.0.7
NETMASK=255.255.255.0
GATEWAY=10.0.0.8
[root@RS2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
NAME=ens33
DEVICE=ens33
ONBOOT=yes
IPADDR=10.0.0.17
NETMASK=255.255.255.0
GATEWAY=10.0.0.8
[root@lvs ~]# cat /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv4.ip_forward = 1
[root@lvs ~]#ipvsadm -A -t 192.168.10.100:80 -s wrr
[root@lvs ~]#ipvsadm -a -t 192.168.10.100:80 -r 10.0.0.7:80 -m
[root@lvs ~]#ipvsadm -a -t 192.168.10.100:80 -r 10.0.0.17:80 -m
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.10.100:80 wrr
-> 10.0.0.7:80 Masq 1 0 0
-> 10.0.0.17:80 Masq 1 0 0
[root@lvs ~]#
#结果验证
root@internet ~]# curl 192.168.10.100
10.0.0.17 RS2
[root@internet ~]# curl 192.168.10.100
RS1 10.0.0.7
[root@internet ~]# curl 192.168.10.100
10.0.0.17 RS2
[root@internet ~]# curl 192.168.10.100
RS1 10.0.0.7
[root@internet ~]# curl 192.168.10.100
10.0.0.17 RS2
[root@internet ~]# curl 192.168.10.100
RS1 10.0.0.7
[root@internet ~]# curl 192.168.10.100
10.0.0.17 RS2
[root@internet ~]# curl 192.168.10.100
RS1 10.0.0.7
[root@internet ~]# curl 192.168.10.100
10.0.0.17 RS2
lvs的DR模型实现
#共五台机器
一台:internet client:192.168.10.6/24 GW:192.168.10.200 #仅主机模式,网关指向路由器
一台:Router
ens33 nat 10.0.0.200/24
ens37 仅主机 192.168.10.200/24
一台:lvs
ens33 NAT DIP:10.0.0.8/24 GW:10.0.0.200
两台RS:
RS1:10.0.0.7/24 GW:10.0.0.200 NAT
RS1:10.0.0.17/24 GW:10.0.0.200 NAT
网络环境配置
#internet
[root@internet ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
NAME=ens33
UUID=d185405d-15dc-46c6-b68a-f159101fd39d
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.10.6
NETMASK=255.255.255.0
GATEWAY=192.168.10.200
#router
[root@router ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
NAME=ens33
DEVICE=ens33
ONBOOT=yes
IPADDR=10.0.0.200
NETMASK=255.255.255.0
[root@router ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens37
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
NAME=ens37
DEVICE=ens37
ONBOOT=yes
IPADDR=192.168.10.200
NETMASK=255.255.255.0
#RS1
[root@RS1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens37
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
NAME=ens37
DEVICE=ens37
ONBOOT=yes
IPADDR=10.0.0.7
NETMASK=255.255.255.0
GATEWAY=10.0.0.200
#RS2
[root@RS2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
DEFROUTE=yes
NAME=ens33
DEVICE=ens33
ONBOOT=yes
IPADDR=10.0.0.17
NETMASK=255.255.255.0
GATEWAY=10.0.0.200
#LVS
[root@lvs ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=static
NAME=ens33
DEVICE=ens33
ONBOOT=yes
IPADDR=10.0.0.8
NETMASK=255.255.255.0
GATEWAY=10.0.0.200
后端RS的IPVS配置
#RS1
root@RS1 ~]# echo 1 >/proc/sys/net/ipv4/conf/all/arp_ignore
[root@RS1 ~]# echo 2 >/proc/sys/net/ipv4/conf/all/arp_announce
[root@RS1 ~]# echo 1 >/proc/sys/net/ipv4/conf/lo/arp_ignore
[root@RS1 ~]# echo 2 >/proc/sys/net/ipv4/conf/lo/arp_announce
[root@RS1 ~]# ip a a 10.0.0.100/32 dev lo label lo:1
[root@RS1 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 10.0.0.100/32 scope global lo:1
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:43:fd:9d brd ff:ff:ff:ff:ff:ff
inet 10.0.0.7/24 brd 10.0.0.255 scope global noprefixroute ens37
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe43:fd9d/64 scope link
valid_lft forever preferred_lft forever
#RS2
[root@RS2 ~]# echo 1 >/proc/sys/net/ipv4/conf/all/arp_ignore
[root@RS2 ~]# echo 2 >/proc/sys/net/ipv4/conf/all/arp_announce
[root@RS2 ~]# echo 1 >/proc/sys/net/ipv4/conf/lo/arp_ignore
[root@RS2 ~]# echo 2 >/proc/sys/net/ipv4/conf/lo/arp_announce
[root@RS2 ~]# ip a a 10.0.0.100/32 dev lo label lo:1
[root@RS2 ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 10.0.0.100/32 scope global lo:1
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:ff:34:62 brd ff:ff:ff:ff:ff:ff
inet 10.0.0.17/24 brd 10.0.0.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:feff:3462/64 scope link
valid_lft forever preferred_lft forever
lvs主机配置
[root@lvs ~]# ip a a 10.0.0.100/32 dev lo label lo:1
[root@lvs ~]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 10.0.0.100/32 scope global lo:1
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether 00:0c:29:13:79:6c brd ff:ff:ff:ff:ff:ff
inet 10.0.0.8/24 brd 10.0.0.255 scope global noprefixroute ens33
valid_lft forever preferred_lft forever
inet6 fe80::20c:29ff:fe13:796c/64 scope link
valid_lft forever preferred_lft forever
#实现LVS规则
[root@lvs ~]# ipvsadm -A -t 10.0.0.100:80 -s rr
[root@lvs ~]# ipvsadm -a -t 10.0.0.100:80 -r 10.0.0.7:80 -g
[root@lvs ~]# ipvsadm -a -t 10.0.0.100:80 -r 10.0.0.17:80 -g
[root@lvs ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.0.0.100:80 rr
-> 10.0.0.7:80 Route 1 0 0
-> 10.0.0.17:80 Route 1 0 0
测试访问
[root@internet ~]# curl 10.0.0.100
10.0.0.17 RS2
[root@internet ~]# curl 10.0.0.100
RS1 10.0.0.7
[root@internet ~]# curl 10.0.0.100
10.0.0.17 RS2
[root@internet ~]# curl 10.0.0.100
RS1 10.0.0.7
[root@internet ~]# curl 10.0.0.100
10.0.0.17 RS2
[root@internet ~]# curl 10.0.0.100
RS1 10.0.0.7
[root@internet ~]# curl 10.0.0.100
10.0.0.17 RS2
[root@internet ~]# curl 10.0.0.100
RS1 10.0.0.7
[root@internet ~]# curl 10.0.0.100
10.0.0.17 RS2
[root@internet ~]# curl 10.0.0.100
RS1 10.0.0.7
[root@internet ~]# curl 10.0.0.100
10.0.0.17 RS2