本文详细介绍了LVS（Linux Virtual Server）的工作原理，包括Load Balancer、Server Array和Shared Storage的角色。重点讲解了LVS的三种负载均衡机制：VS/NAT、VS/TUN和VS/DR，以及各种调度算法。此外，还讨论了LVS的nat和dr两种工作模式，以及相应的配置步骤，适合需要搭建和理解LVS集群的读者。
lvs配置
lvs工作原理
1.Load Balancer：这是LVS的核心部分，它好比我们网站MVC模型的Controller。它负责将客户的请求按照一定的算法分发到下一层不同的服务器进行处理，自己本身不做具体业务的处理。另外该层还可用监控下一层的状态，如果下一层的某台服务器不能正常工作了，它会自动把其剔除，恢复后又可用加上。该层由一台或者几台Director Server组成。
2.Server Array：该层负责具体业务。可有WEB Server、mail Server、FTP Server、DNS Server等组成。注意，其实上层的Director Server也可以当Real server用的。
3.Shared Storage：主要是提高上一层数据和为上一层保持数据一致。
负载均衡机制
- VS/NAT(Virtual Server via Network Address Translation)，即网络地址翻转技术实现虚拟服务器。当请求来到时，Diretor
server上处理的程序将数据报文中的目标地址（即虚拟IP地址）改成具体的某台Real Server,端口也改成Real
Server的端口，然后把报文发给Real Server。Real Server处理完数据后，需要返回给Diretor
Server，然后Diretor
server将数据包中的源地址和源端口改成VIP的地址和端口，最后把数据发送出去。由此可以看出，用户的请求和返回都要经过Diretor
Server，如果数据过多，Diretor Server肯定会不堪重负 - VS/TUN（Virtual Server via IP
Tunneling）,即IP隧道技术实现虚拟服务器。它跟VS/NAT基本一样，但是Real
server是直接返回数据给客户端，不需要经过Diretor server,这大大降低了Diretor server的压力 - VS/DR（Virtual Server via Direct Routing），即用直接路由技术实现虚拟服务器。跟前面两种方式，它的报文转发方法有所不同，VS/DR通过改写请求报文的MAC地址，将请求发送到Real
Server，而Real
Server将响应直接返回给客户，免去了VS/TUN中的IP隧道开销。这种方式是三种负载调度机制中性能最高最好的，但是必须要求Director
Server与Real Server都有一块网卡连在同一物理网段上。
调度算法
VS的调度算法决定了如何在集群节点之间分布工作负荷。当director调度器收到来自客户端访问VIP的上的集群服务的入站请求时，director调度器必须决定哪个集群节点应该
处理请求。
Director调度器用的调度方法基本分为两类 (如下所列, LVS总共有10种调度算法, 常用的也就四种调度算法, 下面会说到):
静态调度算法：rr，wrr，dh，sh
动态调度算法：wlc，lc，lblc，lblcr, sed, nq
静态算法
- rr（轮询） 轮询调度：这种是最简单的调度算法，就是将请求A一个，B一个，A一个，B一个 …
循环的发。就算A主机挂掉了，调度器还是会将请求发送到A。十分均衡。 - wrr（权重, 即加权轮询）
加权轮询调度：这种算法是在rr基础上实现的，只不过加了权重，权重范围为1-100，假设A的服务器性能好，就给A的权重设置的高一点，设为2，而B主机是1。这样就实现A二个，B一个，A二个，B一个
… 循环的发。这样照顾到了服务器性能。 - sh（源地址哈希）
源地址散列：主要是实现将此前的session（会话）绑定。将此前客户的源地址作为散列键，从静态的散列表中找出对应的服务器，只要目标服务器是没有超负荷的就将请求发送过去。就是说某客户访问过A,现在这个客户又来了，所以客户请求会被发送到服务过他的A主机。 - dh（目的地址哈希） 目的地址散列：以目的地址为关键字查找一个静态hash表来获得需要的RS。以目标地址为标准挑选。
功能是和sh近似的，但应用场景不同;
举个dh调度算法的例子：假设1号客户访问了web集群的一个动态页面，调度器将请求转发个A服务器，A服务器的PHP将这个动态请求运行了一遍，生成了缓存并回应1号客户。这下2号客户也访问了这个动态页面，调度器应该将请求发给A。毕竟A已经跑过这段程序了，有缓存，对吧。所以这既是dh算法）
动态算法
动态算法（6种）：前端的调度器会根据后端真实服务器的实际连接情况来分配请求
- 轮询调度（Round Robin）(简称rr)
调度器通过"轮叫"调度算法将外部请求按顺序轮流分配到集群中的真实服务器上，它均等地对待每一台服务器，而不管服务器上实际的连接数和系统负载。 - 加权轮询（Weighted Round Robin）（简称wrr)
调度器通过"加权轮叫"调度算法根据真实服务器的不同处理能力来调度访问请求。这样可以保证处理能力强的服务器能处理更多的访问流量。调度器可以自动问询真实服务器的负载情况，并动态地调整其权值。 - 最少链接（Least Connections）(LC)
调度器通过"最少连接"调度算法动态地将网络请求调度到已建立的链接数最少的服务器上。如果集群系统的真实服务器具有相近的系统性能，采用 “最小连接” 调度算法可以较好地均衡负载。 - 加权最少链接（Weighted Least Connections）(WLC)
在集群系统中的服务器性能差异较大的情况下，调度器采用“加权最少链接”调度算法优化负载均衡性能，具有较高权值的服务器将承受较大比例的活动连接负载。调度器可以自动问询真实服务器的负载情况，并动态地调整其权值。 - 基于局部性的最少链接（Locality-Based Least Connections）(LBLC)
“基于局部性的最少链接”调度算法是针对目标IP地址的负载均衡，目前主要用于Cache集群系统，因为在Cache集群中客户请求报文的目标IP地址是变化的。该算法根据请求的目标IP地址找出该目标IP地址最近使用的服务器，若该服务器是可用的且没有超载，将请求发送到该服务器；若服务器不存在，或者该服务器超载且有服务器处于一半的工作负载，则用“最少链接” 的原则选出一个可用的服务器，将请求发送到该服务器。 - 带复制的基于局部性最少链接（Locality-Based Least Connections with Replication）(LBLCR)
“带复制的基于局部性最少链接”调度算法也是针对目标IP地址的负载均衡，目前主要用于Cache集群系统。它与LBLC算法的不同之处是它要维护从一个目标
IP地址到一组服务器的映射，而LBLC算法维护从一个目标IP地址到一台服务器的映射。该算法根据请求的目标IP地址找出该目标IP地址对应的服务器组，按
“最小连接”原则从服务器组中选出一台服务器，若服务器没有超载，将请求发送到该服务器；若服务器超载，则按“最小连接”原则从这个集群中选出一台
服务器，将该服务器加入到服务器组中，将请求发送到该服务器。同时，当该服务器组有一段时间没有被修改，将最忙的服务器从服务器组中删除，以降低复
制的程度。
lvs的两种工作模式
nat模式
nat模型实现http和https两种负载均衡集群
注意：各RS都要提供同一个私钥和同一个证书
| ip类型 | ip地址 |
|---|---|
| VIP | 192.168.100.10 |
| DIP | 192.168.147.10 |
| R1ip | 192.168.147.22 |
| R2ip | 192.168.147.33 |
配置
https安装请看往期博客
//关闭防火墙
[root@lb ~]# systemctl disable --now firewalld
[root@lb ~]# setenforce 0
[root@R1 CA]# systemctl disable --now firewalld
[root@R1 CA]# setenforce 0
[root@R2 ~]# systemctl disable --now firewalld
[root@R2 ~]# setenforce 0
[root@localhost ~]# mkdir /etc/pki/CA
[root@localhost ~]# cd /etc/pki/CA/
[root@localhost CA]# mkdir private
[root@localhost CA]# (umask 077;openssl genrsa -out private/cakey.pem 2048)
Generating RSA private key, 2048 bit long modulus (2 primes)
..............+++++
.................+++++
e is 65537 (0x010001)
[root@localhost CA]# openssl req -new -x509 -key private/cakey.pem -out cacert.pem -days 365
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:HB
Locality Name (eg, city) [Default City]:WH
Organization Name (eg, company) [Default Company Ltd]:yh
Organizational Unit Name (eg, section) []:yh
Common Name (eg, your name or your server's hostname) []:xiaoyu.com
Email Address []:12@22.com
[root@localhost CA]# mkdir certs newcerts crl
[root@localhost CA]# touch index.txt && echo 01 > serial
[root@localhost CA]# (umask 077;openssl genrsa -out httpd.key 2048)
Generating RSA private key, 2048 bit long modulus (2 primes)
................+++++
...................+++++
e is 65537 (0x010001)
[root@localhost CA]# openssl req -new -key httpd.key -days 365 -out httpd.csr
Ignoring -days; not generating a certificate
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:HB
Locality Name (eg, city) [Default City]:WH
Organization Name (eg, company) [Default Company Ltd]:yh
Organizational Unit Name (eg, section) []:yh
Common Name (eg, your name or your server's hostname) []:xiaoyu.com
Email Address []:12@22.com
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
[root@localhost CA]# openssl ca -in httpd.csr -out httpd.crt -days 365
Using configuration from /etc/pki/tls/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
Serial Number: 1 (0x1)
Validity
Not Before: Jun 14 20:45:20 2021 GMT
Not After : Jun 14 20:45:20 2022 GMT
Subject:
countryName = CN
stateOrProvinceName = HB
organizationName = yh
organizationalUnitName = yh
commonName = xiaoyu.com
emailAddress = 12@22.com
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
2A:35:34:2C:01:19:0A:ED:D2:23:45:0C:70:02:C1:20:A8:F6:E4:56
X509v3 Authority Key Identifier:
keyid:1B:81:BC:B9:AA:11:D6:68:22:5E:D5:C6:33:E4:AB:25:A9:37:4C:31
Certificate is to be certified until Jun 14 20:45:20 2022 GMT (365 days)
Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
[root@localhost CA]# mv httpd.key httpd.crt /etc/httpd/
[root@localhost CA]# scp /etc/httpd/httpd.key 192.168.147.33:/etc/httpd/
The authenticity of host '192.168.147.33 (192.168.147.33)' can't be established.
ECDSA key fingerprint is SHA256:z3c/27WLwROrI78hoFgydslIBbisv+qFqoe9llRiKLA.
Are you sure you want to continue connecting (yes/no/[fingerprint])? y
Please type 'yes', 'no' or the fingerprint: yes
Warning: Permanently added '192.168.147.33' (ECDSA) to the list of known hosts.
root@192.168.147.33's password:
httpd.key 100% 1675 1.3MB/s 00:00
[root@localhost CA]# scp /etc/httpd/httpd.crt 192.168.147.33:/etc/httpd/
root@192.168.147.33's password:
httpd.crt 100% 4495 5.0MB/s 00:00
[root@R1 CA]# echo "R1" >/usr/local/apache/htdocs/index.html
[root@R2 ~]# echo "R2" >/usr/local/apache/htdocs/index.html
在调度器上开启ip转发功能
[root@lb ~]# echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
[root@lb ~]# sysctl -p
net.ipv4.ip_forward = 1
在调度器上添加并保持规则
[root@lb ~]# ipvsadm -A -t 192.168.100.10:80 -s rr
[root@lb ~]# ipvsadm -a -t 192.168.100.10:80 -r 192.168.147.22:80 -m
[root@lb ~]# ipvsadm -a -t 192.168.100.10:80 -r 192.168.147.33:80 -m
[root@lb ~]# ipvsadm -A -t 192.168.100.10:443 -s rr
[root@lb ~]# ipvsadm -a -t 192.168.100.10:443 -r 192.168.147.22:443 -m
[root@lb ~]# ipvsadm -a -t 192.168.100.10:443 -r 192.168.147.33:443 -m
[root@lb ~]# ipvsadm -Sn > /etc/sysconfig/ipvsadm
[root@lb ~]# cat /etc/sysconfig/ipvsadm
-A -t 192.168.100.10:80 -s rr
-a -t 192.168.100.10:80 -r 192.168.147.22:80 -m -w 1
-a -t 192.168.100.10:80 -r 192.168.147.33:80 -m -w 1
-A -t 192.168.100.10:443 -s rr
-a -t 192.168.100.10:443 -r 192.168.147.22:443 -m -w 1
-a -t 192.168.100.10:443 -r 192.168.147.33:443 -m -w 1
//设置开机自启
[root@lb ~]# systemctl enable ipvsadm
[root@lb ~]# echo "ipvsadm -R < /etc/sysconfig/ipvsadm" >>/etc/rc.d/rc.local
将RIP的网关都指向DIP
[root@R1 CA]# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet
BOOTPROTO=static
NAME=ens33
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.147.22
PREFIX=24
GATEWAY=192.168.147.10
DNS1=114.114.114.114
[root@R2 htdocs]# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet
BOOTPROTO=static
NAME=ens33
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.147.33
PREFIX=24
GATEWAY=192.168.147.10
DNS1=114.114.114.114
测试
[root@lb ~]# curl 192.168.100.10
R1
[root@lb ~]# curl 192.168.100.10
R2
[root@lb ~]# curl 192.168.100.10
R1
[root@lb ~]# curl 192.168.100.10
R2
dr模式
要求：
dr模式实现http和https两种负载均衡居群，各RS都要提供通过一个私钥和同一个证书
| ip类型 | ip地址 |
|---|---|
| VIP | 192.168.147.77 |
| DIP | 192.168.147.10 |
| R1ip | 192.168.147.22 |
| R2ip | 192.168.147.33 |
配置
https安装请看往期博客
//关闭防火墙
[root@lb ~]# systemctl disable --now firewalld
[root@lb ~]# setenforce 0
[root@R1 CA]# systemctl disable --now firewalld
[root@R1 CA]# setenforce 0
[root@R2 ~]# systemctl disable --now firewalld
[root@R2 ~]# setenforce 0
[root@localhost ~]# mkdir /etc/pki/CA
[root@localhost ~]# cd /etc/pki/CA/
[root@localhost CA]# mkdir private
[root@localhost CA]# (umask 077;openssl genrsa -out private/cakey.pem 2048)
Generating RSA private key, 2048 bit long modulus (2 primes)
..............+++++
.................+++++
e is 65537 (0x010001)
[root@localhost CA]# openssl req -new -x509 -key private/cakey.pem -out cacert.pem -days 365
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:HB
Locality Name (eg, city) [Default City]:WH
Organization Name (eg, company) [Default Company Ltd]:yh
Organizational Unit Name (eg, section) []:yh
Common Name (eg, your name or your server's hostname) []:xiaoyu.com
Email Address []:12@22.com
[root@localhost CA]# mkdir certs newcerts crl
[root@localhost CA]# touch index.txt && echo 01 > serial
[root@localhost CA]# (umask 077;openssl genrsa -out httpd.key 2048)
Generating RSA private key, 2048 bit long modulus (2 primes)
................+++++
...................+++++
e is 65537 (0x010001)
[root@localhost CA]# openssl req -new -key httpd.key -days 365 -out httpd.csr
Ignoring -days; not generating a certificate
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:HB
Locality Name (eg, city) [Default City]:WH
Organization Name (eg, company) [Default Company Ltd]:yh
Organizational Unit Name (eg, section) []:yh
Common Name (eg, your name or your server's hostname) []:xiaoyu.com
Email Address []:12@22.com
Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
[root@localhost CA]# openssl ca -in httpd.csr -out httpd.crt -days 365
Using configuration from /etc/pki/tls/openssl.cnf
Check that the request matches the signature
Signature ok
Certificate Details:
Serial Number: 1 (0x1)
Validity
Not Before: Jun 14 20:45:20 2021 GMT
Not After : Jun 14 20:45:20 2022 GMT
Subject:
countryName = CN
stateOrProvinceName = HB
organizationName = yh
organizationalUnitName = yh
commonName = xiaoyu.com
emailAddress = 12@22.com
X509v3 extensions:
X509v3 Basic Constraints:
CA:FALSE
Netscape Comment:
OpenSSL Generated Certificate
X509v3 Subject Key Identifier:
2A:35:34:2C:01:19:0A:ED:D2:23:45:0C:70:02:C1:20:A8:F6:E4:56
X509v3 Authority Key Identifier:
keyid:1B:81:BC:B9:AA:11:D6:68:22:5E:D5:C6:33:E4:AB:25:A9:37:4C:31
Certificate is to be certified until Jun 14 20:45:20 2022 GMT (365 days)
Sign the certificate? [y/n]:y
1 out of 1 certificate requests certified, commit? [y/n]y
Write out database with 1 new entries
Data Base Updated
[root@localhost CA]# mv httpd.key httpd.crt /etc/httpd/
[root@localhost CA]# scp /etc/httpd/httpd.key 192.168.147.33:/etc/httpd/
The authenticity of host '192.168.147.33 (192.168.147.33)' can't be established.
ECDSA key fingerprint is SHA256:z3c/27WLwROrI78hoFgydslIBbisv+qFqoe9llRiKLA.
Are you sure you want to continue connecting (yes/no/[fingerprint])? y
Please type 'yes', 'no' or the fingerprint: yes
Warning: Permanently added '192.168.147.33' (ECDSA) to the list of known hosts.
root@192.168.147.33's password:
httpd.key 100% 1675 1.3MB/s 00:00
[root@localhost CA]# scp /etc/httpd/httpd.crt 192.168.147.33:/etc/httpd/
root@192.168.147.33's password:
httpd.crt 100% 4495 5.0MB/s 00:00
[root@R1 CA]# echo "R1" >/usr/local/apache/htdocs/index.html
[root@R2 ~]# echo "R2" >/usr/local/apache/htdocs/index.html
在调度器上配置VIP和DIP
[root@lb ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet
BOOTPROTO=static
NAME=ens33
DEVICE=ens33
ONBOOT=yes
IPADDR0=192.168.147.10
NETMASK0=255.255.255.0
IPADDR1=192.168.147.77
NETMASK1=255.255.255.0
GATEWAY=192.168.147.2
DNS1=114.114.114.114
在RS上关闭arp包的通告和响应然后配置RIP和VIP
[root@R1 ~]# sysctl -p
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
[root@R2 ~]# cat >> /etc/sysctl.conf <<EOF
> net.ipv4.conf.all.arp_ignore=1
> net.ipv4.conf.all.arp_announce=2
> EOF
[root@R2 ~]# sysctl -p
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
[root@R1 ~]# ifconfig lo:0 192.168.147.77/32 broadcast 192.168.147.77 up
[root@R1 ~]# route add -host 192.168.147.77 dev lo:0
[root@R1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet
BOOTPROTO=static
NAME=ens33
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.147.22
NETMASK=255.255.255.0
GATEWAY=192.168.147.2
DNS1=114.114.114.114
[root@R2 ~]# ifconfig lo:0 192.168.147.77/32 broadcast 192.168.147.77 up
[root@R2 ~]# route add -host 192.168.147.77.66 dev lo:0
[root@R2 ~]# cat /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE=Ethernet
BOOTPROTO=static
NAME=ens33
DEVICE=ens33
ONBOOT=yes
IPADDR=192.168.147.33
NETMASK=255.255.255.0
GATEWAY=192.168.147.2
DNS1=114.114.114.114
在调度器上添加，保存规则并开机自动读取规则
[root@lb ~]# ipvsadm -A -t 192.168.147.77:80 -s rr
[root@lb ~]# ipvsadm -a -t 192.168.147.77:80 -r 192.168.147.22:80 -g
[root@lb ~]# ipvsadm -a -t 192.168.147.77:80 -r 192.168.147.33:80 -g
[root@lb ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.147.77:80 rr
-> 192.168.147.22:80 Route 1 0 0
-> 192.168.147.33:80 Route 1 0 0
[root@lb ~]# ipvsadm -Sn > /etc/sysconfig/ipvsadm
[root@lb ~]# echo "ipvsadm -Sn > /etc/sysconfig/ipvsadm" >>/etc/rc.d/rc.local
测试
[root@lb ~]# curl 192.168.147.77
R1
[root@lb ~]# curl 192.168.147.77
R2
[root@lb ~]# curl 192.168.147.77
R1
[root@lb ~]# curl 192.168.147.77
R2
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
