lvs_nat模式实验

本文详细介绍了一个基于NAT模式的负载均衡环境搭建过程及测试案例。通过具体步骤讲解如何配置虚拟服务器、真实服务器,并利用ipvsadm进行调度。文章还深入解析了NAT模式下数据包转发机制及其与FullNAT模式的区别。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

这里写图片描述

nat模式环境模拟

1.实验环境概览
192.168.1.186   192.168.75.131          双网卡         dir     调度器(vip)        ipvsadm,双网卡
192.168.1.185                           单网卡         rs1     真实服务器1      nginx,网关指定调度器(vip)
192.168.1.190                           单网卡         rs2     真实服务器2      nginx,网关指定调度器(vip)
192.168.75.150                          单网卡(双网卡)    客户端 (可以不用配置,直接使用vip测试,可以跟调度器一样的配置,方便远程连接)
2.环境准备
  • 准备三台虚拟机(网卡为桥接模式,单网卡,直接安装下一步或者克隆;能上网),客户端可以不用配置,关闭iptables和selinux
  • 分别修改hostname(vim /etc/sysconfig/network);分别为vip,real_server1,real_server2
  • 在vip上安装ipvsadm,两台real_server安装nginx
  • 在real_server1上执行 echo “1111111111111master” > /usr/share/nginx/html/index.html,修改网卡网关
  • 在real_server2上执行 echo “2222222222222slave” > /usr/share/nginx/html/index.html,修改网卡网关

vip服务器上增加网卡:在vmware上关闭vip,
1. “编辑虚拟机”>”添加”>”网络适配器”>”下一步”>”NAT模式”>”完成”
2. “虚拟网络编辑器”>”VMnet8” 点击下方 NAT设置查看网关(我的是192.168.75.2)
3. 查看”DHCP设置”查看开启的网段(设置外网网段的设置范围)
4. 开启vip,执行vim /etc/udev/rules.d/70-persistent-net.rules,记录ATTR{address},即MAC地址
5. cp /etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-eth1
6. vim /etc/sysconfig/network-scripts/ifcfg-eth1
7. 设置ip为192.168.75.131,网关为192.168.75.2,HWADDR填写刚刚记录的网卡,设置开机启动,重启网卡

进行试验

1.在调度器上新建脚本
vim /usr/local/sbin/lvs_nat.sh

#! /bin/bash
# director 服务器上开启路由转发功能: 
echo 1 > /proc/sys/net/ipv4/ip_forward
# 关闭icmp的重定向
echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/default/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects
echo 0 > /proc/sys/net/ipv4/conf/eth1/send_redirects

iptables -F
iptables -t nat -F
iptables -t nat -X
#iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -j MASQUERADE
#这条命令并没有关联
ipvsadm -C
#ipvsadm -A -t 192.168.75.131:80 -s rr -p 300   #-p 300   长链接300秒(会导致5分钟不更换服务器)
ipvsadm -A -t 192.168.75.131:80 -s wrr
#wrr表示权重轮询,根据权重来轮询
ipvsadm -a -t 192.168.75.131:80 -r 192.168.1.185:80 -m -w 1
ipvsadm -a -t 192.168.75.131:80 -r 192.168.1.190:80 -m -w 2

2.执行脚本

sh /usr/local/sbin/lvs_nat.sh

3.测试结果
在vip上执行

curl 192.168.75.131

会根据权重来返回结果

抓包结果
[root@real_server1 ~]# tcpdump -nn -i eth0 host 192.168.75.150 and ! port 22
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
01:00:20.379751 IP 192.168.75.150.53919 > 192.168.1.190.80: Flags [S], seq 4207417558, win 14600, options [mss 1460,sackOK,TS val 4562638 ecr 0,nop,wscale 6], length 0
01:00:20.379839 IP 192.168.1.190.80 > 192.168.75.150.53919: Flags [S.], seq 3392244670, ack 4207417559, win 14480, options [mss 1460,sackOK,TS val 5501088 ecr 4562638,nop,wscale 6], length 0
01:00:20.388310 IP 192.168.75.150.53919 > 192.168.1.190.80: Flags [.], ack 1, win 229, options [nop,nop,TS val 4562642 ecr 5501088], length 0
01:00:20.388566 IP 192.168.75.150.53919 > 192.168.1.190.80: Flags [P.], seq 1:176, ack 1, win 229, options [nop,nop,TS val 4562642 ecr 5501088], length 175
01:00:20.388772 IP 192.168.1.190.80 > 192.168.75.150.53919: Flags [.], ack 176, win 243, options [nop,nop,TS val 5501096 ecr 4562642], length 0
01:00:20.389948 IP 192.168.1.190.80 > 192.168.75.150.53919: Flags [P.], seq 1:246, ack 176, win 243, options [nop,nop,TS val 5501098 ecr 4562642], length 245
01:00:20.395679 IP 192.168.75.150.53919 > 192.168.1.190.80: Flags [.], ack 246, win 245, options [nop,nop,TS val 4562654 ecr 5501098], length 0
01:00:20.425383 IP 192.168.75.150.53919 > 192.168.1.190.80: Flags [F.], seq 176, ack 246, win 245, options [nop,nop,TS val 4562657 ecr 5501098], length 0
01:00:20.426466 IP 192.168.1.190.80 > 192.168.75.150.53919: Flags [F.], seq 246, ack 177, win 243, options [nop,nop,TS val 5501134 ecr 4562657], length 0
01:00:20.436678 IP 192.168.75.150.53919 > 192.168.1.190.80: Flags [.], ack 247, win 245, options [nop,nop,TS val 4562695 ecr 5501134], length 0
01:00:57.176788 IP 192.168.75.150.53920 > 192.168.1.185.80: Flags [S], seq 704010524, win 14600, options [mss 1460,sackOK,TS val 4599425 ecr 0,nop,wscale 6], length 0
01:00:57.178908 IP 192.168.1.185.80 > 192.168.75.150.53920: Flags [S.], seq 2021861845, ack 704010525, win 14480, options [mss 1460,sackOK,TS val 5533840 ecr 4599425,nop,wscale 6], length 0
01:00:57.187976 IP 192.168.75.150.53920 > 192.168.1.185.80: Flags [.], ack 1, win 229, options [nop,nop,TS val 4599444 ecr 5533840], length 0
01:00:57.188021 IP 192.168.75.150.53920 > 192.168.1.185.80: Flags [P.], seq 1:176, ack 1, win 229, options [nop,nop,TS val 4599444 ecr 5533840], length 175
01:00:57.188359 IP 192.168.1.185.80 > 192.168.75.150.53920: Flags [.], ack 176, win 243, options [nop,nop,TS val 5533851 ecr 4599444], length 0
01:00:57.189733 IP 192.168.1.185.80 > 192.168.75.150.53920: Flags [P.], seq 1:246, ack 176, win 243, options [nop,nop,TS val 5533852 ecr 4599444], length 245
01:00:57.200746 IP 192.168.75.150.53920 > 192.168.1.185.80: Flags [.], ack 246, win 245, options [nop,nop,TS val 4599454 ecr 5533852], length 0
01:00:57.200787 IP 192.168.75.150.53920 > 192.168.1.185.80: Flags [F.], seq 176, ack 246, win 245, options [nop,nop,TS val 4599456 ecr 5533852], length 0
01:00:57.200820 IP 192.168.1.185.80 > 192.168.75.150.53920: Flags [F.], seq 246, ack 177, win 243, options [nop,nop,TS val 5533861 ecr 4599456], length 0
01:00:57.212718 IP 192.168.75.150.53920 > 192.168.1.185.80: Flags [.], ack 247, win 245, options [nop,nop,TS val 4599460 ecr 5533861], length 0
01:02:09.403093 IP 192.168.75.150.53921 > 192.168.1.190.80: Flags [S], seq 970535959, win 14600, options [mss 1460,sackOK,TS val 4671656 ecr 0,nop,wscale 6], length 0
01:02:09.403203 IP 192.168.1.190.80 > 192.168.75.150.53921: Flags [S.], seq 1701915808, ack 970535960, win 14480, options [mss 1460,sackOK,TS val 5610111 ecr 4671656,nop,wscale 6], length 0
01:02:09.412386 IP 192.168.75.150.53921 > 192.168.1.190.80: Flags [.], ack 1, win 229, options [nop,nop,TS val 4671667 ecr 5610111], length 0
01:02:09.412460 IP 192.168.75.150.53921 > 192.168.1.190.80: Flags [P.], seq 1:176, ack 1, win 229, options [nop,nop,TS val 4671669 ecr 5610111], length 175
01:02:09.412507 IP 192.168.1.190.80 > 192.168.75.150.53921: Flags [.], ack 176, win 243, options [nop,nop,TS val 5610120 ecr 4671669], length 0
01:02:09.423352 IP 192.168.1.190.80 > 192.168.75.150.53921: Flags [P.], seq 1:246, ack 176, win 243, options [nop,nop,TS val 5610131 ecr 4671669], length 245
01:02:09.434206 IP 192.168.75.150.53921 > 192.168.1.190.80: Flags [.], ack 246, win 245, options [nop,nop,TS val 4671689 ecr 5610131], length 0
01:02:09.434294 IP 192.168.75.150.53921 > 192.168.1.190.80: Flags [F.], seq 176, ack 246, win 245, options [nop,nop,TS val 4671689 ecr 5610131], length 0
01:02:09.435994 IP 192.168.1.190.80 > 192.168.75.150.53921: Flags [F.], seq 246, ack 177, win 243, options [nop,nop,TS val 5610143 ecr 4671689], length 0
01:02:09.439232 IP 192.168.75.150.53921 > 192.168.1.190.80: Flags [.], ack 247, win 245, options [nop,nop,TS val 4671697 ecr 5610143], length 0
[root@vip ~]# tcpdump -nn -i eth1 host 192.168.75.150 and ! port 22
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
00:58:58.218430 IP 192.168.75.150.53918 > 192.168.75.131.80: Flags [S], seq 2818675909, win 14600, options [mss 1460,sackOK,TS val 4481342 ecr 0,nop,wscale 6], length 0
00:58:58.219493 IP 192.168.75.131.80 > 192.168.75.150.53918: Flags [S.], seq 4017612900, ack 2818675910, win 14480, options [mss 1460,sackOK,TS val 5419792 ecr 4481342,nop,wscale 6], length 0
00:58:58.223052 IP 192.168.75.150.53918 > 192.168.75.131.80: Flags [.], ack 1, win 229, options [nop,nop,TS val 4481347 ecr 5419792], length 0
00:58:58.247015 IP 192.168.75.150.53918 > 192.168.75.131.80: Flags [P.], seq 1:176, ack 1, win 229, options [nop,nop,TS val 4481349 ecr 5419792], length 175
00:58:58.250997 IP 192.168.75.131.80 > 192.168.75.150.53918: Flags [.], ack 176, win 243, options [nop,nop,TS val 5419821 ecr 4481349], length 0
00:58:58.251869 IP 192.168.75.131.80 > 192.168.75.150.53918: Flags [P.], seq 1:246, ack 176, win 243, options [nop,nop,TS val 5419824 ecr 4481349], length 245
00:58:58.253316 IP 192.168.75.150.53918 > 192.168.75.131.80: Flags [.], ack 246, win 245, options [nop,nop,TS val 4481377 ecr 5419824], length 0
00:58:58.253974 IP 192.168.75.150.53918 > 192.168.75.131.80: Flags [F.], seq 176, ack 246, win 245, options [nop,nop,TS val 4481378 ecr 5419824], length 0
00:58:58.263118 IP 192.168.75.131.80 > 192.168.75.150.53918: Flags [F.], seq 246, ack 177, win 243, options [nop,nop,TS val 5419836 ecr 4481378], length 0
00:58:58.265457 IP 192.168.75.150.53918 > 192.168.75.131.80: Flags [.], ack 247, win 245, options [nop,nop,TS val 4481390 ecr 5419836], length 0
00:59:03.219316 ARP, Request who-has 192.168.75.150 tell 192.168.75.131, length 28
00:59:03.221931 ARP, Reply 192.168.75.150 is-at 00:0c:29:5a:c3:3e, length 46
01:00:19.513727 IP 192.168.75.150.53919 > 192.168.75.131.80: Flags [S], seq 4207417558, win 14600, options [mss 1460,sackOK,TS val 4562638 ecr 0,nop,wscale 6], length 0
01:00:19.514865 IP 192.168.75.131.80 > 192.168.75.150.53919: Flags [S.], seq 3392244670, ack 4207417559, win 14480, options [mss 1460,sackOK,TS val 5501088 ecr 4562638,nop,wscale 6], length 0
01:00:19.520316 IP 192.168.75.150.53919 > 192.168.75.131.80: Flags [.], ack 1, win 229, options [nop,nop,TS val 4562642 ecr 5501088], length 0
01:00:19.520645 IP 192.168.75.150.53919 > 192.168.75.131.80: Flags [P.], seq 1:176, ack 1, win 229, options [nop,nop,TS val 4562642 ecr 5501088], length 175
01:00:19.525721 IP 192.168.75.131.80 > 192.168.75.150.53919: Flags [.], ack 176, win 243, options [nop,nop,TS val 5501096 ecr 4562642], length 0
01:00:19.526217 IP 192.168.75.131.80 > 192.168.75.150.53919: Flags [P.], seq 1:246, ack 176, win 243, options [nop,nop,TS val 5501098 ecr 4562642], length 245
01:00:19.529640 IP 192.168.75.150.53919 > 192.168.75.131.80: Flags [.], ack 246, win 245, options [nop,nop,TS val 4562654 ecr 5501098], length 0
01:00:19.559466 IP 192.168.75.150.53919 > 192.168.75.131.80: Flags [F.], seq 176, ack 246, win 245, options [nop,nop,TS val 4562657 ecr 5501098], length 0
01:00:19.561567 IP 192.168.75.131.80 > 192.168.75.150.53919: Flags [F.], seq 246, ack 177, win 243, options [nop,nop,TS val 5501134 ecr 4562657], length 0
01:00:19.570684 IP 192.168.75.150.53919 > 192.168.75.131.80: Flags [.], ack 247, win 245, options [nop,nop,TS val 4562695 ecr 5501134], length 0
01:00:24.515461 ARP, Request who-has 192.168.75.150 tell 192.168.75.131, length 28
01:00:24.526927 ARP, Reply 192.168.75.150 is-at 00:0c:29:5a:c3:3e, length 46
01:00:56.305273 IP 192.168.75.150.53920 > 192.168.75.131.80: Flags [S], seq 704010524, win 14600, options [mss 1460,sackOK,TS val 4599425 ecr 0,nop,wscale 6], length 0
01:00:56.317205 IP 192.168.75.131.80 > 192.168.75.150.53920: Flags [S.], seq 2021861845, ack 704010525, win 14480, options [mss 1460,sackOK,TS val 5533840 ecr 4599425,nop,wscale 6], length 0
01:00:56.321997 IP 192.168.75.150.53920 > 192.168.75.131.80: Flags [.], ack 1, win 229, options [nop,nop,TS val 4599444 ecr 5533840], length 0
01:00:56.322218 IP 192.168.75.150.53920 > 192.168.75.131.80: Flags [P.], seq 1:176, ack 1, win 229, options [nop,nop,TS val 4599444 ecr 5533840], length 175
01:00:56.323050 IP 192.168.75.131.80 > 192.168.75.150.53920: Flags [.], ack 176, win 243, options [nop,nop,TS val 5533851 ecr 4599444], length 0
01:00:56.324052 IP 192.168.75.131.80 > 192.168.75.150.53920: Flags [P.], seq 1:246, ack 176, win 243, options [nop,nop,TS val 5533852 ecr 4599444], length 245
01:00:56.331228 IP 192.168.75.150.53920 > 192.168.75.131.80: Flags [.], ack 246, win 245, options [nop,nop,TS val 4599454 ecr 5533852], length 0
01:00:56.331974 IP 192.168.75.150.53920 > 192.168.75.131.80: Flags [F.], seq 176, ack 246, win 245, options [nop,nop,TS val 4599456 ecr 5533852], length 0
01:00:56.332944 IP 192.168.75.131.80 > 192.168.75.150.53920: Flags [F.], seq 246, ack 177, win 243, options [nop,nop,TS val 5533861 ecr 4599456], length 0
01:00:56.335788 IP 192.168.75.150.53920 > 192.168.75.131.80: Flags [.], ack 247, win 245, options [nop,nop,TS val 4599460 ecr 5533861], length 0
01:01:01.317186 ARP, Request who-has 192.168.75.150 tell 192.168.75.131, length 28
01:01:01.318083 ARP, Reply 192.168.75.150 is-at 00:0c:29:5a:c3:3e, length 46

nat模式在收到请求后,客户端包进入调度器后,调度器修改目的地址;
发送给后端real_server处理,real_server处理完后再返回给调度器;
调度器再修改源地址,返回给客户

为什么在real_server上能接收到另外的服务器包的信息?相互之间会传播信息(这个不是很理解)

看了抓包后再分析nat与fullnat模式的区别
  • 为什么nat模式下需要将real_server的网关修改成vip的内网地址?
    nat模式下,vip将请求转发到后端,是通过修改dst来发送给后端real_server。
    但real_server返回包却出现问题,因为real_server是单网卡,需要将包发送给另一个网段的机器
    需要经过网关来调度,但默认网关却不能实现这个功能,只有被设为vip的机器能实现。
    只有将real_server网关设置成调度器,返回数据包才会发送给调度器
    同时,网关会将返回包的源ip换成vip。(最终目的)
    所以要将real_server的网关设置为vip的内网地址

  • fullnat就没有这个问题
    fullnat模式下,vip的功能是同时改变src跟dst(源地址和目的地址),那样就不需要将real_server网关设置成vip的内网地址
    因为vip与real_server都在同一网段,所以就不需要修改网关

  • fullnat缺点
    RealServer无法获得用户IP;淘宝通过叫TOA的方式解决的,
    主要原理是:将client address放到了TCP Option里面带给后端RealServer,RealServer收到后保存在socket
    的结构体里并通过toa内核模块hook了getname函数,这样当用户调用getname获取远端地址时,返回的是保
    存在socket的TCPOption的IP. 百度的BVS是通过叫ttm模块实现的,其实现方式跟toa基本一样,只是没有开源
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值