Nginx + Keepalived 配置实例

最新推荐文章于 2023-08-28 21:36:35 发布

玄玄君

最新推荐文章于 2023-08-28 21:36:35 发布

阅读量630

点赞数

分类专栏： nginx,tengine,keepalived

nginx,tengine,keepalived 专栏收录该内容

2 篇文章

订阅专栏

本文介绍如何利用Nginx和Keepalived搭建高可用的负载均衡集群，包括安装配置过程、监控机制及主备切换测试。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Nginx_Master: 192.168.1.103 提供负载均衡

Nginx_BackUp: 192.168.1.104 负载均衡备机

Nginx_VIP_TP: 192.168.1.108 网站的 VIP 地址(虚拟 IP)

原理：

VIP 是外网访问的IP地址，通过 keepalived 设置，以及 VRRP 将 VIP 绑定到主机和备机上，通过权重实现控制。当主机挂掉后，keepalived 释放对主机的控制，备机接管VIP。

扩展：

主机和备机可进一步延伸，它们只通过 Nginx 提供负载均衡。再让其它的机器提供真实的 web 服务。

**********************************************************************************************

安装 Nginx (省略)

-----------------------

http://blog.sina.com.cn/s/blog_5f54f0be0100yqm7.html

Keepalived 安装

-----------------------

地址: http://www.keepalived.org/download.html

wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz

tar -zxf keepalived-1.2.7.tar.gz

cd keepalived-1.2.7

64 位系统：

./configure --sysconf=/etc --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.32-358.2.1.el6.x86_64/

或 32 位系统:

./configure --sysconf=/etc --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.32-358.6.2.el6.i686/

可通过 getconf LONG_BIT 得到系统位数。

参数解释:

--sysconf 指定了配置文件的地址.即：/etc/keepalived/keepalived.conf

--prefix 指定了安装目录

--with-kernel-dir 指定使用内核源码中的头文件,即 include 目录.只有使用 LVS 时才需要这个参数，其它的时候不需要。

如果报错:

configure: error: Popt libraries is required

解决:

yum install popt-devel

再 configue .成功后提示：

Keepalived configuration

------------------------

Keepalived version : 1.2.7

Compiler : gcc

Compiler flags : -g -O2

Extra Lib : -lpopt -lssl -lcrypto

Use IPVS Framework : Yes

IPVS sync daemon support : Yes

IPVS use libnl : No

Use VRRP Framework : Yes

Use VRRP VMAC : Yes

SNMP support : No

Use Debug flags : No

安装：

make

make install

设置成为服务并开机启动:

cp /usr/local/keepalived/sbin/keepalived /usr/sbin/

/etc/rc.d/init.d/keepalived status

chkconfig --add keepalived

chkconfig keepalived on

设置主机上的配置文件内容：

vi /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {

notification_email {

sunyu@easymobi.cn

wuxuegang.123@163.com

}

notification_email_from pub@easymobi.cn

smtp_server 127.0.0.1

smtp_connect_timeout 30

router_id LVS_DEVEL

}

vrrp_instance VI_1 {

state MASTER

interface eth0

virtual_router_id 51

# 此处是主 Nginx 的 IP 地址.

mcast_src_ip 192.168.1.103

# 该机的 priority(优先) 为 100

priority 100

advert_int 1

authentication {

auth_type PASS

auth_pass 1111222

}

virtual_ipaddress {

192.168.1.108

}

前面的结构那里已经规定好了 VIP 和主备机的 IP, 所以这里按上面的填。

备机的配置文件:

! Configuration File for keepalived

global_defs {

notification_email {

sunyu@easymobi.cn

wuxuegang.123@163.com

}

notification_email_from pub@easymobi.cn

smtp_server 127.0.0.1

smtp_connect_timeout 30

router_id LVS_DEVEL

}

vrrp_instance VI_1 {

state SLAVER

interface eth0

virtual_router_id 51

# 此处是备 Nginx 的 IP 地址.

mcast_src_ip 192.168.1.104

# 该机的 priority(优先) 为 99

priority 99

advert_int 1

authentication {

auth_type PASS

auth_pass 1111222

}

virtual_ipaddress {

192.168.1.108

}

这时候 ping 192.168.1.108 是不通的.

然后在两台机器上分别启动 keepalived 服务

这时候再 ping 192.168.1.108 .通了.

实际上这时候 108 是被绑到主机上的。在主机上：

查看系统日志:

tailf /var/log/messages

May 29 18:32:16 localhost Keepalived_vrrp[27731]: Opening file '/etc/keepalived/keepalived.conf'.

May 29 18:32:16 localhost Keepalived_vrrp[27731]: Configuration is using : 62906 Bytes

May 29 18:32:16 localhost Keepalived_vrrp[27731]: Using LinkWatch kernel netlink reflector...

May 29 18:32:16 localhost Keepalived_healthcheckers[27729]: Using LinkWatch kernel netlink reflector...

May 29 18:32:16 localhost Keepalived_vrrp[27731]: VRRP sockpool: [ifindex(2), proto(112), fd(11,12)]

May 29 18:32:17 localhost Keepalived_vrrp[27731]: VRRP_Instance(VI_1) Transition to MASTER STATE

May 29 18:32:18 localhost Keepalived_vrrp[27731]: VRRP_Instance(VI_1) Entering MASTER STATE

May 29 18:32:18 localhost Keepalived_vrrp[27731]: VRRP_Instance(VI_1) setting protocol VIPs.

May 29 18:32:18 localhost Keepalived_vrrp[27731]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.1.108

May 29 18:32:18 localhost Keepalived_healthcheckers[27729]: Netlink reflector reports IP 192.168.1.108 added

可以看到. VRRP(虚拟路由冗余协议)已经启动.我们可以通过命令 ip addr 来检查主 Nginx 上的 IP 分配情况.

[root@localhost ~]# ip addr

1: lo: mtu 16436 qdisc noqueue state UNKNOWN

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: eth0: mtu 1500 qdisc mq state UP qlen 1000

link/ether 00:15:c5:ef:53:8c brd ff:ff:ff:ff:ff:ff

inet 192.168.1.103/25 brd 192.168.1.255 scope global eth0

inet 192.168.1.108/32 scope global eth0

inet6 fe80::215:c5ff:feef:538c/64 scope link

valid_lft forever preferred_lft forever

3: eth1: mtu 1500 qdisc noop state DOWN qlen 1000

link/ether 00:15:c5:ef:53:8e brd ff:ff:ff:ff:ff:ff

可以看到 VIP 地址已经绑定到主 Nginx 机器上: inet 192.168.1.108/32 scope global eth0

我们通过 tcpdump 抓包：

[root@localhost ~]# tcpdump vrrp

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode

listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes

13:38:27.797982 IP htuidc.bgp.ip > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20

13:38:28.794693 IP htuidc.bgp.ip > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20

13:38:29.794518 IP htuidc.bgp.ip > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20

13:38:30.798581 IP htuidc.bgp.ip > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20

13:38:31.795902 IP htuidc.bgp.ip > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20

13:38:32.804050 IP htuidc.bgp.ip > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20

13:38:33.801191 IP htuidc.bgp.ip > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20

13:38:34.798793 IP htuidc.bgp.ip > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 100, authtype simple, intvl 1s, length 20

这样，一个 Nginx + Keepalived 的架构就完成了。

监控和主备切换

接下来可以完善一下，加上实时监控，如果发现负载均衡的 Nginx 出现问题，就将该机器上的 Keepalived 服务停掉。

nginx_check.sh:

#!/bin/bash

while :

nginxpid = 'ps -C nginx --no-header | wc -l'

if[ $nginxpid -eq 0 ];then

service nginx start

sleep 3

nginxpid = 'ps -C nginx --no-header | wc -l'

echo $nginxpid

if[ $nginxpid -eq 0 ];then

service keepalived stop

sleep 3

done

然后让该脚本一直在后台运行:

nohup /etc/nginx_check.sh

或者将它添加成服务，让它开机自启动:

http://blog.sina.com.cn/s/blog_5f54f0be0101b3bs.html

测试:

--------------------

在两台机器的 web 服务器上分别放一个 index.html, 里面内容分别是自己机器的IP.

通过VIP访问：

http://192.168.1.108/index.html 发现显示的是主机的IP.

此时，关掉主机的 nginx, 这时候由于上面的监控脚本。主机的 keepalived 也会关闭。这时候再访问上面地址，发现显示的是备机的IP。可见，切换成功。

负载均衡

--------------------

http://blog.sina.com.cn/s/blog_5f54f0be0100zvf5.html

通过 ip_hash 实现会话保持