nginx+keepalived实现高可用负载均衡

最新推荐文章于 2025-02-14 15:51:58 发布

原创最新推荐文章于 2025-02-14 15:51:58 发布 · 951 阅读

1 ·

CC 4.0 BY-SA版权

linux服务篇同时被 2 个专栏收录

41 篇文章

订阅专栏

web架构篇

16 篇文章

订阅专栏

本文介绍如何使用 Keepalived 和 Nginx 构建高可用集群，包括安装配置步骤、检测脚本编写及负载均衡配置，确保服务在主节点故障时能无缝切换至备用节点。

1.获取keepalived

[root@master src]# wget http://www.keepalived.org/software/keepalived-1.2.9.tar.gz

2.安装依赖包

yum -y install openssl-devel gcc gcc-c++ glibc make

3.安装

[root@master src]# cd keepalived-1.2.9
[root@master keepalived-1.2.9]# ./configure --prefix=/usr/local/keepalived --sysconf=/etc
[root@master keepalived-1.2.9]# make && make install

注意：nginx的安装不再赘述，请移步此处

4.配置

备机上需要修改的配置文件注释出来了

[root@slave keepalived]# pwd
/etc/keepalived
[root@slave keepalived]# cat keepalived.conf 
! Configuration File for keepalived
vrrp_script Monitor_Nginx {
 script "/opt/sh/monitor_nginx.sh"
 interval 2
  weight 2
}

global_defs {
notification_email {
994414615@qq.com
}
notification_email_from AdunMasterLVS.com
smtp_server localhost
smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state BACKUP //主机此值为MASTER interface eth0 virtual_router_id 51 //虚拟路由的ID，主备必须一样

priority 80 //优先级，主大于备，如果多备的话这个参数大的首先成为主 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.10.4 //虚拟IP } track_script { Monitor_Nginx}}

检测nginx的脚本如下

[root@master keepalived]# cat /opt/sh/monitor_nginx.sh
#!/bin/bash
if [ "$(ps -ef | grep "nginx: master process"| grep -v grep )" == "" ] #过滤nginx的住进程为空
then
 /usr/local/nginx/sbin/nginx             #尝试启动nginx
 if [ $? != 0 ] #检测nginx是否正常启动 如果$?==0
 then
 killall keepalived     #如果nginx没有政策启动，则杀掉keepalived的进程，实现VIP转移
 fi
fi

5.启动服务

[root@master keepalived]# /usr/local/keepalived/sbin/keepalived -D -f /etc/keepalived/keepalived.conf

6.查看虚拟IP

[root@master keepalived]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:50:56:8a:2c:1c brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.2/24 brd 192.168.10.255 scope global eth0
    inet 192.168.10.4/32 scope global eth0                       //虚拟IP已经备绑定到此处
    inet6 fe80::250:56ff:fe8a:2c1c/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:50:56:8a:2c:1d brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.2/24 brd 172.16.0.255 scope global eth1
    inet6 fe80::250:56ff:fe8a:2c1d/64 scope link 
       valid_lft forever preferred_lft forever

注意：将域名解析到这个可以漂移的虚拟ip就实现HA

7.主down服务测试

[root@master nginx]# pkill -9 nginx && ps -ef|grep nginx && date   //杀掉nginx的进程
root     10804  1250  0 04:09 pts/0    00:00:00 grep nginx        //nginx的进程被杀死
2013年 11月 21日 星期四 04:09:37 EST                                 //时间是09时37秒
[root@master nginx]# ps -ef|grep nginx && date       //再次查看nginx进程和时间
root     10807 10806  0 04:09 ?        00:00:00 /bin/bash /opt/sh/monitor_nginx.sh
root     10813     1  0 04:09 ?        00:00:00 nginx: master process /usr/local/nginx/sbin/nginx
nobody   10814 10813  4 04:09 ?        00:00:00 nginx: worker process      
nobody   10816 10813  4 04:09 ?        00:00:00 nginx: worker process      
nobody   10817 10813  4 04:09 ?        00:00:00 nginx: worker process      
nobody   10818 10813  4 04:09 ?        00:00:00 nginx: worker process      
root     10820  1250  0 04:09 pts/0    00:00:00 grep nginx
2013年 11月 21日 星期四 04:09:38 EST             //9时38秒的时候 nginx已经自动起来了
[root@master nginx]# pkill -9 keep   //主机将keepalived down掉
[root@master nginx]# ps -ef|grep keep
root     11334  1250  0 04:13 pts/0    00:00:00 grep keep

备机接管VIP效果

[root@slave keepalived]# ip add
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:50:56:8a:2c:1e brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.3/24 brd 192.168.10.255 scope global eth0
    inet 192.168.10.4/32 scope global eth0                  //备机已经接管此ip，已提供继续服务
    inet6 fe80::250:56ff:fe8a:2c1e/64 scope link 
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:50:56:8a:2c:1f brd ff:ff:ff:ff:ff:ff
    inet 172.16.0.3/24 brd 172.16.0.255 scope global eth1
    inet6 fe80::250:56ff:fe8a:2c1f/64 scope link 
       valid_lft forever preferred_lft forever

主机启动keepalived查看资源接管情况

主机

[root@master log]# /usr/local/keepalived/sbin/keepalived -D -f /etc/keepalived/keepalived.conf  && tail -0f /var/log/messages 
Nov 21 04:16:55 master Keepalived_healthcheckers[11342]: Registering Kernel netlink reflector
Nov 21 04:16:55 master Keepalived_healthcheckers[11342]: Registering Kernel netlink command channel
Nov 21 04:16:55 master Keepalived_healthcheckers[11342]: Opening file '/etc/keepalived/keepalived.conf'.
Nov 21 04:16:55 master Keepalived_healthcheckers[11342]: Configuration is using : 7207 Bytes
Nov 21 04:16:55 master Keepalived_vrrp[11344]: Opening file '/etc/keepalived/keepalived.conf'.
Nov 21 04:16:55 master Keepalived_vrrp[11344]: Configuration is using : 65334 Bytes
Nov 21 04:16:55 master Keepalived_vrrp[11344]: Using LinkWatch kernel netlink reflector...
Nov 21 04:16:55 master Keepalived_vrrp[11344]: VRRP sockpool: [ifindex(2), proto(112), unicast(0), fd(11,12)]
Nov 21 04:16:55 master Keepalived_healthcheckers[11342]: Using LinkWatch kernel netlink reflector...
Nov 21 04:16:55 master Keepalived_vrrp[11344]: VRRP_Script(Monitor_Nginx) succeeded
Nov 21 04:16:55 master Keepalived_vrrp[11344]: VRRP_Instance(VI_1) Transition to MASTER STATE
Nov 21 04:16:56 master Keepalived_vrrp[11344]: VRRP_Instance(VI_1) Entering MASTER STATE
Nov 21 04:16:56 master Keepalived_vrrp[11344]: VRRP_Instance(VI_1) setting protocol VIPs.  //设置VIP 到eth0
Nov 21 04:16:56 master Keepalived_vrrp[11344]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.10.4
Nov 21 04:17:01 master Keepalived_vrrp[11344]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.10.4

备机

[root@slave keepalived]# tail -10 /var/log/messages 
Nov 21 04:13:33 slave Keepalived_vrrp[5898]: VRRP_Instance(VI_1) Transition to MASTER STATE
Nov 21 04:13:34 slave Keepalived_vrrp[5898]: VRRP_Instance(VI_1) Entering MASTER STATE
Nov 21 04:13:34 slave Keepalived_vrrp[5898]: VRRP_Instance(VI_1) setting protocol VIPs.
Nov 21 04:13:34 slave Keepalived_healthcheckers[5897]: Netlink reflector reports IP 192.168.10.4 added
Nov 21 04:13:34 slave Keepalived_vrrp[5898]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.10.4
Nov 21 04:13:39 slave Keepalived_vrrp[5898]: VRRP_Instance(VI_1) Sending gratuitous ARPs on eth0 for 192.168.10.4
Nov 21 04:16:55 slave Keepalived_vrrp[5898]: VRRP_Instance(VI_1) Received higher prio advert
Nov 21 04:16:55 slave Keepalived_vrrp[5898]: VRRP_Instance(VI_1) Entering BACKUP STATE
Nov 21 04:16:55 slave Keepalived_vrrp[5898]: VRRP_Instance(VI_1) removing protocol VIPs.
Nov 21 04:16:55 slave Keepalived_healthcheckers[5897]: Netlink reflector reports IP 192.168.10.4 removed  //将VIP移除

支持高可用集群部署成功

8.负载均衡配置

upstream HTTPDTEST    //在nginx配置文件的server段前加上此负载均衡列表 名叫HTTPDTEST
     {
     server   192.168.10.2:8080 max_fails=3 fail_timeout=30s ;
     server   192.168.10.3:8080 max_fails=3 fail_timeout=30s ;
     ip_hash;      //测试时去掉此项，线上必须加上以保存session
     }
location /slave {
             proxy_set_header       Host $host;
             proxy_set_header  X-Real-IP  $remote_addr;
             proxy_set_header  X-Forwarded-For $proxy_add_x_forwarded_for;
             proxy_pass  http://HTTPDTEST;         //此处指向如上定义的upstream
        }

至此，负载均衡配置成功

##########################

迷途小运维原创

作者：john

转载请注明出处