32.0Keepalived高可用

原创于 2018-09-02 21:02:46 发布 · 266 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#keepalived

linux 专栏收录该内容

98 篇文章

订阅专栏

本文介绍如何使用 KeepAlived 实现双主的 IPVS 的直接路由 (DR) 模型高可用性集群。文中详细记录了配置步骤，包括设置时间同步、禁用 iptables 和 selinux、主机名解析、SSH 密钥认证等。此外，还提供了 keepalived.conf 配置文件示例，展示了 VRRP 实例配置、健康检查、日志通知等功能。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

KeepAlived实现

keepalived基于lvs，功能包含lvs，解决了lvs不具备健康性检查，单点失败问题；lvs是4层调度，基于内核级，并发访问量400万左右

LB Cluster:
四层：lvs, nginx(stream)，haproxy(mode tcp)   l 
七层：http: nginx(http), haproxy(mode http), httpd...   
lvs号称400万并发，内核级；haproxy 1万，功能多；nginx当代理2到3万并发

HA Cluster 配置准备：       #高可用集群 high availability
(1) 各节点时间必须同步
        ntp, chrony
(2) 确保iptables及selinux不会成为阻碍 
(3) 各节点之间可通过主机名互相通信（对KA并非必须）  
        建议使用/etc/hosts文件实现
(4) 各节点之间的root用户可以基于密钥认证的ssh服务完成互相通信（对KA并非必须）

keepalived的nat模式下如果rs服务器和keepalived连接，需要指向keepalived的网关，需要两个浮动地址来完成请求报文和响应报文(基于ip的原因)；而dr模式下，只需要请求报文时的浮动地址，响应报文无需浮动地址（基于mac地址的原因），keepalived的高可用集群需要多播，即D类地址，multicast用来沟通监控或者交换浮动ip地址

keepalived： #集成了lvs的功能，使其具备高可用性，健康性检查
　　vrrp协议：Virtual Router Redundancy Protocol
术语：
　　虚拟路由器：Virtual Router　
　　虚拟路由器标识：VRID(0-255)，唯一标识虚拟路由器
　　物理路由器：
　　　　master：主设备
　　　　backup：备用设备
　　　　priority：优先级
　　VIP：Virtual IP
　　VMAC：Virutal MAC (00-00-5e-00-01-VRID)

KeepAlived
　　通告：心跳，优先级等；周期性
　　工作方式：抢占式，非抢占式
　　安全工作：
　　　　认证：
　　　　　　无认证
　　　　　　简单字符认证：预共享密钥
　　　　　　MD5
　　　　工作模式：
　　　　　　主/备：单虚拟路径器
　　　　　　主/主：主/备（虚拟路径器1），备/主（虚拟路径器2） #互为主备，如果一个宕机，则另一个需要承担两个的工作量

keepalived:
　　vrrp协议的软件实现，原生设计目的为了高可用ipvs服务
　　功能：
　　　　vrrp协议完成地址流动
　　　　为vip地址所在的节点生成ipvs规则(在配置文件中预先定义)
　　　　为ipvs集群的各RS做健康状态检测
　　　　基于脚本调用接口通过执行脚本完成脚本中定义的功能，进而影响集群事务，以此支持nginx、haproxy等服务
　　组件：
　　核心组件：
　　　　vrrp stack
　　　　ipvs wrapper ipvs规则
　　　　checkers
　　控制组件：配置文件分析器
　　IO复用器
　　内存管理组件

实验

[root@cos7 ~ ]#hostnamectl set-hostname ka1
[root@ka1 ~ ]#cd /etc/keepalived/
[root@ka1 keepalived ]#vim keepalived.conf

[root@ka1 keepalived ]#ssh-genkey
[root@ka1 keepalived ]#ssh-copy-id 192.168.31.17
[root@ka2 ~ ]#ssh-keygen
[root@ka2 ~ ]#ssh-copy-id 192.168.31.7

[root@ka1 keepalived ]#vim /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 ka1
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.31.17 ka2

[root@ka2 etc ]#vim /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4 ka2
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.31.7 ka1      

[root@ka1 etc ]#vim /etc/keepalived/keepalived.conf
"keepalived.conf" 157L, 3598C                                                             1,1           Top
! Configuration File for keepalived

global_defs {
   notification_email {
    root@localhost                 #通知邮件
   }
   notification_email_from ka@localhost  #发送邮件的地址
   smtp_server 127.0.0.1                 #邮件服务器
   smtp_connect_timeout 30               #邮件超时时长
   router_id ka1             #ka2为ka2   #router id名称
   vrrp_mcast_group4 230.10.10.10        #多播地址
}
vrrp_instance VI_1 {                     #vrrp实例1
    state MASTER                         #主
    interface eth0                       #绑定网卡eth0
    virtual_router_id 50                 #id号
    priority 100         #ka2为80        #优先级100
    advert_int 1                         #检查可用性间隔1s
    nopreempt                            #非抢占式选举
    authentication {
        auth_type PASS                   #认证
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.31.100/24                #浮动ip地址
    }
    notify_master "/etc/keepalived/notify.sh master"  #脚本
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}

[root@ka2 etc ]#systemctl start keepalived.service 
[root@ka2 etc ]#systemctl status keepalived.service 
Aug 31 20:54:11 ka2 Keepalived_vrrp[3147]: VRRP_Instance(VI_1) Transition to MASTER STATE

[root@ka2 etc ]#ip a
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 00:50:56:22:38:c9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.31.17/24 brd 192.168.31.255 scope global noprefixroute eth0
       valid_lft forever preferred_lft forever
    inet 192.168.31.100/24 scope global secondary eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::4619:664b:f1e4:d5b3/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever

[root@client ~ ]#ping 192.168.31.100
[root@cos27 ~ ]#tcpdump -i eth0 -nn dst 230.10.10.10
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
20:57:14.870785 IP 192.168.31.17 > 230.10.10.10: VRRPv2, Advertisement, vrid 50, prio 80, authtype simple, intvl 1s, length 20
20:57:15.873160 IP 192.168.31.17 > 230.10.10.10: VRRPv2, Advertisement, vrid 50, prio 80, authtype simple, intvl 1s, length 20
若启动ka1，则100优先级高于ka2的80,192.168.31.100的ip地址将浮动到ka1的eth0网卡上

日志

[root@ka1 ~ ]#rpm -ql keepalived
[root@ka1 ~ ]#vim /etc/sysconfig/keepalived
# Options for keepalived. See `keepalived --help' output and keepalived(8) and
# keepalived.conf(5) man pages for a list of all options. Here are the most
# common ones :
#
# --vrrp               -P    Only run with VRRP subsystem.
# --check              -C    Only run with Health-checker subsystem.
# --dont-release-vrrp  -V    Dont remove VRRP VIPs & VROUTEs on daemon stop.
# --dont-release-ipvs  -I    Dont remove IPVS topology on daemon stop.
# --dump-conf          -d    Dump the configuration data.
# --log-detail         -D    Detailed log messages.
# --log-facility       -S    0-7 Set local syslog facility (default=LOG_DAEMON)
#

#KEEPALIVED_OPTIONS="-D"
KEEPALIVED_OPTIONS="-D -S 3"


[root@ka1 ~ ]#vim /etc/rsyslog.conf
# Save boot messages also to boot.log
local7.*                                                /var/log/boot.log
local3.*                                               /var/log/keepalived.log
[root@ka1 ~ ]#systemctl restart rsyslog
[root@ka1 ~ ]#systemctl restart keepalived
[root@ka1 ~ ]#tail -f /var/log/keepalived.log

通知邮件脚本

[root@ka1 keepalived ]#vim notify.sh
#!/bin/bash
#
contact='root@localhost'
notify() {
mailsubject="$(hostname) to be $1, vip floating"
mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
master)
notify master
;;
backup)
notify backup
;;
fault)
notify fault
;;
*)
echo "Usage: $(basename $0) {master|backup|fault}"
exit 1
;;
esac

[root@ka1 keepalived ]#systemctl reload keepalived
#把ka1的网络连接断开，用另一个网卡连接xshell，如下
[root@ka1 ~ ]#mail
Heirloom Mail version 12.5 7/5/10.  Type ? for help.
"/var/spool/mail/root": 1 message 1 new
>N  1 root                  Sun Sep  2 14:17  18/665   "ka1 to be fault, vip floating"
>& 1
>Message  1:
>From root@ka1.localdomain  Sun Sep  2 14:17:15 2018
>Return-Path: <root@ka1.localdomain>
>X-Original-To: root@localhost
>Delivered-To: root@localhost.localdomain
>Date: Sun, 02 Sep 2018 14:17:15 +0800
>To: root@localhost.localdomain
>Subject: ka1 to be fault, vip floating
>User-Agent: Heirloom mailx 12.5 7/5/10
>Content-Type: text/plain; charset=us-ascii
>From: root@ka1.localdomain (root)
>Status: R

keepalived双主配置

[root@ka1 keepalived ]#cat keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
    root@localhost
   }
   notification_email_from ka@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka1
   vrrp_mcast_group4 230.10.10.10
}

vrrp_instance VI_1 {
    state MASTER                   #ka2为BACKUP
    interface eth0
    virtual_router_id 50
    priority 100                   #ka2为80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        192.168.31.100/24      
    }
}
vrrp_instance VI_2 {
    state BACKUP              #ka2为MASTER
    interface eth0
    virtual_router_id 60
    priority 80               #ka2位100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 654321
    }
    virtual_ipaddress {
        192.168.31.200/24
    }
}
[root@ka1 keepalived ]#systemctl reload keepalived
[root@ka2 keepalived ]#systemctl reload keepalived

实验：利用keepalived实现双主的IPVS 的DR模型高可用性

图keepalived-dr 　　　　 30.2lvs-dr
这里写图片描述

ka1
[root@ka1 keepalived]#cat keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
    root@localhost 
   }
   notification_email_from ka@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka1
   vrrp_mcast_group4 230.10.10.10
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 50
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        10.0.0.100/24 
    }
}
vrrp_instance VI_2 {                                                                                                          
    state BACKUP
    interface eth0
    virtual_router_id 60
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 654321
    }
    virtual_ipaddress {
        10.0.0.200/24 
    }
}

virtual_server 10.0.0.100 80 {
    delay_loop 6
    lb_algo wrr
    lb_kind DR
    #persistence_timeout 50
    protocol TCP
    sorry_server 127.0.0.1 80
    real_server 192.168.31.27 80 {
        weight 1
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

virtual_server 10.0.0.200 80 {
    delay_loop 6
    lb_algo wrr
    lb_kind DR
    #persistence_timeout 50
    protocol TCP
    sorry_server 127.0.0.1 80
    real_server 192.168.31.37 80 {
        weight 2
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
------------------------------------------------
ka2
[root@ka2 keepalived]#cat keepalived.conf
! Configuration File for keepalived

global_defs {
   notification_email {
    root@localhost 
   }
   notification_email_from ka@localhost
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id ka2
   vrrp_mcast_group4 230.10.10.10
}

vrrp_instance VI_1 {
    state BACKUP
    interface eth0
    virtual_router_id 50
    priority 80
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 123456
    }
    virtual_ipaddress {
        10.0.0.100/24 
    }
}

vrrp_instance VI_2 {                                                                                                          
    state MASTER 
    interface eth0
    virtual_router_id 60
    priority 100
    advert_int 1
    authentication {
        auth_type PASS
        auth_pass 654321
    }
    virtual_ipaddress {
        10.0.0.200/24 
    }
}
virtual_server 10.0.0.100 80 {
    delay_loop 6
    lb_algo wrr
    lb_kind DR
    #persistence_timeout 50
    protocol TCP
    sorry_server 127.0.0.1 80
    real_server 192.168.31.27 80 {
        weight 2
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
virtual_server 10.0.0.200 80 {
    delay_loop 6
    lb_algo wrr
    lb_kind DR
    #persistence_timeout 50
    protocol TCP
    sorry_server 127.0.0.1 80
    real_server 192.168.31.37 80 {
        weight 10
        HTTP_GET {
            url {
              path /
              status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}
rs服务器的公网ip要配置不响应，不宣传拥有ip

[root@ka1 keepalived]#ipvsadm  -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.0.0.100:80 wrr
  -> 192.168.31.27:80             Route   1      0          0         
TCP  10.0.0.200:80 wrr
  -> 192.168.31.37:80             Route   2      0          0   

#rs服务器的htptd服务都停止后，如下
[root@ka2 keepalived ]#echo sorry_server 129.0.0.1 80 > /var/www/html/index.html
[root@client ~ ]#curl 10.0.0.100
sorry_server 127.0.0.1 80