Haproxy+keepalived TCP 层的负载均衡

一、安装环境准备：

l  [root@slave3 ~]# yum install ipvsadm kernel-developenssl-devel popt-devel libnl-devel gcc make –y

l  Makebinding to non-local Virtual IPs on all nodes

vi /etc/sysctl.conf  在各haproxy的机器上，在/etc/sysctl.conf下添加如下设置

net.ipv4.ip_nonlocal_bind = 1

 否则：会出现 scoket cannot connect  xxxx.xxxx.xxxx.xxxx:port

二、背景介绍

 

Keepalived是一个基于VRRP协议来实现的WEB 服务高可用方案，可以利用其来避免单点故障。一个WEB服务至少会有2台服务器运行Keepalived，一台为主服务器（MASTER），一台为备份服务器（BACKUP），但是对外表现为一个虚拟IP，主服务器会发送特定的消息给备份服务器，当备份服务器收不到这个消息的时候，即主服务器宕机的时候，备份服务器就会接管虚拟IP，继续提供服务，从而保证了高可用性

HAProxy介绍

反向代理服务器,支持双机热备支持虚拟主机, ,拥有非常不错的服务器健康检查功能,当其代理的后端服务器出现故障, HAProxy会自动将该服务器摘除,故障恢复后再自动将该服务器加入, 新的1.3引入了frontend,backend；frontend根据任意 HTTP请求头内容做规则匹配,然后把请求定向到相关的backend

 

三、安装keepalived 与haproxy

 在

[root@slave3 ~]# yum -y install haproxy keepalived

四、配置haproxy

 

[root@slave3 ~]# cd /etc/haproxy/

[root@slave3 haproxy]# cp  haproxy.cfg  haproxy.cfg.orig

[root@slave3 haproxy] vi  haproxy.cfg

global  

    log         127.0.0.1 local2

    chroot      /var/lib/haproxy

    pidfile      /var/run/haproxy.pid

    maxconn    4000

    user        haproxy

    group       haproxy

    daemon

    #turn onstats unix socket

    stats socket/var/lib/haproxy/stats

 

defaults

    mode                   tcp

    log                     global

    option                  dontlognull

    option                  http-server-close

    option                  redispatch

    option  abortonclose

    retries                  3

    timeout queue           1m

    timeoutconnect          10s

    timeoutclient            1m

    timeoutserver            1m

    timeoutcheck             10s

    maxconn                 3000

 

listen proxy_apalad

   bind192.168.211.200:8082

   mode tcp

   option  httpchk

   balanceroundrobin

   server slave3192.168.211.185:21000 weight 1 maxconn 1000 

 server slave4192.168.211.253:21000 weight 1 maxconn 1000

listen admin_stats

   bind192.168.211.200:8081

   mode http

   optionhttplog

   maxconn 10

   stats refresh30s

   stats uri/stats

 

备注：加日志支持

1.       编辑/etc/rsyslog.conf在最下边增加

local3.* /var/log/haproxy.log

local0.* /var/log/haproxy.log

local2.* /var/log/haproxy.log

2.       编辑 /etc/sysconfig/rsyslog修改：

SYSLOGD_OPTIONS="-r -m 0"

3.       重启日志服务

#service rsyslog restart 

五、配置keepalived

 

! Configuration File for keepalived

vrrp_script chk_http_port {

    script"/etc/keepalived/check_haproxy.sh"

    interval  2

    weight   2

}

#vrrp_script chk_haproxy{

#    script"killall -0 haproxy"

#    interval 2

#    weight -2

#}

vrrp_instance VI_1 {

    state MASTER

    interfaceeth1

   virtual_router_id 51

    priority 100

    advert_int 1

   authentication {

       auth_type PASS

       auth_pass 1111

    }

   virtual_ipaddress {

       192.168.211.200

    }

    track_script{

       #chk_haproxy

      chk_http_port

    }

   #notify_master "/etc/keepalived/scripts/start_haproxy.sh"

   #notify_backup "/etc/keepalived/scripts/stop_keepalived.sh"

   #notify_fault  "/etc/keepalived/scripts/stop_keepalived.sh"

   #notify_stop   "/etc/keepalived/scripts/stop_haproxy.sh"

 

配置keepalived 备用节点，与主配节点的区别有两处

Ø  将state  MASTER 修改为state  BACKUP

Ø  将priority  100   修改为priority  99

 

六、附加脚本

 

[root@slave3 keepalived]#  cd /etc/keepalived/

[root@slave3 keepalived]#  vi check_haproxy.sh

#!/bin/bash

if [ $(ps -C haproxy --no-header | wc-l) -eq 0 ]; then

    /etc/init.d/haproxy  start

fi

sleep 2

if [ $(ps -C haproxy --no-header | wc-l) -eq 0 ]; then

      /etc/init.d/keepalived stop

fi

 

修改脚本执行权限:

[root@slave3 keepalived]# chmod +x /etc/keepalived/check_haproxy.sh

[root@slave3 keepalived]# vi /etc/keepalived/script/start_haproxy.sh

 

#!/bin/bash

 

sleep 5

get=`ip addr |grep 192.168.211.200 |wc -l`

echo $get >>/etc/keepalived/scripts/start_ha.log

 

if [ $get -eq 1 ]

then

        echo"`date +%c` success to get vip" >> /etc/keepalived/scripts/start_ha.log

       /etc/init.d/haproxy start

else

        echo"`date +%c` can not get vip" >>/etc/keepalived/scripts/start_ha.log

fi

 

[root@slave3 keepalived]# vi /etc/keepalived/script/stop_haproxy.sh

 /etc/init.d/haproxystop

[root@slave3 keepalived]# vi /etc/keepalived/script/stop_keepalived.sh

#!/bin/bash

 

pid=`pidof keepalived`

if [ "$pid"x == ""x ]

then

    echo"`date +%c` no keepalived process id" >> /etc/keepalived/scripts/stop_keep.log

else

    echo"`date +%c` will stop keepalived " >> /etc/keepalived/scripts/stop_keep.log

   /etc/init.d/keepalived  stop

fi

 

/etc/init.d/keepalived stop

 

七、测试

启动haproxy:     [root@slave4~]#  /etc/init.d/haproxy start

启动keepalived：[root@slave3 scripts]# /etc/init.d/keepalived start

 

 

 

八、参考文档

http://blog.laimbock.com/2014/10/01/howto-setup-high-available-haproxy-with-keepalived/

Howtosetup High-Available HAProxy with Keepalived

http://blog.youkuaiyun.com/zzhongcy/article/details/46443765

haproxy做TCP层的负载均衡

http://my.oschina.net/davehe/blog/162560

keepalived+haproxy高可用负载均衡

http://blog.liuts.com/post/223/

基于Keepalived+Haproxy搭建四层负载均衡器[原创]