cluster集群的搭建与使用

初始环境的配置

用作集群的几个虚拟机最好时比较纯净的环境，还有必须要时间同步

1.首先配置好yum源以用来安装所需要的ricci服务与luci服务
配置的yum源添加部分如下

[HighAvailability]  （高可用）
name=HighAvailability
baseurl=http://172.25.39.250/rhel6.5/HighAvailability
gppcheck=0

[LoadBalancer]      （负载均衡）
name=LoadBalancer
baseurl=http://172.25.39.250/rhel6.5/LoadBalancer
gpgcheck=0

[ResilientStorage]  （存储）
name=ResilientStorage
baseurl=http://172.25.39.250/rhel6.5/ResilientStorage
gpgcheck=0

[ScalableFileSystem]    （大文件系统支持）
name=ScalableFileSystem
baseurl=http://172.25.39.250/rhel6.5/ScalableFileSystem
gpgcheck=0

yum repolist #刷新yum仓库

---

*repo id repo name status
*HighAvailability HighAvailability 56
*LoadBalancer LoadBalancer 4
*ResilientStorage ResilientStorage 62
*ScalableFileSystem ScalableFileSystem 7
*rhel6.5 Red Hat Enterprise Linux 3,690
*repolist: 3,819

---

2.几个集群的虚拟机中都需要安装ricci服务
yum install -y ricci
passwd ricci #修改ricci密码，最好密码设置为一致

1. /etc/init.d/ricci start ##开启服务
   chkconfig ricci on ##开机自启动

4.clustat #用于测试查看集群当前状态

5.用于提供luci图形管理界面的主机需要安装luci服务具体操作如下

  yum install -y luci   ##图形操作界面luci服务安装
  /etc/init.d/luci start
  chkconfig luci on
 clustat

在浏览器：
https://172.25.40.1:8084 #安装了luci的主机，服务使用的端口为8084.
进入图形界面后点击create将几个集群节点往里头添加即可完成基本的搭建。

fence的配置过程

在真机山如下操作
1.用合适的镜像yum源,安装以下所需软件包

    yum install fence-virtd-multicast.x86_64  fence-virtd.x86_64 fence-virtd-libvirt.x86_64 -y（网络监听器、电源管理器、内核层面控制虚拟机，关掉的话用户空间不能访问）

2.修改fence的配置：

  fence_virtd -c

以下是出现的信息，部分直接回车即可

Module search path [/usr/lib64/fence-virt]: 

Available backends:
    libvirt 0.1
Available listeners:
    multicast 1.2

Listener modules are responsible for accepting requests
from fencing clients.

Listener module [multicast]: 

The multicast listener module is designed for use environments
where the guests and hosts may communicate over a network using
multicast.

The multicast address is the address that a client will use to
send fencing requests to fence_virtd.

Multicast IP Address [225.0.0.12]: 

Using ipv4 as family.

Multicast IP Port [1229]: 

Setting a preferred interface causes fence_virtd to listen only
on that interface.  Normally, it listens on all interfaces.
In environments where the virtual machines are using the host
machine as a gateway, this *must* be set (typically to virbr0).
Set to 'none' for no interface.

Interface [virbr0]: br0

The key file is the shared key information which is used to
authenticate fencing requests.  The contents of this file must
be distributed to each physical host and virtual machine within
a cluster.

Key File [/etc/cluster/fence_xvm.key]: 

Backend modules are responsible for routing requests to
the appropriate hypervisor or management layer.

Backend module [libvirt]: 

Configuration complete.

=== Begin Configuration ===
backends {
    libvirt {
        uri = "qemu:///system";
    }

}

listeners {
    multicast {
        port = "1229";
        family = "ipv4";
        interface = "br0";
        address = "225.0.0.12";
        key_file = "/etc/cluster/fence_xvm.key";
    }

}

fence_virtd {
    module_path = "/usr/lib64/fence-virt";
    backend = "libvirt";
    listener = "multicast";
}

=== End Configuration ===
Replace /etc/fence_virt.conf with the above [y/N]? y

1. a) cd /etc/cluster
   b) dd if=/dev/urandom of=/etc/cluster/fence_xvm.key bs=128 count=1

2. systemctl restart fence_virtd.service (确保火墙允许服务iptables -L)

   netstat -anulp | grep :1229
   udp 0 0 0.0.0.0:1229 0.0.0.0:* 8163/fence_virtd

3. scp fence_xvm.key root@172.25.39.1:/etc/cluster/

   scp fence_xvm.key root@172.25.39.4:/etc/cluster/

配置过后可以在/etc/cluster目录下的配置文件来查看配置的情况

cd /etc/cluster/
cat cluster.conf

<?xml version="1.0"?>
<cluster config_version="7" name="clus">
    <clusternodes>
        <clusternode name="server1" nodeid="1">
            <fence>
                <method name="fence1">
                    <device domain="vm1" name="vmfence"/>
                </method>
            </fence>
        </clusternode>
        <clusternode name="server4" nodeid="2">
            <fence>
                <method name="fence4">
                    <device domain="vm4" name="vmfence"/>
                </method>
            </fence>
        </clusternode>
    </clusternodes>
    <cman expected_votes="1" two_node="1"/>
    <fencedevices>
        <fencedevice agent="fence_xvm" name="vmfence"/>
    </fencedevices>
</cluster>

两个集群主机上安装apache服务

yum install -y httpd
vim /var/www/html/index.html  

然后使用浏览器对其进行测试

网络设备iscis的配置

目的在于添加存储设备，以供两个集群的共享使用

在用于提供设备的主机上如下配置

 yum install -y iscsi-* #安装iscsi所需要的服务
vim /etc/tgt/targets.conf  #编写服务的配置文件如下

(38行)<target iqn.2007-07.com.example:server.target1>
             backing-store /dev/vdb  #存储设备
             initiator-address 172.25.40.1 #ACL
             initiator-address 172.25.40.4
         </target>

编写完毕后推出保存

/etc/init.d/tgtd start  #开启服务
tgt-admin -s        ##测试配置文件修改是否生效

在集群上如下配置

yum install -y iscsi-*  #安装iscsi的输出端

然后搜索网络设备

iscsiadm -m discovery -t st -p 172.25.39.2

然后登录到搜索到的设备上

iscsiadm -m node -l

成功登陆上设备时，会多出一个sda。
可以使用fdisk -l来查看
比较保险是cat /proc/partions(内核级别)
然后对这个sda进行分区，开始构造逻辑卷组

fdisk -cu /dev/sda   ##给sda分区 n->1 t ->8e->wq
vim /etc/lvm/lvm.conf   ##里面的状态会变成集群状态

然后开始构造逻辑卷,期间必须确保几个主机间是同步的

pvcreate /dev/sda1
vgcreate clustervg /dev/sda1
 lvcreate -L +2G -n demo clustervg
 mkfs.ext4 /dev/clustervg/demo  ##格式化为ext4文件系统

最后检测

clustat ##查看集群的信息