####Pacemaker+corosyn+pcs####

Pacemaker+corosyn+pcs:rhel7.3 实现高可用集群管理
##Pacemaker是什么？
Pacemaker是一个集群资源管理器。它利用集群基础构件（OpenAIS 、heartbeat或corosync）提供的消息和成员管理能力来探测并从节点或资源级别的故障中恢复，以实现群集服务（亦称资源）的最大可用性。

实验环境配置：
1.配置seerver1、server2、server3的网络及高级yum源(4830个软件包)

2.编辑地址解析文件

server1 和server2
3. yum install -y pacemaker pcs corosync ##安装pacemaker pcs corosync
4.设置免密连接

[root@server1 ~]# ssh-keygen    ##创建密钥
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
57:39:aa:29:6a:50:cd:bb:0b:79:89:05:ff:83:f8:c8 root@server1
The key's randomart image is:
+--[ RSA 2048]----+
|                 |
|             .   |
|    .o      +    |
|    .oo    o .   |
|   .  o.S o      |
|  .  =.+ +       |
|   .= =.=        |
|   ..*.. .       |
|   .E o.         |
+-----------------+
[root@server1 ~]# ssh-copy-id server1  ##将公钥和密钥给自己
The authenticity of host 'server1 (172.25.46.1)' can't be established.
ECDSA key fingerprint is f1:b5:0e:f2:ee:55:d3:75:c6:c0:27:c9:d5:31:b1:83.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@server1's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'server1'"
and check to make sure that only the key(s) you wanted were added.

[root@server1 ~]# ssh-copy-id server2   ##将公钥和密钥给 server2 
The authenticity of host 'server2 (172.25.46.2)' can't be established.
ECDSA key fingerprint is 79:9e:f1:d2:e9:35:07:f8:ac:d1:51:ba:f9:a0:d3:19.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@server2's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh 'server2'"
and check to make sure that only the key(s) you wanted were added.

[root@server1 ~]# ssh 172.25.46.2  ##免密连接测试是否成功
Last login: Tue Jun 25 20:40:22 2019 from 172.25.46.250
[root@server2 ~]# logout  ##连接到server2
Connection to 172.25.46.2 closed.

server2
[root@server2 ~]#  yum install -y pacemaker pcs corosync
[root@server2 ~]# ssh-keygen 
[root@server2 ~]# ssh-copy-id server1
[root@server2 ~]# ssh-copy-id server2
[root@server2 ~]#  ssh 172.25.46.1

5.开启并设置开机自启服务，设置密码（server1 和server2密码一致）

[root@server1 ~]# systemctl start pcsd
[root@server1 ~]# systemctl enable pcsd
Created symlink from /etc/systemd/system/multi-user.target.wants/pcsd.service to /usr/lib/systemd/system/pcsd.service.
[root@server1 ~]# id hacluster
uid=189(hacluster) gid=189(haclient) groups=189(haclient)
[root@server1 ~]# passwd hacluster
Changing password for user hacluster.
New password:   #这里设置的westos和server2一致 
BAD PASSWORD: The password is shorter than 8 characters
Retype new password: 
passwd: all authentication tokens updated successfully.

server2
 [root@server2 ~]#systemctl start pcsd
 [root@server2 ~]#  systemctl enable pcsd
 [root@server2 ~]# id hacluster
 [root@server2 ~]# passwd  hacluster  #设置的密码：westos

6.认证server1、server2 ##自任意一个服务机上认证(server1、server2）
yum install -y bash-* ##安装Tab 补齐命令（server1 、server2）

[root@server1 ~]# pcs cluster auth server1 server2   ##认证server1、server2 用 户hacluster   密码 是刚设置的密码westos
Username: hacluster
Password: 
server1: Authorized
server2: Authorized
[root@server1 ~]# pcs cluster setup --name mycluster server1 server2  ##将server1 server2 加到群组
Destroying cluster on nodes: server1, server2...
server1: Stopping Cluster (pacemaker)...
server2: Stopping Cluster (pacemaker)...
server1: Successfully destroyed cluster
server2: Successfully destroyed cluster
[root@server1 ~]# pcs cluster start --all #d打开群组
server1: Starting Cluster...
server2: Starting Cluster...
^[[A[root@server1 ~]# pcs cluster enable --all  #设置开机自启
server1: Cluster Enabled
server2: Cluster Enabled
[root@server1 ~]# pcs status  ##查看群组状态
Cluster name: mycluster
WARNING: no stonith devices and stonith-enabled is not false
Stack: corosync
Current DC: server2 (version 1.1.15-11.el7-e174ec8) - partition with quorum
Last updated: Tue Jun 25 21:39:01 2019		Last change: Tue Jun 25 21:38:07 2019 by hacluster via crmd on server2

2 nodes and 0 resources configured

Online: [ server1 server2 ]

No resources


Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled
[root@server1 ~]# crm_verify -L -V  ##查看组群版本错误
   error: unpack_resources:	Resource start-up disabled since no STONITH resources have been defined
   error: unpack_resources:	Either configure some or disable STONITH with the stonith-enabled option
   error: unpack_resources:	NOTE: Clusters with shared data need STONITH to ensure data integrity
Errors found during check: config not valid
[root@server1 ~]# pcs property set stonith-enabled=false  ##关闭组群服务
[root@server1 ~]# pcs property set no-quorum-policy=ignore  ##忽略投票制（就两个服务，当一个宕机后无法进行投票运行）
[root@server1 ~]# crm_verify -L -V
[root@server1 ~]# pcs resource standards  ##查看资源类型
ocf
lsb
service
systemd

yum install -y httpd ##安装httpd(server1, server2)
server1:

[root@server1 ~]# pcs resource create vip ocf:heartbeat:IPaddr2 ip=172.25.46.100 cidr_netmask=32 op monitor interval=30s  ##创建vip(可以使用命令 pcs stonith deleve vip 删除已经建立的vip)

建立好以上所有步骤检测可以看见如下图 crm_mon

检测服务的转换

[root@server1 ~]# pcs cluster stop server1  ##关闭server1服务
server1: Stopping Cluster (pacemaker)...
server1: Stopping Cluster (corosync)...

[root@server2 ~]# crm_mon  ##检测命令
Connection to the CIB terminated

[root@server1 ~]# pcs resource create apache systemd:httpd op monitor interval=1min ##添加脚本服务
[root@server1 ~]# pcs resource group add apache-group vip apache  #添加资源组
[root@server1 ~]# pcs constraint order vip then apache  #定义服务的运行顺序
Adding vip apache (kind: Mandatory) (Options: first-action=start then-action=start)

yum install -y fence-virt ##安装fence（server1、server2）
mkdir /etc/cluster ##建立一个放钥匙的文件
真机：

[root@foundation46 ~]# yum search fence
[root@foundation46 ~]# yum install -y fence-virtd.x86_64 fence-virtd-libvirt.x86_64 fence-virtd-multicast.x86_64  ##安装fence的包
[root@foundation46 ~]# fence_virtd -c
Interface [virbr0]: br0  ##只改这一处
[root@foundation46 ~]# vim /etc/fence_virt.conf   ##进入该文件查看是否改成br0
[root@foundation46 ~]# mkdir /etc/cluster  #创建一个放钥匙的文件
[root@foundation46 ~]#cd  /etc/cluster 
[root@foundation46 cluster ]# dd if=/dev/urandom of=/etc/cluster/fence_xvm.key bs=128
count=1  ##截取钥匙
[root@foundation46 cluster ]#scp /etc/cluster/fence_xvm.key  root@172.25.46.1:/etc/cluster/
[root@foundation46 cluster ]#scp /etc/cluster/fence_xvm.key  root@172.25.46.2:/etc/cluster/

[root@server1 ~]# systemctl status fence_virted.server
[root@server1 ~]# systemctl start fence_virted.server
[root@server1 ~]# cd /etc/cluster
[root@server1 cluster]#stonith_admin I
[root@server1 cluster]#pcs stonith create vmfence fence_xvm pcmk_host_map=“server1:server1;server2:server2” op monitor intercal=1min