Kubernetes 集群部署之ETCD集群部署（二）

目录

ETCD简介：

官方文档：

1、基础环境准备：

2、创建集群 CA 与 Certificates （master）

下载证书工具

3、建立/etc/etcd/ssl，然后进入目录完成以下操作

4、创建ETCD数据目录（所有节点）

5、创建CA证书

切换目录

编辑CA配置文件

CA签名请求文件

生成 CA 证书和私钥

6、创建ETCD证书

etcd证书请求文件

生成etcd证书和私钥

7、分发证书和私钥到etcd各个节点

查看

8、ETCD安装与设置（所有节点）

下载

解压

安装

创建etcd的systemd unit文件

拷贝启动文件及配置文件到node1，node2并修改对应的IP地址

9、启动（所有节点）

10、验证

---

---

ETCD简介：

etcd是Kubernetes提供默认的存储系统，保存所有集群数据，使用时需要为etcd数据提供备份计划。

官方文档：

Kubernetes 1.8.x 全手动安装教程_Kubernetes中文社区

1、基础环境准备：

Kubernetes 集群部署之基础环境准备_集群部署基本的环境_abel_dwh的博客-优快云博客

2、创建集群 CA 与 Certificates （master）

在开始安装 Kubernetes 之前，需要先将一些必要系统创建完成，其中 Etcd 就是 Kubernetes 最重要的一环，Kubernetes 会将大部分信息储存于 Etcd 上，来提供给其他节点索取，以确保整个集群运作与沟通正常

• 下载证书工具

[root@master ~]# wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64
[root@master ~]# wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64
[root@master ~]# wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64
[root@master ~]# chmod +x cfssl_linux-amd64 cfssljson_linux-amd64 cfssl-certinfo_linux-amd64
[root@master ~]# mv cfssl_linux-amd64 /usr/local/bin/cfssl
[root@master ~]# mv cfssljson_linux-amd64 /usr/local/bin/cfssljson
[root@master ~]# mv cfssl-certinfo_linux-amd64 /usr/bin/cfssl-certinfo

3、建立/etc/etcd/ssl，然后进入目录完成以下操作

在这部分，将会需要产生 client 与 server 的各组件 certificates，并且替 Kubernetes admin user 产生 client 证书。

[root@master ~]# mkdir -p ~/TLS/{etcd,k8s}

4、创建ETCD数据目录（所有节点）

[root@master ~]#  mkdir /var/lib/etcd -p

5、创建CA证书

• 切换目录

[root@master ssl]# cd TLS/etcd

• 编辑CA配置文件

[root@master etcd]# cat ca-config.json 
{ 
"signing": { 
"default": { 
"expiry": "87600h" },
"profiles": { 
"www": { 
"expiry": "87600h", 
"usages": [ 
"signing", 
"key encipherment", 
"server auth", 
"client auth" 
] 
}
} 
} 
}

• CA签名请求文件

[root@master etcd]# cat ca-csr.json 
{ 
"CN": "etcd CA", 
"key": { 
"algo": "rsa", 
"size": 2048 
},
"names": [ 
{ 
"C": "CN", 
"L": "Beijing", 
"ST": "Beijing" 
} 
] 
}

• 生成 CA 证书和私钥

[root@master etcd]# cfssl gencert -initca ca-csr.json | cfssljson -bare ca -
[root@master etcd]# ls *pem
ca-key.pem ca.pem

6、创建ETCD证书

• etcd证书请求文件

[root@master etcd]# cat server-csr.json 
{ 
"CN": "etcd", 
"hosts": [ 
"192.168.44.128", 
"192.168.44.129", 
"192.168.44.130" 
],
"key": { 
"algo": "rsa", 
"size": 2048 
},
"names": [ 
{ 
"C": "CN", 
"L": "BeiJing", 
"ST": "BeiJing" 
} 
] 
}

• 生成etcd证书和私钥

[root@k8s-master etcd]# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=www server-csr.json |cfssljson -bare server
[root@k8s-master etcd]# ls server*pem
server-key.pem server.pem

7、分发证书和私钥到etcd各个节点

[root@master etcd]# mkdir /opt/etcd/{bin,cfg,ssl} -p
[root@master etcd]# cp *.pem /opt/etcd/ssl/
[root@master etcd]# for node_ip in node1 node2;do scp -r *.pem root@$node_ip:/opt/etcd/ssl/; done 
The authenticity of host 'node1 (192.168.44.130)' can't be established.
ECDSA key fingerprint is SHA256:XwTx4YllGruncWe10QvPnbxbhdVDsVo8nDROdO4ToKg.
ECDSA key fingerprint is MD5:ba:71:78:e3:cb:6a:38:e2:d6:96:25:c2:f5:5d:9e:1c.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'node1,192.168.44.130' (ECDSA) to the list of known hosts.
root@node1's password: 

• 查看

[root@k8s-node1 ~]# ll /opt/etcd/ssl
total 16
-rw------- 1 root root 1675 May 21 14:49 ca-key.pem
-rw-r--r-- 1 root root 1265 May 21 14:49 ca.pem
-rw------- 1 root root 1679 May 21 14:49 server-key.pem
-rw-r--r-- 1 root root 1338 May 21 14:49 server.pem 


[root@k8s-node2 ~]# ll /opt/etcd/ssl
total 16
-rw------- 1 root root 1675 May 21 14:49 ca-key.pem
-rw-r--r-- 1 root root 1265 May 21 14:49 ca.pem
-rw------- 1 root root 1679 May 21 14:49 server-key.pem
-rw-r--r-- 1 root root 1338 May 21 14:49 server.pem

8、ETCD安装与设置（所有节点）

• 下载

[root@master ~]# wget https://github.com/coreos/etcd/releases/download/v3.3.2/etcd-v3.3.2-linux-amd64.tar.gz

• 解压

[root@master ~]# tar -zf etcd-v3.3.2-linux-amd64.tar.gz

• 安装

[root@master ~]# mv etcd-v3.3.2-linux-amd64/etcd /opt/etcd/bin/
[root@master ~]# mv etcd-v3.3.2-linux-amd64/etcdctl /usr/bin
 

创建ETCD配置文件

[root@master etcd]# cat /opt/etcd/cfg/etcd.conf 
#[Member] 
ETCD_NAME="etcd-1" 
ETCD_DATA_DIR="/var/lib/etcd/default.etcd" 
ETCD_LISTEN_PEER_URLS="https://192.168.44.128:2380" 
ETCD_LISTEN_CLIENT_URLS="https://192.168.44.128:2379" 
#[Clustering] 
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.44.128:2380" 
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.44.128:2379" 
ETCD_INITIAL_CLUSTER="etcd-1=https://192.168.44.128:2380,etcd- 
2=https://192.168.44.129:2380,etcd-3=https://192.168.44.130:2380" 
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster" 
ETCD_INITIAL_CLUSTER_STATE="new"

• 创建etcd的systemd unit文件

[root@master etcd]# cat /usr/lib/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
[Service]
Type=notify
EnvironmentFile=/opt/etcd/cfg/etcd.conf
ExecStart=/opt/etcd/bin/etcd --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem --peer-key-file=/opt/etcd/ssl/server-key.pem --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem --logger=zap
Restart=on-failure
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target

• 拷贝启动文件及配置文件到node1，node2并修改对应的IP地址

[root@master ~]# for i in node1 node2
> do
> scp -r /usr/lib/systemd/system/etcd.service root@$i:/usr/lib/systemd/system/
> done



[root@master ~]# for i in node1 node2
> do
> scp -r /opt/etcd/cfg/etcd.conf  root@$i:/opt/etcd/cfg/etcd.conf 
> done

9、启动（所有节点）

[root@master ~]# systemctl daemon-reload;systemctl enable etcd;systemctl start etcd
[root@master etcd]# ps -ef |grep etcd
root       1160      1  1 14:48 ?        00:00:51 /opt/etcd/bin/etcd --cert-file=/opt/etcd/ssl/server.pem --key-file=/opt/etcd/ssl/server-key.pem --peer-cert-file=/opt/etcd/ssl/server.pem --peer-key-file=/opt/etcd/ssl/server-key.pem --trusted-ca-file=/opt/etcd/ssl/ca.pem --peer-trusted-ca-file=/opt/etcd/ssl/ca.pem --logger=zap

10、验证

[root@master ~]# ETCDCTL_API=3 etcdctl --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem --endpoints="https://192.168.44.128:2379,https://192.168.44.129:2379,https://192.168.44.130:2379" endpoint health
https://192.168.44.129:2379 is healthy: successfully committed proposal: took = 42.670255ms
https://192.168.44.130:2379 is healthy: successfully committed proposal: took = 45.641195ms
https://192.168.44.128:2379 is healthy: successfully committed proposal: took = 23.560676ms

软件包地址：https://download.youkuaiyun.com/download/abel_dwh/18938145

ETCD集群部署完毕。