1. 准备环境
本次部署采用Centos 7.9系统,cpu最小>2核
主机名 | IP | mem |
---|---|---|
k8s-master01 | 192.168.10.3 | 3G |
k8s-node01 | 192.168.10.4 | 4G |
k8s-node02 | 192.168.10.5 | 4G |
1.1 修改hostname主机名并添加hosts解析
hostnamectl set-hostname k8s-master01
hostnamectl set-hostname k8s-node01
hostnamectl set-hostname k8s-node02
1.2 安装依赖包
yum -y install conntrack ntpdate ntp ipvsadm ipset jq iptables curl sysstat libseccomp wget vim net-tools git
1.3 设置防火墙为iptables并设置空规则
systemctl stop firewalld && systemctl disable firewalld
yum -y install iptables-serives && systemctl start iptables && systemctl enable iptables && iptables -F && systemctl iptables save
swapoff -a
1.4 关闭seliunx
1.5. 调整内核参数,对于k8s
cat > kubernetes.conf <<EOF
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
net.ipv4.tcp_tw_recycle=0
vm.swappiness=0
# 禁止使用 swap 空间,只有当系统 OOM 时才允许使用它
vm.overcommit_memory=1
# 不检查物理内存是否够用
vm.panic_on_oom=0
# 开启 OOM
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720
EOF
cp kubernetes.conf /etc/sysctl.d/kubernetes.conf
sysctl -p /etc/sysctl.d/kubernetes.conf
1.6. 调整时区
# 设置系统时区为中国/上海
timedatectl set-timezone Asia/Shanghai
# 将当前的 UTC 时间写入硬件时钟
timedatectl set-local-rtc 0
# 重启依赖于系统时间的服务
systemctl restart rsyslog
systemctl restart crond
1.7 关闭系统不需要的服务
systemctl stop postfix && systemctl disable postfix
1.8 升级系统内核为 4.44
CentOS 7.x 系统自带的 3.10.x 内核存在一些 Bugs,导致运行的 Docker、Kubernetes 不稳定,例如: rpm -Uvhhttp://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-3.el7.elrepo.noarch.rpm
# 安装完成后检查 /boot/grub2/grub.cfg 中对应内核 menuentry 中是否包含 initrd16 配置,如果没有,再安装一次!
yum --enablerepo=elrepo-kernel install -y kernel-lt
# 设置开机从新内核启动
grub2-set-default 'CentOS Linux (4.4.189-1.el7.elrepo.x86_64) 7 (Core)'
1.9 kube-proxy开启ipvs的前置条件
modprobe br_netfilter
cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF
chmod 755 /etc/sysconfig/modules/ipvs.modules && bash /etc/sysconfig/modules/ipvs.modules &&lsmod | grep -e ip_vs -e nf_conntrack_ipv4
2. 所有节点安装 Docker/kubeadm/kubelet
Kubernetes 默认 CRI(容器运行时)为 Docker,因此先安装 Docker。
2.1 安装Docker 软件
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum install -y docker-ce
## 创建 /etc/docker 目录
mkdir /etc/docker
# 配置 daemon.
cat > /etc/docker/daemon.json <<EOF
{
"registry-mirrors": ["https://b9pmyelo.mirror.aliyuncs.com"]
}
EOF
mkdir -p /etc/systemd/system/docker.service.d
# 重启docker服务
systemctl daemon-reload && systemctl restart docker && systemctl enable docker
2.2 添加阿里云yum软件源
cat > /etc/yum.repos.d/kubernets.repo <<-EOF[kubernets]name=kubernetsbaseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64enabled=1gpgcheck=0repo_gpgcheck=0gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpghttps://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpgEOF
2.3 安装 Kubeadm(v1.18)
yum -y install kubeadm-1.18.0 kubectl-1.18.0 kubelet-1.18.0
systemctl enable kubelet.service
2.4 部署 Kubernetes Master
(1) 在192.168.10.3(Master)执行
kubeadm init --apiserver-advertise-address=192.168.10.3 --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.18.0 --service-cidr=10.96.0.0/12 --pod-network-cidr=10.244.0.0/16 | tee kube-init.conf
由于默认拉取镜像地址k8s.gcr.io国内无法访问。这里指定阿里云镜像仓库地址
(2)使用 kubectl 工具:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
2.5 加入 Kubernetes Node
(1) 在所有node执行
向集群添加新节点,执行在 kubeadm init 输出的 kubeadm join 命令:
kubeadm join 192.168.10.3:6443 --token zozs4l.nq865i7mmuitorbp \ --discovery-token-ca-cert-hash sha256:15b7774f74aff096af1fc5e4e8d19d4b8d140564daf7a7bd8300f509dee3c86d
默认token有效为24小时,当过期之后该token就不能使用了,这是就需要重新创建token,操作如下:
kubeadm token create --print-join-command
2.6 安装 Pod 网络插件(flannel)
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
检查
[root@k8s-master01 ~]# kubectl get all -n kube-system
NAME READY STATUS RESTARTS
AGEpod/coredns-7ff77c879f-sjswq 1/1 Running 0 14m
pod/coredns-7ff77c879f-szzwc 1/1 Running 0 14m
pod/etcd-k8s-master01.didi.cn 1/1 Running 0 14m
pod/kube-apiserver-k8s-master01.didi.cn 1/1 Running 0 14m
pod/kube-controller-manager-k8s-master01.didi.cn 1/1 Running 0 14
mpod/kube-flannel-ds-rj2ld 1/1 Running 0 47s
pod/kube-flannel-ds-spn2z 1/1 Running 0 47s
pod/kube-flannel-ds-v8qhz 1/1 Running 0 47s
pod/kube-proxy-jlkd4 1/1 Running 0 3m
59spod/kube-proxy-nbljh 1/1 Running 0 14m
pod/kube-proxy-tclbr 1/1 Running 0 4m16s
pod/kube-scheduler-k8s-master01.didi.cn 1/1 Running 0 14m
[root@k8s-master01 ~]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8s-master01.didi.cn Ready master 14m v1.18.0
k8s-node01.didi.cn Ready <none> 4m20s v1.18.0
k8s-node02.didi.cn Ready <none> 4m3s v1.18.0
2.7 测试 kubernetes 集群
[root@k8s-master01 ~]# kubectl create deployment nginx --image=nginx
[root@k8s-master01 ~]# kubectl expose deployment nginx --port=80 --type=NodePort service/nginx exposed
[root@k8s-master01 ~]# kubectl get pod,svc
NAME READY STATUS RESTARTS AGE
pod/nginx-f89759699-8z68t 1/1 Running 0 2m35s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 20m
service/nginx NodePort 10.105.156.53 <none> 80:30957/TCP 9s
访问http://nodeIP:NodePort获取的端口