Kubernetes学习笔记一《搭建Kubernetes本地集群环境》

目的

    在安装了win10操作系统个人笔记本电脑上搭建Kubernetes集群实验环境，做为Kubernetes学习之用。

虚拟环境

    Oracle VM VirtualBox”建立三台虚拟机，虚拟机安装CentOs（http://mirrors.aliyun.com/centos/7.9.2009/isos/x86_64/CentOS-7-x86_64-DVD-2009.iso），每个虚拟机设置2块网卡，一块网卡通过NAT连接外网，另一块用于三台虚拟机之间通信。

软件	版本
CentOS	Linux version 3.10.0-957.el7.x86_64
Kubernetes	1.23.16

一、配置虚拟机与安装所需要的软件

1.1 配置VirtualBox的NAT网络，VirtualBox 菜单：管理->全局设定->网络->增加一新的NAT网络

1.2 配置VM的网卡

   每台VM设两块网卡，一块为NAT网络，一块为仅主机，如下图

  

1.3 安装操作系统centos

        选择最小化安装，主机名设为master.k8s

     

1.4  安装dockerce(Install Docker Engine on CentOS | Docker Documentation 和Kubernetes

      更改repo

     解决containerd.io-1.6.33-3.1.el7.x86_64: [Errno 256] No more mirrors to try.

1. yum install epel-release

2. yum makecache

3. yum install container-selinux

repo_file=/etc/yum.repos.d/CentOS-Base.repo
cp ${repo_file} ~/CentOS-Base.repo.backup
sudo sed -i s/#baseurl/baseurl/ ${repo_file}
sudo sed -i s/mirrorlist.centos.org/vault.centos.org/ ${repo_file}
sudo sed -i s/mirror.centos.org/vault.centos.org/ ${repo_file}
sudo yum clean all

        安装完成后登录，禁用两个安全功能：SELinux以及防火墙

•       禁用SELinux，执行以下命令

      修改 /etc/selinux/config文件，把SELINUX=enforcing行改为SELINUX=disabled

•      禁用防火墙

 # systemctl disable firewalld && systemctl stop firewalld

•     加入kubernetes阿里云的源

  # vi /etc/yum.repos.d/kubernetes.repo
 [kubernetes]
 name=Kubernetes
 baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
 enabled=1
 gpgcheck=0

• 安装DOCKER, KUBELET, KUBEADM, KUBECTL, AND KUBERNETES-CNI

# yum install -y kubelet-1.23.16-0 kubeadm-1.23.16-0 kubectl-1.23.16-0 kubernetes-cni-1.1.1-0 --nogpgcheck

# systemctl enable docker && systemctl start docker

# systemctl enable kubelet 


# curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io

• 启用net.bridge.bridge-nf-call-iptables内核选项

#  sysctl -w net.bridge.bridge-nf-call-iptables=1

# echo "net.bridge.bridge-nf-call-iptables=1" > /etc/sysctl.d/k8s.conf

#  echo "1" > /proc/sys/net/ipv4/ip_forward

• 禁用 SWAP

# swapoff -a && sed -i '/ swap / s/^/#/' /etc/fstab

•  docker 加入systemd ,解决Failed connect to localhost:10248
• DockerHub国内镜像源列表（2024年6月18日 亲测可用） - 资源荟萃 - LINUX DO docker hub国内镜像

vi /etc/docker/daemon.json 加入
{
"exec-opts": ["native.cgroupdriver=systemd"]
}

systemctl daemon-reload
 systemctl restart docker
 systemctl restart kubelet

•   执行以下操作，解决[ERROR CRI]: container runtime is not running: output: E0606

rm -rf /etc/containerd/config.toml

systemctl restart containerd

1.5   复制VM

•       关闭VM         

# shutdown now

• VirtualBox 界面选择复制，复制2个虚拟机：node1.k8s,node2.k8s

二、配置三台虚拟机环境

1.    启动 master.k8s,node1.k8s,node2.k83
2.    设置node1.k8s, node2.k8s的主机名

# hostnamectl --static set-hostname node1.k8s

# hostnamectl --static set-hostname node2.k8s

   3.  修改三台虚拟机网络配置 

        三台虚拟机的内网配置如下：

主机名	IP
master.k8s	192.168.56.101
node1.k8s	192.168.56.102
node2.k83	192.168.56.103

  # cat /etc/sysconfig/network-scripts/ifcfg-enp0s8 

  TYPE=Ethernet
  PROXY_METHOD=none
  BROWSER_ONLY=no
  BOOTPROTO=none
  DEFROUTE=yes
  IPV4_FAILURE_FATAL=no
  IPV6INIT=yes
  IPV6_AUTOCONF=yes
  IPV6_DEFROUTE=yes
  IPV6_FAILURE_FATAL=no
  IPV6_ADDR_GEN_MODE=stable-privacy
  NAME=enp0s8
  UUID=9993cb88-7de1-4038-bd36-f64e681f4d54
  DEVICE=enp0s8
  ADDRES=192.168.56.101
  IPADDR=192.168.56.101
  ONBOOT=yes

 4. 设置三台VM的hosts文件，

# cat /etc/hosts
192.168.56.101 master.k8s
192.168.56.102 node1.k8s
192.168.56.103 node2.k8s

三、配置Kubernetes

         3.1  拉取docker images ，编写脚本拉取所需要的image,用kubeadm config images list 查看当前kube版本

 # vi  pullimages.sh

#!/bin/bash
images=(
  kube-apiserver:v1.23.16
  kube-controller-manager:v1.23.16
  kube-scheduler:v1.23.16
  kube-proxy:v1.23.16
  pause:3.6
  etcd:3.5.6-0
  coredns:v1.8.6
)

for imageName in ${images[@]}; do
  docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
  docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName registry.k8s.io/$imageName
  docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName
done

3.3 在虚拟机master.k8s用kubeadm配置master  

# sh pullimages.sh

# kubeadm init --kubernetes-version v1.23.16 --apiserver-advertise-address 1 92.168.56.101 --pod-network-cidr 10.244.0.0/16

[init] Using Kubernetes version: v1.14.0
[preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Activating the kubelet service


Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.56.101:6443 --token pllbqq.at7ky0jtv4exkt1h \
    --discovery-token-ca-cert-hash sha256:aeef41f9631853a270a7a4640e6332cdad3ba1f5fefef5e158502e9509375b47

•  建立和修改所需要的目录

# mkdir -p $HOME/.kube
# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
# sudo chown $(id -u):$(id -g) $HOME/.kube/config

3.4 配置虚拟机node1.k8s,与node2.k8s

# vi pullimages.sh

#!/bin/bash                                                                                          
images=(                                                                                             
   kube-proxy:v1.23.16                                                                               
   pause:3.6                                                                                         
)                                                                                                    
for imageName in ${images[@]}; do                                                                    
   docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName                        
   docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName 
    registry.k8s.io/$imageName                                                                                                
  docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName                          
done                                                                                                 

• 加入集群

# kubeadm join 192.168.56.101:6443 --token pllbqq.at7ky0jtv4exkt1h \
    --discovery-token-ca-cert-hash sha256:aeef41f9631853a270a7a4640e6332cdad3ba1f5fefef5e158502e9509375b47

四、配置Kubernetes网络

4.1  master主机安装Weave Net container networking plugin

    

kubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml

4.2 查询节点情况

[root@master ~]# kubectl get node
NAME         STATUS   ROLES    AGE    VERSION
master.k8s   Ready    master   178m   v1.14.0
node1.k8s    Ready    <none>   162m   v1.14.0
node2.k8s    Ready    <none>   113m   v1.14.0

[root@master ~]# kubectl get pods -n kube-system -l name=weave-net
NAME              READY   STATUS    RESTARTS   AGE
weave-net-9vcqs   2/2     Running   1          156m
weave-net-c5gp2   2/2     Running   2          156m
weave-net-dh48f   2/2     Running   0          108m

五、解决token过期

  master节点初始化生成的token有有效期，过期后要加入新的节点要重新生成。

5.1  生成一条永久有效的token

 # kubeadm token create --ttl 0
 # kubeadm token list

5. 2 获取ca证书sha256编码hash值

# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'

5.3  加入节点

# kubeadm join 10.167.11.153:6443 --token o4avtg.65ji6b778nyacw68 --discovery-token-ca-cert-hash sha256:2cc3029123db737f234186636330e87b5510c173c669f513a9c0e0da395515b0