k8s证书过期问题处理

k8s证书过期问题处理

1、文档目的

处理k8s证书过期问题，使用 kubelet get node 后报错： x509: certificate has expired or is not yet valid ，提示证书过期。

2、系统要求

kubernetes：V1.19.10

3、修改过期时间

如果不需要自定义证书时间，可以直接下载：kubeadm，从3.4开始执行即可，默认证书过期时间为99年。

3.1 查看当前证书过期时间

[root@master ~]# kubeadm alpha certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 May 21, 2025 06:29 UTC   356d                                    no      
apiserver                  May 21, 2025 06:29 UTC   356d            ca                      no      
apiserver-etcd-client      May 21, 2025 06:29 UTC   356d            etcd-ca                 no      
apiserver-kubelet-client   May 21, 2025 06:29 UTC   356d            ca                      no      
controller-manager.conf    May 21, 2025 06:29 UTC   356d                                    no      
etcd-healthcheck-client    May 21, 2025 06:29 UTC   356d            etcd-ca                 no      
etcd-peer                  May 21, 2025 06:29 UTC   356d            etcd-ca                 no      
etcd-server                May 21, 2025 06:29 UTC   356d            etcd-ca                 no      
front-proxy-client         May 21, 2025 06:29 UTC   356d            front-proxy-ca          no      
scheduler.conf             May 21, 2025 06:29 UTC   356d                                    no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      May 19, 2034 06:29 UTC   9y              no      
etcd-ca                 May 19, 2034 06:29 UTC   9y              no      
front-proxy-ca          May 19, 2034 06:29 UTC   9y              no  

3.2 下载并修改k8s源码

找到cmd/kubeadm/app/constants/constants.go文件，并修改CertificateValidity为time.Hour * 24 * 365 * 99，其中99为你想修改的证书日期，可以是其他年份。

const (
    // KubernetesDir is the directory Kubernetes owns for storing various configuration files
    KubernetesDir = "/etc/kubernetes"
    // ManifestsSubDirName defines directory name to store manifests
    ManifestsSubDirName = "manifests"
    // TempDirForKubeadm defines temporary directory for kubeadm
    // should be joined with KubernetesDir.
    TempDirForKubeadm = "tmp"

    // CertificateValidity defines the validity for all the signed certificates generated by kubeadm
    CertificateValidity = time.Hour * 24 * 365

    // CACertAndKeyBaseName defines certificate authority base name
    CACertAndKeyBaseName = "ca"
    // CACertName defines certificate name
    CACertName = "ca.crt"
    // CAKeyName defines certificate name
    CAKeyName = "ca.key"
    ...
}

3.3 编译kubeadm

# 加载go运行环境并启动
docker load -i golang.tar.gz
docker run -itd --name golang golang

# 解压k8s源码
unzip kubernetes-release-1.19.zip

# 复制代码到容器中
docker cp kubernetes-release-1.19/kubernetes-release-1.19/ golang:/opt/

# 进入容器
docker exec -it golang bash

# 安装编译工具
apt update && apt-get install rsync

# 开始编译
cd /opt/kubernetes-release-1.19 && make WHAT=cmd/kubeadm

# 退出容器，并复制编译好的kubeadm
docker cp golang:/opt/kubernetes-release-1.19/_output/local/bin/linux/amd64/kubeadm ./

3.4 重新生成证书

# 备份
cp /usr/bin/kubeadm{,.bak20240520}
cp -r /etc/kubernetes/pki{,.bak20240520}

# 替换kubeadm
cp ./kubeadm /usr/bin/kubeadm

# 生成新的证书
cd /etc/kubernetes/pki
kubeadm alpha certs renew all

输出如下：

[root@master pki]# kubeadm alpha certs renew all
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'

certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed

刷新配置：

 cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
 docker ps |grep kube-apiserver|grep -v pause|awk '{print $1}'|xargs -i docker restart {}

 docker ps |grep kube-controller-manage|grep -v pause|awk '{print $1}'|xargs -i docker restart {}

 docker ps |grep kube-scheduler|grep -v pause|awk '{print $1}'|xargs -i docker restart {}

验证：

kubeadm alpha certs check-expiration

输出如下：

[root@master pki]# kubeadm alpha certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 May 07, 2123 04:34 UTC   98y                                     no      
apiserver                  May 07, 2123 04:34 UTC   98y             ca                      no      
apiserver-etcd-client      May 07, 2123 04:34 UTC   98y             etcd-ca                 no      
apiserver-kubelet-client   May 07, 2123 04:34 UTC   98y             ca                      no      
controller-manager.conf    May 07, 2123 04:34 UTC   98y                                     no      
etcd-healthcheck-client    May 07, 2123 04:34 UTC   98y             etcd-ca                 no      
etcd-peer                  May 07, 2123 04:34 UTC   98y             etcd-ca                 no      
etcd-server                May 07, 2123 04:34 UTC   98y             etcd-ca                 no      
front-proxy-client         May 07, 2123 04:34 UTC   98y             front-proxy-ca          no      
scheduler.conf             May 07, 2123 04:34 UTC   98y                                     no      

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      May 19, 2034 06:29 UTC   9y              no      
etcd-ca                 May 19, 2034 06:29 UTC   9y              no      
front-proxy-ca          May 19, 2034 06:29 UTC   9y              no

检查集群状态：

[root@master pki]# kubectl get nodes
NAME     STATUS   ROLES    AGE   VERSION
master   Ready    master   8d    v1.19.10
node1    Ready    <none>   8d    v1.19.10
node2    Ready    <none>   8d    v1.19.10

[root@master pki]# kubectl get po -A
NAMESPACE     NAME                                      READY   STATUS    RESTARTS   AGE
default       nfs-client-provisioner-5ddcc4547d-9rbs9   1/1     Running   0          35m
default       nfs-client-provisioner-5ddcc4547d-fgwk6   1/1     Running   1          8d
kube-system   calico-etcd-8dbfx                         1/1     Running   0          8d
kube-system   calico-kube-controllers-9b4cd4c97-wrf4x   1/1     Running   0          8d
kube-system   calico-node-dtnng                         1/1     Running   0          8d
kube-system   calico-node-s52fp                         1/1     Running   0          8d
kube-system   calico-node-wzzdp                         1/1     Running   0          8d
kube-system   coredns-f9fd979d6-8krvk                   1/1     Running   0          8d
kube-system   coredns-f9fd979d6-tv56g                   1/1     Running   0          8d
kube-system   etcd-master                               1/1     Running   0          8d
kube-system   heapster-55bb46945c-bhqsc                 1/1     Running   0          8d
kube-system   kube-apiserver-master                     1/1     Running   0          8d
kube-system   kube-controller-manager-master            1/1     Running   2          8d
kube-system   kube-proxy-cz4w7                          1/1     Running   0          8d
kube-system   kube-proxy-g9vvk                          1/1     Running   0          8d
kube-system   kube-proxy-nd888                          1/1     Running   0          8d
kube-system   kube-scheduler-master                     1/1     Running   2          8d
kube-system   kubernetes-dashboard-75bf8468f8-g6r88     1/1     Running   0          8d
kube-system   monitoring-grafana-649955cdf7-52rcv       1/1     Running   0          8d
kube-system   monitoring-influxdb-5cf7f5bf76-2hqhv      1/1     Running   0          8d

到此证书修改完成。