kube-apiserver源码学习-2 debug调试

本文目标：

(1) 创建k8s token，通过postman触发接口

(2) 通过dlv，断电调试apiserver源码

启动k8s本地集群

通过./hack/local-up-cluster.sh启动本地集群后，注入环境变量才能通过cluster/kubectl.sh操作资源

kubectl调取k8s接口时可能存在缓存问题，因此debug时需要用postman调本地k8s集群的6443端口，但是需要先配置token

 

创建k8s token

通过创建ServiceAccount，绑定足够权限的ClusterRole，下发接口时带上ServiceAccount的token，对应的yaml为

 

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: k8s-api-sa
  namespace: default
  
---
apiVersion: v1
kind: Secret
metadata:
  name: k8s-api-secret
  namespace: default
  annotations:
    kubernetes.io/service-account.name: "k8s-api-sa"
type: kubernetes.io/service-account-token

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: k8s-api
subjects:
- kind: ServiceAccount
  name: k8s-api-sa
  namespace: default
roleRef:
  kind: ClusterRole
  # 内置的k8s 集群角色，超级管理员
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

通过/cluster/kubectl.sh apply 上述的yaml

获取token

./cluster/kubectl.sh get secret k8s-api-secret -o jsonpath='{.data.token}' | base64 --decode

获取ca

./cluster/kubectl.sh get secret k8s-api-secret -o jsonpath='{.data.ca\.crt}' | base64 --decode

配置postman下发时的ca和token

正确设置之后，能够正确返回数据

源码单独启动apiserver组件

在local-up-cluster.sh启动本地集群后，查看apiserver进程，获取启动参数

记录上述的参数后，kill掉当前这个apiserver进程，后面改用源码运行

vscode远程调试apiserver源码

vscode通过remote ssh插件，远程连接虚机上的代码目录，并配置好免密登录(相关配置过程本文不再赘述)。打开vscode的运行和调试，会提示安装go dlv调试工具，也可以手动在虚机上安装dlv，如果dlv安装时下载依赖失败，配置go代理还失败情况下可通过go clean --modcache清除依赖重新安装试下。

对应launch.json供参考，其中args是上面apiserver运行时的参数

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "apiserver调试",
            "type": "go",
            "request": "launch",
            "args": [
                "--cloud-provider=",
                "--cloud-config=",
                "--v=3",
                "--vmodule=",
                "--audit-policy-file=/tmp/kube-audit-policy-file",
                "--audit-log-path=/tmp/kube-apiserver-audit.log",
                "--authorization-webhook-config-file=",
                "--authentication-token-webhook-config-file=",
                "--cert-dir=/var/run/kubernetes",
                "--egress-selector-config-file=/tmp/kube_egress_selector_configuration.yaml",
                "--client-ca-file=/var/run/kubernetes/client-ca.crt",
                "--kubelet-client-certificate=/var/run/kubernetes/client-kube-apiserver.crt",
                "--kubelet-client-key=/var/run/kubernetes/client-kube-apiserver.key",
                "--service-account-key-file=/tmp/kube-serviceaccount.key",
                "--service-account-lookup=true",
                "--service-account-issuer=https://kubernetes.default.svc",
                "--service-account-jwks-uri=https://kubernetes.default.svc/openid/v1/jwks",
                "--service-account-signing-key-file=/tmp/kube-serviceaccount.key",
                "--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,Priority,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota,NodeRestriction",
                "--disable-admission-plugins=",
                "--admission-control-config-file=",
                "--bind-address=0.0.0.0",
                "--secure-port=6443",
                "--tls-cert-file=/var/run/kubernetes/serving-kube-apiserver.crt",
                "--tls-private-key-file=/var/run/kubernetes/serving-kube-apiserver.key",
                "--storage-backend=etcd3",
                "--storage-media-type=application/vnd.kubernetes.protobuf",
                "--etcd-servers=http://127.0.0.1:2379",
                "--service-cluster-ip-range=10.0.0.0/24",
                "--feature-gates=AllAlpha=false",
                "--external-hostname=localhost",
                "--requestheader-username-headers=X-Remote-User",
                "--requestheader-group-headers=X-Remote-Group",
                "--requestheader-extra-headers-prefix=X-Remote-Extra-",
                "--requestheader-client-ca-file=/var/run/kubernetes/request-header-ca.crt",
                "--requestheader-allowed-names=system:auth-proxy",
                "--proxy-client-cert-file=/var/run/kubernetes/client-auth-proxy.crt",
                "--proxy-client-key-file=/var/run/kubernetes/client-auth-proxy.key",
                "--cors-allowed-origins=\"/127.0.0.1(:[0-9]+)?$,/localhost(:[0-9]+)?$\""
            ],
            "program": "cmd/kube-apiserver/apiserver.go",
        }
    ]
}

这样我们就能通过debug方式运行apiserver源码

运行之后能够正常获取k8s资源

当设置断点之后，通过postman下发之后，能被拦截到