外部prometheus+grafana监控k8s（附所需文件资源）——保姆级教程

背景：我们在上一篇讲的部署好普罗米修斯后得到一个监控k8s集群的需求，但因为prometheus是用docker部署在k8s外部，所以无法直接获取k8s的内部pod信息，这时候我们有2种办法来取得监控k8s。

普罗米修斯部署文档链接：docker部署prometheus和grafana监控容器和主机-优快云博客

第一种：使用集成的kube-state-metrics部署到k8s内部，获取pod信息，对接prometheus

第二种：通过授权，Cadvisor内置在kubelet当中,通过kubelet暴露的api去访问的，2种办法我都会讲，但是推荐使用第一种，简洁方便。

第一种：部署kube-state-metrics

准备文件和镜像：

yaml文件下载地址：kube-state-metrics/examples/standard at main · kubernetes/kube-state-metrics · GitHub

这里你们不用下载，我下面直接配置好给你们，在服务器上vim保存就行

kube-state-metrics最新版镜像下载地址：

链接: https://caiyun.139.com/m/i?2jexBsdYH3Em9  提取码:zd7r  

编写配置文件

1.    vim cluster-role.yaml文件

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.12.0
  name: kube-state-metrics
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - secrets
  - nodes
  - pods
  - services
  - serviceaccounts
  - resourcequotas
  - replicationcontrollers
  - limitranges
  - persistentvolumeclaims
  - persistentvolumes
  - namespaces
  - endpoints
  verbs:
  - list
  - watch
- apiGroups:
  - apps
  resources:
  - statefulsets
  - daemonsets
  - deployments
  - replicasets
  verbs:
  - list
  - watch
- apiGroups:
  - batch
  resources:
  - cronjobs
  - jobs
  verbs:
  - list
  - watch
- apiGroups:
  - autoscaling
  resources:
  - horizontalpodautoscalers
  verbs:
  - list
  - watch
- apiGroups:
  - authentication.k8s.io
  resources:
  - tokenreviews
  verbs:
  - create
- apiGroups:
  - authorization.k8s.io
  resources:
  - subjectaccessreviews
  verbs:
  - create
- apiGroups:
  - policy
  resources:
  - poddisruptionbudgets
  verbs:
  - list
  - watch
- apiGroups:
  - certificates.k8s.io
  resources:
  - certificatesigningrequests
  verbs:
  - list
  - watch
- apiGroups:
  - discovery.k8s.io
  resources:
  - endpointslices
  verbs:
  - list
  - watch
- apiGroups:
  - storage.k8s.io
  resources:
  - storageclasses
  - volumeattachments
  verbs:
  - list
  - watch
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - mutatingwebhookconfigurations
  - validatingwebhookconfigurations
  verbs:
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - networkpolicies
  - ingressclasses
  - ingresses
  verbs:
  - list
  - watch
- apiGroups:
  - coordination.k8s.io
  resources:
  - leases
  verbs:
  - list
  - watch
- apiGroups:
  - rbac.authorization.k8s.io
  resources:
  - clusterrolebindings
  - clusterroles
  - rolebindings
  - roles
  verbs:
  - list
  - watch

2.  vim  cluster-role-binding.yaml文件

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.12.0
  name: kube-state-metrics
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-state-metrics
subjects:
- kind: ServiceAccount
  name: kube-state-metrics
  namespace: kube-system

3.  vim  deployment.yaml文件

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.12.0
  name: kube-state-metrics
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: kube-state-metrics
  template:
    metadata:
      labels:
        app.kubernetes.io/component: exporter
        app.kubernetes.io/name: kube-state-metrics
        app.kubernetes.io/version: 2.12.0
    spec:
      automountServiceAccountToken: true
      containers:
      - image: 172.22.16.20:8080/kubesphere/kube-state-metrics:latest
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
          initialDelaySeconds: 5
          timeoutSeconds: 5
        name: kube-state-metrics
        ports:
        - containerPort: 8080
          name: http-metrics
        - containerPort: 8081
          name: telemetry
        readinessProbe:
          httpGet:
            path: /
            port: 8081
          initialDelaySeconds: 5
          timeoutSeconds: 5
        securityContext:
          allowPrivilegeEscalation: false
          capabilities:
            drop:
            - ALL
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 65534
          seccompProfile:
            type: RuntimeDefault
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: kube-state-metrics

重点注意：

这里的image：地址必须是服务器本地的私有仓库地址，即后面我们上传的kube-state-metrics地址，不然镜像拉取不到

4. vim service-account.yaml文件

apiVersion: v1
automountServiceAccountToken: false
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.12.0
  name: kube-state-metrics
  namespace: kube-system

5.  vim  service.yaml 文件

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: exporter
    app.kubernetes.io/name: kube-state-metrics
    app.kubernetes.io/version: 2.12.0
  name: kube-state-metrics
  namespace: kube-system
spec:
  type: NodePort
  ports:
  - name: http-metrics
    port: 8080
    targetPort: 8080
    nodePort: 31666
  - name: telemetry
    port: 8081
    targetPort: 8081
  selector:
    app.kubernetes.io/name: kube-state-metrics

上传kube-state-metrics 镜像

将下载好的kube-state-metrics.tar上传至服务器

加载镜像

$ docker load -i kube-state-metrics.tar

将镜像上传至本地私有仓库（这里自己找一下自己的本地仓库），这里必须先上传到仓库，否则部署会显示镜像拉取失败。即

  docker login 172.22.16.20:8080   #登录自己私有仓库
  docker images 
  docker tag bitnami/kube-state-metrics:latest 172.22.16.20:8080/kubesphere/kube-state-metrics:latest  #tag镜像
  docker images
  docker push 172.22.16.20:8080/kubesphere/kube-state-metrics:latest #上传镜像

#注意将这里的地址换成自己的仓库地址

部署kube-state-metrics

pwd
~/kube-state-metrics
[K8S@k8s-master kube-state-metrics]$ kubectl apply -f ./

[K8S@k8s-master kube-state-metrics]$ kubectl get all -n kube-system  #查看部署结果

验证

访问Service暴露的31666端口，能成功返回，或者curl 能返回结果

配置普罗米修斯数据源对接k8s

vim  prometheus.yml

scrape_configs:
  - job_name: 'kube-state-metrics'
    static_configs:
      - targets: ['<kube-state-metrics-service-ip>:<kube-state-metrics-service-port>']

访问prometheus,验证结果

配置grafana对接显示

,这里很简单就不多说了，忘了的看完上一篇文章https://blog.youkuaiyun.com/weixin_47531290/article/details/144538170

第二种

！！！太累了，第二种方式后面更新，更新后链接会放在这里