Core Metrics in kubelet

本文探讨了Kubernetes中核心度量标准的概念,包括kubelet、cAdvisor、SummaryAPI的作用,以及如何通过这些组件实现资源隔离和利用特性。介绍了监视体系结构提案的目的,即为系统组件提供关键资源度量。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

参考文献:Core Metrics in kubelet

简介:

定义

“Kubelet”:运行在每个kubernetes节点上的守护进程,控制pod和容器生命周期,以及许多其他东西。

“cAdvisor”:一个开放源码的容器监控解决方案,它只监控容器,没有像pod或volume这样的kubernetes构造的概念。

“Summary API”: 是一个kubelet API,它目前公开了供系统组件和监视系统使用的节点指标。

CRI:容器运行时接口,设计用于在运行时(docker、rkt等)上提供抽象。

“Core Metrics”:监视体系结构中描述的一组度量标准,其目的是为一流的资源隔离和利用特性(包括资源可行性检查和节点资源管理)提供度量标准。

“Resouces”:节点的可消耗元素(如内存、磁盘空间、CPU时间等)。

“First-class Resources”:对调度至关重要的资源,其请求和限制可以(或即将)通过Pod/容器规范进行设置。

“Metric”:一种资源消耗的度量/指标。

背景

监视体系结构(Monitoring Architecture)提案包含一组称为“核心度量标准”的度量标准的蓝图。此提案的目的是指定这些度量标准是什么,以便通过kubelet支持与度量标准的集合相关的工作。Kubernetes将cAdvisor提供给它的代码库,kubelet使用cAdvisor作为库,使它能够收集容器上的指标。然后,kubelet可以将来自cAdvisor的容器级指标与kubelet对kubernetes构造(例如pods)的知识结合起来,生成kubelet汇总统计信息,该统计信息为kubelet或用户通过Summary API使用提供指标。cAdvisor的工作方式是每隔一段时间(默认情况下是10秒)收集指标,然后kubelet只要需要就可以查询这些缓存的指标。

目前,cAdvisor收集了大量与系统和容器性能相关的指标。然而,kubelet summary API只使用了其中的一些指标,而很多指标都没有使用。kubelet Summary API发布到kubelet summary API端点(stats/ Summary)。summary API提供的一些指标由kubernetes系统组件使用,但其中许多指标的唯一目的是提供用于监视的指标。

动机(Motivations)

监视体系结构提案解释了为什么需要单独的监视管道。

通过发布核心度量标准,kubelet就不必为监视提供度量标准了。第三方监视管道还可以免除向系统组件提供这些指标的任何责任。

cAdvisor的结构是按一定的时间间隔收集指标,这对于独立的指标收集器是合适的。然而,kubelet中的许多函数是延迟敏感的(例如,回收),并且可以从更“随需应变”的度量集合设计中获益。

提案(Proposal)

此提案使用由kubelet收集的这组核心度量标准,并仅由kubernetes系统组件使用,以支持“First-Class Resource Isolation and Utilization Features”。这个提案不是设计成由kubelet发布的API,而是由kubelet收集的一组度量标准,这些度量标准将被转换,并在将来发布。

这组指标的目标“用户”是kubernetes组件(尽管不一定是直接的)。这组指标本身并不是设计为面向用户的,而是设计为足够通用以支持面向用户的组件。

设计

这种设计只覆盖了核心度量管道中包含的度量。设计的高级需求如下:

kubelet收集尽可能少的指标,以提供“First-Class Resource Isolation and Utilization Features”。

可以“按需”获取指标,为kubelet提供更多最新的统计数据。

此提案有意省略了许多最终可能成为核心度量标准的度量标准。这是故意的。一旦需要度量来支持一流的资源隔离和利用特性,就可以将它们添加到核心度量API中。

指标要求(Metric Requirements)

 

[root@k8s-master-node1 ~]# kubectl get nodes -o wide NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME k8s-master-node1 Ready control-plane,master,worker 12m v1.22.1 192.168.200.13 <none> CentOS Linux 7 (Core) 3.10.0-1062.el7.x86_64 docker://20.10.9 [root@k8s-master-node1 ~]# kubectl describe node master Error from server (NotFound): nodes "master" not found [root@k8s-master-node1 ~]# kubectl describe node 192.168.200.13 Error from server (NotFound): nodes "192.168.200.13" not found [root@k8s-master-node1 ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION k8s-master-node1 Ready control-plane,master,worker 13m v1.22.1 [root@k8s-master-node1 ~]# kubectl describe node k8s-master-node1 Name: k8s-master-node1 Roles: control-plane,master,worker Labels: beta.kubernetes.io/arch=amd64 beta.kubernetes.io/os=linux kubernetes.io/arch=amd64 kubernetes.io/hostname=k8s-master-node1 kubernetes.io/os=linux node-role.kubernetes.io/control-plane= node-role.kubernetes.io/master= node-role.kubernetes.io/worker= node.kubernetes.io/exclude-from-external-load-balancers= Annotations: flannel.alpha.coreos.com/backend-data: {"VNI":1,"VtepMAC":"ba:7a:02:98:65:de"} flannel.alpha.coreos.com/backend-type: vxlan flannel.alpha.coreos.com/kube-subnet-manager: true flannel.alpha.coreos.com/public-ip: 192.168.200.13 kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock node.alpha.kubernetes.io/ttl: 0 volumes.kubernetes.io/controller-managed-attach-detach: true CreationTimestamp: Thu, 31 Jul 2025 10:15:21 +0800 Taints: <none> Unschedulable: false Lease: HolderIdentity: k8s-master-node1 AcquireTime: <unset> RenewTime: Thu, 31 Jul 2025 10:29:30 +0800 Conditions: Type Status LastHeartbeatTime LastTransitionTime Reason Message ---- ------ ----------------- ------------------ ------ ------- NetworkUnavailable False Thu, 31 Jul 2025 10:15:44 +0800 Thu, 31 Jul 2025 10:15:44 +0800 FlannelIsUp Flannel is running on this node MemoryPressure False Thu, 31 Jul 2025 10:25:35 +0800 Thu, 31 Jul 2025 10:15:19 +0800 KubeletHasSufficientMemory kubelet has sufficient memory available DiskPressure False Thu, 31 Jul 2025 10:25:35 +0800 Thu, 31 Jul 2025 10:15:19 +0800 KubeletHasNoDiskPressure kubelet has no disk pressure PIDPressure False Thu, 31 Jul 2025 10:25:35 +0800 Thu, 31 Jul 2025 10:15:19 +0800 KubeletHasSufficientPID kubelet has sufficient PID available Ready True Thu, 31 Jul 2025 10:25:35 +0800 Thu, 31 Jul 2025 10:15:34 +0800 KubeletReady kubelet is posting ready status Addresses: InternalIP: 192.168.200.13 Hostname: k8s-master-node1 Capacity: cpu: 2 ephemeral-storage: 51175Mi hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 1863100Ki pods: 110 Allocatable: cpu: 2 ephemeral-storage: 48294789041 hugepages-1Gi: 0 hugepages-2Mi: 0 memory: 1760700Ki pods: 110 System Info: Machine ID: aed7f180110d4207925b8b6ffb5e04e7 System UUID: 326C4D56-3645-80DE-DA14-9E7FCF9952A9 Boot ID: f9cacd8a-e24c-4f56-9e30-7388c70acfe9 Kernel Version: 3.10.0-1062.el7.x86_64 OS Image: CentOS Linux 7 (Core) Operating System: linux Architecture: amd64 Container Runtime Version: docker://20.10.9 Kubelet Version: v1.22.1 Kube-Proxy Version: v1.22.1 PodCIDR: 10.244.0.0/24 PodCIDRs: 10.244.0.0/24 Non-terminated Pods: (12 in total) Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits Age --------- ---- ------------ ---------- --------------- ------------- --- dashboard-cn dashboard-agent-cd88cf454-qg6mv 0 (0%) 0 (0%) 0 (0%) 0 (0%) 13m dashboard-cn dashboard-cn-64bd46887f-kmjwm 0 (0%) 0 (0%) 0 (0%) 0 (0%) 13m dashboard-en dashboard-en-55596d469-cc22g 0 (0%) 0 (0%) 0 (0%) 0 (0%) 13m kube-system coredns-78fcd69978-8s9xt 100m (5%) 0 (0%) 70Mi (4%) 170Mi (9%) 13m kube-system coredns-78fcd69978-mkg6d 100m (5%) 0 (0%) 70Mi (4%) 170Mi (9%) 13m kube-system etcd-k8s-master-node1 100m (5%) 0 (0%) 100Mi (5%) 0 (0%) 14m kube-system kube-apiserver-k8s-master-node1 250m (12%) 0 (0%) 0 (0%) 0 (0%) 14m kube-system kube-controller-manager-k8s-master-node1 200m (10%) 0 (0%) 0 (0%) 0 (0%) 14m kube-system kube-flannel-ds-dm28d 100m (5%) 100m (5%) 50Mi (2%) 50Mi (2%) 13m kube-system kube-proxy-4hznc 0 (0%) 0 (0%) 0 (0%) 0 (0%) 13m kube-system kube-scheduler-k8s-master-node1 100m (5%) 0 (0%) 0 (0%) 0 (0%) 14m kube-system metrics-server-77564bc84d-ddcbl 0 (0%) 0 (0%) 0 (0%) 0 (0%) 13m Allocated resources: (Total limits may be over 100 percent, i.e., overcommitted.) Resource Requests Limits -------- -------- ------ cpu 950m (47%) 100m (5%) memory 290Mi (16%) 390Mi (22%) ephemeral-storage 0 (0%) 0 (0%) hugepages-1Gi 0 (0%) 0 (0%) hugepages-2Mi 0 (0%) 0 (0%) Events: Type Reason Age From Message ---- ------ ---- ---- ------- Warning SystemOOM 14m (x9 over 14m) kubelet (combined from similar events): System OOM encountered, victim process: portainer, pid: 11953 Normal Starting 14m kubelet Starting kubelet. Warning SystemOOM 14m kubelet System OOM encountered, victim process: virt-controller, pid: 6419 Warning SystemOOM 14m kubelet System OOM encountered, victim process: virt-controller, pid: 6503 Warning SystemOOM 14m kubelet System OOM encountered, victim process: portainer, pid: 9051 Warning SystemOOM 14m kubelet System OOM encountered, victim process: virt-api, pid: 6382 Warning SystemOOM 14m kubelet System OOM encountered, victim process: virt-handler, pid: 9060 Warning SystemOOM 14m kubelet System OOM encountered, victim process: runc:[2:INIT], pid: 9670 Warning SystemOOM 14m kubelet System OOM encountered, victim process: runc:[2:INIT], pid: 9678 Warning SystemOOM 14m kubelet System OOM encountered, victim process: portainer, pid: 9426 Warning SystemOOM 14m kubelet System OOM encountered, victim process: portainer, pid: 9534 Warning SystemOOM 14m (x15 over 14m) kubelet (combined from similar events): System OOM encountered, victim process: virt-api, pid: 10256 [root@k8s-master-node1 ~]# ssh master systemctl status kubelet ssh: Could not resolve hostname master: Name or service not known [root@k8s-master-node1 ~]#
最新发布
08-01
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值