2018年05月04日 17:16:53 迷途的攻城狮（798570156） 阅读数：2258
版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.youkuaiyun.com/chenleiking/article/details/80197975
K8S的应用更新、回滚、伸缩
1、应用结构
以一个简单的deployment为例来进行说明：
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 4
revisionHistoryLimit: 10
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: 192.168.101.88:5000/nginx:1.7.9
ports:
- containerPort: 80- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
一个相对完整的应用可能还包括：volume、env、service、ingress等
root@master-1:~/kubernetes/example# kubectl apply -f nginx-deployment.yaml
deployment.apps "nginx-deployment" created
root@master-1:~/kubernetes/example# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-deployment-85b548b894-ct66d 1/1 Running 0 20s
pod/nginx-deployment-85b548b894-pfmvl 1/1 Running 0 20s
pod/nginx-deployment-85b548b894-v88qq 1/1 Running 0 20s
pod/nginx-deployment-85b548b894-zxml8 1/1 Running 0 20s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7d
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-deployment 4 4 4 4 20s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-deployment-85b548b894 4 4 4 20s- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
忽略service/kubernetes～～
graph TB;
deployment.apps/nginx-deployment --> replicaset.apps/nginx-deployment-85b548b894
replicaset.apps/nginx-deployment-85b548b894 --> pod/nginx-deployment-85b548b894-ct66d
replicaset.apps/nginx-deployment-85b548b894 --> pod/nginx-deployment-85b548b894-pfmvl
replicaset.apps/nginx-deployment-85b548b894 --> pod/nginx-deployment-85b548b894-v88qq
replicaset.apps/nginx-deployment-85b548b894 --> pod/nginx-deployment-85b548b894-zxml8- 1
- 2
- 3
- 4
- 5
- 6
- 每个deployment会包含一个或者多个replicaset
- 每个replicaset会可以包含零个或者多个pod
- 每个replicaset对应deployment的一个revision
- 每次更新时，pod总是在一个replicaset中创建，然后在现有的replicaset中销毁
2、更新与回滚
只有spec.template.spec.containers.image或者spec.template.metadata.labels发生变化时才会出发更新rolout操作～～
- 更新
root@master-1:~/kubernetes/example# kubectl set image deployment nginx-deployment nginx=192.168.101.88:5000/nginx:1.9.1
deployment.apps "nginx-deployment" image updated- 1
- 2
> gif图片太大，连接：https://pan.baidu.com/s/1cR7l4anB6FrQ27UNvJ1h0g
更新过程如图所示：
- 首先，因为没有匹配的replicaset，创建一个新的replicaset
replicaset.apps/nginx-deployment-767fdf8df6- 然后根据spec.strategy.rollingUpdate.maxUnavailable的值（默认25%）终止掉
replicaset.apps/nginx-deployment-85b548b894中对应数量的pod- 最后根据spec.strategy.rollingUpdate.maxSurge的值（默认25%）在
replicaset.apps/nginx-deployment-767fdf8df6中启动对应数量多pod- 最终，
replicaset.apps/nginx-deployment-85b548b894中的pod全部终止，replicaset.apps/nginx-deployment-767fdf8df6中的pod达到预期值
- replicaset的名字组成：deploymentName + podTemplate的Hash值
- maxUnavailable：表示最大不可用pod数量，可以是整数或者百分比。这里的4个pod，25%就是一个，也就是说一次只能停止一个pod
- maxSurge：表示可超过预期值的pod数量，可以是整数或者百分比。预期值4，25%就是可以超出一个，也就是说同时可以存在5个可用的pod
graph TB;
deployment.apps/nginx-deployment --> replicaset.apps/nginx-deployment-85b548b894
deployment.apps/nginx-deployment --> replicaset.apps/nginx-deployment-767fdf8df6
replicaset.apps/nginx-deployment-767fdf8df6 --> pod/nginx-deployment-767fdf8df6-5zj8c
replicaset.apps/nginx-deployment-767fdf8df6 --> pod/nginx-deployment-767fdf8df6-7668l
replicaset.apps/nginx-deployment-767fdf8df6 --> pod/nginx-deployment-767fdf8df6-s7rpq
replicaset.apps/nginx-deployment-767fdf8df6 --> pod/nginx-deployment-767fdf8df6-vgm2m- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 回滚
### 查看更新记录 ###
root@master-1:~# kubectl rollout history deployment nginx-deployment
deployments "nginx-deployment"
REVISION CHANGE-CAUSE
1 <none>
2 <none>- 1
- 2
- 3
- 4
- 5
- 6
默认的部署方式不会记录CHANGE-CAUSE，需要在部署的时候添加
--record选项～～spec.revisionHistoryLimit被用来设置保留的历史记录数量，默认为2，设置为0时将不能回滚～～
root@master-1:~/kubernetes/example# kubectl delete -f nginx-deployment.yaml
deployment.apps "nginx-deployment" deleted
root@master-1:~/kubernetes/example# kubectl apply -f nginx-deployment.yaml --record
deployment.apps "nginx-deployment" created
root@master-1:~/kubernetes/example# kubectl set image deployment nginx-deployment nginx=192.168.101.88:5000/nginx:1.9.1
deployment.apps "nginx-deployment" image updated
root@master-1:~/kubernetes/example# kubectl rollout history deployment nginx-deployment
deployments "nginx-deployment"
REVISION CHANGE-CAUSE
1 kubectl apply --filename=nginx-deployment.yaml --record=true
2 kubectl set image deployment nginx-deployment nginx=192.168.101.88:5000/nginx:1.9.1- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
### 回滚到上一个版本 ###
root@master-1:~/kubernetes/example# kubectl rollout undo deployment nginx-deployment
deployment.apps "nginx-deployment"
root@master-1:~/kubernetes/example# kubectl rollout history deployment nginx-deployment
deployments "nginx-deployment"
REVISION CHANGE-CAUSE
2 kubectl set image deployment nginx-deployment nginx=192.168.101.88:5000/nginx:1.9.1
3 kubectl apply --filename=nginx-deployment.yaml --record=true- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
回滚操作的执行流程与更新一样，都是按照spec.strategy.rollingUpdate.maxUnavailable和spec.strategy.rollingUpdate.maxSurge在当前replicaset中终止pod，在符合条件的replicaset中启动新的pod～～
### 回滚到指定版本 ###
root@master-1:~/kubernetes/example# kubectl rollout undo deployment nginx-deployment --to-revision=2
deployment.apps "nginx-deployment"
root@master-1:~/kubernetes/example# kubectl rollout history deployment nginx-deployment
deployments "nginx-deployment"
REVISION CHANGE-CAUSE
3 kubectl apply --filename=nginx-deployment.yaml --record=true
4 kubectl set image deployment nginx-deployment nginx=192.168.101.88:5000/nginx:1.9.1- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
root@master-1:~/kubernetes/example# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-deployment-767fdf8df6-56bmx 1/1 Running 0 42s
pod/nginx-deployment-767fdf8df6-bjbx4 1/1 Running 0 42s
pod/nginx-deployment-767fdf8df6-p6xll 1/1 Running 0 45s
pod/nginx-deployment-767fdf8df6-rf5tm 1/1 Running 0 44s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7d
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-deployment 4 4 4 4 6m
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-deployment-767fdf8df6 4 4 4 6m
replicaset.apps/nginx-deployment-85b548b894 0 0 0 6m- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
因为上面的更新与回滚一直在两个版本之间来回修改，deploymentName + podTemplate的Hash值并没有改变，所以没有产生新的replicaset，但是版本号回一直递增～～～
3、伸缩
3.1、手动伸缩
root@master-1:~/kubernetes/example# kubectl scale deployment nginx-deployment --replicas=10
deployment.extensions "nginx-deployment" scaled
root@master-1:~/kubernetes/example# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-deployment-767fdf8df6-4zz4k 1/1 Running 0 11s
pod/nginx-deployment-767fdf8df6-56bmx 1/1 Running 0 5m
pod/nginx-deployment-767fdf8df6-9rqk6 1/1 Running 0 11s
pod/nginx-deployment-767fdf8df6-bjbx4 1/1 Running 0 5m
pod/nginx-deployment-767fdf8df6-ksrc2 1/1 Running 0 11s
pod/nginx-deployment-767fdf8df6-p6xll 1/1 Running 0 5m
pod/nginx-deployment-767fdf8df6-qcj9q 1/1 Running 0 11s
pod/nginx-deployment-767fdf8df6-rf5tm 1/1 Running 0 5m
pod/nginx-deployment-767fdf8df6-rljpn 1/1 Running 0 11s
pod/nginx-deployment-767fdf8df6-ts8sn 1/1 Running 0 11s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 7d
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-deployment 10 10 10 10 11m
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-deployment-767fdf8df6 10 10 10 11m
replicaset.apps/nginx-deployment-85b548b894 0 0 0 11m- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
3.2、自动伸缩（Horizontal Pod Autoscaling）
自动伸缩需要部署Heapster和metrics-server～
root@master-1:~/kubernetes/example# cat nginx-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 4
revisionHistoryLimit: 10
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: 192.168.101.88:5000/nginx:1.7.9
ports:
- containerPort: 80
resources:
requests:
memory: "64Mi"
cpu: "25m"
limits:
memory: "128Mi"
cpu: "50m"
---
kind: Service
apiVersion: v1
metadata:
labels:
app: nginx-deployment
name: nginx-deployment-service
spec:
ports:
- port: 80
targetPort: 80
name: http
selector:
app: nginx
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-deployment-service-ingress
spec:
rules:
- host: nginx-deployment.chenlei.com
http:
paths:
- backend:
serviceName: nginx-deployment-service
servicePort: 80- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 使用HPA需要在部署文件中明确指定资源配置情况
resources，最少需要配置resources.requests.cpu- 为了测试，向部署文件中增加了service和ingress
root@master-1:~/kubernetes/example# kubectl apply -f nginx-deployment.yaml
deployment.apps "nginx-deployment" created
service "nginx-deployment-service" created
ingress.extensions "nginx-deployment-service-ingress" created
root@master-1:~/kubernetes/example# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-deployment-5f79d4fc87-7k6cn 1/1 Running 0 9s
pod/nginx-deployment-5f79d4fc87-f5v27 1/1 Running 0 9s
pod/nginx-deployment-5f79d4fc87-ls6hz 1/1 Running 0 9s
pod/nginx-deployment-5f79d4fc87-rlwh4 1/1 Running 0 9s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 8d
service/nginx-deployment-service ClusterIP 10.97.127.155 <none> 80/TCP 9s
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-deployment 4 4 4 4 10s
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-deployment-5f79d4fc87 4 4 4 9s- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
root@master-1:~/kubernetes/example# kubectl autoscale deployment nginx-deployment --min=1 --max=10 --cpu-percent=20
deployment.apps "nginx-deployment" autoscaled
root@master-1:~/kubernetes/example# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-deployment-5f79d4fc87-7k6cn 1/1 Running 0 2m
pod/nginx-deployment-5f79d4fc87-f5v27 0/1 Terminating 0 2m
pod/nginx-deployment-5f79d4fc87-ls6hz 0/1 Terminating 0 2m
pod/nginx-deployment-5f79d4fc87-rlwh4 0/1 Terminating 0 2m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 8d
service/nginx-deployment-service ClusterIP 10.97.127.155 <none> 80/TCP 2m
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-deployment 1 1 1 1 2m
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-deployment-5f79d4fc87 1 1 1 2m
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
horizontalpodautoscaler.autoscaling/nginx-deployment Deployment/nginx-deployment 0%/20% 1 10 4 34s- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
HPA创建之后，会在30秒（根据controller-manager的参数配置）内根据CPU的负载情况自动调整POD数量。因为负载为零，所以自动终止了三个pod，仅保留一个～～～
kube-controller-manage的调度日志如下：
I0504 08:25:48.431218 1 horizontal.go:512] Successful rescale of nginx-deployment, old size: 4, new size: 1, reason: All metrics below target I0504 08:25:48.431412 1 event.go:218] Event(v1.ObjectReference{Kind:"HorizontalPodAutoscaler", Namespace:"default", Name:"nginx-deployment", UID:"adb4b7a9-4f74-11e8-9807-0050568e3344", APIVersion:"autoscaling/v2beta1", ResourceVersion:"1314846", FieldPath:""}): type: 'Normal' reason: 'SuccessfulRescale' New size: 1; reason: All metrics below target I0504 08:25:48.475085 1 event.go:218] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"default", Name:"nginx-deployment", UID:"79043755-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1314901", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled down replica set nginx-deployment-5f79d4fc87 to 1 I0504 08:25:48.599598 1 event.go:218] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"default", Name:"nginx-deployment-5f79d4fc87", UID:"79075132-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1314904", FieldPath:""}): type: 'Normal' reason: 'SuccessfulDelete' Deleted pod: nginx-deployment-5f79d4fc87-ls6hz I0504 08:25:48.599622 1 event.go:218] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"default", Name:"nginx-deployment-5f79d4fc87", UID:"79075132-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1314904", FieldPath:""}): type: 'Normal' reason: 'SuccessfulDelete' Deleted pod: nginx-deployment-5f79d4fc87-rlwh4 I0504 08:25:48.599634 1 event.go:218] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"default", Name:"nginx-deployment-5f79d4fc87", UID:"79075132-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1314904", FieldPath:""}): type: 'Normal' reason: 'SuccessfulDelete' Deleted pod: nginx-deployment-5f79d4fc87-f5v27 W0504 08:27:12.650959 1 reflector.go:341] k8s.io/kubernetes/pkg/controller/garbagecollector/graph_builder.go:125: watch of <nil> ended with: very short watch: k8s.io/kubernetes/pkg/controller/garbagecollector/graph_builder.go:125: Unexpected watch close - watch lasted less than a second and no items received
- 1
- 2
- 3
- 4
- 5
- 6
- 7
增加负载：
LiondeMacBook-Pro:~ lion$ for i in {1..10000}; do curl http://nginx-deployment.chenlei.com/ > /dev/null -s; done;- 1
这里打开四个终端分别执行，然后观察pod伸缩情况
root@master-1:~/kubernetes/example# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/nginx-deployment-5f79d4fc87-7k6cn 1/1 Running 0 12m
pod/nginx-deployment-5f79d4fc87-gvfpr 1/1 Running 0 4s
pod/nginx-deployment-5f79d4fc87-mzd58 1/1 Running 0 4s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 8d
service/nginx-deployment-service ClusterIP 10.97.127.155 <none> 80/TCP 12m
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx-deployment 3 3 3 3 12m
NAME DESIRED CURRENT READY AGE
replicaset.apps/nginx-deployment-5f79d4fc87 3 3 3 12m
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
horizontalpodautoscaler.autoscaling/nginx-deployment Deployment/nginx-deployment 44%/20% 1 10 1 11m- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
当CPU压力增加时，kubernetes自动创建了两个新的pod来分摊压力，新增pod的数量取决于CPU压力打下
kube-controller-maanage的调度日志：
I0504 08:36:14.996758 1 horizontal.go:512] Successful rescale of nginx-deployment, old size: 1, new size: 3, reason: cpu resource utilization (percentage of request) above target I0504 08:36:14.997180 1 event.go:218] Event(v1.ObjectReference{Kind:"HorizontalPodAutoscaler", Namespace:"default", Name:"nginx-deployment", UID:"adb4b7a9-4f74-11e8-9807-0050568e3344", APIVersion:"autoscaling/v2beta1", ResourceVersion:"1315533", FieldPath:""}): type: 'Normal' reason: 'SuccessfulRescale' New size: 3; reason: cpu resource utilization (percentage of request) above target I0504 08:36:15.061094 1 event.go:218] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"default", Name:"nginx-deployment", UID:"79043755-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1316131", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled up replica set nginx-deployment-5f79d4fc87 to 3 I0504 08:36:15.163244 1 event.go:218] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"default", Name:"nginx-deployment-5f79d4fc87", UID:"79075132-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1316134", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: nginx-deployment-5f79d4fc87-mzd58 I0504 08:36:15.208366 1 event.go:218] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"default", Name:"nginx-deployment-5f79d4fc87", UID:"79075132-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1316134", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: nginx-deployment-5f79d4fc87-gvfpr
- 1
- 2
- 3
- 4
- 5
压力较低之后又会消减pod数量
3.3、自定义指标
这一块内容留在下次补充～～～～
3.4、异常记录
- 问题一：
root@master-1:~/kubernetes/example# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx-deployment Deployment/nginx-deployment <unknown>/10% 1 10 4 37s
root@master-1:~/kubernetes/example# kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
nginx-deployment Deployment/nginx-deployment <unknown>/10% 1 10 4 37s
root@master-1:~/kubernetes/example# kubectl describe hpa nginx-deployment
Name: nginx-deployment
Namespace: default
Labels: <none>
Annotations: <none>
CreationTimestamp: Fri, 04 May 2018 04:46:16 -0400
Reference: Deployment/nginx-deployment
Metrics: ( current / target )
resource cpu on pods (as a percentage of request): <unknown> / 10%
Min replicas: 1
Max replicas: 10
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True SucceededGetScale the HPA controller was able to get the target's current scale
ScalingActive False FailedGetResourceMetric the HPA was unable to compute the replica count: unable to get metrics for resource cpu: no metrics returned from resource metrics API
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedGetResourceMetric 21s horizontal-pod-autoscaler unable to get metrics for resource cpu: no metrics returned from resource metrics API
Warning FailedComputeMetricsReplicas 21s horizontal-pod-autoscaler failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
kube-controller-manage日志：
I0504 08:47:16.381435 1 event.go:218] Event(v1.ObjectReference{Kind:"HorizontalPodAutoscaler", Namespace:"default", Name:"nginx-deployment", UID:"9b7a46ff-4f77-11e8-9807-0050568e3344", APIVersion:"autoscaling/v2beta1", ResourceVersion:"1317451", FieldPath:""}): type: 'Warning' reason: 'FailedGetResourceMetric' missing request for cpu on container nginx in pod default/nginx-deployment-85b548b894-2lzg8
E0504 08:47:16.391281 1 horizontal.go:189] failed to compute desired number of replicas based on listed metrics for Deployment/default/nginx-deployment: failed to get cpu utilization: missing request for cpu on container nginx in pod default/nginx-deployment-85b548b894-2lzg8
I0504 08:47:16.391308 1 event.go:218] Event(v1.ObjectReference{Kind:"HorizontalPodAutoscaler", Namespace:"default", Name:"nginx-deployment", UID:"9b7a46ff-4f77-11e8-9807-0050568e3344", APIVersion:"autoscaling/v2beta1", ResourceVersion:"1317451", FieldPath:""}): type: 'Warning' reason: 'FailedComputeMetricsReplicas' failed to get cpu utilization: missing request for cpu on container nginx in pod default/nginx-deployment-85b548b894-2lzg8- 1
- 2
- 3
原因：
定义pod的时候，没有设置resources，HPA取不到CPU当前值
resources:
requests:
memory: "64Mi"
cpu: "25m"
limits:
memory: "128Mi"
cpu: "50m"- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 问题二
root@master-1:~/kubernetes/example# kubectl autoscale deployment nginx-deployment --min=1 --max=10 --cpu-percent=10
deployment.apps "nginx-deployment" autoscaled
root@master-1:~/kubernetes/example# kubectl describe hpa nginx-deployment
Name: nginx-deployment
Namespace: default
Labels: <none>
Annotations: <none>
CreationTimestamp: Fri, 04 May 2018 04:53:20 -0400
Reference: Deployment/nginx-deployment
Metrics: ( current / target )
resource cpu on pods (as a percentage of request): <unknown> / 10%
Min replicas: 1
Max replicas: 10
Conditions:
Type Status Reason Message
---- ------ ------ -------
AbleToScale True SucceededGetScale the HPA controller was able to get the target's current scale
ScalingActive False FailedGetResourceMetric the HPA was unable to compute the replica count: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedGetResourceMetric 27s horizontal-pod-autoscaler unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)
Warning FailedComputeMetricsReplicas 27s horizontal-pod-autoscaler failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)
root@master-1:~/kubernetes/example# - 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
kube-controller-manage日志：
E0504 08:54:20.852828 1 horizontal.go:189] failed to compute desired number of replicas based on listed metrics for Deployment/default/nginx-deployment: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)
I0504 08:54:20.852958 1 event.go:218] Event(v1.ObjectReference{Kind:"HorizontalPodAutoscaler", Namespace:"default", Name:"nginx-deployment", UID:"987a4389-4f78-11e8-9807-0050568e3344", APIVersion:"autoscaling/v2beta1", ResourceVersion:"1318418", FieldPath:""}): type: 'Warning' reason: 'FailedGetResourceMetric' unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)- 1
- 2
原因：
没有部署metrics-server服务，部署后任然无法解决的可参考GitHub上的这个问题。
4、参考资料
https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/
https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/
2万+
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
