k8s live

最新推荐文章于 2024-10-09 20:28:24 发布

转载最新推荐文章于 2024-10-09 20:28:24 发布 · 107 阅读

0 ·

CC 4.0 BY-SA版权

原文链接：https://my.oschina.net/xiaominmin/blog/1598586

文章标签：

#运维 #爬虫 #python

本文记录了一次在Kubernetes中部署Nginx容器时遇到的健康检查失败问题。通过详细的事件日志，展示了容器从启动到因健康检查失败而被不断重启的过程。最终由于HTTP状态码返回404导致健康检查失败，并触发了回退重启机制。

2019独角兽企业重金招聘Python工程师标准>>>

k8s live 博客分类：搜索引擎，爬虫 Kubernetes

---------	--------	-----	----					-------------		--------	------		-------
  11m		11m		1	{default-scheduler }						Normal		Scheduled	Successfully assigned pod-with-healthcheck to dev-mg4.internal.tude.com
  11m		11m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Created		Created container with docker id ee8718fcfa36; Security:[seccomp=unconfined]
  11m		11m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Started		Started container with docker id ee8718fcfa36
  9m		9m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Created		Created container with docker id 030a5e32e952; Security:[seccomp=unconfined]
  9m		9m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Killing		Killing container with docker id ee8718fcfa36: pod "pod-with-healthcheck_default(f298e45a-0089-11e7-a3ad-005056ae01be)" container "nginx" is unhealthy, it will be killed and re-created.
  9m		9m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Started		Started container with docker id 030a5e32e952
  7m		7m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Started		Started container with docker id 28f81ea65711
  7m		7m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Killing		Killing container with docker id 030a5e32e952: pod "pod-with-healthcheck_default(f298e45a-0089-11e7-a3ad-005056ae01be)" container "nginx" is unhealthy, it will be killed and re-created.
  7m		7m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Created		Created container with docker id 28f81ea65711; Security:[seccomp=unconfined]
  6m		6m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Killing		Killing container with docker id 28f81ea65711: pod "pod-with-healthcheck_default(f298e45a-0089-11e7-a3ad-005056ae01be)" container "nginx" is unhealthy, it will be killed and re-created.
  6m		6m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Created		Created container with docker id f186b8dda69c; Security:[seccomp=unconfined]
  6m		6m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Started		Started container with docker id f186b8dda69c
  4m		4m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Killing		Killing container with docker id f186b8dda69c: pod "pod-with-healthcheck_default(f298e45a-0089-11e7-a3ad-005056ae01be)" container "nginx" is unhealthy, it will be killed and re-created.
  4m		4m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Created		Created container with docker id 008a30536782; Security:[seccomp=unconfined]
  4m		4m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Started		Started container with docker id 008a30536782
  3m		3m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Created		Created container with docker id 030f5bccb32b; Security:[seccomp=unconfined]
  3m		3m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Started		Started container with docker id 030f5bccb32b
  3m		3m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Killing		Killing container with docker id 008a30536782: pod "pod-with-healthcheck_default(f298e45a-0089-11e7-a3ad-005056ae01be)" container "nginx" is unhealthy, it will be killed and re-created.
  1m		1m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Started		Started container with docker id 04046c952761
  11m		1m		7	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Pulled		Container image "reg.docker.tude.com/cmall/business:test-1-master" already present on machine
  1m		1m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Killing		Killing container with docker id 030f5bccb32b: pod "pod-with-healthcheck_default(f298e45a-0089-11e7-a3ad-005056ae01be)" container "nginx" is unhealthy, it will be killed and re-created.
  1m		1m		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Created		Created container with docker id 04046c952761; Security:[seccomp=unconfined]
  10m		40s		9	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Warning		Unhealthy	Liveness probe failed: HTTP probe failed with statuscode: 404
  9s		9s		1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal		Killing		Killing container with docker id 04046c952761: pod "pod-with-healthcheck_default(f298e45a-0089-11e7-a3ad-005056ae01be)" container "nginx" is unhealthy, it will be killed and re-created.
  9s		8s		3	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Warning		BackOff		Back-off restarting failed docker container
  9s		8s		3	{kubelet dev-mg4.internal.tude.com}				Warning		FailedSync	Error syncing pod, skipping: failed to "StartContainer" for "nginx" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=nginx pod=pod-with-healthcheck_default(f298e45a-0089-11e7-a3ad-005056ae01be)"

  1m	1m	1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal	Created		Created container with docker id 5c4597fa06b9; Security:[seccomp=unconfined]
  15m	1m	8	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal	Pulled		Container image "reg.docker.tude.com/cmall/business:test-1-master" already present on machine
  1m	1m	1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal	Started		Started container with docker id 5c4597fa06b9
  14m	54s	10	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Warning	Unhealthy	Liveness probe failed: HTTP probe failed with statuscode: 404
  23s	23s	1	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Normal	Killing		Killing container with docker id 5c4597fa06b9: pod "pod-with-healthcheck_default(f298e45a-0089-11e7-a3ad-005056ae01be)" container "nginx" is unhealthy, it will be killed and re-created.
  4m	6s	18	{kubelet dev-mg4.internal.tude.com}	spec.containers{nginx}	Warning	BackOff		Back-off restarting failed docker container
  23s	6s	4	{kubelet dev-mg4.internal.tude.com}				Warning	FailedSync	Error syncing pod, skipping: failed to "StartContainer" for "nginx" with CrashLoopBackOff: "Back-off 5m0s restarting failed container=nginx pod=pod-with-healthcheck_default(f298e45a-0089-11e7-a3ad-005056ae01be)"

apiVersion: v1
kind: Pod
metadata:
  name: pod-with-healthcheck
spec:
  containers:
  - name: nginx
    image: reg.docker.tude.com/cmall/business:test-1-master
    # defines the health checking
    livenessProbe:
      # an http probe
      httpGet:
        path: /111213asd
        port: 80
      # length of time to wait for a pod to initialize
      # after pod startup, before applying health checking
      initialDelaySeconds: 30
      timeoutSeconds: 10
      # 检查周期
      periodSeconds: 30 
      # 临界值 
      # successThreshold: 1
      # 失败次数报 	Unhealthy	Liveness probe failed: HTTP probe failed with statuscode: 404，不检测了 , kubernetes-dashboard 出现错误
      # failureThreshold: 3
    ports:
    - containerPort: 80

Liveness probe failed: HTTP probe failed with statuscode: 404
Back-off restarting failed docker container
Error syncing pod, skipping: failed to "StartContainer" for "nginx" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=nginx pod=pod-with-healthcheck_default(f298e45a-0089-11e7-a3ad-005056ae01be)"

pod-with-healthcheck

Waiting: CrashLoopBackOff

16 minutes

Liveness probe failed: HTTP probe failed with statuscode: 404

Back-off restarting failed docker container

Error syncing pod, skipping: failed to "StartContainer" for "nginx" with CrashLoopBackOff: "Back-off 2m40s restarting failed container=nginx pod=pod-with-healthcheck_default(f298e45a-0089-11e7-a3ad-005056ae01be)"

转载于:https://my.oschina.net/xiaominmin/blog/1598586