【kubernetes】记录一次namespace 处于Terminating状态的处理方法

最新推荐文章于 2024-02-16 14:58:20 发布

王树民

最新推荐文章于 2024-02-16 14:58:20 发布

阅读量1.5k

点赞数 1

分类专栏：多线程/高并发 docker/k8s

docker/k8s 同时被 2 个专栏收录

169 篇文章

订阅专栏

多线程/高并发

96 篇文章

订阅专栏

本文介绍了一种在Kubernetes中遇到命名空间状态为Terminating时的处理方法，详细记录了从尝试多种方法到最终成功删除命名空间的过程，包括使用kubectl proxy和curl命令直接修改命名空间的finalizers。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

【kubernetes】记录一次namespace 处于Terminating状态的处理方法

2019年02月22日 14:00:55 fengwuxichen 阅读数：401

记录一次k8s namespace 处于Terminating状态的处理方法

问题背景

使用k8s部署rook-ceph后，想要删除集群重新部署的，结果方法不对，导致rook-ceph命名空间状态一直处于Terminating，使用kubectl get all -n rook-ceph命令显示已经没有资源，但仍无法删除。

尝试过的方法

删除时带上–force --grace-period=0参数 ,无法删除；
kubectl delete namespace rook-ceph --force --grace-period=0
使用kubectl edit namespaces rook-ceph将finalizer的value删除，显示命名空间成功被编辑，但是kubectl get ns看到rook-ceph命名空间仍然存在，且状态依旧是Terminating；
参考文章：https://segmentfault.com/a/1190000016924414

最终成功的方法

NAMESPACE=rook-ceph
kubectl proxy &
kubectl get namespace $NAMESPACE -o json |jq '.spec = {"finalizers":[]}' >temp.json
curl -k -H "Content-Type: application/json" -X PUT --data-binary @temp.json 127.0.0.1:8001/api/v1/namespaces/$NAMESPACE/finalize

将上面的代码执行完成后，再使用kubectl get ns命令，发现rook-ceph命名空间已经没了，最后别忘了把后台运行的kubectl proxy进程kill掉。
参考：
https://stackoverflow.com/questions/52369247/namespace-stucked-as-terminating-how-i-removed-it
https://github.com/kubernetes/kubernetes/issues/60807#issuecomment-466278490

问题分析

命名空间无法删除通常是因为还有资源在使用这个命名空间，这个资源可能自己的状态也已经不正常了。
在通过kubectl delete -f operator.yaml删除operator的时候，虽然都显示deleted，发现命令夯死。手动逐个删除的时候发现，在删除clusters.ceph.rook.io的时候命令（kubectl delete crd clusters.ceph.rook.io）夯死，查看资源使用情况发现cluster.ceph.rook.io占用着rook-ceph命名空间：

[root@k8s-master rook]# kubectl api-resources --namespaced=true -o name | xargs -n 1 kubectl get --show-kind --ignore-not-found -n rook-ceph
Error from server (MethodNotAllowed): the server does not allow this method on the requested resource
Error from server (MethodNotAllowed): the server does not allow this method on the requested resource
NAME                             AGE
cluster.ceph.rook.io/rook-ceph   1d

使用kubectl edit crd clusters.ceph.rook.io编辑clusters.ceph.rook.io：

# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"apiextensions.k8s.io/v1beta1","kind":"CustomResourceDefinition","metadata":{"annotations":{},"name":"clusters.ceph.rook.io","namespace":""},"spec":{"group":"ceph.rook.io","names":{"kind":"Cluster","listKind":"ClusterList","plural":"clusters","shortNames":["rcc"],"singular":"cluster"},"scope":"Namespaced","version":"v1beta1"}}
  creationTimestamp: 2019-02-21T02:00:26Z
  deletionGracePeriodSeconds: 0
  deletionTimestamp: 2019-02-22T09:01:44Z
  finalizers:
  - customresourcecleanup.apiextensions.k8s.io
  generation: 1
  name: clusters.ceph.rook.io
  resourceVersion: "6630441"
  selfLink: /apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions/clusters.ceph.rook.io
  uid: 74fdfb7b-357c-11e9-9add-fa163efae430
spec:
  additionalPrinterColumns:
  - JSONPath: .metadata.creationTimestamp
    description: |-
      CreationTimestamp is a timestamp representing the server time when this object was created. It is not guaranteed to be set in happens-before order across separate operations. Clients may not set this value. It is represented in RFC3339 form and is in UTC.

      Populated by the system. Read-only. Null for lists. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#metadata
    name: Age
    type: date
  group: ceph.rook.io
  names:
    kind: Cluster
    listKind: ClusterList
    plural: clusters
    shortNames:
    - rcc
    singular: cluster
  scope: Namespaced
  version: v1beta1
  versions:
  - name: v1beta1
    served: true
    storage: true
status:
  acceptedNames:
    kind: Cluster
    listKind: ClusterList
    plural: clusters
    shortNames:
    - rcc
    singular: cluster
  conditions:
  - lastTransitionTime: 2019-02-21T02:00:26Z
    message: no conflicts found
    reason: NoConflicts
    status: "True"
    type: NamesAccepted
  - lastTransitionTime: null
    message: the initial names have been accepted
    reason: InitialNamesAccepted
    status: "True"
    type: Established
  - lastTr，ansitionTime: 2019-02-21T07:28:49Z
    message: CustomResource deletion is in progress
    reason: InstanceDeletionInProgress
    status: "True"
    type: Terminating
  storedVersions:
  - v1beta1

可以看到cluster.ceph.rook.io也Terminating，把finalizers的值删掉，保存，cluster.ceph.rook.io便会自己删掉。此时rook-ceph便也会自动删除了。