kubernetes存储（三）——K8S的Volumes配置管理（动态卷+StorageClass存储类（class）+部署mysql主从集群）

1.StorageClass介绍

StorageClass提供了一种描述存储类（class）的方法，不同的class可能会映射到不同的服务质量等级和备份策略或其他策略等。

每个 StorageClass 都包含 provisioner、parameters 和 reclaimPolicy 字段， 这些字段会在 StorageClass需要动态分配 PersistentVolume 时会使用到。

StorageClass的属性
• Provisioner（存储分配器）：用来决定使用哪个卷插件分配 PV，该字段必须指定。可以指 定内部分配器，也可以指定外部分配器。外部分配器的代码地址为： kubernetesincubator/external-storage，其中包括NFS和Ceph等。
• Reclaim Policy（回收策略）：通过reclaimPolicy字段指定创建的Persistent Volume的回收 策略，回收策略包括：Delete 或者 Retain，没有指定默认为Delete。
• 更多属性查看：https://kubernetes.io/zh/docs/concepts/storage/storage-classes/

2.配置NFS Client Provisioner

NFS Client Provisioner是一个automatic provisioner，使用NFS作为存储，自动创建PV和对应的 PVC，本身不提供NFS存储，需要外部先有一套NFS存储服务。

• PV以 ${namespace}-${pvcName}-${pvName}的命名格式提供（在NFS服务器上） 
• PV回收的时候以 archieved-${namespace}-${pvcName}-${pvName} 的命名格式（在NFS 服务器上） 

nfs-client-provisioner源码地址：https://github.com/kubernetes-incubator/external-storage/tree/master/nfs-client/deploy

[root@harbor ~]# vim /etc/exports
[root@harbor ~]# cat /etc/exports
/nfsdata   *(rw,sync,no_root_squash)
[root@harbor ~]# exportfs -rv
exporting *:/nfsdata
[root@harbor ~]# showmount -e
Export list for harbor:
/nfsdata *

[kubeadm@server1 ~]$ cd vol/
[kubeadm@server1 vol]$ mkdir nfs-client
[kubeadm@server1 vol]$ cd nfs-client/
[kubeadm@server1 nfs-client]$ vim rbac.yml ##添加角色绑定的名称空间
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
subjects:
  - kind: ServiceAccount
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

[kubeadm@server1 nfs-client]$ vim deployment.yml ##删除重复的选择器字段
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: nfs-client-provisioner:latest ##镜像需要下载并上传到仓库
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: red.org/nfs
            - name: NFS_SERVER
              value: 172.25.1.11
            - name: NFS_PATH
              value: /nfsdata
      volumes:
        - name: nfs-client-root
          nfs:
            server: 172.25.1.11
            path: /nfsdata

[kubeadm@server1 nfs-client]$ vim class.yml ##创建存储类
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: managed-nfs-storage
provisioner: red.org/nfs
parameters:
  archiveOnDelete: "false"

[kubeadm@server1 nfs-client]$ vim pvc.yml ##创建pvc
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: pvc1
  annotations:
    volume.beta.kubernetes.io/storage-class: "managed-nfs-storage"
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 100Mi

[kubeadm@server1 nfs-client]$ vim pod.yml
kind: Pod
apiVersion: v1
metadata:
  name: test-pod
spec:
  containers:
  - name: test-pod
    image: busybox
    command:
      - "/bin/sh"
    args:
      - "-c"
      - "touch /mnt/SUCCESS && exit 0 || exit 1"
    volumeMounts:
      - name: nfs-pvc
        mountPath: "/mnt"
  restartPolicy: "Never"
  volumes:
    - name: nfs-pvc
      persistentVolumeClaim:
        claimName: pvc1 

再次改为false

以上为动态卷的存储方式

---

默认的 StorageClass 将被用于动态的为没有特定 storage class 需求的 PersistentVolumeClaims 配置存储：（只能有一个默认StorageClass）
如果没有默认StorageClass，PVC 也没有指定storageClassName 的值，那么意味着它只能够跟 storageClassName 也是“”的 PV 进行绑定。

[kubeadm@server1 nfs-client]$ kubectl get pvc
No resources found in default namespace.
[kubeadm@server1 nfs-client]$ kubectl get pv
No resources found in default namespace.
[kubeadm@server1 nfs-client]$ kubectl get storageclasses.storage.k8s.io 
NAME                  PROVISIONER   RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
managed-nfs-storage   red.org/nfs   Delete          Immediate           false                  6m33s
[