k8s-2:nfs搭建

目录

方式一:nfs挂载到宿主机目录

方法二：在机器上搭建nfs，其余使用就可以手动静态创建所要的pv和pvc，或创建pvc动态创建pv

修改kube-apiserver.yaml ,增加 - --feature-gates=RemoveSelfLink=false

创建RBAC授权

---

两种方式：

方式一:nfs挂载到宿主机目录

优点: 简单易用，无需额外支持
缺点：依赖宿主机磁盘容量，pod与宿主机存在强耦合，不利于管理。需要指定标签,当pod部署多个副本并分配到不同host时，数据不共享；当要驱赶pod时，数据不同步；当node故障时，数据易丢失；

1.node打标签

kubectl label nodes master25 monitor=nfsserver

2.all.yaml

apiVersion: apps/v1 kind: Deployment metadata: name: nfs-busybox spec: replicas: 2 template: metadata: labels: name: nfs-busybox spec: containers: - name: busybox image: busybox command: - sh - -c - 'while true; do date > /mnt/index.html; hostname >> /mnt/index.html; sleep $(($RANDOM % 5 + 5)); done' imagePullPolicy: IfNotPresent volumeMounts: # name must match the volume name below - name: nfs mountPath: "/mnt" volumes: - name: nfs persistentVolumeClaim: claimName: nfs --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: nfs spec: accessModes: - ReadWriteMany storageClassName: "" resources: requests: storage: 1Gi --- apiVersion: v1 kind: PersistentVolume metadata: name: nfs spec: capacity: storage: 1Gi accessModes: - ReadWriteMany nfs: # FIXME: use the right IP # get the server ip from the kube service server: 10.107.89.35 #server: nfs-server.default.svc.cluster.local path: "/nfsdemo/" --- apiVersion: v1 kind: Service metadata: name: nfs-server spec: ports: - name: nfs port: 2049 targetPort: 2049 protocol: TCP - name: mountd port: 20048 - name: rpcbind port: 111 selector: name: nfs-server --- apiVersion: apps/v1 kind: Deployment metadata: name: nfs-server namespace: default spec: replicas: 1 template: metadata: labels: name: nfs-server spec: nodeSelector: label: nfsserver containers: - name: nfs-server image: googlecontainer/volume-nfs:0.8 ports: - name: nfs containerPort: 2049 - name: mountd containerPort: 20048 - name: rpcbind containerPort: 111 securityContext: privileged: true volumeMounts: - mountPath: /exports name: nfs-folder volumes: - name: nfs-folder hostPath: path: "/home/dkongjian/nfs-share"

方法二：在机器上搭建nfs，其余使用就可以手动静态创建所要的pv和pvc，或创建pvc动态创建pv

优点:可做高可用配置,预防单点故障

缺点:切记一定要配置成开机自启动,

架构是:搭建StorageClass+NFS

1.创建一个可用的NFS Serve 2.创建Service Account.这是用来管控NFS provisioner在k8s集群中运行的权限 3.创建StorageClass.负责建立PVC并调用NFS provisioner进行预定的工作,并让PV与PVC建立管理 4.创建NFS provisioner.有两个功能,一个是在NFS共享目录下创建挂载点(volume),另一个则是建了PV并将PV与NFS的挂载点建立关联

1.安装nfs-server

sudo apt install nfs-kernel-server

其余node安装

apt-get install nfs-common

2.创建文件系统

sudo mkdir -p /data/nfsboot/ sudo chmod 777 /data/nfsboot/

3.配置文件系统

vim /etc/exports

/data/nfsboot/ *(rw,sync,no_root_squash)

重启服务

sudo service nfs-kernel-server  restart service nfs-kernel-server  enable

showmount测试

root@master24:/data/nfsboot# showmount -e 127.0.0.1 Export list for 127.0.0.1: /data/nfsboot *

修改kube-apiserver.yaml ,增加 - --feature-gates=RemoveSelfLink=false

原因是:

创建pvc后状态一直是pending,因为原来是1.20版本（我的是1.22.4）默认禁止使用selfLink。

创建RBAC授权

rbac.yaml:      #唯一需要修改的地方只有namespace,根据实际情况定义

wget https://raw.githubusercontent.com/kubernetes-incubator/external-storage/master/nfs-client/deploy/rbac.yaml # kubectl apply -f rbac.yaml

3.创建NFS资源的StorageClass

class.yaml

apiVersion: storage.k8s.io/v1 kind: StorageClass metadata:   name: managed-nfs-storage provisioner: fuseim.pri/ifs # or choose another name, must match deployment's env PROVISIONER_NAME' parameters:   archiveOnDelete: "false"

4.创建nfs-client-provisioner自动配置程序，以便自动创建持久卷(PV)

• 自动创建的 PV 以 ${namespace}-${pvcName}-${pvName} 的命名格式创建在 NFS 上
• 当这个 PV 被回收后会以 archieved-${namespace}-${pvcName}-${pvName} 的命名格式存在 NFS 服务器上

deployment.yaml

apiVersion: apps/v1 kind: Deployment metadata:   name: nfs-client-provisioner   labels:     app: nfs-client-provisioner   # replace with namespace where provisioner is deployed   namespace: default spec:   replicas: 1   strategy:     type: Recreate   selector:     matchLabels:       app: nfs-client-provisioner   template:     metadata:       labels:         app: nfs-client-provisioner     spec:       serviceAccountName: nfs-client-provisioner       containers:         - name: nfs-client-provisioner           image: quay.io/external_storage/nfs-client-provisioner:latest           volumeMounts:             - name: nfs-client-root               mountPath: /persistentvolumes           env:             - name: PROVISIONER_NAME               value: fuseim.pri/ifs             - name: NFS_SERVER               value: 192.168.50.24             - name: NFS_PATH               value: /data/nfsboot       volumes:         - name: nfs-client-root           nfs:             server: 192.168.50.24             path: /data/nfsboot

5.测试

# cat statefulset-nfs.yaml apiVersion: v1 kind: Service metadata:   name: nginx   labels:     app: nginx spec:   ports:   - port: 80     name: web   clusterIP: None   selector:     app: nginx --- apiVersion: apps/v1 kind: StatefulSet metadata:   name: nfs-web spec:   serviceName: "nginx"   replicas: 3   selector:     matchLabels:       app: nfs-web # has to match .spec.template.metadata.labels   template:     metadata:       labels:         app: nfs-web     spec:       terminationGracePeriodSeconds: 10       containers:       - name: nginx         image: nginx:1.7.9         ports:         - containerPort: 80           name: web         volumeMounts:         - name: www           mountPath: /usr/share/nginx/html   volumeClaimTemplates:   - metadata:       name: www       annotations:         volume.beta.kubernetes.io/storage-class: managed-nfs-storage     spec:       accessModes: [ "ReadWriteOnce" ]       resources:         requests:           storage: 1Gi

查看 Pod/PV/PVC

root@master24:/opt/yaml/nfs#  kubectl get pods NAME                                      READY   STATUS    RESTARTS   AGE nfs-client-provisioner-5cd65b7586-tbb7l   1/1     Running   0          24m nfs-web-0                                 1/1     Running   0          10m nfs-web-1                                 1/1     Running   0          9m41s nfs-web-2                                 1/1     Running   0          8m49s root@master24:/opt/yaml/nfs# kubectl get pv NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                   STORAGECLASS          REASON   AGE pvc-c67257ab-3694-47b5-afe1-d1cf5ebe098d   1Gi        RWO            Delete           Bound    default/www-nfs-web-1   managed-nfs-storage            9m49s pvc-e9ecc66a-f4db-4bfb-a66f-56a13a969e1a   1Gi        RWO            Delete           Bound    default/www-nfs-web-0   managed-nfs-storage            11m pvc-ea6fb9d2-c621-4d03-97f7-d54c570a7e22   1Gi        RWO            Delete           Bound    default/www-nfs-web-2   managed-nfs-storage            8m56s root@master24:/opt/yaml/nfs# kubectl get pvc NAME            STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS          AGE www-nfs-web-0   Bound    pvc-e9ecc66a-f4db-4bfb-a66f-56a13a969e1a   1Gi        RWO            managed-nfs-storage   16m www-nfs-web-1   Bound    pvc-c67257ab-3694-47b5-afe1-d1cf5ebe098d   1Gi        RWO            managed-nfs-storage   9m53s www-nfs-web-2   Bound    pvc-ea6fb9d2-c621-4d03-97f7-d54c570a7e22   1Gi        RWO            managed-nfs-storage   9m1s

查看 nfs server 目录中信息，同时各子目录中内容为空

root@master24:/opt/yaml/nfs#  ls -l  /data/nfsboot/ total 20 drwxrwxrwx 2 root root 4096 Nov 29 13:59 default-www-nfs-web-0-pvc-e9ecc66a-f4db-4bfb-a66f-56a13a969e1a drwxrwxrwx 2 root root 4096 Nov 29 14:01 default-www-nfs-web-1-pvc-c67257ab-3694-47b5-afe1-d1cf5ebe098d drwxrwxrwx 2 root root 4096 Nov 29 14:02 default-www-nfs-web-2-pvc-ea6fb9d2-c621-4d03-97f7-d54c570a7e22 drwxr-xr-x 2 root root 4096 Nov 27 15:08 test -rw-r--r-- 1 root root    6 Nov 26 17:47 test.txt