KubeKey创建k8s集群、k8s监控、KubeSphere可视化安装mysql中间件并暴露服务，中间件扩容缩容

以下步骤如未说明，每个节点都要执行

一、前置条件

1、三台或者更多兼容的 Linux 主机（建议CentOS 7.9），每台机器 2 GB 或更多的 RAM，2 CPU 核或更多。
2、各个主机网络互通(公网和内网均可）。
3、 节点之中不可以有重复的主机名、MAC 地址或 product_uuid。

注意：分别在三个主机执行以下三行命令

#各个机器设置自己的域名
hostnamectl set-hostname master
hostnamectl set-hostname node1
hostnamectl set-hostname node2

#查看主机名
hostname

二、使用KubeKey创建集群

1、下载KubeKey

export KKZONE=cn

curl -sfL https://get-kk.kubesphere.io | VERSION=v1.1.1 sh -

chmod +x kk

2、创建集群配置yaml文件

./kk create config --with-kubernetes v1.20.4 --with-kubesphere v3.1.1

执行完成会生成一个config-sample.yaml的配置文件，打开修改以下内容为自己的节点hostname、IP、用户名和密码：

3、创建集群

./kk create cluster -f config-sample.yaml

如果提示没有conntrack执行yum install -y conntrack

正常安装，输入yes继续4、查看安装进度

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f

安装完成访问

三、安装NFS文件系统

NFS就是Network File System的缩写，它最大的功能就是可以通过网络，让不同的机器、不同的操作系统可以共享彼此的文件。NFS服务器可以让主机将网络中的NFS服务器共享的目录挂载到本地端的文件系统中，而在本地端的系统中来看，那个远程主机的目录就好像是自己的一个磁盘分区一样，在使用上相当便利；利用这个特点我们可以保存有状态应用的一些数据或者配置，比如MySQL的数据文件或者配置文件。
1、安装nfs

# 在每个机器。
yum install -y nfs-utils


# 在master 执行以下全部命令 
echo "/nfs/data/ *(insecure,rw,sync,no_root_squash)" > /etc/exports


# 执行以下命令，启动 nfs 服务;创建共享目录
mkdir -p /nfs/data

systemctl enable rpcbind
systemctl enable nfs-server
systemctl start rpcbind
systemctl start nfs-server

# 使配置生效
exportfs -r

#检查配置是否生效
exportfs

2、配置默认存储，保存以下到storage.yaml中，注意修改中间的两个IP为自己master的IP

## 创建了一个存储类
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: nfs-storage
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner
parameters:
  archiveOnDelete: "true"  ## 删除pv的时候，pv的内容是否要备份

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nfs-client-provisioner
  labels:
    app: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nfs-client-provisioner
  template:
    metadata:
      labels:
        app: nfs-client-provisioner
    spec:
      serviceAccountName: nfs-client-provisioner
      containers:
        - name: nfs-client-provisioner
          image: registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/nfs-subdir-external-provisioner:v4.0.2
          # resources:
          #    limits:
          #      cpu: 10m
          #    requests:
          #      cpu: 10m
          volumeMounts:
            - name: nfs-client-root
              mountPath: /persistentvolumes
          env:
            - name: PROVISIONER_NAME
              value: k8s-sigs.io/nfs-subdir-external-provisioner
            - name: NFS_SERVER
              value: 172.31.0.2 ## 指定自己nfs服务器地址
            - name: NFS_PATH  
              value: /nfs/data  ## nfs服务器共享的目录
      volumes:
        - name: nfs-client-root
          nfs:
            server: 172.31.0.2
            path: /nfs/data
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: nfs-client-provisioner-runner
rules:
  - apiGroups: [""]
    resources: ["nodes"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["persistentvolumes"]
    verbs: ["get", "list", "watch", "create", "delete"]
  - apiGroups: [""]
    resources: ["persistentvolumeclaims"]
    verbs: ["get", "list", "watch", "update"]
  - apiGroups: ["storage.k8s.io"]
    resources: ["storageclasses"]
    verbs: ["get", "list", "watch"]
  - apiGroups: [""]
    resources: ["events"]
    verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: run-nfs-client-provisioner
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: ClusterRole
  name: nfs-client-provisioner-runner
  apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
rules:
  - apiGroups: [""]
    resources: ["endpoints"]
    verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: leader-locking-nfs-client-provisioner
  # replace with namespace where provisioner is deployed
  namespace: default
subjects:
  - kind: ServiceAccount
    name: nfs-client-provisioner
    # replace with namespace where provisioner is deployed
    namespace: default
roleRef:
  kind: Role
  name: leader-locking-nfs-client-provisioner
  apiGroup: rbac.authorization.k8s.io

kubectl apply -f storage.yaml

四、KubeSphere可视化界面安装mysql

1、创建企业空间

登录KubeSphere点击企业空间

点击创建

输入企业空间名称

2、创建项目

点击左上角工作台退出来，点击企业空间，点击刚刚创建的企业空间test，点击项目管理，点击创建

输入项目名称，点击创建

创建完成点击项目名称

3、创建mysql工作负载

首先分析一下，k8s最小单元是pod，而pod是一组docker容器，所以安装mysql就是用docker安装mysql，参照第二节：Docker使用命令，第10小节：docker安装mysql，docker的安装命令是：

docker run -p 3306:3306 --name mysql-01 \
-v /mydata/mysql/log:/var/log/mysql \
-v /mydata/mysql/data:/var/lib/mysql \
-v /mydata/mysql/conf:/etc/mysql/conf.d \
-e MYSQL_ROOT_PASSWORD=123456 \
--restart=always \
-d mysql:5.7 

需要考虑：

• 挂载配置文件到主机，在配置中心添加配置
• 执行参数MYSQL_ROOT_PASSWORD来给mysql配置登录密码，添加环境变量
• 挂载mysql数据到主机，添加PVC使用nfs来挂载

对应上面：

（1）、增加mysql配置

点击配置中心=>配置=>创建

输入名称，点击下一步

输入key，输入值，点击对号，点击创建

[client]
default-character-set=utf8mb4
 
[mysql]
default-character-set=utf8mb4
 
[mysqld]
init_connect='SET collation_connection = utf8mb4_unicode_ci'
init_connect='SET NAMES utf8mb4'
character-set-server=utf8mb4
collation-server=utf8mb4_unicode_ci
skip-character-set-client-handshake
skip-name-resolve

（2）、创建工作负载，配置mysql镜像，配置端口号

点击应用负载，选择有状态副本集，点击创建

输入名称，点击下一步

点击添加容器镜像

输入mysql:5.7，点击回车，等到搜索结果出来，点击使用默认端口

（3）、增加环境变量

勾选环境变量输入MYSQL_ROOT_PASSWORD，值为123456（此环境变量是给mysql设置登录密码）；勾选时区，点击对号

点击下一步

（4）、添加存储卷模板，挂载mysql数据到nfs

点击添加存储卷模板

输入新建存储卷名称；存储类型选择nfs-storage（如果没有这个选项，参照第三节：安装nfs文件系统），访问模式选择单个节点读写；容量输入5；挂载路径选择读写，路径为/var/lib/mysql，点击对号

点击挂载配置文件或者秘钥

选择mysql-conf，选择只读，路径为：/etc/mysql/conf.d，点击对号，点击下一步，再点击下一步，点击创建

可以看到已经创建了一个部署mysql

点击容器组，点击mysql

点击事件，等待拉取镜像、容器创建并启动完成

4、创建服务，暴露mysql外网访问接口

点击服务，点击创建

选择指定工作负载

输入名称，点击下一步

点击指定工作负载，选择有状态副本集，选择mysql，点击确定

端口号输入mysql默认端口号3306，点击下一步

勾选外网访问，访问方式选择NodePort，点击创建

可以看到暴露的端口号

5、测试mysql链接

此时用任意节点的外网IP加上刚才端口号即可连接mysql

7、mysql扩容缩容
扩容缩容，点击上下即可实现

扩容完成

五、安装集群指标监控组件Metrics（选做）

Metrics Server是k8s内置的集群范围资源使用情况的数据聚合器，为集群提供Node、Pods资源利用率指标等功能。
保存以下配置到metrics.yaml中

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
    rbac.authorization.k8s.io/aggregate-to-admin: "true"
    rbac.authorization.k8s.io/aggregate-to-edit: "true"
    rbac.authorization.k8s.io/aggregate-to-view: "true"
  name: system:aggregated-metrics-reader
rules:
- apiGroups:
  - metrics.k8s.io
  resources:
  - pods
  - nodes
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
rules:
- apiGroups:
  - ""
  resources:
  - pods
  - nodes
  - nodes/stats
  - namespaces
  - configmaps
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server-auth-reader
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server:system:auth-delegator
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:auth-delegator
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    k8s-app: metrics-server
  name: system:metrics-server
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:metrics-server
subjects:
- kind: ServiceAccount
  name: metrics-server
  namespace: kube-system
---
apiVersion: v1
kind: Service
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  ports:
  - name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    k8s-app: metrics-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    k8s-app: metrics-server
  name: metrics-server
  namespace: kube-system
spec:
  selector:
    matchLabels:
      k8s-app: metrics-server
  strategy:
    rollingUpdate:
      maxUnavailable: 0
  template:
    metadata:
      labels:
        k8s-app: metrics-server
    spec:
      containers:
      - args:
        - --cert-dir=/tmp
        - --kubelet-insecure-tls
        - --secure-port=4443
        - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
        - --kubelet-use-node-status-port
        image: registry.cn-hangzhou.aliyuncs.com/lfy_k8s_images/metrics-server:v0.4.3
        imagePullPolicy: IfNotPresent
        livenessProbe:
          failureThreshold: 3
          httpGet:
            path: /livez
            port: https
            scheme: HTTPS
          periodSeconds: 10
        name: metrics-server
        ports:
        - containerPort: 4443
          name: https
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /readyz
            port: https
            scheme: HTTPS
          periodSeconds: 10
        securityContext:
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          runAsUser: 1000
        volumeMounts:
        - mountPath: /tmp
          name: tmp-dir
      nodeSelector:
        kubernetes.io/os: linux
      priorityClassName: system-cluster-critical
      serviceAccountName: metrics-server
      volumes:
      - emptyDir: {}
        name: tmp-dir
---
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
  labels:
    k8s-app: metrics-server
  name: v1beta1.metrics.k8s.io
spec:
  group: metrics.k8s.io
  groupPriorityMinimum: 100
  insecureSkipTLSVerify: true
  service:
    name: metrics-server
    namespace: kube-system
  version: v1beta1
  versionPriority: 100

kubectl apply -f metrics.yaml