MeterSphere容器化部署：Kubernetes集群配置指南-优快云博客

MeterSphere容器化部署：Kubernetes集群配置指南

【免费下载链接】metersphere MeterSphere 一站式开源持续测试平台，为软件质量保驾护航。搞测试，就选 MeterSphere！项目地址: https://gitcode.com/gh_mirrors/me/metersphere

1. 部署背景与痛点解决

1.1 传统部署模式的挑战

在大规模测试场景下，传统单机部署的MeterSphere面临三大核心痛点：

资源利用率低下：测试任务峰谷差异导致服务器资源闲置率高达40%~60%
扩展性受限：单节点并发能力受限于硬件配置，无法应对突发测试需求
维护成本高：分布式部署需要手动配置负载均衡与数据同步

1.2 Kubernetes部署优势

采用Kubernetes(K8s)容器编排平台可实现：

资源动态调度，测试任务按需分配计算资源
自动扩缩容，根据测试任务量弹性调整Pod数量
服务健康检查与自动恢复，减少人工干预
标准化部署流程，降低环境一致性问题

2. 部署环境准备

2.1 集群环境要求

组件	最低配置	推荐配置	说明
Kubernetes	v1.21+	v1.24+	需支持CRD与StatefulSet
节点数量	2+	3+	包含1个控制节点与至少1个工作节点
CPU	4核	8核	每节点
内存	8GB	16GB	每节点
存储	100GB SSD	200GB SSD	持久化存储支持
网络	Calico/Flannel	Calico	支持NetworkPolicy

2.2 必要工具安装

# 安装kubectl命令行工具
curl -LO "https://dl.k8s.io/release/v1.24.0/bin/linux/amd64/kubectl"
chmod +x kubectl && sudo mv kubectl /usr/local/bin/

# 安装Helm包管理工具
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

2.3 集群网络与存储配置

# 示例：StorageClass配置（需提前创建）
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: metersphere-sc
provisioner: kubernetes.io/aws-ebs
parameters:
  type: gp2
reclaimPolicy: Retain
allowVolumeExpansion: true

3. 部署架构设计

3.1 整体架构图

mermaid

3.2 资源组件说明

组件	功能说明	推荐副本数	资源限制
MeterSphere应用	核心业务逻辑	2+	CPU: 2核, 内存: 4Gi
MySQL	元数据存储	1（主从架构可3）	CPU: 1核, 内存: 2Gi
Redis	缓存与会话存储	1	CPU: 500m, 内存: 1Gi
Kafka	消息队列	3	CPU: 1核, 内存: 2Gi
MinIO	文件存储	4	CPU: 1核, 内存: 2Gi

4. 详细部署步骤

4.1 命名空间创建

kubectl create namespace metersphere

4.2 配置数据库

4.2.1 MySQL部署

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
  namespace: metersphere
spec:
  serviceName: mysql
  replicas: 1
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
      - name: mysql
        image: mysql:8.0
        ports:
        - containerPort: 3306
        env:
        - name: MYSQL_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: password
        - name: MYSQL_DATABASE
          value: metersphere
        volumeMounts:
        - name: mysql-data
          mountPath: /var/lib/mysql
  volumeClaimTemplates:
  - metadata:
      name: mysql-data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "metersphere-sc"
      resources:
        requests:
          storage: 50Gi
---
apiVersion: v1
kind: Service
metadata:
  name: mysql
  namespace: metersphere
spec:
  selector:
    app: mysql
  ports:
  - port: 3306
  clusterIP: None

4.2.2 数据库初始化

# 创建数据库密码密钥
kubectl create secret generic mysql-secret -n metersphere --from-literal=password=Password123!

# 部署MySQL
kubectl apply -f mysql-statefulset.yaml

# 等待MySQL就绪
kubectl rollout status statefulset/mysql -n metersphere

# 执行初始化SQL
kubectl exec -it mysql-0 -n metersphere -- mysql -uroot -pPassword123! metersphere < /docker-entrypoint-initdb.d/init.sql

4.3 中间件部署

4.3.1 Redis配置

apiVersion: v1
kind: ConfigMap
metadata:
  name: redis-config
  namespace: metersphere
data:
  redis.conf: |
    appendonly yes
    protected-mode no
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: redis
  namespace: metersphere
spec:
  replicas: 1
  selector:
    matchLabels:
      app: redis
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:6.2
        command: ["redis-server", "/etc/redis/redis.conf"]
        ports:
        - containerPort: 6379
        volumeMounts:
        - name: redis-config
          mountPath: /etc/redis
        - name: redis-data
          mountPath: /data
  volumes:
  - name: redis-config
    configMap:
      name: redis-config
  - name: redis-data
    persistentVolumeClaim:
      claimName: redis-data-pvc

4.4 MeterSphere应用部署

4.4.1 部署配置

apiVersion: apps/v1
kind: Deployment
metadata:
  name: metersphere
  namespace: metersphere
spec:
  replicas: 2
  selector:
    matchLabels:
      app: metersphere
  template:
    metadata:
      labels:
        app: metersphere
    spec:
      containers:
      - name: metersphere
        image: metersphere/metersphere-ce-allinone:latest
        ports:
        - containerPort: 8081
        env:
        - name: SPRING_DATASOURCE_URL
          value: jdbc:mysql://mysql:3306/metersphere?useUnicode=true&characterEncoding=utf-8&serverTimezone=Asia/Shanghai
        - name: SPRING_DATASOURCE_USERNAME
          value: root
        - name: SPRING_DATASOURCE_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mysql-secret
              key: password
        - name: REDIS_HOST
          value: redis
        - name: REDIS_PORT
          value: "6379"
        resources:
          limits:
            cpu: "2"
            memory: "4Gi"
          requests:
            cpu: "1"
            memory: "2Gi"

4.4.2 服务暴露

apiVersion: v1
kind: Service
metadata:
  name: metersphere
  namespace: metersphere
spec:
  selector:
    app: metersphere
  ports:
  - port: 80
    targetPort: 8081
  type: ClusterIP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: metersphere
  namespace: metersphere
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: ms.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: metersphere
            port:
              number: 80

4.5 K8s资源池配置

4.5.1 资源池参数说明

参数名	类型	描述	示例值
ip	String	K8s API Server地址	https://10.96.0.1:6443
token	String	认证Token	eyJhbGciOiJSUzI1NiIsImtpZCI6Il...
namespace	String	命名空间	metersphere
concurrentNumber	Integer	最大并发数	10
podThreads	Integer	单Pod线程数	50
deployName	String	部署名称	ms-agent

4.5.2 资源池添加步骤

登录MeterSphere系统，进入系统设置 > 测试资源池
点击添加资源池，选择类型为K8s

填写配置参数：

{
  "ip": "https://10.96.0.1:6443",
  "token": "eyJhbGciOiJSUzI1NiIsImtpZCI6Il...",
  "namespace": "metersphere",
  "concurrentNumber": 10,
  "podThreads": 50,
  "deployName": "ms-agent"
}

点击测试连接验证配置正确性
保存并启用资源池

5. 部署验证与运维

5.1 部署状态检查

# 检查Pod状态
kubectl get pods -n metersphere

# 查看服务日志
kubectl logs -f deployment/metersphere -n metersphere

# 检查Ingress规则
kubectl get ingress -n metersphere

5.2 性能优化建议

资源调优：
- 根据测试任务类型调整CPU/内存资源限制
- 为API测试场景设置podThreads=100，UI测试场景设置podThreads=20

自动扩缩容：

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: metersphere
  namespace: metersphere
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: metersphere
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80

5.3 常见问题排查

问题现象	可能原因	解决方案
Pod启动失败	数据库连接错误	检查MySQL服务是否正常， credentials是否正确
测试任务执行超时	资源池配置错误	验证K8s API Server地址与token权限
并发任务数受限	资源池并发参数设置过低	调整`concurrentNumber`参数值

6. 高级配置与最佳实践

6.1 持久化存储配置

推荐使用MinIO作为分布式文件存储：

apiVersion: v1
kind: ConfigMap
metadata:
  name: minio-config
  namespace: metersphere
data:
  MINIO_ROOT_USER: "minio"
  MINIO_ROOT_PASSWORD: "minio123"
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: minio
  namespace: metersphere
spec:
  serviceName: minio
  replicas: 4
  selector:
    matchLabels:
      app: minio
  template:
    metadata:
      labels:
        app: minio
    spec:
      containers:
      - name: minio
        image: minio/minio
        args:
        - server
        - /data
        - --console-address
        - ":9001"
        ports:
        - containerPort: 9000
        - containerPort: 9001
        envFrom:
        - configMapRef:
            name: minio-config
        volumeMounts:
        - name: data
          mountPath: /data
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "metersphere-sc"
      resources:
        requests:
          storage: 10Gi

6.2 监控告警配置

集成Prometheus与Grafana监控：

部署Prometheus Operator
创建ServiceMonitor:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: metersphere
  namespace: metersphere
  labels:
    release: prometheus
spec:
  selector:
    matchLabels:
      app: metersphere
  endpoints:
  - port: http
    path: /actuator/prometheus
    interval: 15s

6.3 备份策略

# 创建数据库备份脚本
cat > backup-mysql.sh << 'EOF'
#!/bin/bash
TIMESTAMP=$(date +%Y%m%d%H%M%S)
kubectl exec -n metersphere mysql-0 -- mysqldump -uroot -p$MYSQL_PASSWORD metersphere > metersphere_$TIMESTAMP.sql
aws s3 cp metersphere_$TIMESTAMP.sql s3://backup-bucket/metersphere/
EOF

# 创建定时任务
kubectl create cronjob mysql-backup -n metersphere \
  --schedule="0 2 * * *" \
  --image=busybox:1.35 \
  --env="MYSQL_PASSWORD=$(kubectl get secret -n metersphere mysql-secret -o jsonpath='{.data.password}' | base64 -d)" \
  --command -- /bin/sh -c "wget -O /tmp/backup.sh https://example.com/backup-mysql.sh && sh /tmp/backup.sh"

7. 总结与展望

通过Kubernetes集群部署MeterSphere，企业可以获得更弹性、可靠的测试基础设施。本文详细介绍了从环境准备到高级配置的全流程，包括：

基于K8s的容器化部署架构设计
核心组件的配置清单与参数说明
资源池配置与性能优化实践
运维监控与备份策略

未来版本将进一步增强K8s原生特性支持，包括：

基于CustomResourceDefinition的资源池管理
测试任务的K8s Job原生调度
与Istio服务网格的深度集成

建议定期关注官方文档与GitHub仓库获取最新部署指南与最佳实践。

附录：参考资源

MeterSphere官方文档: https://metersphere.io/docs/
Kubernetes官方文档: https://kubernetes.io/docs/home/
Helm Charts仓库: https://artifacthub.io/
部署示例代码库: https://gitcode.com/gh_mirrors/me/metersphere

创作声明：本文部分内容由AI辅助生成（AIGC），仅供参考