【Cloud】HELM 最佳实践

下面是一个完整的Helm Chart结构，包含了Tomcat和Nginx的配置，支持多环境部署。这个结构遵循了Kubernetes最佳实践和Helm模板开发规范。

主要改进说明

1. 简化架构设计：

   • 合并了多个Tomcat Deployment为单个Deployment，通过replicas控制实例数量
   • 使用Service实现内部负载均衡，不再需要为每个Tomcat实例创建单独的Service

2. 增强环境配置：

   • 添加了global.environment变量，标识当前环境
   • 为各环境提供了专用配置文件（dev/uat/prod）
   • 支持通过环境变量注入特定配置

3. 生产就绪特性：

   • 添加了健康检查（readinessProbe和livenessProbe）
   • 优化了资源请求和限制配置
   • 支持负载均衡器注解（如AWS NLB）

4. 改进模板设计：

   • 增加了标签生成函数，确保一致性
   • 使用toYaml简化复杂结构的模板编写
   • 添加了测试钩子，便于验证部署

5. 安全性提升：

   • 为所有资源添加了完整的标签
   • 使用ConfigMap管理Nginx配置，避免硬编码
   • 遵循Kubernetes命名规范

部署命令示例

开发环境

helm install my-tomcat-dev ./my-tomcat-chart -f environments/dev-values.yaml

UAT环境

helm install my-tomcat-uat ./my-tomcat-chart -f environments/uat-values.yaml

生产环境

helm install my-tomcat-prod ./my-tomcat-chart -f environments/prod-values.yaml

这个Chart结构遵循了Helm最佳实践，支持多环境部署，并具备良好的可维护性和扩展性。

my-tomcat-chart/
├── Chart.yaml
├── values.yaml
├── environments/
│   ├── dev-values.yaml
│   ├── uat-values.yaml
│   └── prod-values.yaml
├── templates/
│   ├── _helpers.tpl
│   ├── tomcat-deployment.yaml
│   ├── tomcat-service.yaml
│   ├── nginx-configmap.yaml
│   ├── nginx-deployment.yaml
│   ├── nginx-service.yaml
│   └── tests/
│       └── test-connection.yaml
└── .helmignore

my-tomcat-chart/values.yaml

# 全局配置
global:
  imagePullPolicy: IfNotPresent
  environment: dev  # 默认环境

# Tomcat 配置
tomcat:
  enabled: true
  replicas: 2
  image:
    repository: tomcat
    tag: 9.0.71
  ports:
    containerPort: 8080
  hostHtmlPath: /path/on/host/a.html
  resources:
    requests:
      memory: "512Mi"
      cpu: "200m"
    limits:
      memory: "1Gi"
      cpu: "500m"
  env:
    JAVA_OPTS: "-Xms256m -Xmx512m"

# Nginx 配置
nginx:
  enabled: true
  replicas: 1
  image:
    repository: nginx
    tag: 1.25.3
  service:
    type: LoadBalancer
    port: 80
    annotations: {}
  resources:
    requests:
      memory: "128Mi"
      cpu: "100m"
    limits:
      memory: "256Mi"
      cpu: "200m"  

my-tomcat-chart/chart.yaml

apiVersion: v2
name: my-tomcat-chart
description: A Helm chart for Tomcat with Nginx Load Balancer
type: application
version: 0.1.0
appVersion: "9.0.71"  

my-tomcat-chart/environments/dev-values.yaml

# 开发环境配置
global:
  environment: dev

tomcat:
  replicas: 1
  hostHtmlPath: /data/dev/a.html
  resources:
    requests:
      memory: "256Mi"
      cpu: "100m"
    limits:
      memory: "512Mi"
      cpu: "200m"

nginx:
  replicas: 1
  service:
    type: NodePort  

my-tomcat-chart/environments/uat-values.yaml

# UAT环境配置
global:
  environment: uat
  imagePullPolicy: Always

tomcat:
  replicas: 2
  hostHtmlPath: /data/uat/a.html
  resources:
    requests:
      memory: "512Mi"
      cpu: "200m"
    limits:
      memory: "1Gi"
      cpu: "500m"

nginx:
  replicas: 2
  service:
    type: LoadBalancer  

my-tomcat-chart/environments/prod-values.yaml

# 生产环境配置
global:
  environment: production
  imagePullPolicy: Always

tomcat:
  replicas: 5
  hostHtmlPath: /data/prod/a.html
  resources:
    requests:
      memory: "1Gi"
      cpu: "500m"
    limits:
      memory: "2Gi"
      cpu: "1"
  readinessProbe:
    httpGet:
      path: /a.html
      port: 8080
    initialDelaySeconds: 30
    periodSeconds: 10
  livenessProbe:
    httpGet:
      path: /a.html
      port: 8080
    initialDelaySeconds: 60
    periodSeconds: 10

nginx:
  replicas: 3
  service:
    type: LoadBalancer
    annotations:
      service.beta.kubernetes.io/aws-load-balancer-type: nlb
      service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
  resources:
    requests:
      memory: "256Mi"
      cpu: "200m"
    limits:
      memory: "512Mi"
      cpu: "500m"  

my-tomcat-chart\templates_helpers.tpl

{{/*
Expand the name of the chart.
*/}}
{{- define "my-tomcat-chart.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
{{- end -}}

{{/*
Create a default fully qualified app name.
*/}}
{{- define "my-tomcat-chart.fullname" -}}
{{- if .Values.fullnameOverride -}}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
{{- else -}}
{{- printf "%s-%s" .Release.Name (include "my-tomcat-chart.name" .) | trunc 63 | trimSuffix "-" -}}
{{- end -}}
{{- end -}}

{{/*
Create chart labels.
*/}}
{{- define "my-tomcat-chart.labels" -}}
helm.sh/chart: {{ include "my-tomcat-chart.name" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end -}}

{{/*
Create pod labels.
*/}}
{{- define "my-tomcat-chart.podLabels" -}}
{{- include "my-tomcat-chart.labels" . | nindent 4 }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end -}}  

my-tomcat-chart\templates\nginx-configmap.yaml

{{- if and .Values.nginx.enabled .Values.tomcat.enabled }}
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ include "my-tomcat-chart.fullname" . }}-nginx-config
  labels:
    app.kubernetes.io/name: nginx
    {{- include "my-tomcat-chart.labels" . | nindent 4 }}
data:
  nginx.conf: |
    worker_processes 1;
    
    events {
        worker_connections 1024;
    }
    
    http {
        upstream tomcat_backend {
            server {{ include "my-tomcat-chart.fullname" . }}-tomcat:80;
        }
        
        server {
            listen 80;
            
            location / {
                proxy_pass http://tomcat_backend;
                proxy_set_header Host $host;
                proxy_set_header X-Real-IP $remote_addr;
                proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header X-Forwarded-Proto $scheme;
            }
        }
    }
{{- end }}  

my-tomcat-chart\templates\nginx-deployment.yaml

{{- if .Values.nginx.enabled }}
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "my-tomcat-chart.fullname" . }}-nginx
  labels:
    app.kubernetes.io/name: nginx
    {{- include "my-tomcat-chart.labels" . | nindent 4 }}
spec:
  replicas: {{ .Values.nginx.replicas }}
  selector:
    matchLabels:
      app.kubernetes.io/name: nginx
      app.kubernetes.io/instance: {{ .Release.Name }}
  template:
    metadata:
      labels:
        app.kubernetes.io/name: nginx
        app.kubernetes.io/instance: {{ .Release.Name }}
        {{- include "my-tomcat-chart.podLabels" . | nindent 8 }}
    spec:
      containers:
      - name: nginx
        image: "{{ .Values.nginx.image.repository }}:{{ .Values.nginx.image.tag }}"
        imagePullPolicy: {{ .Values.global.imagePullPolicy }}
        ports:
        - name: http
          containerPort: 80
          protocol: TCP
        resources:
          {{- toYaml .Values.nginx.resources | nindent 10 }}
        volumeMounts:
        - name: config-volume
          mountPath: /etc/nginx
      volumes:
      - name: config-volume
        configMap:
          name: {{ include "my-tomcat-chart.fullname" . }}-nginx-config
{{- end }}  

my-tomcat-chart\templates\nginx-service.yaml

{{- if .Values.nginx.enabled }}
apiVersion: v1
kind: Service
metadata:
  name: {{ include "my-tomcat-chart.fullname" . }}-nginx
  labels:
    app.kubernetes.io/name: nginx
    {{- include "my-tomcat-chart.labels" . | nindent 4 }}
  {{- with .Values.nginx.service.annotations }}
  annotations:
    {{- toYaml . | nindent 4 }}
  {{- end }}
spec:
  type: {{ .Values.nginx.service.type }}
  ports:
    - port: {{ .Values.nginx.service.port }}
      targetPort: 80
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: nginx
    app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}  

my-tomcat-chart\templates\tomcat-deployment.yaml

{{- if .Values.tomcat.enabled }}
{{- $replicas := .Values.tomcat.replicas }}
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "my-tomcat-chart.fullname" . }}-tomcat
  labels:
    app.kubernetes.io/name: tomcat
    {{- include "my-tomcat-chart.labels" . | nindent 4 }}
spec:
  replicas: {{ $replicas }}
  selector:
    matchLabels:
      app.kubernetes.io/name: tomcat
      app.kubernetes.io/instance: {{ .Release.Name }}
  template:
    metadata:
      labels:
        app.kubernetes.io/name: tomcat
        app.kubernetes.io/instance: {{ .Release.Name }}
        {{- include "my-tomcat-chart.podLabels" . | nindent 8 }}
    spec:
      containers:
      - name: tomcat
        image: "{{ .Values.tomcat.image.repository }}:{{ .Values.tomcat.image.tag }}"
        imagePullPolicy: {{ .Values.global.imagePullPolicy }}
        ports:
        - name: http
          containerPort: {{ .Values.tomcat.ports.containerPort }}
          protocol: TCP
        env:
        - name: ENVIRONMENT
          value: {{ .Values.global.environment | quote }}
        {{- if .Values.tomcat.env }}
        {{- toYaml .Values.tomcat.env | nindent 8 }}
        {{- end }}
        {{- if .Values.tomcat.readinessProbe }}
        readinessProbe:
          {{- toYaml .Values.tomcat.readinessProbe | nindent 10 }}
        {{- end }}
        {{- if .Values.tomcat.livenessProbe }}
        livenessProbe:
          {{- toYaml .Values.tomcat.livenessProbe | nindent 10 }}
        {{- end }}
        resources:
          {{- toYaml .Values.tomcat.resources | nindent 10 }}
        volumeMounts:
        - name: host-html
          mountPath: /usr/local/tomcat/webapps/ROOT/a.html
          subPath: a.html
      volumes:
      - name: host-html
        hostPath:
          path: {{ .Values.tomcat.hostHtmlPath }}
          type: File
{{- end }}  

my-tomcat-chart\templates\tomcat-service.yaml

{{- if .Values.tomcat.enabled }}
apiVersion: v1
kind: Service
metadata:
  name: {{ include "my-tomcat-chart.fullname" . }}-tomcat
  labels:
    app.kubernetes.io/name: tomcat
    {{- include "my-tomcat-chart.labels" . | nindent 4 }}
spec:
  type: ClusterIP
  ports:
    - port: 80
      targetPort: {{ .Values.tomcat.ports.containerPort }}
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: tomcat
    app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}  

my-tomcat-chart\templates\tests

{{- if and .Values.nginx.enabled .Values.tomcat.enabled }}
apiVersion: v1
kind: Pod
metadata:
  name: "{{ include "my-tomcat-chart.fullname" . }}-test-connection"
  labels:
    {{- include "my-tomcat-chart.labels" . | nindent 4 }}
  annotations:
    "helm.sh/hook": test
spec:
  containers:
    - name: wget
      image: busybox
      command: ['wget']
      args: ['{{ include "my-tomcat-chart.fullname" . }}-nginx:80/a.html']
  restartPolicy: Never
{{- end }}