K8s集群+Rancher Server：部署DolphinScheduler 3.2.2集群

本文手把手演示了如何在 K8s + Rancher 环境中快速拉起一套生产可用的 Apache DolphinScheduler 3.2.2 集群。全文围绕“镜像加速、依赖本地化、存储持久化”三大痛点展开，附有详细的代码解析，收藏细看吧！

环境准备

1、软件准备

2、环境规划

部署

1、官网下载apache-dolphinscheduler源码

[root@master ~]# mkdir /opt/dolphinscheduler
#
[root@master ~]# cd /opt/dolphinscheduler
[root@master dolphinscheduler]# curl -Lo https://dlcdn.apache.org/dolphinscheduler/3.2.2/apache-dolphinscheduler-3.2.2-src.tar.gz
[root@master dolphinscheduler]# tar -xzvf ./apache-dolphinscheduler-3.2.2-src.tar.gz

2、修改Chart的镜像地址配置

（1）、修改Chart.yaml配置文件
将repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami改为repository: https://raw.gitmirror.com/bitnami/charts/archive-full-index/bitnami

[root@master dolphinscheduler]# cd /opt/dolphinscheduler/apache-dolphinscheduler-3.2.2-src/deploy/kubernetes/dolphinscheduler
[root@master dolphinscheduler]# vim Chart.yaml

# Application charts are a collection of templates that can be packaged into versioned archives
# to be deployed.
#
# Library charts provide useful utilities or functions for the chart developer. They're included as
# a dependency of application charts to inject those utilities and functions into the rendering
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
type: application

# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
version: 3.3.0-alpha

# This is the version number of the application being deployed. This version number should be
# incremented each time you make changes to the application.
appVersion: 3.3.0-alpha

dependencies:
- name: postgresql
  version: 10.3.18
  # Due to a change in the Bitnami repo, https://charts.bitnami.com/bitnami was truncated only
  # containing entries for the latest 6 months (from January 2022 on).
  # This URL: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami
  # contains the full 'index.yaml'.
  # See detail here: https://github.com/bitnami/charts/issues/10833
  #repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami
  repository: https://raw.gitmirror.com/bitnami/charts/archive-full-index/bitnami
  condition: postgresql.enabled
- name: zookeeper
  version: 6.5.3
  # Same as above.
  #repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami
  repository: https://raw.gitmirror.com/bitnami/charts/archive-full-index/bitnami
  condition: zookeeper.enabled
- name: mysql
  version: 9.4.1
  #repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami
  repository: https://raw.gitmirror.com/bitnami/charts/archive-full-index/bitnami
  condition: mysql.enabled
- name: minio
  version: 11.10.13
  #repository: https://raw.githubusercontent.com/bitnami/charts/archive-full-index/bitnami
  repository: https://raw.gitmirror.com/bitnami/charts/archive-full-index/bitnami
  condition: minio.enabled

（2）、执行helm更新
Chart.yaml配置文件修改完成后执行下面的命令：

[root@master dolphinscheduler]# helm repo add bitnami-full-index  https://raw.gitmirror.com/bitnami/charts/archive-full-index/bitnami
[root@master dolphinscheduler]# helm repo add bitnami https://charts.bitnami.com/bitnami
[root@master dolphinscheduler]# helm dependency update .

（3）、解压依赖文件
helm更新完成后，在当前目录会新生成【charts】目录，该目录下就是过helm更新下载到本地的依赖文件，如下图所示：

解压依赖包文件，因为本次安装除了Mysql都需要同步安装，所以要解压minio-11.10.13.tgz，postgresql-10.3.18.tgz，zookeeper-6.5.3.tgz这几个压缩包

[root@master dolphinscheduler]# tar -zxvf minio-11.10.13.tgz
[root@master dolphinscheduler]# tar -zxvf postgresql-10.3.18.tgz
[root@master dolphinscheduler]# tar -zxvf zookeeper-6.5.3.tgz

3、修改Dolphinscheduler的Helm配置文件

cd /opt/dolphinscheduler/apache-dolphinscheduler-3.2.2-src/deploy/kubernetes/dolphinscheduler
vim values.yaml

（1）、修改初始镜像
需要修改busybox和dolphinscheduler各个组件的镜像地址，修改后如下：

# -- World time and date for cities in all time zones
timezone: "Asia/Shanghai"

# -- Used to detect whether dolphinscheduler dependent services such as database are ready
initImage:
  # -- Image pull policy. Options: Always, Never, IfNotPresent
  pullPolicy: "IfNotPresent"
  # -- Specify initImage repository
  #busybox: "busybox:1.30.1"
  busybox: "registry.cn-hangzhou.aliyuncs.com/docker_image-ljx/busybox:1.30.1"

image:
  # -- Docker image repository for the DolphinScheduler
  registry: registry.cn-hangzhou.aliyuncs.com/docker_image-ljx
  # -- Docker image version for the DolphinScheduler
  tag: 3.2.2
  # -- Image pull policy. Options: Always, Never, IfNotPresent
  pullPolicy: "IfNotPresent"
  # -- Specify a imagePullSecrets
  pullSecret: ""
  # -- master image
  master: dolphinscheduler-master
  # -- worker image
  worker: dolphinscheduler-worker
  # -- api-server image
  api: dolphinscheduler-api
  # -- alert-server image
  alert: dolphinscheduler-alert-server
  # -- tools image
  tools: dolphinscheduler-tools

（2）、修改postgresql配置

• 本文安装依赖的数据库配置了默认的数据库postgresql，参见配置datasource.profile: postgresql，但是values.yaml的默认配置中并没有配置postgresql镜像，会导致postgresql镜像拉取失败。
• 本文dolphinscheduler依赖于bitnami postgresql的Chart，且已经在上一步 解压了。获取镜像地址的方式只需进入解压目录（/opt/dolphinscheduler/apache-dolphinscheduler-3.2.2-src/deploy/kubernetes/dolphinscheduler/charts/postgresql）。

查看postgresql的values.yaml（/opt/dolphinscheduler/apache-dolphinscheduler-3.2.2-src/deploy/kubernetes/dolphinscheduler/charts/postgresql/values.yaml）找到镜像地址：

image:
  registry: docker.io
  repository: bitnami/postgresql
  tag: 11.11.0-debian-10-r71
  ## Specify a imagePullPolicy
  ## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'
  ## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
  ##
  pullPolicy: IfNotPresent
  ## Optionally specify an array of imagePullSecrets.
  ## Secrets must be manually created in the namespace.
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
  ##
  # pullSecrets:
  #   - myRegistryKeySecretName

  ## Set to true if you would like to see extra information on logs
  ## It turns BASH and/or NAMI debugging in the image
  ##
  debug: false

复制上述配置文件中的image配置信息，如下图所示：

将上步复制到image粘贴至/opt/dolphinscheduler/apache-dolphinscheduler-3.2.2-src/deploy/kubernetes/dolphinscheduler/values.yaml配置文件中postgresql配置项下，如下所示：

datasource:
  # -- The profile of datasource
  profile: postgresql

postgresql:
  image:
    registry: docker.io
    repository: bitnami/postgresql
    tag: 11.11.0-debian-10-r71
  # -- If not exists external PostgreSQL, by default, the DolphinScheduler will use a internal PostgreSQL
  enabled: true
  # -- The username for internal PostgreSQL
  postgresqlUsername: "root"
  # -- The password for internal PostgreSQL
  postgresqlPassword: "root"
  # -- The database for internal PostgreSQL
  postgresqlDatabase: "dolphinscheduler"
  # -- The driverClassName for internal PostgreSQL
  driverClassName: "org.postgresql.Driver"
  # -- The params for internal PostgreSQL
  params: "characterEncoding=utf8"
  persistence:
    # -- Set postgresql.persistence.enabled to true to mount a new volume for internal PostgreSQL
    enabled: false
    # -- `PersistentVolumeClaim` size
    size: "20Gi"
    # -- PostgreSQL data persistent volume storage class. If set to "-", storageClassName: "", which disables dynamic provisioning
    storageClass: "-"

（3）、修改minio配置
查看minio的values.yaml（/opt/dolphinscheduler/apache-dolphinscheduler-3.2.2-src/deploy/kubernetes/dolphinscheduler/charts/minio/values.yaml）找到镜像地址：

image:
  registry: docker.io
  repository: bitnami/minio
  tag: 2022.10.29-debian-11-r0
  digest: ""
  ## Specify a imagePullPolicy
  ## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'
  ## ref: https://kubernetes.io/docs/user-guide/images/#pre-pulling-images
  ##
  pullPolicy: IfNotPresent
  ## Optionally specify an array of imagePullSecrets.
  ## Secrets must be manually created in the namespace.
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
  ## e.g:
  ## pullSecrets:
  ##   - myRegistryKeySecretName
  ##
  pullSecrets: []
  ## Set to true if you would like to see extra information on logs
  ##
  debug: false

复制上述配置文件中的image配置信息，如下图所示：

将上步复制到image粘贴至/opt/dolphinscheduler/apache-dolphinscheduler-3.2.2-src/deploy/kubernetes/dolphinscheduler/values.yaml配置文件中minio配置项下，如下所示：

minio:
  image:
    registry: docker.io
    repository: bitnami/minio
    tag: 2022.10.29-debian-11-r0

  # -- Deploy minio and configure it as the default storage for DolphinScheduler, note this is for demo only, not for production.
  enabled: true
  auth:
    # -- minio username
    rootUser: minioadmin
    # -- minio password
    rootPassword: minioadmin
  persistence:
    # -- Set minio.persistence.enabled to true to mount a new volume for internal minio
    enabled: false
  # -- minio default buckets
  defaultBuckets: "dolphinscheduler"

（4）、修改zookeeper配置
查看zookeeper的values.yaml（/opt/dolphinscheduler/apache-dolphinscheduler-3.2.2-src/deploy/kubernetes/dolphinscheduler/charts/zookeeper/values.yaml）找到镜像地址：

image:
  registry: docker.io
  repository: bitnami/zookeeper
  tag: 3.6.2-debian-10-r185

  ## Specify a imagePullPolicy
  ## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'
  ## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
  ##
  pullPolicy: IfNotPresent
  ## Optionally specify an array of imagePullSecrets.
  ## Secrets must be manually created in the namespace.
  ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
  ##
  # pullSecrets:
  #   - myRegistryKeySecretName
  ## Set to true if you would like to see extra information on logs
  ## It turns BASH and/or NAMI debugging in the image
  ##
  debug: false

复制上述配置文件中的image配置信息，如下图所示：

将上步复制到image粘贴至/opt/dolphinscheduler/apache-dolphinscheduler-3.2.2-src/deploy/kubernetes/dolphinscheduler/values.yaml配置文件中zookeeper配置项下，如下所示：

zookeeper:
  image:
    registry: docker.io
    repository: bitnami/zookeeper
    tag: 3.6.2-debian-10-r185

  # -- If not exists external registry, the zookeeper registry will be used by default.
  enabled: true
  service:
    # -- The port of zookeeper
    port: 2181
  # -- A list of comma separated Four Letter Words commands to use
  fourlwCommandsWhitelist: "srvr,ruok,wchs,cons"
  persistence:
    # -- Set `zookeeper.persistence.enabled` to true to mount a new volume for internal ZooKeeper
    enabled: false
    # -- PersistentVolumeClaim size
    size: "20Gi"
    # -- ZooKeeper data persistent volume storage class. If set to "-", storageClassName: "", which disables dynamic provisioning
    storageClass: "-"

（5）、修改master配置

master:
  # -- Enable or disable the Master component
  enabled: true
  replicas: "1"
  resources:
    limits:
      memory: "4Gi"
      cpu: "4"
    requests:
      memory: "2Gi"
      cpu: "500m"  
  persistentVolumeClaim:
    # -- Set `worker.persistentVolumeClaim.enabled` to `true` to enable `persistentVolumeClaim` for `worker`
    enabled: true
    ## dolphinscheduler data volume
    dataPersistentVolume:
      # -- Set `worker.persistentVolumeClaim.dataPersistentVolume.enabled` to `true` to mount a data volume for `worker`
      enabled: true
      # -- `PersistentVolumeClaim` access modes
      accessModes:
      - "ReadWriteOnce"
      # -- `Worker` data persistent volume storage class. If set to "-", storageClassName: "", which disables dynamic provisioning
      storageClassName: "-"
      # -- `PersistentVolumeClaim` size
      storage: "20Gi"
  env:
    # -- The jvm options for master server
    JAVA_OPTS