kubernetes中安装Jenkins实现ci/cd

最新推荐文章于 2025-09-13 12:59:37 发布

转载最新推荐文章于 2025-09-13 12:59:37 发布 · 477 阅读

0 ·

CC 4.0 BY-SA版权

原文链接：https://my.oschina.net/u/1271447/blog/3084197

文章标签：

#运维

本文详述了如何在Kubernetes集群上部署Jenkins，包括构建定制化的Jenkins镜像、配置持久化存储、创建服务账户、部署Jenkins服务及配置流程。通过实践操作，实现了CI/CD流程的自动化，适用于微服务开发环境。

微服务开发必不可少的就是k8s和docker了，有了这些之后你就会发现部署又是个问题，简直了。花了两天时间才完整的搭建一个Jenkins环境，由于避免复杂性，快速搭建ci的需求，本次只搭建Jenkins-master和并手动触发构建，去掉那些繁琐的NFS，证书生成啥的，本文绝对让你有感觉（前提要了解k8s和docker仓库工作模式）。下面列下创建Jenkins用的文件：

1、Jenkins镜像

2、持久化存储卷

3、找个demo

4、编写Jenkins

==========================================================

准备工作：你得有个k8s集群，然后自己构建镜像上传到自己的私库，如果直接使用Jenkins的镜像，安装完一系列的操作之后就会发现，执行构建会报错，报错的内容诸如：docker:not found之类，即使将宿主机的docker挂载到Jenkins中也无法执行，原因是没有建用户添加到docker的组中。自己的私库可以自己建也可以用阿里云的，我用的是阿里云的。

1、k8s部署Jenkins

1）构建Jenkins镜像，使用阿里云的构建镜像Dockerfile如下：

FROM jenkins/jenkins #这个是官方镜像，必须基于官方镜像构建，否则安装之后还是要更新，又报错

USER root
#清除了基础镜像设置的源，切换成阿里云的jessie源
RUN echo '' > /etc/apt/sources.list.d/jessie-backports.list \
  && echo "deb http://mirrors.aliyun.com/debian jessie main contrib non-free" > /etc/apt/sources.list \
  && echo "deb http://mirrors.aliyun.com/debian jessie-updates main contrib non-free" >> /etc/apt/sources.list \
  && echo "deb http://mirrors.aliyun.com/debian-security jessie/updates main contrib non-free" >> /etc/apt/sources.list
#更新源并安装缺少的包
RUN apt-get update && apt-get install -y libltdl7

ARG dockerGid=999

RUN echo "docker:x:${dockerGid}:jenkins" >> /etc/group \
USER jenkins

2）部署Jenkins镜像

A、创建pv持久化存储卷，本次用的是宿主机文件系统

apiVersion: "v1"
kind: "PersistentVolume"
metadata:
  name: jenkins-0
spec:
  capacity:
    storage: "10Gi"
  accessModes:
    - "ReadWriteMany"
  hostPath :
    path: /tmp
    volumeName: jenkins

B、创建账户

kubectl create -f service-account.yml

# In GKE need to get RBAC permissions first with
# kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin [--user=<user-name>|--group=<group-name>]

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins
  namespace: default

---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkins
  namespace: default
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: default
  
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkinsClusterRole
  namespace: default
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkinsClusterRuleBinding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: jenkinsClusterRole
subjects:
- kind: ServiceAccount
  name: jenkins
  namespace: default

C、创建Jenkins服务

kubectl create -f jenkins.yml

# jenkins

---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: jenkins
  namespace: default
  labels:
    name: jenkins
spec:
  serviceName: jenkins
  replicas: 1
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      name: jenkins
      labels:
        name: jenkins
    spec:
      terminationGracePeriodSeconds: 10
      serviceAccountName: jenkins
      imagePullSecrets:
            - name: registry-secret
      containers:
        - name: jenkins
          # image: jenkins/jenkins:lts-alpine
          # image: chadmoon/jenkins-docker-kubectl:latest h1kkan/jenkins-docker
          image: registry.cn-shanghai.aliyuncs.com/pml/jenkins:v1.0
          imagePullPolicy: Always
          ports:
            - containerPort: 8080
            - containerPort: 50000
          resources:
            limits:
              cpu: 1
              memory: 1Gi
            requests:
              cpu: 0.5
              memory: 500Mi
          env:
            - name: LIMITS_MEMORY
              valueFrom:
                resourceFieldRef:
                  resource: limits.memory
                  divisor: 1Mi
            - name: JAVA_OPTS
              # value: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
              value: -Xmx800m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
          volumeMounts:
            - name: jenkins-home
              mountPath: /var/jenkins_home
            - name: docker
              mountPath: /usr/bin/docker
            - name: docker-sock
              mountPath: /var/run/docker.sock
          securityContext:
            privileged: true
      volumes:
        - name: docker
          hostPath:
            path: /usr/bin/docker
        - name: docker-sock
          hostPath:
            path: /var/run/docker.sock
  volumeClaimTemplates:
  - metadata:
      name: jenkins-home
      # annotations:
      #   volume.beta.kubernetes.io/storage-class: anything
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi

---
apiVersion: v1
kind: Service
metadata:
  name: jenkins
  namespace: default
  annotations:
    # ensure the client ip is propagated to avoid the invalid crumb issue (k8s <1.7)
    # service.beta.kubernetes.io/external-traffic: OnlyLocal
spec:
  type: NodePort
  selector:
    name: jenkins
  # k8s 1.7+
  externalTrafficPolicy: Local
  ports:
    -
      name: http
      port: 80
      targetPort: 8080
      protocol: TCP
    -
      name: agent
      port: 50000
      protocol: TCP

2、登录并配置Jenkins

1）登录咋说呢，上图吧

找到服务端口

2）浏览器输入node的ip加上端口，诸如：http://192.168.16.131:32453，就可以了

3）装插件，kubernetes plugin

4）配凭据，一个是登录gitee的（下代码），一个是登录docker仓库的（pull/push镜像），还一个是ssh登录k8s的(deploy)

3、找个demo项目吧

看到这了，star/fork一下行不行，看提交记录就知道差点搞疯了！！！

https://gitee.com/chenqq/petclinic/

4、配下流水线

5、enjoy it!!!

如果持久化存储卷声明一直在pending，参考如下，重新设置状态：

{
  "kind": "PersistentVolumeClaim",
  "apiVersion": "v1",
  "metadata": {
    "name": "jenkins-home-jenkins-0",
    "namespace": "default",
    "selfLink": "/api/v1/namespaces/default/persistentvolumeclaims/jenkins-home-jenkins-0",
    "uid": "e0e85f5b-b64b-11e9-acc8-000c29e92529",
    "resourceVersion": "649259",
    "creationTimestamp": "2019-08-04T00:07:42Z",
    "labels": {
      "name": "jenkins"
    },
    "annotations": {
      "pv.kubernetes.io/bind-completed": "yes",
      "pv.kubernetes.io/bound-by-controller": "yes"
    },
    "finalizers": [
      "kubernetes.io/pvc-protection"
    ]
  },
  "spec": {
    "accessModes": [
      "ReadWriteOnce"
    ],
    "resources": {
      "requests": {
        "storage": "1Gi"
      }
    },
    "volumeName": "jenkins-1",
    "volumeMode": "Filesystem",
    "dataSource": null
  },
  "status": {
    "phase": "Bound",
    "accessModes": [
      "ReadWriteOnce"
    ],
    "capacity": {
      "storage": "10Gi"
    }
  }
}

转载于:https://my.oschina.net/u/1271447/blog/3084197