简介
官网： https://karmada.io/zh/docs/
github： https://github.com/karmada-io/karmada
Karmada：Kubernetes Armada，k8s 舰队。由华为云、工商银行、小红书、中国一汽等8家企业联合发起，2021年4月25日正式开源。同年9月，Karmada 成为 CNCF 首个多云容器编排项目。是 Kubernetes 管理系统，能跨多个 k8s 集群（包括公有云、私有云、边缘计算集群），提供集中式管理、高可用性、故障恢复和流量调度等功能（其实就是可以实现两地三中心、同城双活、异地容灾、把一个 deploy 的副本按照区域/集群等调度策略调度到不同的 k8s 集群）。
优势：
- K8s 原生 API 兼容：无需修改 k8s 代码
- 开箱即用
- 避免与单一供应商锁定
- 集中管理
- 多集群调度策略
架构：
一个 deploy 应用通过 karmada control-plane 把多个副本调度到不同的 k8s 集群：
karmada control-plane：karmada 控制平面
- karmada apiserver：karmada 自己的 apiserver，可以通过 kubectl、client-go 等调用
- etcd：karmada 自己的 etcd
- karmada scheduler：karmada 自己的 scheduler
- karmada controllers：karmada 自己的 controller，包括不限于：
- cluster controller：集群控制器
- policy controller：策略控制器
- binding controller：绑定控制器
- execution controller：执行控制器
安装karmadactl
karmadactl：就是 karmada 命令行工具
[root@k8s-master01 ~]# wget https://github.com/karmada-io/karmada/releases/download/v1.4.1/karmadactl-linux-amd64.tgz
[root@k8s-master01 ~]# tar xf karmadactl-linux-amd64.tgz
[root@k8s-master01 ~]# mv karmadactl /usr/local/bin/
安装 karmada
前提：需要有 k8s 集群，karmada 以 k8s 的方式安装的
[root@k8s-master01 ~]# cat karmada.sh
karmadactl init \
--namespace='karmada-system' \ #karmada所在的ns
--port 30000 \ #karmada端口
--etcd-image='registry.k8s.io/etcd:3.5.6-0' \ #karmada所需的etcd
--etcd-pvc-size='10Gi' \ #以下三行是karmada所需的etcd的存储信息
--etcd-storage-mode='PVC' \
--storage-classes-name='nfs-client' \
--etcd-replicas=1 \
--karmada-aggregated-apiserver-replicas=1 \ #karmada组件的副本数
--karmada-apiserver-replicas=1 \
--karmada-controller-manager-replicas=1 \
--karmada-kube-controller-manager-replicas=1 \
--karmada-scheduler-replicas=1 \
--karmada-webhook-replicas=1 \
--karmada-aggregated-apiserver-image='docker.io/karmada/karmada-aggregated-apiserver:latest' \
--karmada-apiserver-image='registry.k8s.io/kube-apiserver:v1.24.9' \ #karmada-apiserver的镜像地址。因为它用的就是k8s apiserver的镜像，所以需要和k8s的版本保持一致
--karmada-controller-manager-image='docker.io/karmada/karmada-controller-manager:latest' \
--karmada-kube-controller-manager-image='registry.k8s.io/kube-controller-manager:v1.24.9' \ #karmada-kube-controller-manager的镜像地址。因为它用的就是k8s controller-manager的镜像，所以需要和k8s的版本保持一致
--karmada-scheduler-image='docker.io/karmada/karmada-scheduler:latest' \
--karmada-webhook-image='docker.io/karmada/karmada-webhook:latest'
注意：karmada创建完成后会创建自己的kubeconfig文件
[root@k8s-master01 ~]# sh karmada.sh
# 把以下信息保存一下：
Karmada is installed successfully.
Register Kubernetes cluster to Karmada control plane.
# 采用push模式注册k8s集群
Register cluster with 'Push' mode
Step 1: Use "karmadactl join" command to register the cluster to Karmada control plane. --cluster-kubeconfig is kubeconfig of the member cluster.
(In karmada)~# MEMBER_CLUSTER_NAME=$(cat ~/.kube/config | grep current-context | sed 's/: /\n/g'| sed '1d')
(In karmada)~# karmadactl --kubeconfig /etc/karmada/karmada-apiserver.config join ${MEMBER_CLUSTER_NAME} --cluster-kubeconfig=$HOME/.kube/config
Step 2: Show members of karmada
(In karmada)~# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get clusters
# 采用pull模式注册k8s集群
Register cluster with 'Pull' mode
Step 1: Use "karmadactl register" command to register the cluster to Karmada control plane. "--cluster-name" is set to cluster of current-context by default.
(In member cluster)~# karmadactl register 192.168.10.160:30000 --token lw3mto.jv3qeoqtt7hm1vxl --discovery-token-ca-cert-hash sha256:11d5ff60b1fbeba0a4df9926157a4ac86d89ac649b5138a81bd763dbf6a7b7b0
Step 2: Show members of karmada
(In karmada)~# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get clusters
[root@k8s-master01 ~]# kubectl get pods -n karmada-system
NAME READY STATUS RESTARTS AGE
etcd-0 1/1 Running 0 13h
karmada-agent-7d85f6cc69-x957m 1/1 Running 0 13h
karmada-aggregated-apiserver-6577d5769f-h8nmp 1/1 Running 0 13h
karmada-apiserver-664446c84c-l5t26 1/1 Running 0 13h
karmada-controller-manager-776f8fcbff-fwjqv 1/1 Running 0 13h
karmada-scheduler-55dfc748b7-bw2t8 1/1 Running 0 13h
karmada-webhook-6cc85d8f86-hr52h 1/1 Running 0 13h
kube-controller-manager-7767d948d4-2s4vx 1/1 Running 0 13h
用 karmadactl 创建联邦
push 模式是 karmada 推送到 k8s 集群；
pull 模式是 k8s 集群从 karmada 拉取
使用时，可以使用 push 模式、pull 模式，或者两者混合使用
karmada push模式创建
这一步其实就是获取k8s集群的名字：
[root@k8s-master01 kubedir]# MEMBER_CLUSTER_NAME=$(cat ~/kubeconfigdir/config12 | grep current-context | sed 's/: /\n/g'| sed '1d')
[root@k8s-master01 kubedir]# echo ${MEMBER_CLUSTER_NAME}
member1
[root@k8s-master01 kubedir]# karmadactl --kubeconfig /etc/karmada/karmada-apiserver.config join ${MEMBER_CLUSTER_NAME} --cluster-kubeconfig=$HOME/kubeconfigdir/config12
cluster(member1) is joined successfully
[root@k8s-master01 kubedir]# kubectl --kubeconfig=config12 config use-context member2
Switched to context "member2".
[root@k8s-master01 kubedir]# MEMBER_CLUSTER_NAME=$(cat ~/kubeconfigdir/config12 | grep current-context | sed 's/: /\n/g'| sed '1d')
[root@k8s-master01 kubedir]# echo ${MEMBER_CLUSTER_NAME}
member2
[root@k8s-master01 kubedir]# karmadactl --kubeconfig /etc/karmada/karmada-apiserver.config join ${MEMBER_CLUSTER_NAME} --cluster-kubeconfig=$HOME/kubeconfigdir/config12
cluster(member2) is joined successfully
[root@k8s-master01 kubedir]# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get clusters
NAME VERSION MODE READY AGE
member1 v1.24.9 Push True 96s
member2 v1.24.9 Push True 59s
karmada pull模式创建
pull 模式需要在要加入联邦的 k8s 集群操作，其原理就是在当前 k8s 注入了一个 karmada-agent
加入第一个 k8s 集群：在第一个 k8s 集群操作
#192.168.10.160:30000：联邦集群的地址
#--cluster-name：指定要加入联邦的k8s集群的名字，一般跟k8s集群上下文里的名字一致。如果不指定，加入联邦后显示的名字会是kubernetes等，不易于管理
[root@k8s-master01 ~]# karmadactl register 192.168.10.160:30000 --token 2sh3py.li1p4o8gz4wqudrw --discovery-token-ca-cert-hash sha256:b1913d9f6410c2a4e19623223b141000a807cd2b15bbc3096ffc9a1dc39a5fe5 --cluster-name member1
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR]: /etc/karmada/pki/ca.crt already exists #报错：删除掉就行
[preflight] Please check the above errors
[root@k8s-master01 ~]# rm -rf /etc/karmada/pki/ca.crt
[root@k8s-master01 ~]# karmadactl register 192.168.10.160:30000 --token 2sh3py.li1p4o8gz4wqudrw --discovery-token-ca-cert-hash sha256:b1913d9f6410c2a4e19623223b141000a807cd2b15bbc3096ffc9a1dc39a5fe5 --cluster-name member1
[preflight] Running pre-flight checks
[prefligt] All pre-flight checks were passed
[karmada-agent-start] Waiting to perform the TLS Bootstrap
[karmada-agent-start] Waiting to construct karmada-agent kubeconfig
[karmada-agent-start] Waiting the necessary secret and RBAC
[karmada-agent-start] Waiting karmada-agent Deployment
W0107 13:21:28.781319 28891 check.go:52] pod: karmada-agent-5dd59f9696-5tt82 not ready. status: ContainerCreating
W0107 13:21:29.784425 28891 check.go:52] pod: karmada-agent-5dd59f9696-5tt82 not ready. status: ContainerCreating
W0107 13:21:30.785356 28891 check.go:52] pod: karmada-agent-5dd59f9696-5tt82 not ready. status: ContainerCreating
W0107 13:21:31.785006 28891 check.go:52] pod: karmada-agent-5dd59f9696-5tt82 not ready. status: ContainerCreating
W0107 13:21:32.784396 28891 check.go:52] pod: karmada-agent-5dd59f9696-5tt82 not ready. status: ContainerCreating
W0107 13:21:33.784441 28891 check.go:52] pod: karmada-agent-5dd59f9696-5tt82 not ready. status: ContainerCreating
W0107 13:21:34.784517 28891 check.go:52] pod: karmada-agent-5dd59f9696-5tt82 not ready. status: ContainerCreating
W0107 13:21:35.785503 28891 check.go:52] pod: karmada-agent-5dd59f9696-5tt82 not ready. status: ContainerCreating
W0107 13:21:36.784929 28891 check.go:52] pod: karmada-agent-5dd59f9696-5tt82 not ready. status: ContainerCreating
W0107 13:21:37.784301 28891 check.go:52] pod: karmada-agent-5dd59f9696-5tt82 not ready. status: ContainerCreating
W0107 13:21:38.785135 28891 check.go:52] pod: karmada-agent-5dd59f9696-5tt82 not ready. status: ContainerCreating
I0107 13:21:39.784795 28891 check.go:49] pod: karmada-agent-5dd59f9696-5tt82 is ready. status: Running
cluster(member1) is joined successfully
加入第二个 k8s 集群：在第二个 k8s 集群操作
[root@k8s-master01 ~]# karmadactl register 192.168.10.160:30000 --token 2sh3py.li1p4o8gz4wqudrw --discovery-token-ca-cert-hash sha256:b1913d9f6410c2a4e19623223b141000a807cd2b15bbc3096ffc9a1dc39a5fe5 --cluster-name member2
[preflight] Running pre-flight checks
[prefligt] All pre-flight checks were passed
[karmada-agent-start] Waiting to perform the TLS Bootstrap
[karmada-agent-start] Waiting to construct karmada-agent kubeconfig
[karmada-agent-start] Waiting the necessary secret and RBAC
[karmada-agent-start] Waiting karmada-agent Deployment
W0107 13:21:44.795888 28340 check.go:52] pod: karmada-agent-76b7857fb4-zdnc9 not ready. status: ContainerCreating
W0107 13:21:45.798850 28340 check.go:52] pod: karmada-agent-76b7857fb4-zdnc9 not ready. status: ContainerCreating
W0107 13:21:46.798943 28340 check.go:52] pod: karmada-agent-76b7857fb4-zdnc9 not ready. status: ContainerCreating
W0107 13:21:47.800993 28340 check.go:52] pod: karmada-agent-76b7857fb4-zdnc9 not ready. status: ContainerCreating
W0107 13:21:48.799594 28340 check.go:52] pod: karmada-agent-76b7857fb4-zdnc9 not ready. status: ContainerCreating
W0107 13:21:49.799841 28340 check.go:52] pod: karmada-agent-76b7857fb4-zdnc9 not ready. status: ContainerCreating
W0107 13:21:50.799034 28340 check.go:52] pod: karmada-agent-76b7857fb4-zdnc9 not ready. status: ContainerCreating
W0107 13:21:51.799729 28340 check.go:52] pod: karmada-agent-76b7857fb4-zdnc9 not ready. status: ContainerCreating
W0107 13:21:52.801867 28340 check.go:52] pod: karmada-agent-76b7857fb4-zdnc9 not ready. status: ContainerCreating
I0107 13:21:53.799843 28340 check.go:49] pod: karmada-agent-76b7857fb4-zdnc9 is ready. status: Running
cluster(member2) is joined successfully
查看联邦信息：
[root@k8s-master01 ~]# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get clusters
NAME VERSION MODE READY AGE
member1 v1.24.9 Pull True 119s
member2 v1.24.9 Pull True 104s
删除 k8s 集群
从 karmada 中删除 k8s 集群：
删除member1集群：
[root@k8s-master01 ~]# karmadactl --kubeconfig /etc/karmada/karmada-apiserver.config unjoin member1
删除member2集群：
[root@k8s-master01 ~]# karmadactl --kubeconfig /etc/karmada/karmada-apiserver.config unjoin member2
部署应用
部署应用
要部署的 deploy nginx 应用：
[root@k8s-master01 nginxdir]# cat n1.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx1
spec:
replicas: 1
selector:
matchLabels:
app: nginx1
template:
metadata:
labels:
app: nginx1
spec:
containers:
- name: nginx1
image: nginx:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
不能这样部署：这样是咋k8s集群里部署：
[root@k8s-master01 nginxdir]# kubectl apply -f n1.yaml
而是这样部署：通过karmada部署：
==注意==：命令执行后，执行的是模板，其实并没有运行在k8s中
[root@k8s-master01 nginxdir]# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config apply -f n1.yaml
deployment.apps/nginx1 created
karmadactl：能够查看pod是部署在哪个k8s集群
查看：发现nginx deploy 确实没有被创建：
[root@k8s-master01 nginxdir]# karmadactl --kubeconfig /etc/karmada/karmada-apiserver.config get pods
NAME CLUSTER READY STATUS RESTARTS AGE
nfs-client-provisioner-dfd8488d7-hhmwh member1 1/1 Running 3 (24h ago) 30h
nfs-client-provisioner-dfd8488d7-rtmdj member2 1/1 Running 2 (24h ago) 30h
传播策略文件：
[root@k8s-master01 pdir]# cat p1.yaml
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
name: example-policy #传播策略资源的名字
spec:
resourceSelectors:
- apiVersion: apps/v1
kind: Deployment
name: nginx1
placement:
clusterAffinity: #指定部署在哪个k8s集群
clusterNames:
- member1
[root@k8s-master01 pdir]# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config apply -f p1.yaml
propagationpolicy.policy.karmada.io/example-policy created
通过karmada查看部署nginx deploy：
[root@k8s-master01 pdir]# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config get deployment
NAME READY UP-TO-DATE AVAILABLE AGE
nginx1 1/1 1 1 3m9s
通过karmada查看nginx pod：
[root@k8s-master01 pdir]# karmadactl --kubeconfig /etc/karmada/karmada-apiserver.config get pod
NAME CLUSTER READY STATUS RESTARTS AGE
nfs-client-provisioner-dfd8488d7-hhmwh member1 1/1 Running 3 (25h ago) 30h
nginx1-7bff55d4bd-lcbcn member1 1/1 Running 0 2m40s
nfs-client-provisioner-dfd8488d7-rtmdj member2 1/1 Running 2 (25h ago) 30h
在部署的k8s集群查看：
[root@k8s-master01 pdir]# kubectl get pods
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-dfd8488d7-hhmwh 1/1 Running 3 (25h ago) 30h
nginx1-7bff55d4bd-lcbcn 1/1 Running 0 44s
修改传播策略
修改传播策略：
[root@k8s-master01 pdir]# cat p2.yaml
apiVersion: policy.karmada.io/v1alpha1
kind: PropagationPolicy
metadata:
name: example-policy
spec:
resourceSelectors:
- apiVersion: apps/v1
kind: Deployment
name: nginx1
placement:
clusterAffinity:
clusterNames: # 从member1修改为member2
- member2
执行：
[root@k8s-master01 pdir]# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config apply -f p2.yaml
propagationpolicy.policy.karmada.io/example-policy configured
通过karmada查看：nginx deploy已经运行在member2了：
[root@k8s-master01 pdir]# karmadactl --kubeconfig /etc/karmada/karmada-apiserver.config get pod
NAME CLUSTER READY STATUS RESTARTS AGE
nfs-client-provisioner-dfd8488d7-hhmwh member1 1/1 Running 3 (25h ago) 30h
nfs-client-provisioner-dfd8488d7-rtmdj member2 1/1 Running 2 (25h ago) 30h
nginx1-7bff55d4bd-fm7kj member2 1/1 Running 0 28s
修改应用模板
修改应用模板：
[root@k8s-master01 nginxdir]# cat n1.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx1
spec:
replicas: 2 #从1修改为2
selector:
matchLabels:
app: nginx1
template:
metadata:
labels:
app: nginx1
spec:
containers:
- name: nginx1
image: nginx:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
执行：
[root@k8s-master01 nginxdir]# kubectl --kubeconfig /etc/karmada/karmada-apiserver.config apply -f n1.yaml
deployment.apps/nginx1 configured
通过karmada查看：nginx deploy的 pod由1个变为2个：
[root@k8s-master01 nginxdir]# karmadactl --kubeconfig /etc/karmada/karmada-apiserver.config get pod
NAME CLUSTER READY STATUS RESTARTS AGE
nfs-client-provisioner-dfd8488d7-hhmwh member1 1/1 Running 3 (25h ago) 30h
nfs-client-provisioner-dfd8488d7-rtmdj member2 1/1 Running 2 (25h ago) 30h
nginx1-7bff55d4bd-fm7kj member2 1/1 Running 0 8m46s
nginx1-7bff55d4bd-llhv2 member2 1/1 Running 0 5m12s
版权声明
原创不易，转载请标注原出处！！！
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
