玩k8s？安全机制与RBAC使用方法

最新推荐文章于 2024-11-12 17:32:53 发布

原创最新推荐文章于 2024-11-12 17:32:53 发布 · 374 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#kubernetes #linux

Docker容器与K8s 专栏收录该内容

27 篇文章

订阅专栏

k8s的安全机制与RBAC使用方法

文章目录

前言
一：k8s安全机制

前言

一：k8s安全机制

安全框架、

传输安全、认证，授权，准入控制

使用rbac授权

1.1：kubernetes安全框架

mark

安全框架的流程

流程：kubectl先请求api资源，然后是过三关，第一关是认证（Authentication），第二关是授权（Authorization），第三关是准入控制（Admission Control），只有通过这三关才可能会被k8s创建资源。

K8S安全控制框架主要由下面3个阶段进行控制，每一个阶段都支持插件方式，通过API Server配置来启用插件。

普通用户若要安全访问集群API Server，往往需要证书、Token或者用户名+密码；Pod访问，需要ServiceAccount

apiserver使用的是token认证

[root@master ~]# ps aux |grep apiserver
root      12636  3.0 23.6 420460 235620 ?       Ssl  5月17 156:09 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.233.131:2379,https://192.168.233.132:2379,https://192.168.233.133:2379 --bind-address=192.168.233.131 --secure-port=6443 --advertise-address=192.168.233.131 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction --authorization-mode=RBAC,Node --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem
root      22849  0.0  0.0 112728   976 pts/0    S+   10:20   0:00 grep --color=auto apiserver
'//其中能够查询到token认证等信息'

查看ServiceAccount，可以通过ServiceAccount在pod中去访问apiserver

[root@master ~]# kubectl get sa
NAME      SECRETS   AGE
default   1         21d
'//Service Account它并不是给kubernetes集群的用户使用的，而是给pod里面的进程使用的，它为pod提供必要的身份认证。'

传输安全方面：8080用于内部通讯，6443是提供给外部访问的端口

[root@master ~]# netstat -ntap |grep 8080 |grep LISTEN	'//默认8080监听本地（是通过master及其他组件连接使用）'
tcp        0      0 127.0.0.1:8080          0.0.0.0:*               LISTEN      12636/kube-apiserve 
[root@master ~]# netstat -ntap |grep 6443 |grep LISTEN	'//对外提供服务端口是6443'
tcp        0      0 192.168.233.131:6443    0.0.0.0:*               LISTEN      12636/kube-apiserve

1.2：第一关：Authentication认证

三种客户端身份认证：

1、HTTPS 证书认证：基于CA证书签名的数字证书认证

2、HTTP Token认证：通过一个Token来识别用户（生产环境中使用广泛）

3、HTTP Base认证：用户名+密码的方式认证

1、HTTPS证书认证

[root@master ~]# cat k8s/k8s-cert/k8s-cert.sh
cat > ca-config.json <<EOF
{
  "signing": {
    "default": {
      "expiry": "87600h"
    },
    "profiles": {
      "kubernetes": {
         "expiry": "87600h",
         "usages": [
            "signing",
            "key encipherment",
            "server auth",
            "client auth"
        ]
      }
    }
  }
}
EOF

cat > ca-csr.json <<EOF
{
    "CN": "kubernetes",
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "Beijing",
            "ST": "Beijing",
      	    "O": "k8s",
            "OU": "System"
        }
    ]
}
EOF

cfssl gencert -initca ca-csr.json | cfssljson -bare ca -

#-----------------------

cat > server-csr.json <<EOF
{
    "CN": "kubernetes",
    "hosts": [
      "10.0.0.1",
      "127.0.0.1",
      "192.168.233.131",	'//此处直接指定了负载均衡和master的节点'
      "192.168.233.130",
      "192.168.233.100",
      "192.168.233.128",
      "192.168.233.129",
      "kubernetes",
      "kubernetes.default",
      "kubernetes.default.svc",
      "kubernetes.default.svc.cluster",
      "kubernetes.default.svc.cluster.local"
    ],
    "key": {
        "algo": "rsa",
        "size": 2048
    },
    "names": [
        {
            "C": "CN",
            "L": "BeiJing",
            "ST": "BeiJing",
            "O": "k8s",
            "OU": "System"
        }
    ]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes server-csr.json | cfssljson -bare server

#-----------------------

cat > admin-csr.json <<EOF
{
  "CN": "admin",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "BeiJing",
      "ST": "BeiJing",
      "O": "system:masters",
      "OU": "System"
    }
  ]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes admin-csr.json | cfssljson -bare admin

#-----------------------

cat > kube-proxy-csr.json <<EOF
{
  "CN": "system:kube-proxy",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "BeiJing",
      "ST": "BeiJing",
      "O": "k8s",
      "OU": "System"
    }
  ]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kube-proxy-csr.json | cfssljson -bare kube-proxy

2、httpd的token认证

[root@master ~]# cat /opt/kubernetes/cfg/token.csv 
7ea8f86b157225fd4b9273765e88a3ca,kubelet-bootstrap,10001,"system:kubelet-bootstrap"

1.3：第二关：Authorization授权

RBAC（Role-Based Access Control，基于角色的访问控制）：负责完成授权（Authorization）工作，允许通过Kubernetes API动态配置策略。
使用RBAC授权

mark

角色：

1、Role：授权特定命名空间的访问权限

2、ClusterRole：授权所有命名空间的访问权限

角色绑定

1、RoleBinding：将角色绑定到主体（即subject）

2、ClusterRoleBinding：将集群角色绑定到主体

主体（subject）

1、User：用户

2、Group：用户组

3、ServiceAccount：服务账号

1.3.1：RBAC使用测试

1、创建名称空间dabao

[root@master ~]# kubectl create ns dabao
namespace/dabao created
[root@master ~]# kubectl get ns
NAME          STATUS   AGE
dabao         Active   7s
default       Active   21d
kube-public   Active   21d
kube-system   Active   21d

2、创建测试pod

[root@master ~]# kubectl run nginx01  --image=nginx -n dabao
kubectl run --generator=deployment/apps.v1beta1 is DEPRECATED and will be removed in a future version. Use kubectl create instead.
deployment.apps/nginx-test created
[root@master ~]# kubectl get pod -n dabao
NAME                         READY   STATUS    RESTARTS   AGE
nginx01-77c44977fd-p4xbh     1/1     Running   0          15m

3、扩容成3个副本，使用scale命令

[root@master ~]# kubectl scale deploy/nginx01 --replicas=3 -n dabao
deployment.extensions/nginx01 scaled
[root@master ~]# kubectl get pod -n dabao
NAME                         READY   STATUS    RESTARTS   AGE
nginx01-77c44977fd-98jc4     1/1     Running   0          31s
nginx01-77c44977fd-nt424     1/1     Running   0          31s
nginx01-77c44977fd-p4xbh     1/1     Running   0          17m

4、创建角色

[root@master ~]# vim rbac-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: dabao
  name: pod-reader
rules:
- apiGroups: [""] # "" indicates the core API group
  resources: ["pods"]	'//创建角色只有pod资源的操作权限'
  verbs: ["get", "watch", "list"]	'//只有这些操作可以使用'
[root@master ~]# kubectl apply -f rbac-role.yaml 
role.rbac.authorization.k8s.io/pod-reader created
[root@master ~]# kubectl get role -n dabao
NAME         AGE
pod-reader   8s
[root@master ~]#

5、角色绑定

[root@master ~]# vim rbac-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: dabao
subjects:
- kind: User
  name: zhangsan
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role   
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io
[root@master ~]# kubectl apply -f rbac-rolebinding.yaml 
rolebinding.rbac.authorization.k8s.io/read-pods created
[root@master ~]# kubectl get role,rolebinding -n dabao
NAME                                        AGE
role.rbac.authorization.k8s.io/pod-reader   2m14s

NAME                                              AGE
rolebinding.rbac.authorization.k8s.io/read-pods   13s

6、创建两个rbac的文件

[root@master ~]# mkdir zhangsan
[root@master ~]# cd zhangsan/
[root@master zhangsan]# vim rbac.yaml 
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: default
  name: pod-reader
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "watch", "list"]

---

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: read-pods
  namespace: default
subjects:
- kind: User
  name: jane
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io




apiVersion: v1
kind: ServiceAccount
metadata:
  name: pod-reader
  namespace: default

---

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: sa-read-pods
  namespace: default
subjects:
- kind: ServiceAccount
  name: pod-reader
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io
[root@master zhangsan]#  vim rbac-user.sh
cat > zhangsan-csr.json <<EOF
{
  "CN": "zhangsan",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "BeiJing",
      "ST": "BeiJing"
    }
  ]
}
EOF

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes zhangsan-csr.json | cfssljson -bare zhangsan 

kubectl config set-cluster kubernetes \
  --certificate-authority=ca.pem \
  --embed-certs=true \
  --server=https://192.168.233.100:6443 \	'//修改为负载均衡VIP地址'
  --kubeconfig=zhangsan-kubeconfig
  
kubectl config set-credentials zhangsan \
  --client-key=zhangsan-key.pem \
  --client-certificate=zhangsan.pem \
  --embed-certs=true \
  --kubeconfig=zhangsan-kubeconfig

kubectl config set-context default \
  --cluster=kubernetes \
  --user=zhangsan \
  --kubeconfig=zhangsan-kubeconfig

kubectl config use-context default --kubeconfig=zhangsan-kubeconfig

7、拷贝证书到张三目录并安装格式化工具

[root@master zhangsan]# cp /root/k8s/k8s-cert/ca* ./
[root@master zhangsan]# ls
ca-config.json  ca.csr  ca-csr.json  ca-key.pem  ca.pem  rbac-user.sh  rbac.yaml
[root@master zhangsan]# yum install dos2unix -y
[root@master zhangsan]# dos2unix rbac-user.sh 	'//执行格式化'
dos2unix: converting file rbac-user.sh to Unix format ...
[root@master zhangsan]# bash rbac-user.sh 	'//运行脚本'

8、使用zhangsan-kubeconfig访问资源测试

[root@master zhangsan]# cat zhangsan-kubeconfig
[root@master zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get pod -n dabao
NAME                       READY   STATUS    RESTARTS   AGE
nginx01-77c44977fd-98jc4   1/1     Running   0          23m
nginx01-77c44977fd-nt424   1/1     Running   0          23m
nginx01-77c44977fd-p4xbh   1/1     Running   0          40m
'//但是访问不到除了pod的其他资源'
[root@master zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get svc -n dabao	'//service无法访问'
Error from server (Forbidden): services is forbidden: User "zhangsan" cannot list resource "services" in API group "" in the namespace "dabao"
[root@master zhangsan]# kubectl --kubeconfig=zhangsan-kubeconfig get svc	'//默认名称空间也无法访问'
Error from server (Forbidden): services is forbidden: User "zhangsan" cannot list resource "services" in API group "" in the namespace "default"

UI访问控制

[root@master zhangsan]# kubectl get svc -n kube-system
NAME                   TYPE       CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
kubernetes-dashboard   NodePort   10.0.0.139   <none>        443:30005/TCP   13d
[root@master zhangsan]# kubectl get pod -n kube-system -o wide
NAME                                    READY   STATUS    RESTARTS   AGE   IP            NODE              NOMINATED NODE
kubernetes-dashboard-7dffbccd68-58qms   1/1     Running   5          13d   172.17.78.3   192.168.233.132   <none>

'//接下来访问之前搭建的UI界面'

mark

查看令牌

[root@master zhangsan]# kubectl get secret -n kube-system	'//这是管理员的令牌'
NAME                               TYPE                                  DATA   AGE
dashboard-admin-token-zwktc        kubernetes.io/service-account-token   3      13d
default-token-cnnbv                kubernetes.io/service-account-token   3      21d
kubernetes-dashboard-certs         Opaque                                11     13d
kubernetes-dashboard-key-holder    Opaque                                2      13d
kubernetes-dashboard-token-qgppd   kubernetes.io/service-account-token   3      13d
[root@master zhangsan]# vim sa.yaml	'//编辑认证yaml文件'
apiVersion: v1
kind: ServiceAccount
metadata:
  name: pod-reader
  namespace: dabao

---

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: sa-read-pods
  namespace: dabao
subjects:
- kind: ServiceAccount
  name: pod-reader
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io
[root@master zhangsan]# kubectl apply -f sa.yaml 
serviceaccount/pod-reader created
rolebinding.rbac.authorization.k8s.io/sa-read-pods created
[root@master zhangsan]# kubectl get sa -n dabao
NAME         SECRETS   AGE
default      1         60m
pod-reader   1         11s
[root@master zhangsan]# kubectl get secret -n dabao
NAME                     TYPE                                  DATA   AGE
default-token-5lkxq      kubernetes.io/service-account-token   3      61m
pod-reader-token-l49zb   kubernetes.io/service-account-token   3      76s
[root@master zhangsan]# kubectl describe secret pod-reader-token-l49zb -n dabao
Name:         pod-reader-token-l49zb
Namespace:    dabao
Labels:       <none>
Annotations:  kubernetes.io/service-account.name: pod-reader
              kubernetes.io/service-account.uid: b262ec71-9b16-11ea-8c4f-000c294b2dd3

Type:  kubernetes.io/service-account-token

Data
====
ca.crt:     1359 bytes
namespace:  5 bytes
token:      eyJhbGciOiJSUzI1NiIsImtpZCI6IiJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkYWJhbyIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJwb2QtcmVhZGVyLXRva2VuLWw0OXpiIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6InBvZC1yZWFkZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJiMjYyZWM3MS05YjE2LTExZWEtOGM0Zi0wMDBjMjk0YjJkZDMiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6ZGFiYW86cG9kLXJlYWRlciJ9.vUdzOLAxq4GE6sRGmS4RkK6_s-NeF7mAeNnuZf9zkacYXZSxxWR6yz7h2-xpSPFJJIvZ1IAmP_G0XO4pazlS_DkLn_cNzLwAT7moOp8CxalATySLY-KtXxbRslwpcyLfyxaZ-PSEiI3fKt5f66C0eL3aoFYIM-xukVQNx_UJE2vsmu93WTuUb8XQjMVGUkQW-p7Mw_2f2wCyGbk_Y__LzXv3dRj0df7EoANHPiadYbntO-_k04LNSfrA1cHRZKBgIddO5tF8olw6rTs99IkSWfMUoF4-qCBDROOBf2h8tSMgz3jrOlhAbvq8kuvR4zjBtwUfI4l0c2IS00HZFW3K3g
[root@master zhangsan]#

9、使用令牌登陆UI界面

1.4：第三关：准入控制Admission Control

Adminssion Control实际上是一个准入控制器插件列表，发送到API Server的请求都需要经过这个列表中的每个准入控制器插件的检查，检查不通过，则拒绝请求。

查看进程信息

[root@master zhangsan]# ps aux |grep apiserver
root      12636  3.0 20.9 420460 208584 ?       Ssl  5月17 160:57 /opt/kubernetes/bin/kube-apiserver --logtostderr=true --v=4 --etcd-servers=https://192.168.233.131:2379,https://192.168.233.132:2379,https://192.168.233.133:2379 --bind-address=192.168.233.131 --secure-port=6443 --advertise-address=192.168.233.131 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction '//此行是准入控制的插件信息'--authorization-mode=RBAC,Node --kubelet-https=true --enable-bootstrap-token-auth --token-auth-file=/opt/kubernetes/cfg/token.csv --service-node-port-range=30000-50000 --tls-cert-file=/opt/kubernetes/ssl/server.pem --tls-private-key-file=/opt/kubernetes/ssl/server-key.pem --client-ca-file=/opt/kubernetes/ssl/ca.pem --service-account-key-file=/opt/kubernetes/ssl/ca-key.pem --etcd-cafile=/opt/etcd/ssl/ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem
root      23615  0.0  0.0 112728   972 pts/0    S+   12:18   0:00 grep --color=auto apiserver

NamespaceLifecycle：命令空间回收

LimitRanger：配额管理

ServiceAccount：每个pod中导入方便访问apiserver

ResourceQuota：基于命名空间的高级配额管理

NodeRestriction： Node加入到k8s群集中以最小权限运行

官网推荐的插件：

1.11版本以上推荐使用的插件：

–enable-admission-plugins= \ NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds, ResourceQuota

ca.pem --etcd-certfile=/opt/etcd/ssl/server.pem --etcd-keyfile=/opt/etcd/ssl/server-key.pem
root 23615 0.0 0.0 112728 972 pts/0 S+ 12:18 0:00 grep --color=auto apiserver


- NamespaceLifecycle： 命令空间回收

LimitRanger：配额管理

ServiceAccount： 每个pod中导入方便访问apiserver

ResourceQuota： 基于命名空间的高级配额管理

NodeRestriction： Node加入到k8s群集中以最小权限运行

官网推荐的插件：

1.11版本以上推荐使用的插件：

–enable-admission-plugins= \ NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds, ResourceQuota

### 如有疑问可评论区交流！