Secret是用来保存小片敏感数据的k8s资源,例如密码,token,或者秘钥。这类数据当然也可以存放在Pod或者镜像中,但是放在Secret中是为了更方便的控制如何使用数据,并减少暴露的风险。
用户可以创建自己的secret,系统也会有自己的secret。
Pod需要先引用才能使用某个secret,Pod有2种方式来使用secret:作为volume的一个域被一个或多个容器挂载;在拉取镜像的时候被kubelet引用。
Service Account
Service Account 场景:pod里的进程需要调用 Kubernetes API 以及非 Kubernetes API 的其它服务。Service Account 是给 pod 里面的进程使用的,它为pod提供必要的身份认证
如果kubernetes 启动参数设置 --admission_control=ServiceAccount,会在每个namespace下创建一个默认的 default 的service account
# kubectl get serviceaccount default -oyaml
apiVersion: v1
kind: ServiceAccount
metadata:
creationTimestamp: "2019-05-29T06:59:44Z"
name: default
namespace: default
resourceVersion: "225"
selfLink: /api/v1/namespaces/default/serviceaccounts/default
uid: 56de4d32-81df-11e9-884b-0800271c9f15
secrets:
- name: default-token-shmrt
在看看这个secrets是啥内容
apiVersion: v1
data:
ca.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUR2akNDQXFhZ0F3SUJBZ0lVZFBJbVBPUVhGL2g0MzNGRHAvTWI4MnN2NWVZd0RRWUpLb1pJaHZjTkFRRUwKQlFBd1pURUxNQWtHQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbAphVXBwYm1jeEREQUtCZ05WQkFvVEEyczRjekVQTUEwR0ExVUVDeE1HVTNsemRHVnRNUk13RVFZRFZRUURFd3ByCmRXSmxjbTVsZEdWek1CNFhEVEU1TURReU5URXdNelF3TUZvWERUSTBNRFF5TXpFd016UXdNRm93WlRFTE1Ba0cKQTFVRUJoTUNRMDR4RURBT0JnTlZCQWdUQjBKbGFVcHBibWN4RURBT0JnTlZCQWNUQjBKbGFVcHBibWN4RERBSwpCZ05WQkFvVEEyczRjekVQTUEwR0ExVUVDeE1HVTNsemRHVnRNUk13RVFZRFZRUURFd3ByZFdKbGNtNWxkR1Z6Ck1JSUJJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0NBUUVBcnJUQW5tR2pGRWoxUmVLVG0vOXgKZlhQSXJ1NXZtVG5GYTYrUmR6bGhRTldSdW11Tmc4eCtPNnZ3U0tHSzdORUMzdEN4UkxsOHZKQmpJR0o5TlJ4egpLcWVtamZaR2E2NERGT3g0N1RlaTRkUHRlVDBqNjFsWjFxeFcwcmhxSHFzRWJySVZYWE14bExhVnYyYVorVUZwCitkOFg0Q1dKVVQ2OW1zcDZDdzhIZHYvMnNLRGsraVlPY2dzei9EbzlPaTBKbThrb3B3ak5sd1lqZ012ZVdkcnMKTS8vS2Q3MVQwY040bzhhYnZwZng2V2pCTWNuUzVxb01XTy94bzFPUXZyZmpFUmkxL2dPSmdZRHpRamFESXRNeQp4cWZEZ2lZZmJmTktTSVNpc3FPRG9BaVZ5VkY1SS8xajVrWFBSTldiUURxUHl0N0VZWnk5aXN4Mm0zRTFKRkE0CmpRSURBUUFCbzJZd1pEQU9CZ05WSFE4QkFmOEVCQU1DQVFZd0VnWURWUjBUQVFIL0JBZ3dCZ0VCL3dJQkFqQWQKQmdOVkhRNEVGZ1FVbHlrT0ZrMThPZ3dBa2JLNnZaVWNDai9hSEZJd0h3WURWUjBqQkJnd0ZvQVVseWtPRmsxOApPZ3dBa2JLNnZaVWNDai9hSEZJd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFHR0UrODEwY2hUNUVpUnI3K1VsCmRmeHB6byswNFJxbCt3RUhFaThuY1pYTFoxS25xeDUya1M4UnpsdHR4bzRDRGpyRE1nakkzSmpJZXIwcFFwcmYKdklCcnBkdFJCQm0wNTI0SEorUThldG1md0FiR0xOVjhyYmowdDhVc1hPZ000RjkzelhsV0NNWDdLT3Bzc2xsVgpxTEk1Qi80aUh1LzFmOU9RU0ViejhSWmtkeTY3RlZPSjJYMlNTMXJldjZzbTlERVE0RnZ5WWlJejFJWXhjY2tuClRGbGNwMVVFTzZOOGRsWnpuTTNnL2N2OVNSZVJiMXUyYktnL0hKZm1nTTZTVU9CYU9XYmR6SkM2Z2s4UkhCbW8KdWtpWFNkOHNWeGJTb1lYL011S2grS3Z0K3NENWNaUitKQXVnd2lJcFQvKzRlaUNyc0F6akNicHlsY29WR2NQeAp1N009Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K
namespace: ZGVmYXVsdA==
token: ZXlKaGJHY2lPaUpTVXpJMU5pSXNJbXRwWkNJNklpSjkuZXlKcGMzTWlPaUpyZFdKbGNtNWxkR1Z6TDNObGNuWnBZMlZoWTJOdmRXNTBJaXdpYTNWaVpYSnVaWFJsY3k1cGJ5OXpaWEoyYVdObFlXTmpiM1Z1ZEM5dVlXMWxjM0JoWTJVaU9pSmtaV1poZFd4MElpd2lhM1ZpWlhKdVpYUmxjeTVwYnk5elpYSjJhV05sWVdOamIzVnVkQzl6WldOeVpYUXVibUZ0WlNJNkltUmxabUYxYkhRdGRHOXJaVzR0YzJodGNuUWlMQ0pyZFdKbGNtNWxkR1Z6TG1sdkwzTmxjblpwWTJWaFkyTnZkVzUwTDNObGNuWnBZMlV0WVdOamIzVnVkQzV1WVcxbElqb2laR1ZtWVhWc2RDSXNJbXQxWW1WeWJtVjBaWE11YVc4dmMyVnlkbWxqWldGalkyOTFiblF2YzJWeWRtbGpaUzFoWTJOdmRXNTBMblZwWkNJNklqVTJaR1UwWkRNeUxUZ3haR1l0TVRGbE9TMDRPRFJpTFRBNE1EQXlOekZqT1dZeE5TSXNJbk4xWWlJNkluTjVjM1JsYlRwelpYSjJhV05sWVdOamIzVnVkRHBrWldaaGRXeDBPbVJsWm1GMWJIUWlmUS5TSnY1b2tpYXVHMHg2UGJYT0hWZHdYbkZxYlNKNUNSdk9zOVRWbVpGY3dNUUdMTnpROUlLOXJ6MmRSQjhxUmhTdUcwS2ZDemxCeW0tYTRuaFp2MHNvMURndkV2MVY3LVRhRklzLW9HbXp2bVpoUzh0YVZvdVI2MDYyLWlKcmVmVHg5VkgzX1pLbkdNY241VXp4UEFSNkV1Xy1hZXFBd244UzJSN2xDZEtiNEZmNkt2Zl9Bdk5uWnhLdmt1QVJNYmg3Z2tsNm55LUItNnJiQ0w0V0F6R3JMVFUyYmNtSVhTZEwyWkFmbUhpcGxnMWhBUWtmcnZVVDFRZjJQQnZSczAtQ0xUYU1OeTFRMF9CMHc5SlN1ZWJBYjYycE41WXE4UmhFcjR1dEZuNW83Y3lnRWFoQlloZGFMdEdram1wdFNlZVRucE80UFJPLU1FdXVCZHdtdTRNZVE=
kind: Secret
metadata:
annotations:
kubernetes.io/service-account.name: default
kubernetes.io/service-account.uid: 56de4d32-81df-11e9-884b-0800271c9f15
creationTimestamp: "2019-05-29T06:59:44Z"
name: default-token-shmrt
namespace: default
resourceVersion: "219"
selfLink: /api/v1/namespaces/default/secrets/default-token-shmrt
uid: 56f435ab-81df-11e9-884b-0800271c9f15
type: kubernetes.io/service-account-token
使用Secret
secret可以作为数据卷挂载或者作为环境变量暴露给Pod中的容器使用。比如可以用secret导入与外部系统交互需要的证书文件等。
volumeMounts:
- mountPath: /var/lib/mysql
name: mysql-persistent-storage
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-shmrt
readOnly: truevolumes:
- hostPath:
path: /tmp/mysql
type: ""
name: mysql-persistent-storage
- name: default-token-shmrt
secret:
defaultMode: 420
secretName: default-token-shmrt
将ca.crt 、namespace和token三个文件写入value在容器的/var/run/secrets/kubernetes.io/serviceaccount目录下,这样容器通过https 访问apiserver
Pod中以文件的形式使用secret
- 创建一个Secret,多个Pod可以引用同一个Secret
- 修改Pod的定义,在
spec.volumes[]
加一个volume- 在每个需要使用Secret的容器中添加一项
spec.containers[].volumeMounts
- Secret中
data
字段的每一个key都是指定路径下面的一个文件名
类型
Opaque任意字符串,默认类型
kubernetes.io/service-account-token:作用于ServiceAccount
每一个用户对API资源进行操作都需要通经过以下三个步骤:
第一步:对客户端访问进行认证操作,确认是否具有访问k8s权限
token(共享秘钥)
SSL(双向SSL认证)
通过任何一个认证即表示认证通过,进入下一步
第二步:授权检查,确认是否对资源具有相关的权限
ABAC(基于属性的访问控制)
RBAC(基于角色的访问控制)
NODE(基于节点的访问控制)
WEB HOOK(自定义HTTP回调方法的访问控制)
第三步:准入控制(对操作资源相关联的其他资源是否有权限操作)
- Pod在创建后都会自动设置spec.serviceAccount为default(除非指定了ServiceAccout)
- 验证Pod引用的service account存在,否则拒绝创建
- 如果Pod没有指定ImagePullSecrets,则把service account的ImagePullSecrets加到Pod中
- container启动后都会挂载该service account的namespace token和ca.crt到/var/run/secrets/kubernetes.io/serviceaccount
下面是第一部secret分析
1. NewMounter 函数
插件名为:kubernetes.io/secret,实例化secretVolumeMounter,名,pod 信息等
func (plugin *secretPlugin) NewMounter(spec *volume.Spec, pod *v1.Pod, opts volume.VolumeOptions) (volume.Mounter, error) {
return &secretVolumeMounter{
secretVolume: &secretVolume{
spec.Name(),
pod.UID,
plugin,
plugin.host.GetMounter(plugin.GetPluginName()),
volume.NewCachedMetrics(volume.NewMetricsDu(getPath(pod.UID, spec.Name(), plugin.host))),
},
source: *spec.Volume.Secret,
pod: *pod,
opts: &opts,
getSecret: plugin.getSecret,
}, nil
}
2. SetUpAt 函数
/var/lib/kubelet/pods/7f3ca9fc-8435-11e9-a5a0-0800271c9f15/volumes/kubernetes.io~secret/default-token-w25t2
func getPath(uid types.UID, volName string, host volume.VolumeHost) string { return host.GetPodVolumeDir(uid, strings.EscapeQualifiedNameForDisk(secretPluginName), volName) }
每一个 namespace 都有一default serviceaccount,存储在 kubernetes API 中的 Secrets 资源
secret, err := b.getSecret(b.pod.Namespace, b.source.SecretName)
if err != nil {
if !(errors.IsNotFound(err) && optional) {
klog.Errorf("Couldn't get secret %v/%v: %v", b.pod.Namespace, b.source.SecretName, err)
return err
}
secret = &v1.Secret{
ObjectMeta: metav1.ObjectMeta{
Namespace: b.pod.Namespace,
Name: b.source.SecretName,
},
}
}
MakePayload 主要是读取secret 中的data数据,包括ca.crt namespace token,value为需要写入这三个文件的内容
// MakePayload function is exported so that it can be called from the projection volume driver
func MakePayload(mappings []v1.KeyToPath, secret *v1.Secret, defaultMode *int32, optional bool) (map[string]volumeutil.FileProjection, error) {
if defaultMode == nil {
return nil, fmt.Errorf("No defaultMode used, not even the default value for it")
}
payload := make(map[string]volumeutil.FileProjection, len(secret.Data))
var fileProjection volumeutil.FileProjection
if len(mappings) == 0 {
for name, data := range secret.Data {
fileProjection.Data = []byte(data)
fileProjection.Mode = *defaultMode
payload[name] = fileProjection
}
} else {
for _, ktp := range mappings {
content, ok := secret.Data[ktp.Key]
if !ok {
if optional {
continue
}
errMsg := "references non-existent secret key"
klog.Errorf(errMsg)
return nil, fmt.Errorf(errMsg)
}
fileProjection.Data = []byte(content)
if ktp.Mode != nil {
fileProjection.Mode = *ktp.Mode
} else {
fileProjection.Mode = *defaultMode
}
payload[ktp.Path] = fileProjection
}
}
return payload, nil
}
使用wrapped emptydir插件
setupSuccess := false
if err := wrapped.SetUpAt(dir, fsGroup); err != nil {
return err
}
也就是写入文件包括ca.crt namespace token,设置文件owner,完活
writerContext := fmt.Sprintf("pod %v/%v volume %v", b.pod.Namespace, b.pod.Name, b.volName)
writer, err := volumeutil.NewAtomicWriter(dir, writerContext)
if err != nil {
klog.Errorf("Error creating atomic writer: %v", err)
return err
}
err = writer.Write(payload)
if err != nil {
klog.Errorf("Error writing payload to dir: %v", err)
return err
}
err = volume.SetVolumeOwnership(b, fsGroup)
if err != nil {
klog.Errorf("Error applying volume ownership settings for group: %v", fsGroup)
return err
}
setupSuccess = true
总结:
secret 这个主要是 pod内进程访问的凭证,将 ca.crt namespace token 三个文件挂载到pod内