1、认识Ingress
1.1 什么是Ingress?
通常情况下,service和pod仅可在集群内部网络中通过IP地址访问。所有到达边界路由器的流量或被丢弃或被转发到其他地方。从概念上讲,可能像下面这样:
1
2
3
4
|
internet
|
------------
[ Services ]
|
Ingress是授权入站连接到达集群服务的规则集合。
1
2
3
4
5
|
internet
|
[ Ingress ]
--|-----|--
[ Services ]
|
你可以给Ingress配置提供外部可访问的URL、负载均衡、SSL、基于名称的虚拟主机等。用户通过POST Ingress资源到API server的方式来请求ingress。 Ingress controller负责实现Ingress,通常使用负载平衡器,它还可以配置边界路由和其他前端,这有助于以HA方式处理流量。
1.2 Ingress工作示意图
1.3先决条件
在使用Ingress resource之前,有必要先了解下面几件事情。Ingress是beta版本的resource,在kubernetes1.1之前还没有。你需要一个Ingress Controller来实现Ingress,单纯的创建一个Ingress没有任何意义。
GCE/GKE会在master节点上部署一个ingress controller。你可以在一个pod中部署任意个自定义的ingress controller。你必须正确地annotate每个ingress,比如 运行多个ingress controller 和 关闭glbc.
确定你已经阅读了Ingress controller的beta版本限制。在非GCE/GKE的环境中,你需要在pod中部署一个controller。
1.4 Ingress定义资源清单几个字段
- apiVersion: v1 版本
- kind: Ingress 类型
- metadata 元数据
- spec 期望状态
- backend: 默认后端,能够处理与任何规则不匹配的请求
- rules:用于配置Ingress的主机规则列表
- tls:目前Ingress仅支持单个TLS端口443
- status 当前状态
2、部署一个Ingress
(1)在gitlab上下载yaml文件,并创建部署
gitlab ingress-nginx项目:https://github.com/kubernetes/ingress-nginx
ingress安装指南:https://kubernetes.github.io/ingress-nginx/deploy/
因为需要拉取镜像,所以需要等一段时间
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
---下载需要的yaml文件
[root@master ingress-nginx]
# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml
---查询下载成功
[root@master ingress-nginx]
# ls
mandatory.yaml
---创建ingress
[root@master ingress-nginx]
# kubectl apply -f mandatory.yaml
namespace
/ingress-nginx
created
configmap
/nginx-configuration
created
configmap
/tcp-services
created
configmap
/udp-services
created
serviceaccount
/nginx-ingress-serviceaccount
created
clusterrole.rbac.authorization.k8s.io
/nginx-ingress-clusterrole
created
role.rbac.authorization.k8s.io
/nginx-ingress-role
created
rolebinding.rbac.authorization.k8s.io
/nginx-ingress-role-nisa-binding
created
clusterrolebinding.rbac.authorization.k8s.io
/nginx-ingress-clusterrole-nisa-binding
created
deployment.apps
/nginx-ingress-controller
created
|
(2)如果是裸机,还需要安装service
1
2
3
|
[root@master ingress-nginx]
# wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/baremetal/service-nodeport.yaml
[root@master ingress-nginx]
# kubectl apply -f service-nodeport.yaml
service
/ingress-nginx
created
|
(3)验证
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
|
---查询生产的pod
[root@master ~]
# kubectl get pods -n ingress-nginx
NAME READY STATUS RESTARTS AGE
nginx-ingress-controller-648c7bb65b-df9qz 1
/1
Running 0 34m
---查询生产的svc
[root@master ingress-nginx]
# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx NodePort 10.109.244.123 <none> 80:30080
/TCP
,443:30443
/TCP
21s
---查询svc的详细信息
[root@master ~]
# kubectl describe svc ingress-nginx -n ingress-nginx
Name: ingress-nginx
Namespace: ingress-nginx
Labels: app.kubernetes.io
/name
=ingress-nginx
app.kubernetes.io
/part-of
=ingress-nginx
Annotations: kubectl.kubernetes.io
/last-applied-configuration
={
"apiVersion"
:
"v1"
,
"kind"
:
"Service"
,
"metadata"
:{
"annotations"
:{},
"labels"
:{
"app.kubernetes.io/name"
:
"ingress-nginx"
,
"app.kubernetes.io/part-of"
:"ingres...
Selector: app.kubernetes.io
/name
=ingress-nginx,app.kubernetes.io
/part-of
=ingress-nginx
Type: NodePort
IP: 10.111.143.90
Port: http 80
/TCP
TargetPort: 80
/TCP
NodePort: http 30080
/TCP
Endpoints: 10.244.1.104:80
Port: https 443
/TCP
TargetPort: 443
/TCP
NodePort: https 30443
/TCP
Endpoints: 10.244.1.104:443
Session Affinity: None
External Traffic Policy: Cluster
Events: <none>
|
3、创建Ingress,代理到后端nginx服务
3.1 准备后端pod和service
(1)编写yaml文件,并创建
创建3个nginx服务的pod,并创建一个service绑定
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
|
[root@master ingress]
# vim deploy-damo.yaml
apiVersion: v1
kind: Service
metadata:
name: myapp
namespace: default
spec:
selector:
app: myapp
release: canary
ports:
- name: http
targetPort: 80
port: 80
---
apiVersion: apps
/v1
kind: Deployment
metadata:
name: myapp-deploy
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: myapp
release: canary
template:
metadata:
labels:
app: myapp
release: canary
spec:
containers:
- name: myapp
image: ikubernetes
/myapp
:v2
ports:
- name: http
containerPort: 80
[root@master ingress]
# kubectl apply -f deploy-damo.yaml
service
/myapp
created
deployment.apps
/myapp-deploy
created
|
(2)查询验证
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
[root@master ~]
# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443
/TCP
146d
myapp ClusterIP 10.103.137.126 <none> 80
/TCP
6s
[root@master ~]
# kubectl get pods
NAME READY STATUS RESTARTS AGE
myapp-deploy-67f6f6b4dc-2vzjn 1
/1
Running 0 14s
myapp-deploy-67f6f6b4dc-c7f76 1
/1
Running 0 14s
myapp-deploy-67f6f6b4dc-x79hc 1
/1
Running 0 14s
[root@master ~]
# kubectl describe svc myapp
Name: myapp
Namespace: default
Labels: <none>
Annotations: kubectl.kubernetes.io
/last-applied-configuration
={
"apiVersion"
:
"v1"
,
"kind"
:
"Service"
,
"metadata"
:{
"annotations"
:{},
"name"
:
"myapp"
,
"namespace"
:
"default"
},
"spec"
:{
"ports"
:[{
"name"
:
"http"
,
"port"
:80,"targe...
Selector: app=myapp,release=canary
Type: ClusterIP
IP: 10.103.137.126
Port: http 80
/TCP
TargetPort: 80
/TCP
Endpoints: 10.244.1.102:80,10.244.1.103:80,10.244.2.109:80
Session Affinity: None
Events: <none>
|
3.2 创建ingress,绑定后端nginx服务
(1)编写yaml文件,并创建
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
[root@master ingress]
# vim ingress-myapp.yaml
apiVersion: extensions
/v1beta1
kind: Ingress
metadata:
name: ingress-myapp
namespace: default
spec:
rules:
- host: myapp.along.com
http:
paths:
- path:
backend:
serviceName: myapp
servicePort: 80
[root@master ingress]
# kubectl apply -f ingress-myapp.yaml
ingress.extensions
/ingress-myapp
created
|
(2)查询验证
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
|
[root@master ~]
# kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
ingress-myapp myapp.along.com 80 140d
[root@master ~]
# kubectl describe ingress ingress-myapp
Name: ingress-myapp
Namespace: default
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
myapp.along.com
myapp:80 (<none>)
Annotations:
kubectl.kubernetes.io
/last-applied-configuration
: {
"apiVersion"
:
"extensions/v1beta1"
,
"kind"
:
"Ingress"
,
"metadata"
:{
"annotations"
:{},
"name"
:
"ingress-myapp"
,
"namespace"
:
"default"
},
"spec"
:{
"rules"
:[{
"host"
:
"myapp.along.com"
,
"http"
:{
"paths"
:[{
"backend"
:{
"serviceName"
:
"myapp"
,
"servicePort"
:80},
"path"
:null}]}}]}}
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 37s nginx-ingress-controller Ingress default
/ingress-myapp
|
(3)在集群外,查询服务验证
① 可以先修改一下主机的hosts,因为不是公网域名
192.168.130.103 myapp.along.com
② 访问业务成功
4、创建Ingress,代理到后端tomcat服务
4.1 准备后端pod和service
(1)编写yaml文件,并创建
创建3个tomcat服务的pod,并创建一个service绑定
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
|
[root@master ingress]
# vim tomcat-deploy.yaml
apiVersion: v1
kind: Service
metadata:
name: tomcat
namespace: default
spec:
selector:
app: tomcat
release: canary
ports:
- name: http
targetPort: 8080
port: 8080
- name: ajp
targetPort: 8009
port: 8009
---
apiVersion: apps
/v1
kind: Deployment
metadata:
name: tomcat-deploy
namespace: default
spec:
replicas: 3
selector:
matchLabels:
app: tomcat
release: canary
template:
metadata:
labels:
app: tomcat
release: canary
spec:
containers:
- name: tomcat
image: tomcat:8.5.37-jre8-alpine
ports:
- name: http
containerPort: 8080
- name: ajp
containerPort: 8009
[root@master ingress]
# kubectl apply -f tomcat-deploy.yaml
service
/tomcat
created
deployment.apps
/tomcat-deploy
created
|
(2)查询验证
1
2
3
4
5
6
7
8
9
|
[root@master ~]
# kubectl get pods
NAME READY STATUS RESTARTS AGE
tomcat-deploy-97d6458c5-hrmrw 1
/1
Running 0 1m
tomcat-deploy-97d6458c5-ngxxx 1
/1
Running 0 1m
tomcat-deploy-97d6458c5-xchgn 1
/1
Running 0 1m
[root@master ~]
# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443
/TCP
146d
tomcat ClusterIP 10.98.193.252 <none> 8080
/TCP
,8009
/TCP
1m
|
4.2 创建ingress,绑定后端tomcat服务
(1)编写yaml文件,并创建
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
[root@master ingress]
# vim ingress-tomcat.yaml
apiVersion: extensions
/v1beta1
kind: Ingress
metadata:
name: ingress-tomcat
namespace: default
spec:
rules:
- host: tomcat.along.com
http:
paths:
- path:
backend:
serviceName: tomcat
servicePort: 8080
[root@master ingress]
# kubectl apply -f ingress-tomcat.yaml
ingress.extensions
/ingress-tomcat
created
|
(2)查询验证
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
[root@master ~]
# kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
ingress-myapp myapp.along.com 80 17m
ingress-tomcat tomcat.along.com 80 6s
[root@master ~]
# kubectl describe ingress ingress-tomcat
Name: ingress-tomcat
Namespace: default
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
tomcat.along.com
tomcat:8080 (<none>)
Annotations:
kubectl.kubernetes.io
/last-applied-configuration
: {
"apiVersion"
:
"extensions/v1beta1"
,
"kind"
:
"Ingress"
,
"metadata"
:{
"annotations"
:{},
"name"
:
"ingress-tomcat"
,
"namespace"
:
"default"
},
"spec"
:{
"rules"
:[{
"host"
:
"tomcat.along.com"
,
"http"
:{
"paths"
:[{
"backend"
:{
"serviceName"
:
"tomcat"
,
"servicePort"
:8080},
"path"
:null}]}}]}}
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 17s nginx-ingress-controller Ingress default
/ingress-tomcat
|
(3)在集群外,查询服务验证
① 可以先修改一下主机的hosts,因为不是公网域名
192.168.130.103 tomcat.along.com
② 访问业务成功
4.3 使用https协议访问服务
4.3.1 创建证书、私钥和secret
(1)创建私钥
1
2
3
4
5
6
7
|
[root@master ingress]
# openssl genrsa -out tls.key 2048
Generating RSA private key, 2048 bit long modulus
.............................................+++
...............+++
e is 65537 (0x10001)
[root@master ingress]
# ls *key
tls.key
|
(2)创建证书
1
2
3
|
[root@master ingress]
# openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=tomcat.along.com
[root@master ingress]
# ls tls.*
tls.crt tls.key
|
(3)创建secret
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
[root@master ingress]
# kubectl create secret tls tomcat-ingress-secret --cert=tls.crt --key=tls.key
secret
/tomcat-ingress-secret
created
[root@master ingress]
# kubectl get secret
NAME TYPE DATA AGE
tomcat-ingress-secret kubernetes.io
/tls
2 8s
[root@master ingress]
# kubectl describe secret tomcat-ingress-secret
Name: tomcat-ingress-secret
Namespace: default
Labels: <none>
Annotations: <none>
Type: kubernetes.io
/tls
Data
====
tls.key: 1675 bytes
tls.crt: 1294 bytes
|
4.3.2 重新创建ingress,使用https协议绑定后端tomcat服务
(1)编写yaml文件,并创建
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
|
[root@master ingress]
# vim ingress-tomcat-tls.yaml
apiVersion: extensions
/v1beta1
kind: Ingress
metadata:
name: ingress-tomcat-tls
namespace: default
spec:
tls:
- hosts:
- tomcat.along.com
secretName: tomcat-ingress-secret
rules:
- host: tomcat.along.com
http:
paths:
- path:
backend:
serviceName: tomcat
servicePort: 8080
|
(2)查询验证
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
[root@master ~]
# kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
ingress-myapp myapp.along.com 80 34m
ingress-tomcat tomcat.along.com 80 16m
ingress-tomcat-tls tomcat.along.com 80, 443 8s
[root@master ~]
# kubectl describe ingress ingress-tomcat-tls
Name: ingress-tomcat-tls
Namespace: default
Address:
Default backend: default-http-backend:80 (<none>)
TLS:
tomcat-ingress-secret terminates tomcat.along.com
Rules:
Host Path Backends
---- ---- --------
tomcat.along.com
tomcat:8080 (<none>)
Annotations:
kubectl.kubernetes.io
/last-applied-configuration
: {
"apiVersion"
:
"extensions/v1beta1"
,
"kind"
:
"Ingress"
,
"metadata"
:{
"annotations"
:{},
"name"
:
"ingress-tomcat-tls"
,
"namespace"
:
"default"
},
"spec"
:{
"rules"
:[{
"host"
:
"tomcat.along.com"
,
"http"
:{
"paths"
:[{
"backend"
:{
"serviceName"
:
"tomcat"
,
"servicePort"
:8080},
"path"
:null}]}}],
"tls"
:[{
"hosts"
:[
"tomcat.along.com"
],
"secretName"
:
"tomcat-ingress-secret"
}]}}
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal CREATE 14s nginx-ingress-controller Ingress default
/ingress-tomcat-tls
|
(3)在集群外,查询服务验证
使用https协议,访问业务成功