istio笔记02--istio流量镜像最佳实践

介绍

流量镜像是Istio流量管理的一个重要特性，它允许将生产环境的流量复制一份并发送到镜像服务，而不会影响实际的生产流量。
如下图，客户端请求服务A的sidecar，它会镜像流量镜像到A-v2中，镜像服务处理请求后响应被丢弃。
本文主要介绍三种常见的流量镜像的常见配置方法，方便大家在实际业务中按需配置。

流量镜像案例

同集群流量镜像

前提条件：部署两个服务 nginx-v1和nginx-v2, 它们的svc也分别为nginx-v1和nginx-v2

deploy-nginx-v1.yaml

nginx-v2类似，不再重复贴一遍

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v2
  namespace: default
  labels:
    app: nginx-v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx-v2
  template:
    metadata:
      labels:
        app: nginx-v2
    spec:
      containers:
        - name: nginx
          image: nginx:1.23
          ports:
            - containerPort: 80
              protocol: TCP
          resources: {}

svc-nginx-v1.yaml

nginx-v2类似，此处不重复

apiVersion: v1
kind: Service
metadata:
  name: nginx-v1
  namespace: default
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: nginx-v1
  type: ClusterIP

cluster-a-gw.yaml

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: default-gw
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
    - hosts:
        - nginx.xg.com
      port:
        name: http
        number: 80
        protocol: HTTP

vs-nginx-v1.yaml

以下流量镜像配置将nginx-v1 50%的流量镜像到nginx-v2

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: my-nginx
  namespace: default
spec:
  gateways:
    - istio-system/default-gw
  hosts:
    - nginx.xg.com
  http:
  - mirror:
      host: nginx-v2
      port:
        number: 80
    mirrorPercentage:
      value: 50
    name: http-nginx-v1
    route:
    - destination:
        host: nginx-v1
        port:
          number: 80

如果跨命名空间，在destination.host中指定为svc-name.ns-name即可

本集群内部svc流量镜像到外部集群

前提条件: cluster-a和cluster-b的default命名空间都部署了nginx-v1应用, 且cluster-b服务的域名为 nginx-v1-cluster-b.xg.com

cluster-a-vs-nginx-v1.yaml

配置gateways为mesh的VS后，可以将集群内部通过svc-name的请求转发到cluster-b集群的 nginx-v1服务

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: cluster-a-nginx-v1-vs-mesh
  namespace: default
spec:
  gateways:
  - mesh
  hosts:
  - nginx-v1.default.svc.cluster.local
  http:
  - mirror:
      host: nginx-v1-cluster-b.cn-bj.xg.com
      port:
        number: 80
    mirrorPercent: 50
    route:
    - destination:
        host: nginx-v1.default.svc.cluster.local
        port:
          number: 80

cluster-a-serviceentry.yaml

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
  name: cluster-b-mirror-serviceentry
  namespace: default
spec:
  hosts:
  - nginx-v1-cluster-b.xg.com
  location: MESH_EXTERNAL
  ports:
  - name: http-80
    number: 80
    protocol: HTTP
  resolution: DNS

cluster-b-external-gateway.yaml

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: external-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway-external
  servers:
  - hosts:
    - nginx-v1-cluster-b.xg.com
    - nginx-v1.default.svc.cluster.local-shadow
    port:
      name: http-80
      number: 80
      protocol: HTTP

cluster-b-vs-nginx-v1.yaml

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: cluster-b-nginx-v1-vs
  namespace: default
spec:
  gateways:
  - istio-system/external-gateway
  hosts:
  - nginx-v1.default.svc.cluster.local-shadow
  - nginx-v1-cluster-b.xg.com
  http:
  - route:
    - destination:
        host: nginx-v1.default.svc.cluster.local
        port:
          number: 80

验证方法：
在cluster-a集群内部通过svc访问nginx-v1服务，curl nginx-v1.default ，同时在cluster-b 看istio-proxy的日志，能正常看到日志即配置成功

cluster-b 看istio-proxy的日志
kubectl logs -f --tail=20 -l app=nginx-v1 -n default -c istio-proxy | grep nginx-v1.default.svc.cluster.local-shadow

本集群gw域名流量镜像到外部集群

前提条件 cluster-a和cluster-b的default命名空间都部署了nginx-v1应用, cluster-a服务的域名为 nginx.xg.com ,cluster-b服务的域名为 nginx-v1-cluster-b.xg.com

cluster-a-vs-nginx-v1.yaml

配置gateways为istio-system/default-gw的VS后，可以将集群外部通过svc-name的nginx.xg.com访问的请求转发到cluster-b集群的nginx-v1服务

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: cluster-a-nginx-v1-vs
  namespace: default
spec:
  gateways:
  - istio-system/default-gw
  hosts:
  - nginx.xg.com
  http:
  - mirror:
      host: nginx-v1-cluster-b.cn-bj.xg.com
      port:
        number: 80
    mirrorPercent: 50
    route:
    - destination:
        host: nginx-v1.default.svc.cluster.local
        port:
          number: 80

cluster-a-gw.yaml

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: default-gw
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
    - hosts:
        - nginx.xg.com
      port:
        name: http
        number: 80
        protocol: HTTP

cluster-a-serviceentry.yaml

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
  name: cluster-b-mirror-serviceentry
  namespace: default
spec:
  hosts:
  - nginx-v1-cluster-b.xg.com
  location: MESH_EXTERNAL
  ports:
  - name: http-80
    number: 80
    protocol: HTTP
  resolution: DNS

cluster-b-external-gateway.yaml

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: external-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway-external
  servers:
  - hosts:
    - nginx-v1-cluster-b.xg.com
    - nginx.xg.com-shadow
    port:
      name: http-80
      number: 80
      protocol: HTTP

cluster-b-vs-nginx-v1.yaml

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: cluster-b-nginx-v1-vs
  namespace: default
spec:
  gateways:
  - istio-system/external-gateway
  hosts:
  - nginx.xg.com-shadow
  - nginx-v1-cluster-b.xg.com
  http:
  - route:
    - destination:
        host: nginx-v1.default.svc.cluster.local
        port:
          number: 80

验证方法：
在外部curl nginx.xg.com 域名，同时在cluster-b 看istio-proxy的日志，能正常看到日志即配置成功

cluster-b 看istio-proxy的日志
kubectl logs -f --tail=20 -l app=nginx-v1 -n default -c istio-proxy | grep nginx.xg.com-shadow

注意事项

1. 跨集群svc访问流量镜像需要使用 gateways -> mesh。
2. istio流量镜像使用 -shadow 后缀标识镜像流量，确保镜像流量不会与原始流量混淆。
3. 跨集群流量调试方式
   如果发现流量异常的时候，可以优先排查下目标集群配置是否正常，然后就再查源头集群配置。可以在本地配置dns解析到目标集群的gw对应的lb，然后访问 xxx-shadow，并观察gw和目标pod的istio-proxy日志是否有xxx-shadow的请求。

说明

系统软件：
ubuntu 22.04 Server,
istio 1.23.2
K8s v1.30.0
参考文档：
docs/concepts/traffic-management
docs/tasks/traffic-management/mirroring