灰度路由的实现

玩命cofing中

已于 2023-07-07 10:06:06 修改

阅读量437

点赞数 1

CC 4.0 BY-SA版权

于 2023-07-07 10:02:55 首次发布

本文链接：https://blog.youkuaiyun.com/weixin_41014727/article/details/131590742

书接上文，上次介绍基于gateway，nacos，redis实现动态路由。本次介绍灰度路由的实现。
首先介绍gateway中的一个全局过滤器类ReactiveLoadBalancerClientFilter

ReactiveLoadBalancerClientFilter

此类为gateway提供的负载均衡过滤器。在配置的断言命中时触发。其中和新方法filter中的逻辑为：

获取上下文中的请求URI，并判断模式是否是"lb"，即多节点负载模式
在上下文的originalRequestUrl中加入当前url
调用choose方法，进行负载均衡，返回应用实例
重新构建可执行示例
构建URI，换成ip和端口模式
过滤器释放，接着执行其他过滤器链

以下是该过滤器核心方法源码，我已添加注释：

public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        //从上下文中获取请求地址
        URI url = (URI)exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR);
        //判断请求模式。
        String schemePrefix = (String)exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_SCHEME_PREFIX_ATTR);
    	//当模式支持负载均衡
        if (url != null && ("lb".equals(url.getScheme()) || "lb".equals(schemePrefix))) { 
            //在上下文中加入原始url
            ServerWebExchangeUtils.addOriginalRequestUrl(exchange, url);
            if (log.isTraceEnabled()) {
                log.trace(ReactiveLoadBalancerClientFilter.class.getSimpleName() + " url before: " + url);
            }
            //负载均衡返回serviceInstance
            return this.choose(exchange).doOnNext((response) -> {
                if (!response.hasServer()) {
                    throw NotFoundException.create(this.properties.isUse404(), "Unable to find instance for " + url.getHost());
                } else {
                    URI uri = exchange.getRequest().getURI();
                    String overrideScheme = null;
                    if (schemePrefix != null) {
                        overrideScheme = url.getScheme();
                    }

                    DelegatingServiceInstance serviceInstance = new DelegatingServiceInstance((ServiceInstance)response.getServer(), overrideScheme);
                    URI requestUrl = this.reconstructURI(serviceInstance, uri);
                    if (log.isTraceEnabled()) {
                        log.trace("LoadBalancerClientFilter url chosen: " + requestUrl);
                    }

                    exchange.getAttributes().put(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR, requestUrl);
                }
            }).then(chain.filter(exchange));
        } else {
            return chain.filter(exchange);
        }
    }

由上述代码可见，负载均衡策略发生在choose方法，下面我们看一下choose方法。


    private Mono<Response<ServiceInstance>> choose(ServerWebExchange exchange) {
        //获取源url
        URI uri = (URI)exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR);
        //后获取ServiceInstance 列表
        ReactorLoadBalancer<ServiceInstance> loadBalancer = (ReactorLoadBalancer)this.clientFactory.getInstance(uri.getHost(), ReactorLoadBalancer.class, new Class[]{ServiceInstance.class});
        if (loadBalancer == null) {
            throw new NotFoundException("No loadbalancer available for " + uri.getHost());
        } else {
            //负载均衡策略获取其中一个ServiceInstance
            return loadBalancer.choose(this.createRequest());
        }
    }

以上便是gateway提供的负载均衡过滤器。

灰度路由介绍

首先，灰度概念，介于对与错之间的，便是灰度。详细概念可自行查阅。而灰度路由的产生，实在强大的负载均衡策略下，选中你想要的那一个路由。若抛开gateway组件不谈，可以理解为，如果在强大的负载均衡策略下，所有请求都到达同一个服务实例。一句话——谈何容易。若真的实现，那就可以实现定向攻击了。
可是，gateway为我们提供了支持编辑的“元数据”标签。如下图所示：

这个自定义的元数据标签，就会让我们的负载均衡策略变得更加灵活。可以做到按照标签匹配。
也有人说，这样不安全，容易受到攻击。但是不要忘记，这个配置界面，只有内部可见，只要保证网络的内外隔离，就保证元数据标签不会被外人可见，那么就会被认为，是安全的。这极大地方便了开发者调试和定位线上问题，且不影响线上使用。

灰度路由方案实现

下面，我将实现灰度路由方案

首先，基于上述知识，若想实现灰度路由方案，我们只需要按我们需求修改负载均衡逻辑即可。方案有两种，一种是修改ribbon的负载均衡逻辑，一种是我们继承ReactiveLoadBalancerClientFilter类，并重载他的filter逻辑，重载他的choose方法。
而重新修改ribbon负载均衡逻辑由于涉及到源码，修改起来并不容易。而基于以上了解到的知识，采用方案二，将会变得容易。
所以，我们将采用方案二，创建新的filter，继承ReactiveLoadBalancerClientFilter类，重写choose方法。
总结一下，我们需要修改ReactiveLoadBalancerClientFilter 中loadBalance的choose方案，根据请求待的元数据标签和naco所带的元数据标签比较，命中则返回，不命中，则走其他的策略。

本方案实现的核心代码如下：

  @Override
    public ServiceInstance choose(String serviceId, ServerHttpRequest request) {
        List<ServiceInstance> instances = discoveryClient.getInstances(serviceId);

        //注册中心无实例 抛出异常
        if (CollectionUtils.isEmpty(instances)) {
            System.out.println(("No instance available for {}" + serviceId));
            ;
            throw new NotFoundException("No instance available for " + serviceId);
        }

        // 获取请求version，无则随机返回可用实例
        String reqVersion = request.getHeaders().getFirst("VERSION");
        if (StringUtils.isBlank(reqVersion)) {
            return instances.get(RandomUtils.nextInt(instances.size()));
        }

        // 遍历可以实例元数据，若匹配则返回此实例
        for (ServiceInstance instance : instances) {
            Map<String, String> metadata = instance.getMetadata();
            String targetVersion = metadata.get("VERSION");
            if (reqVersion.equalsIgnoreCase(targetVersion)) {
                return instance;
            }
        }
        return instances.get(RandomUtils.nextInt(instances.size()));
    }

核心代码可见，在不命中时，方便理解，我随机返回了一个。在这里，大家可以按照自己的业务逻辑进行改写，也可以重新调用loadbalabce的choose方法。
至此，整个灰度路由的内容介绍完了。

工程代码

下面我将放置实现的过程源码。

package com.css.dynamic.config;

import com.css.dynamic.filter.GrayReactiveLoadBalancerClientFilter;
import com.css.dynamic.rule.GrayLoadBalancer;
import com.css.dynamic.rule.VersionGrayLoadBalancer;
import org.springframework.boot.autoconfigure.AutoConfigureBefore;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.boot.autoconfigure.condition.ConditionalOnWebApplication;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.cloud.gateway.config.GatewayReactiveLoadBalancerClientAutoConfiguration;
import org.springframework.cloud.gateway.config.LoadBalancerProperties;
import org.springframework.cloud.gateway.filter.ReactiveLoadBalancerClientFilter;
import org.springframework.cloud.loadbalancer.support.LoadBalancerClientFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Scope;

/**
 * Mica ribbon rule auto configuration.
 *
 * @author L.cm
 * @link https://github.com/lets-mica/mica
 */
@Configuration
@EnableConfigurationProperties(LoadBalancerProperties.class)
@ConditionalOnProperty(value = "route.gray.rule.enabled", havingValue = "true")
@AutoConfigureBefore(GatewayReactiveLoadBalancerClientAutoConfiguration.class)
@ConditionalOnWebApplication(type = ConditionalOnWebApplication.Type.REACTIVE)
public class GrayLoadBalancerClientConfiguration {

    @Bean
    public GrayReactiveLoadBalancerClientFilter gatewayLoadBalancerClientFilter(GrayLoadBalancer grayLoadBalancer,
                                                                                LoadBalancerProperties properties) {
        return new GrayReactiveLoadBalancerClientFilter(properties, grayLoadBalancer);
    }

//    @Bean
//    public ReactiveLoadBalancerClientFilter gatewayLoadBalancerClientFilter(LoadBalancerClientFactory clientFactory,
//                                                                            LoadBalancerProperties properties) {
//        return new ReactiveLoadBalancerClientFilter(clientFactory,properties);
//    }


    @Bean
    public GrayLoadBalancer grayLoadBalancer(DiscoveryClient discoveryClient) {
        return new VersionGrayLoadBalancer(discoveryClient);
    }

}

package com.css.dynamic.filter;

import com.alibaba.nacos.client.naming.utils.RandomUtils;
import com.css.gatewaynacos.rule.GrayLoadBalancer;
import org.apache.commons.lang3.StringUtils;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.cloud.client.loadbalancer.LoadBalancerUriTools;
import org.springframework.cloud.client.loadbalancer.reactive.DefaultResponse;
import org.springframework.cloud.gateway.config.LoadBalancerProperties;
import org.springframework.cloud.gateway.filter.GatewayFilterChain;
import org.springframework.cloud.gateway.filter.ReactiveLoadBalancerClientFilter;
import org.springframework.cloud.gateway.support.DelegatingServiceInstance;
import org.springframework.cloud.gateway.support.NotFoundException;
import org.springframework.cloud.gateway.support.ServerWebExchangeUtils;
import org.springframework.stereotype.Component;
import org.springframework.web.server.ServerWebExchange;
import reactor.core.publisher.Mono;

import java.net.URI;
import java.util.List;
import java.util.Map;

/**
 * @author 郭辉
 * @className TODO
 * @description TODO
 * @date 2022/8/20 8:40
 * @company 海康威视
 * @since 1.0.0
 */
@Component
public class MyGrayLoadBalancer extends ReactiveLoadBalancerClientFilter {
    private static final int LOAD_BALANCER_CLIENT_FILTER_ORDER = 10150;
    private LoadBalancerProperties properties;

    private GrayLoadBalancer grayLoadBalancer;

    private DiscoveryClient discoveryClient;

    public MyGrayLoadBalancer(LoadBalancerProperties properties, GrayLoadBalancer grayLoadBalancer,DiscoveryClient discoveryClient) {
        super(null, properties);
        this.properties = properties;
        this.grayLoadBalancer = grayLoadBalancer;
        this.discoveryClient = discoveryClient;
    }

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        URI url = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR);
        String schemePrefix = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_SCHEME_PREFIX_ATTR);
        if (!"lb".equalsIgnoreCase(schemePrefix) && !url.getScheme().equalsIgnoreCase("lb")){
            return chain.filter(exchange);
        }
        ServiceInstance serviceInstance = getServiceInstance(exchange, url);

        return Mono.just(new DefaultResponse(serviceInstance)).doOnNext((response)->{
            if (!response.hasServer()){
                throw NotFoundException.create(properties.isUse404(),
                        "Unable to find instance for " + url.getHost());
            }

            URI uri = exchange.getRequest().getURI();

            // if the `lb:<scheme>` mechanism was used, use `<scheme>` as the default,
            // if the loadbalancer doesn't provide one.
            String overrideScheme = null;
            if (schemePrefix != null) {
                overrideScheme = url.getScheme();
            }

            DelegatingServiceInstance delegatingServiceInstance = new DelegatingServiceInstance(
                    response.getServer(), overrideScheme);

            URI requestUrl = LoadBalancerUriTools.reconstructURI(delegatingServiceInstance, uri);
            exchange.getAttributes().put(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR, requestUrl);
        }).then(chain.filter(exchange));
    }

    private ServiceInstance getServiceInstance(ServerWebExchange exchange, URI url) {
        List<ServiceInstance> instances = discoveryClient.getInstances(url.getHost());
        String version = exchange.getRequest().getHeaders().getFirst("VERSION");
        if (StringUtils.isEmpty(version)){//随机取一个实例
            return instances.get(RandomUtils.nextInt(instances.size()));
        }
        ServiceInstance serviceInstance = null;
        for (ServiceInstance service : instances) {
            Map<String, String> metadata = service.getMetadata();
            if (metadata != null && metadata.get("VERSION") .equalsIgnoreCase(version)){
                serviceInstance = service;
            }
        }
        return serviceInstance;
    }

    @Override
    public int getOrder() {
        return LOAD_BALANCER_CLIENT_FILTER_ORDER;
    }

}

package com.css.dynamic.filter;

import com.css.dynamic.rule.GrayLoadBalancer;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.loadbalancer.LoadBalancerUriTools;
import org.springframework.cloud.client.loadbalancer.reactive.DefaultResponse;
import org.springframework.cloud.client.loadbalancer.reactive.Response;
import org.springframework.cloud.gateway.config.LoadBalancerProperties;
import org.springframework.cloud.gateway.filter.GatewayFilterChain;
import org.springframework.cloud.gateway.filter.ReactiveLoadBalancerClientFilter;
import org.springframework.cloud.gateway.support.DelegatingServiceInstance;
import org.springframework.cloud.gateway.support.NotFoundException;
import org.springframework.cloud.gateway.support.ServerWebExchangeUtils;
import org.springframework.stereotype.Component;
import org.springframework.web.server.ServerWebExchange;
import reactor.core.publisher.Mono;

import java.net.URI;

/**
 * @description: 灰度路由策略，若没有灰度，则随机返回实例
 * @author: guohui13
 * @date: 2022/8/19 17:24
 **/
@Component
public class GrayReactiveLoadBalancerClientFilter extends ReactiveLoadBalancerClientFilter {
    private static final int LOAD_BALANCER_CLIENT_FILTER_ORDER = 0;
    private LoadBalancerProperties properties;

    private GrayLoadBalancer grayLoadBalancer;

    public GrayReactiveLoadBalancerClientFilter(LoadBalancerProperties properties, GrayLoadBalancer grayLoadBalancer) {
        super(null, properties);
        this.properties = properties;
        this.grayLoadBalancer = grayLoadBalancer;
    }

    @Override
    public int getOrder() {
        return LOAD_BALANCER_CLIENT_FILTER_ORDER;
    }

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        URI url = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR);
        String schemePrefix = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_SCHEME_PREFIX_ATTR);
        if (url == null
                || (!"lb".equals(url.getScheme()) && !"lb".equals(schemePrefix))) {
            return chain.filter(exchange);
        }
        // preserve the original url
        ServerWebExchangeUtils.addOriginalRequestUrl(exchange, url);

//        if (log.isTraceEnabled()) {
//            log.trace(ReactiveLoadBalancerClientFilter.class.getSimpleName()
//                    + " url before: " + url);
//        }
        return choose(exchange).doOnNext(response -> {

            if (!response.hasServer()) {
                throw NotFoundException.create(properties.isUse404(),
                        "Unable to find instance for " + url.getHost());
            }
            URI uri = exchange.getRequest().getURI();
            // if the `lb:<scheme>` mechanism was used, use `<scheme>` as the default,
            // if the loadbalancer doesn't provide one.
            String overrideScheme = null;
            if (schemePrefix != null) {
                overrideScheme = url.getScheme();
            }
            DelegatingServiceInstance serviceInstance = new DelegatingServiceInstance(
                    response.getServer(), overrideScheme);

            URI requestUrl = LoadBalancerUriTools.reconstructURI(serviceInstance, uri);

//            if (log.isTraceEnabled()) {
//                log.trace("LoadBalancerClientFilter url chosen: " + requestUrl);
//            }
            exchange.getAttributes().put(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR, requestUrl);
        }).then(chain.filter(exchange));
    }

    private Mono<Response<ServiceInstance>> choose(ServerWebExchange exchange) {
        URI uri = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR);
        ServiceInstance serviceInstance = grayLoadBalancer.choose(uri.getHost(), exchange.getRequest());
        return Mono.just(new DefaultResponse(serviceInstance));
    }
}

package com.css.dynamic.rule;

import com.alibaba.nacos.client.naming.utils.RandomUtils;
import lombok.AllArgsConstructor;
import org.apache.commons.lang3.StringUtils;
import org.springframework.cloud.client.ServiceInstance;
import org.springframework.cloud.client.discovery.DiscoveryClient;
import org.springframework.cloud.gateway.support.NotFoundException;
import org.springframework.http.server.reactive.ServerHttpRequest;
import org.springframework.util.CollectionUtils;

import java.util.List;
import java.util.Map;

/**
 * 基于客户端版本号灰度路由
 *
 * @description:
 * @author: guohui13
 * @date: 2022/8/19 17:24
 **/
@AllArgsConstructor
public class VersionGrayLoadBalancer implements GrayLoadBalancer {
    private DiscoveryClient discoveryClient;


    /**
     * 根据serviceId 筛选可用服务
     *
     * @param serviceId 服务ID
     * @param request   当前请求
     * @return
     */
    @Override
    public ServiceInstance choose(String serviceId, ServerHttpRequest request) {
        List<ServiceInstance> instances = discoveryClient.getInstances(serviceId);

        //注册中心无实例 抛出异常
        if (CollectionUtils.isEmpty(instances)) {
            System.out.println(("No instance available for {}" + serviceId));
            ;
            throw new NotFoundException("No instance available for " + serviceId);
        }

        // 获取请求version，无则随机返回可用实例
        String reqVersion = request.getHeaders().getFirst("VERSION");
        if (StringUtils.isBlank(reqVersion)) {
            return instances.get(RandomUtils.nextInt(instances.size()));
        }

        // 遍历可以实例元数据，若匹配则返回此实例
        for (ServiceInstance instance : instances) {
            Map<String, String> metadata = instance.getMetadata();
//            String targetVersion = MapUtil.getStr(metadata, "VERSION");
            String targetVersion = metadata.get("VERSION");
            if (reqVersion.equalsIgnoreCase(targetVersion)) {
//                log.debug("gray requst match success :{} {}", reqVersion, instance);
                return instance;
            }
        }
//        return instances.get(RandomUtil.randomInt(instances.size()));
        return instances.get(RandomUtils.nextInt(instances.size()));
    }
}