利用Apollo进行Sentinel规则持久化

什么是sentinel？

Sentinel 是面向分布式服务架构的流量控制组件，主要以流量为切入点，从限流、流量整形、熔断降级、系统负载保护、热点防护等多个维度来帮助开发者保障微服务的稳定性。

源码地址：https://github.com/alibaba/Sentinel

官方文档：https://github.com/alibaba/Sentinel/wiki

为什么要持久化

我们下载好sentinel之后,每次要使用都需要启动sentinel的jar包,在sentinel中添加规则之后,每次重启,所有规则都会没了!所以我们需要把这些限流规则存到Apollo中统一保存!这样就可以永久保存了!

变更与原理

如图这是Sentinelt原始的工作方式，Sentinelt的Dashboard将我们在控制台配置的各种规则直接推送至客户端并直接更新到内存中，这种做法的好处是简单，无依赖；坏处是应用重启规则就会消失，仅用于简单测试，不能用于生产环境。

如图这是改造之后的原理图。我们将Sentinelt Dashboard跟Sentinelt客户端的关系进行了解耦。首先我们的Sentinelt Dashboard可以从远程配置中心读取到配置数据，并实时将更新的配置信心推送到配置中心进行持久化。然后Sentinelt客户端实时监听配置中心变化，当配置数据变化时Sentinelt客户端及时进行内存刷新。

通过分析原理我们知道整个改造要分两步：1 Sentinelt Dashboard将数据推送到配置中心 2 Sentinelt客户端对配置中心进行监听

Sentinelt Dashboard将数据推送到配置中心

1 从官网拉去Sentinelt Dashboard源码并用idea打开

2 找到下面的Apollo的代码：

这部分代码是在test包下，我们需要复制到main包下才行：

复制过去后会发现一个问题，所有Apollo相关的import都报错，需要改下pom.xml中的配置，把scope注释掉：

<!-- for Apollo rule publisher sample -->
  <dependency>
        <groupId>com.ctrip.framework.apollo</groupId>
        <artifactId>apollo-openapi</artifactId>
        <version>1.2.0</version>
        <!--<scope>test</scope>-->
  </dependency>

3 ApolloConfig中需要修改的有PortalUrl和token:

@Bean
 public ApolloOpenApiClient apolloOpenApiClient() {
     ApolloOpenApiClient client = ApolloOpenApiClient.newBuilder()
            .withPortalUrl("http://localhost:10034")
            .withToken("token")
            .build();
     return client;

 }

token的生成需要去Apollo的Portal中进行生成:https://blog.youkuaiyun.com/lizz861109/article/details/107426057

4 这样改完其实还没效果，因为页面上用的是FlowControllerV1的代码，你可以直接在访问uri后面加上v2来访问，但这不是很好，我们可以直接改下V1代码：

@RestController
@RequestMapping(value = "/v1/flow")
public class FlowControllerV1 {

    private final Logger logger = LoggerFactory.getLogger(FlowControllerV1.class);

    @Autowired
    private InMemoryRuleRepositoryAdapter<FlowRuleEntity> repository;

    @Autowired
    private SentinelApiClient sentinelApiClient;

    @Autowired
    @Qualifier("flowRuleApolloProvider")
    private DynamicRuleProvider<List<FlowRuleEntity>> ruleProvider;
    @Autowired
    @Qualifier("flowRuleApolloPublisher")
    private DynamicRulePublisher<List<FlowRuleEntity>> rulePublisher;

    @GetMapping("/rules")
    @AuthAction(PrivilegeType.READ_RULE)
    public Result<List<FlowRuleEntity>> apiQueryMachineRules(@RequestParam String app,
                                                             @RequestParam String ip,
                                                             @RequestParam Integer port) {

        if (StringUtil.isEmpty(app)) {
            return Result.ofFail(-1, "app can't be null or empty");
        }
        if (StringUtil.isEmpty(ip)) {
            return Result.ofFail(-1, "ip can't be null or empty");
        }
        if (port == null) {
            return Result.ofFail(-1, "port can't be null");
        }
        try {
//            List<FlowRuleEntity> rules = sentinelApiClient.fetchFlowRuleOfMachine(app, ip, port);
            List<FlowRuleEntity> rules = ruleProvider.getRules(app);
            rules = repository.saveAll(rules);
            return Result.ofSuccess(rules);
        } catch (Throwable throwable) {
            logger.error("Error when querying flow rules", throwable);
            return Result.ofThrowable(-1, throwable);
        }
    }

    private <R> Result<R> checkEntityInternal(FlowRuleEntity entity) {
        if (StringUtil.isBlank(entity.getApp())) {
            return Result.ofFail(-1, "app can't be null or empty");
        }
        if (StringUtil.isBlank(entity.getIp())) {
            return Result.ofFail(-1, "ip can't be null or empty");
        }
        if (entity.getPort() == null) {
            return Result.ofFail(-1, "port can't be null");
        }
        if (StringUtil.isBlank(entity.getLimitApp())) {
            return Result.ofFail(-1, "limitApp can't be null or empty");
        }
        if (StringUtil.isBlank(entity.getResource())) {
            return Result.ofFail(-1, "resource can't be null or empty");
        }
        if (entity.getGrade() == null) {
            return Result.ofFail(-1, "grade can't be null");
        }
        if (entity.getGrade() != 0 && entity.getGrade() != 1) {
            return Result.ofFail(-1, "grade must be 0 or 1, but " + entity.getGrade() + " got");
        }
        if (entity.getCount() == null || entity.getCount() < 0) {
            return Result.ofFail(-1, "count should be at lease zero");
        }
        if (entity.getStrategy() == null) {
            return Result.ofFail(-1, "strategy can't be null");
        }
        if (entity.getStrategy() != 0 && StringUtil.isBlank(entity.getRefResource())) {
            return Result.ofFail(-1, "refResource can't be null or empty when strategy!=0");
        }
        if (entity.getControlBehavior() == null) {
            return Result.ofFail(-1, "controlBehavior can't be null");
        }
        int controlBehavior = entity.getControlBehavior();
        if (controlBehavior == 1 && entity.getWarmUpPeriodSec() == null) {
            return Result.ofFail(-1, "warmUpPeriodSec can't be null when controlBehavior==1");
        }
        if (controlBehavior == 2 && entity.getMaxQueueingTimeMs() == null) {
            return Result.ofFail(-1, "maxQueueingTimeMs can't be null when controlBehavior==2");
        }
        if (entity.isClusterMode() && entity.getClusterConfig() == null) {
            return Result.ofFail(-1, "cluster config should be valid");
        }
        return null;
    }

    @PostMapping("/rule")
    @AuthAction(PrivilegeType.WRITE_RULE)
    public Result<FlowRuleEntity> apiAddFlowRule(@RequestBody FlowRuleEntity entity) {
        Result<FlowRuleEntity> checkResult = checkEntityInternal(entity);
        if (checkResult != null) {
            return checkResult;
        }
        entity.setId(null);
        Date date = new Date();
        entity.setGmtCreate(date);
        entity.setGmtModified(date);
        entity.setLimitApp(entity.getLimitApp().trim());
        entity.setResource(entity.getResource().trim());
        try {
            entity = repository.save(entity);

//            publishRules(entity.getApp(), entity.getIp(), entity.getPort()).get(5000, TimeUnit.MILLISECONDS);
            if (!publishRules(entity.getApp(), entity.getIp(), entity.getPort())) {
                logger.error("Publish flow rules failed after rule add");
            }
//            return Result.ofSuccess(entity);
        } catch (Throwable t) {
            Throwable e = t instanceof ExecutionException ? t.getCause() : t;
            logger.error("Failed to add new flow rule, app={}, ip={}", entity.getApp(), entity.getIp(), e);
            return Result.ofFail(-1, e.getMessage());
        }
        return Result.ofSuccess(entity);
    }

    @PutMapping("/save.json")
    @AuthAction(PrivilegeType.WRITE_RULE)
    public Result<FlowRuleEntity> apiUpdateFlowRule(HttpServletRequest request,
                                                    Long id, String app,
                                                    String limitApp, String resource, Integer grade,
                                                    Double count, Integer strategy, String refResource,
                                                    Integer controlBehavior, Integer warmUpPeriodSec,
                                                    Integer maxQueueingTimeMs) {

        if (id == null) {
            return Result.ofFail(-1, "id can't be null");
        }
        FlowRuleEntity entity = repository.findById(id);
        if (entity == null) {
            return Result.ofFail(-1, "id " + id + " dose not exist");
        }
        if (StringUtil.isNotBlank(app)) {
            entity.setApp(app.trim());
        }
        if (StringUtil.isNotBlank(limitApp)) {
            entity.setLimitApp(limitApp.trim());
        }
        if (StringUtil.isNotBlank(resource)) {
            entity.setResource(resource.trim());
        }
        if (grade != null) {
            if (grade != 0 && grade != 1) {
                return Result.ofFail(-1, "grade must be 0 or 1, but " + grade + " got");
            }
            entity.setGrade(grade);
        }
        if (count != null) {
            entity.setCount(count);
        }
        if (strategy != null) {
            if (strategy != 0 && strategy != 1 && strategy != 2) {
                return Result.ofFail(-1, "strategy must be in [0, 1, 2], but " + strategy + " got");
            }
            entity.setStrategy(strategy);
            if (strategy != 0) {
                if (StringUtil.isBlank(refResource)) {
                    return Result.ofFail(-1, "refResource can't be null or empty when strategy!=0");
                }
                entity.setRefResource(refResource.trim());
            }
        }
        if (controlBehavior != null) {
            if (controlBehavior != 0 && controlBehavior != 1 && controlBehavior != 2) {
                return Result.ofFail(-1, "controlBehavior must be in [0, 1, 2], but " + controlBehavior + " got");
            }
            if (controlBehavior == 1 && warmUpPeriodSec == null) {
                return Result.ofFail(-1, "warmUpPeriodSec can't be null when controlBehavior==1");
            }
            if (controlBehavior == 2 && maxQueueingTimeMs == null) {
                return Result.ofFail(-1, "maxQueueingTimeMs can't be null when controlBehavior==2");
            }
            entity.setControlBehavior(controlBehavior);
            if (warmUpPeriodSec != null) {
                entity.setWarmUpPeriodSec(warmUpPeriodSec);
            }
            if (maxQueueingTimeMs != null) {
                entity.setMaxQueueingTimeMs(maxQueueingTimeMs);
            }
        }
        Date date = new Date();
        entity.setGmtModified(date);
        try {
            entity = repository.save(entity);
            if (entity == null) {
                return Result.ofFail(-1, "save entity fail: null");
            }

//            publishRules(entity.getApp(), entity.getIp(), entity.getPort()).get(5000, TimeUnit.MILLISECONDS);
//            return Result.ofSuccess(entity);
            if (!publishRules(entity.getApp(), entity.getIp(), entity.getPort())) {
                logger.info("publish flow rules fail after rule update");
            }
        } catch (Throwable t) {
            Throwable e = t instanceof ExecutionException ? t.getCause() : t;
            logger.error("Error when updating flow rules, app={}, ip={}, ruleId={}", entity.getApp(),
                entity.getIp(), id, e);
            return Result.ofFail(-1, e.getMessage());
        }

        return Result.ofSuccess(entity);
    }

    @DeleteMapping("/delete.json")
    @AuthAction(PrivilegeType.WRITE_RULE)
    public Result<Long> apiDeleteFlowRule(Long id) {

        if (id == null) {
            return Result.ofFail(-1, "id can't be null");
        }
        FlowRuleEntity oldEntity = repository.findById(id);
        if (oldEntity == null) {
            return Result.ofSuccess(null);
        }

        try {
            repository.delete(id);
            if (!publishRules(oldEntity.getApp(), oldEntity.getIp(), oldEntity.getPort())) {
                logger.info("publish flow rules fail after rule delete");
            }
        } catch (Exception e) {
            return Result.ofFail(-1, e.getMessage());
        }
//        try {
//            publishRules(oldEntity.getApp(), oldEntity.getIp(), oldEntity.getPort()).get(5000, TimeUnit.MILLISECONDS);
//            return Result.ofSuccess(id);
//        } catch (Throwable t) {
//            Throwable e = t instanceof ExecutionException ? t.getCause() : t;
//            logger.error("Error when deleting flow rules, app={}, ip={}, id={}", oldEntity.getApp(),
//                oldEntity.getIp(), id, e);
//            return Result.ofFail(-1, e.getMessage());
//        }
        return Result.ofSuccess(id);
    }

//    private CompletableFuture<Void> publishRules(String app, String ip, Integer port) {
//        List<FlowRuleEntity> rules = repository.findAllByMachine(MachineInfo.of(app, ip, port));
//        return sentinelApiClient.setFlowRuleOfMachineAsync(app, ip, port, rules);
//    }

    private boolean publishRules(String app, String ip, Integer port) throws Exception {
        List<FlowRuleEntity> rules = repository.findAllByApp(app);
        rulePublisher.publish(app, rules);
        return true;
    }
}

5 启动DashboardApplication应用。需要添加启动参数

-Dserver.port=8080
-Dcsp.sentinel.dashboard.server=localhost:8080
-Dproject.name=apollo-test

Sentinelt客户端监听配置中心

1开启监听配置

@Configuration
public class Sentinel4ApolloConfig {

    @Value("${spring.application.name}")
    private String applicationName = "";

    @Bean
    @ConditionalOnMissingBean
    public SentinelResourceAspect sentinelResourceAspect() {
        return new SentinelResourceAspect();
    }
    @Bean
    @ConditionalOnMissingBean
    public ApolloDataSourceListener apolloDataSourceListener() {
        return new ApolloDataSourceListener(applicationName);
    }


}

2 开启监听器

public class ApolloDataSourceListener implements InitializingBean {

    private static final String FLOW_RULE_TYPE = "flow";

    private static final String DEGRADE_RULE_TYPE = "degrade";
    //  *-flow-rules
    private static final String FLOW_DATA_ID_POSTFIX = "-" + FLOW_RULE_TYPE + "-rules";
    //  *-degrade-rules
    private static final String DEGRADE_DATA_ID_POSTFIX = "-" + DEGRADE_RULE_TYPE + "-rules";

    private String applicationName ;

    public ApolloDataSourceListener(String applicationName) {
        this.applicationName = applicationName;
    }

    @Override
    public void afterPropertiesSet() throws Exception {
        initFlowRules();
    }

    private void initFlowRules() {
        //apollo-test-flow-rules
        String flowRuleKey = applicationName + FLOW_DATA_ID_POSTFIX;
        // 动态监听
        ReadableDataSource<String, List<FlowRule>> flowRuleDataSource =
                new ApolloDataSource<>("application",
                        flowRuleKey,
                        "[]",
                        source -> JSON.parseObject(source, new TypeReference<List<FlowRule>>() {
                        }));
        // 刷新内存
        FlowRuleManager.register2Property(flowRuleDataSource.getProperty());
    }

}

3 这样客户端对配置中心的监听就配置好了，客户端启动后就可以实时关注到配置中心的变化。注意启动客户端时需要通过配置启动参数

-Dapollo.configService=http://ip:8080

以上代码地址