presto热加载catalog配置

最新推荐文章于 2025-04-14 18:42:06 发布

xiaoping_1988

最新推荐文章于 2025-04-14 18:42:06 发布

阅读量3.3k

点赞数

CC 4.0 BY-SA版权

分类专栏： presto 文章标签： java presto catalog热加载大数据

本文链接：https://blog.youkuaiyun.com/xiaoping_1988/article/details/122198309

presto 专栏收录该内容

2 篇文章

订阅专栏

本文介绍如何在Presto启动时通过远程获取catalog配置，并实现动态加载以实现实时更新。涉及代码变更、配置文件管理和定时刷新策略。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

presto启动从远程获取catalog配置做加载

启动类Server的代码变更

        try {
            Injector injector = app.strictConfig().initialize();

            logLocation(log, "Working directory", Paths.get("."));
            logLocation(log, "Etc directory", Paths.get("etc"));

            injector.getInstance(PluginManager.class).loadPlugins();
            // 变更代码 presto原生加载catalog
            // injector.getInstance(StaticCatalogStore.class).loadCatalogs();
            // 变更代码 动态加载catalog
            injector.getInstance(CatalogStore.class).loadCatalogs();

            // 
            // 变更代码  去掉在Server类加载ConnectorIds信息,统一在热加载模版更新
//            updateConnectorIds(injector.getInstance(Announcer.class), injector.getInstance(CatalogManager.class));

            injector.getInstance(SessionPropertyDefaults.class).loadConfigurationManager();
            injector.getInstance(ResourceGroupManager.class).loadConfigurationManager();
            injector.getInstance(AccessControlManager.class).loadSystemAccessControl();
            injector.getInstance(PasswordAuthenticatorManager.class).loadPasswordAuthenticator();
            injector.getInstance(EventListenerManager.class).loadEventListeners();
            injector.getInstance(GroupProviderManager.class).loadConfiguredGroupProvider();
            injector.getInstance(CertificateAuthenticatorManager.class).loadCertificateAuthenticator();

            injector.getInstance(Announcer.class).start();

            injector.getInstance(ServerInfoResource.class).startupComplete();

            log.info("======== SERVER STARTED ========");
        }
        catch (Throwable e) {
            log.error(e);
            System.exit(1);
        }

远程加载流程

在这里插入图片描述

presto监听远程catalog配置变化做热加载

在这里插入图片描述

配置类，配置写在config.properties

/**
 * 动态更新catalog信息的配置数据
 *
 * @Author 刘江平10269373
 * @Date 2020-12-25 8:45
 */
public class DynamicCatalogStoreConfig {
    private boolean dynamic;
    private String restUrl;
    private long timerPeriod= 5L;
    private String secretKey;

    @Config("catalog.dynamic")
    public DynamicCatalogStoreConfig setDynamic(boolean dynamic) {
        this.dynamic = dynamic;
        return this;
    }

    @Config("catalog.dynamic.rest-url")
    public DynamicCatalogStoreConfig setRestUrl(String restUrl) {
        this.restUrl = restUrl;
        return this;
    }

    @Config("catalog.dynamic.timer-period-minutes")
    public DynamicCatalogStoreConfig setTimerPeriod(long timerPeriod) {
        this.timerPeriod = timerPeriod * 60 * 1000;
        return this;
    }

    @Config("catalog.dynamic.secret-key")
    public DynamicCatalogStoreConfig setSecretKey(String secretKey) {
        this.secretKey = secretKey;
        return this;
    }

    public boolean isDynamic() {
        return dynamic;
    }

    public String getRestUrl() {
        return restUrl;
    }

    public long getTimerPeriod() {
        return timerPeriod;
    }

    public String getSecretKey() {
        return secretKey;
    }
}

key	描述	默认值
catalog.dynamic	是否动态热加载数据源	false
catalog.dynamic.rest-url	获取远程catalog数据源信息的接口	空
catalog.dynamic.secret-key	解密从远程获取的catalog信息	空
catalog.dynamic.timer-period-minutes	定时从远程获取catalog数据源并刷新本地内存catalog频率，单位分	5