doris创建异步物化视图(加速数据低频变更的复杂实时计算)

异步物化视图,可以把那些每次实时计算非常耗时的,而需要计算的数据变更比较低频的这些计算创建对应的异步物化视图,当相关数据变化的时候触发异步任务去更新计算结果,或者定时计算也可以。例如该处示范为计算订单的订单标识,订单标识使用bit位储存在一个整数内,每一个比特位存储了一个标识。目前已有二十余个标识了,如果实时计算在做筛选项的自动列出可选元素时就会崩,会去扫所有数据看有哪些可选值,因此弄一个异步物化视图来异步计算好结果连表查询拿到结果。
异步物化视图官方文档位置:
https://doris.apache.org/zh-CN/docs/query/view-materialized-view/async-materialized-view

-- 使用数据库test2
use test2;
-- 创建异步物化视图
CREATE MATERIALIZED VIEW order_identification_view
        BUILD DEFERRED REFRESH AUTO ON MANUAL
        DISTRIBUTED BY RANDOM BUCKETS 2
        PROPERTIES ('replication_num' = '1') 
        AS 
            select oi.order_id ,
            concat_ws(','
	,if(oi.order_identification & 2 > 0,'0元购',null)
	,if(oi.order_identification & 4 > 0,'子母件',null)
	,if(oi.order_identification & 8 > 0,'保价成交',null)
	) identification
           FROM
	test2.ods_order_order_info oi;

-- 先强制刷新一次视图,让数据计算好,后续的走数据变更钩子刷新
REFRESH MATERIALIZED VIEW order_identification_view ;
-- 查询数据看看,后续可以把这个视图当一个实际的表一样使用
select * from test2.order_identification_view v where LENGTH (v.identification) >0;
-- 删除物化视图
DROP MATERIALIZED VIEW order_identification_view;
-- 查看异步物化视图任务
select * from jobs("type"="mv") order by CreateTime;






查询这个异步物化视图

在这里插入图片描述
查看异步物化视图任务
在这里插入图片描述

CREATE MATERIALIZED VIEW dim_cw04_ss_prcie_temp_0108 (cinventoryid,caccountperiod,nabprice,rn) BUILD IMMEDIATE REFRESH AUTO ON SCHEDULE EVERY 5 MINUTE DUPLICATE KEY(`cinventoryid`, `caccountperiod`) DISTRIBUTED BY RANDOM BUCKETS 16 PROPERTIES ( "replication_allocation" = "tag.location.default: 3", "min_load_replica_num" = "-1", "is_being_synced" = "false", "storage_medium" = "hdd", "storage_format" = "V2", "inverted_index_storage_format" = "V1", "light_schema_change" = "true", "disable_auto_compaction" = "false", "enable_single_replica_compaction" = "false", "group_commit_interval_ms" = "10000", "group_commit_data_bytes" = "134217728" ) AS select distinct `t`.`CINVENTORYID`, ifnull(`t1`.`CACCOUNTPERIOD`,`t2`.`CACCOUNTPERIOD`)as `CACCOUNTPERIOD` , ifnull(`t1`.`NABPRICE`,ifnull(`t2`.`NABPRICE`,`t3`.`NABPRICE`)) as `NABPRICE`, ifnull(`t1`.`rn`,`t2`.`rn`) as `rn` from ( select distinct `internal`.`CW`.`ods_cw04_ic_flow`.`cmaterialoid` as `CINVENTORYID` from `internal`.`CW`.`ods_cw04_ic_flow` union select distinct `internal`.`CW`.`ods_cw04_ia_monthnab`.`CINVENTORYID` from `internal`.`CW`.`ods_cw04_ia_monthnab` ) `t` left join ( select `internal`.`CW`.`ods_cw04_ia_monthnab`.`cinventoryid` as `CINVENTORYID` , `internal`.`CW`.`ods_cw04_ia_monthnab`.`caccountperiod` as `CACCOUNTPERIOD` , sum( `internal`.`CW`.`ods_cw04_ia_monthnab`.`nabmny` ) / sum( `internal`.`CW`.`ods_cw04_ia_monthnab`.`nabnum` ) as `NABPRICE` , row_number() over (partition by `internal`.`CW`.`ods_cw04_ia_monthnab`.`cinventoryid` order by `internal`.`CW`.`ods_cw04_ia_monthnab`.`caccountperiod` desc) as `rn` from `internal`.`CW`.`ods_cw04_ia_monthnab` where `internal`.`CW`.`ods_cw04_ia_monthnab`.`NABPRICE` is not null and `internal`.`CW`.`ods_cw04_ia_monthnab`.`pk_org` in ( select distinct PK_COSTREGION from `internal`.`CW`.`dim_cw04_org_costregion` where NAME like '%和泰%' ) group by `internal`.`CW`.`ods_cw04_ia_monthnab`.`cinventoryid` , `internal`.`CW`.`ods_cw04_ia_monthnab`.`caccountperiod` having `rn` =1 ) `t1` on `t`.`CINVENTORYID`=`t1`.`CINVENTORYID` left join ( select `internal`.`CW`.`ods_cw04_ia_monthnab`.`cinventoryid` as `CINVENTORYID` , `internal`.`CW`.`ods_cw04_ia_monthnab`.`caccountperiod` as `CACCOUNTPERIOD` , sum( `internal`.`CW`.`ods_cw04_ia_monthnab`.`nabmny` ) / sum( `internal`.`CW`.`ods_cw04_ia_monthnab`.`nabnum` ) as `NABPRICE` , row_number() over (partition by `internal`.`CW`.`ods_cw04_ia_monthnab`.`cinventoryid` order by `internal`.`CW`.`ods_cw04_ia_monthnab`.`caccountperiod` desc) as `rn` from `internal`.`CW`.`ods_cw04_ia_monthnab` where `internal`.`CW`.`ods_cw04_ia_monthnab`.`NABPRICE` is not null and `internal`.`CW`.`ods_cw04_ia_monthnab`.`pk_org` not in ( select distinct PK_COSTREGION from `internal`.`CW`.`dim_cw04_org_costregion` where NAME like '%和泰%' ) group by `internal`.`CW`.`ods_cw04_ia_monthnab`.`cinventoryid` , `internal`.`CW`.`ods_cw04_ia_monthnab`.`caccountperiod` having `rn` =1 )`t2` on `t`.`CINVENTORYID`=`t2`.`CINVENTORYID` left join ( select `internal`.`CW`.`ods_cw04_ic_flow`.`cmaterialoid` as `cinventoryid`, sum(`internal`.`CW`.`ods_cw04_ic_flow`.`ncostmny`) / sum( `internal`.`CW`.`ods_cw04_ic_flow`.`noutnum`) as `NABPRICE` from `internal`.`CW`.`ods_cw04_ic_flow` where `internal`.`CW`.`ods_cw04_ic_flow`.`pk_org` = "0001A31000000010J88O" and `internal`.`CW`.`ods_cw04_ic_flow`.`ncostmny` is not null and `internal`.`CW`.`ods_cw04_ic_flow`.`noutnum` is not null group by `internal`.`CW`.`ods_cw04_ic_flow`.`cmaterialoid` )`t3` on `t`.`CINVENTORYID`=`t3`.`CINVENTORYID` 按以上标准修改一下
最新发布
07-26
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

HumorChen99

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值