使用with as优化sql解决filter

最新推荐文章于 2022-10-25 13:11:58 发布

原创最新推荐文章于 2022-10-25 13:11:58 发布 · 3.8k 阅读

2 ·

CC 4.0 BY-SA版权

文章标签：

#filter #sql #优化 #action #crm #产品

SQL TUNING 专栏收录该内容

16 篇文章

订阅专栏

本文详细阐述了一个地市报表系统中数据出得慢的问题，通过优化SQL语句，使用WITH as方法，最终将查询时间缩短至12秒内。包括收集统计信息、查看执行计划、识别性能瓶颈并采取相应措施的过程。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

最近一个地市的报表系统老是说数据出得慢，基本上要到中午才能出数据，但是查看过程是比较简单的。后来发现一条sql有问题，居然要跑5个多小时。

查看跑2个小时以上的sql
SELECT SQ.SQL_TEXT,
       S.SQL_ID,
       S.SID,
       S.SERIAL#,
       S.LAST_CALL_ET,
       TO_CHAR(S.LOGON_TIME, 'yyyymmdd hh24:mi:ss') LOGON_TIME
FROM   V$SESSION S,
             V$SQLTEXT SQ
WHERE S.SQL_ID = SQ.SQL_ID AND
                 LAST_CALL_ET > 10000 AND
                TO_CHAR(LOGON_TIME,'yyyymmdd') =TO_CHAR(SYSDATE,'yyyymmdd');

NTERVAL '7' DAY < D.COMPLETE_DT) 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
移机','资费变更') AND                          A.COMPLETE_DT - I 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
         A.BO_ACTION_NAME IN                          ('新装', ' 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
                     A.PROD_ID != D.PROD_ID AND                 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
ARTY_ID AND                          A.BO_ID != D.BO_ID AND     4kyzp58tz0xya 232 811 15400 20111110 06:27:38
TAFF_ID LIKE '36%' AND                          A.PARTY_ID = D.P 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
               FROM   DW_BO_ORDER D                   WHERE D.S 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
欢乐送之E6套餐') AND                 NOT EXISTS (SELECT *     4kyzp58tz0xya 232 811 15400 20111110 06:27:38
   '新春欢乐送之E8套餐',                                  '新春 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
) 全省_紧密融合型E9套餐产品规格',                               4kyzp58tz0xya 232 811 15400 20111110 06:27:38
紧密融合型E9套餐产品规格',                                  '(新 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
'普通E9','普通新版E8',                                  '全省_ 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
2S','E6移动版', 'E9版1M(老版)',                                 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
', 'ADSL','LAN', '手机',                                  'E8 - 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
移机','资费变更') AND                  B.PROD_SPEC IN ('普通电话 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
D LIKE '36%' AND                  A.BO_ACTION_NAME IN ('新装',' 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
   A.CHANNEL_ID = C.CHANNEL_ID AND                  A.SO_STAFF_I 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
     A           WHERE A.PROD_ID = B.PROD_ID AND               4kyzp58tz0xya 232 811 15400 20111110 06:27:38
                 DW_CRM_DAY_USER B,                  DW_BO_ORDER 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
                 A.PROD_ID            FROM   DW_CHANNEL      C, 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
              A.SO_STAFF_ID,                  A.ATOM_ACTION_ID, 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
              B.START_DT,                  A.BO_ACTION_NAME,    4kyzp58tz0xya 232 811 15400 20111110 06:27:38
              B.ACCESS_NUMBER,                  B.PROD_SPEC,    4kyzp58tz0xya 232 811 15400 20111110 06:27:38
CHANNEL_NAME,                  B.NAME           PARTY_NAME,    4kyzp58tz0xya 232 811 15400 20111110 06:27:38
                  B.AREA_NAME,                  C.NAME          4kyzp58tz0xya 232 811 15400 20111110 06:27:38
SELECT                   B.AREA_ID,                  A.PARTY_ID, 4kyzp58tz0xya 232 811 15400 20111110 06:27:38
         CREATE TABLE TMP_DTBB_FMT_10000_1 NOLOGGING AS         4kyzp58tz0xya 232 811 15400 20111110 06:27:38

找到的sql为如下：

SELECT
               B.AREA_ID,
               A.PARTY_ID,
               B.AREA_NAME,
               C.NAME           CHANNEL_NAME,
               B.NAME           PARTY_NAME,
               B.ACCESS_NUMBER,
               B.PROD_SPEC,
               B.START_DT,
               A.BO_ACTION_NAME,
               A.SO_STAFF_ID,
               A.ATOM_ACTION_ID,
               A.PROD_ID
        FROM   DW_CHANNEL      C,
               DW_CRM_DAY_USER B,
               DW_BO_ORDER     A
        WHERE A.PROD_ID = B.PROD_ID AND
               A.CHANNEL_ID = C.CHANNEL_ID AND
               A.SO_STAFF_ID LIKE '36%' AND
               A.BO_ACTION_NAME IN ('新装','移机','资费变更') AND
               B.PROD_SPEC IN ('普通电话', 'ADSL','LAN', '手机',
                               'E8 - 2S','E6移动版', 'E9版1M(老版)',
                               '普通E9','普通新版E8',
                               '全省_紧密融合型E9套餐产品规格',
                               '(新) 全省_紧密融合型E9套餐产品规格',
                               '新春欢乐送之E8套餐',
                               '新春欢乐送之E6套餐') AND
              NOT EXISTS (SELECT *
                FROM   DW_BO_ORDER D
                WHERE D.STAFF_ID LIKE '36%' AND
                       A.PARTY_ID = D.PARTY_ID AND
                       A.BO_ID != D.BO_ID AND
                       A.PROD_ID != D.PROD_ID AND
                       A.BO_ACTION_NAME IN
                       ('新装', '移机','资费变更') AND
                       A.COMPLETE_DT - INTERVAL '7' DAY < D.COMPLETE_DT)
下面根据sql语句去查找相关的session，发现该语句已经跑了将近4个小时，从早上6点多开始跑。

SQL> SELECT S.SQL_ID,to_char(S.LOGON_TIME,'yyyymmdd hh24:mi:ss') LOGON_TIME,S.LAST_CALL_ET,S.STATE FROM V$SESSION S WHERE S.SID=232;

SQL_ID LOGON_TIME LAST_CALL_ET STATE
------------- ----------------- ------------ -------------------
4kyzp58tz0xya 20111110 06:27:38 13647 WAITED SHORT TIME

SQL> select 13647/3600 from dual;

13647/3600
----------
3.79083333

SQL> select to_char(sysdate,'yyyymmdd hh24:mi:ss') sysdat from dual;

SYSDAT
-----------------
20111110 10:18:39

OK，现在知道导致数据出的慢的原因,现在就开始来优化这条sql，首先收集一下统计图，然后看执行计划

BEGIN
DBMS_STATS.GATHER_TABLE_STATS(ownname          => 'LBAS',
                                tabname          => 'DW_BO_ORDER',
                                estimate_percent => 20,
                                method_opt       => 'for all columns size auto',
                                no_invalidate    => FALSE,
                                degree           => 4,
                                granularity      => 'ALL',
                                cascade          => TRUE);
END;
/

Plan hash value: 2142862569
----------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
----------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 905 | 121K| 4152K (2)| 13:50:32 | | |
|* 1 | FILTER | | | | | | | |
|* 2 | HASH JOIN | | 905 | 121K| 12616 (2)| 00:02:32 | | |
|* 3 | HASH JOIN | | 905 | 99550 | 12448 (2)| 00:02:30 | | |
| 4 | PARTITION RANGE ALL| | 1979 | 108K| 9168 (2)| 00:01:51 | 1 | 5 |
|* 5 | TABLE ACCESS FULL | DW_BO_ORDER | 1979 | 108K| 9168 (2)| 00:01:51 | 1 | 5 |
|* 6 | TABLE ACCESS FULL | DW_CRM_DAY_USER | 309K| 15M| 3277 (2)| 00:00:40 | | |
| 7 | TABLE ACCESS FULL | DW_CHANNEL | 48425 | 1276K| 168 (1)| 00:00:03 | | |
|* 8 | FILTER | | | | | | | |
| 9 | PARTITION RANGE ALL | | 1 | 29 | 9147 (2)| 00:01:50 | 1 | 5 |
|* 10 | TABLE ACCESS FULL | DW_BO_ORDER | 1 | 29 | 9147 (2)| 00:01:50 | 1 | 5 |
----------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter( NOT EXISTS (SELECT /*+ */ 0 FROM "DW_BO_ORDER" "D" WHERE (:B1='新装' OR :B2='移机' OR
:B3='资费变更') AND "D"."PARTY_ID"=:B4 AND TO_CHAR("D"."STAFF_ID") LIKE '36%' AND
"D"."COMPLETE_DT">:B5-INTERVAL'+07 00:00:00' DAY(2) TO SECOND(0) AND "D"."PROD_ID"<>:B6 AND
"D"."BO_ID"<>:B7))
2 - access("A"."CHANNEL_ID"="C"."CHANNEL_ID")
3 - access("A"."PROD_ID"="B"."PROD_ID")
5 - filter("A"."PROD_ID" IS NOT NULL AND ("A"."BO_ACTION_NAME"='新装' OR
"A"."BO_ACTION_NAME"='移机' OR "A"."BO_ACTION_NAME"='资费变更') AND TO_CHAR("A"."SO_STAFF_ID") LIKE
'36%')
6 - filter("B"."PROD_SPEC"='(新) 全省_紧密融合型E9套餐产品规格' OR "B"."PROD_SPEC"='ADSL' OR
"B"."PROD_SPEC"='E6移动版' OR "B"."PROD_SPEC"='E8 - 2S' OR "B"."PROD_SPEC"='E9版1M(老版)' OR
"B"."PROD_SPEC"='LAN' OR "B"."PROD_SPEC"='普通E9' OR "B"."PROD_SPEC"='普通电话' OR
"B"."PROD_SPEC"='普通新版E8' OR "B"."PROD_SPEC"='全省_紧密融合型E9套餐产品规格' OR "B"."PROD_SPEC"='手机' OR
"B"."PROD_SPEC"='新春欢乐送之E6套餐' OR "B"."PROD_SPEC"='新春欢乐送之E8套餐')
8 - filter(:B1='新装' OR :B2='移机' OR :B3='资费变更')
10 - filter("D"."PARTY_ID"=:B1 AND TO_CHAR("D"."STAFF_ID") LIKE '36%' AND
"D"."COMPLETE_DT">:B2-INTERVAL'+07 00:00:00' DAY(2) TO SECOND(0) AND "D"."PROD_ID"<>:B3 AND
              "D"."BO_ID"<>:B4)

执行计划里面有个filter在里面，这个是导致这条sql跑的慢的原因。主要语句就是在not exists 里面居然存在!=，哎，这sql语句写得真坑爹，由于有!=的存在，CBO不能选择 HASH_AJ join的方式，准备修改sql语句，但是不管怎么改，业务逻辑总不对，最后的结果也不对。得想办法吧filter搞掉，加多种hint，最后的结果还是一样的，在朋友的帮下，最后使用了了with as. good最后居然实现了在12秒以内出数据。下面看一下

WITH D AS
(SELECT /*+ materialize */
   A.PARTY_ID,
   A.BO_ID,
   A.PROD_ID,
   A.COMPLETE_DT
FROM   DW_BO_ORDER A
WHERE STAFF_ID LIKE '36%' AND
         A.BO_ACTION_NAME IN ('新装',
                              '移机',
                              '资费变更'))
SELECT B.AREA_ID,
       A.PARTY_ID,
       B.AREA_NAME,
       C.NAME           CHANNEL_NAME,
       B.NAME           PARTY_NAME,
       B.ACCESS_NUMBER,
       B.PROD_SPEC,
       B.START_DT,
       A.BO_ACTION_NAME,
       A.SO_STAFF_ID,
       A.ATOM_ACTION_ID,
       A.PROD_ID
FROM   DW_CHANNEL      C,
       DW_CRM_DAY_USER B,
       DW_BO_ORDER     A
WHERE A.PROD_ID = B.PROD_ID AND
       A.CHANNEL_ID = C.CHANNEL_ID AND
       A.SO_STAFF_ID LIKE '36%' AND
       A.BO_ACTION_NAME IN ('新装',
                            '移机',
                            '资费变更') AND
       B.PROD_SPEC IN ('普通电话',
                       'ADSL',
                       'LAN',
                       '手机',
                       'E8 - 2S',
                       'E6移动版',
                       'E9版1M(老版)',
                       '普通E9',
                       '普通新版E8',
                       '全省_紧密融合型E9套餐产品规格',
                       '(新) 全省_紧密融合型E9套餐产品规格',
                       '新春欢乐送之E8套餐',
                       '新春欢乐送之E6套餐') AND
       NOT EXISTS (SELECT *
        FROM   D
        WHERE A.PARTY_ID = D.PARTY_ID AND
               A.BO_ID != D.BO_ID AND
               A.PROD_ID != D.PROD_ID AND
               A.COMPLETE_DT - INTERVAL '7'
         DAY < D.COMPLETE_DT);
已用时间: 00: 00: 12.37
执行计划
--------------------------------------------------------------------------------------------------------------------------
Plan hash value: 2591883460
--------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time | Pstart| Pstop |
--------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 905 | 121K| 62428 (2)| 00:12:30 | | |
| 1 | TEMP TABLE TRANSFORMATION | | | | | | | |
| 2 | LOAD AS SELECT | DW_BO_ORDER | | | | | | |
| 3 | PARTITION RANGE ALL | | 114K| 3228K| 9127 (2)| 00:01:50 | 1 | 5 |
|* 4 | TABLE ACCESS FULL | DW_BO_ORDER | 114K| 3228K| 9127 (2)| 00:01:50 | 1 | 5 |
|* 5 | FILTER | | | | | | | |
|* 6 | HASH JOIN | | 905 | 121K| 12616 (2)| 00:02:32 | | |
|* 7 | HASH JOIN | | 905 | 99550 | 12448 (2)| 00:02:30 | | |
| 8 | PARTITION RANGE ALL | | 1979 | 108K| 9168 (2)| 00:01:51 | 1 | 5 |
|* 9 | TABLE ACCESS FULL | DW_BO_ORDER | 1979 | 108K| 9168 (2)| 00:01:51 | 1 | 5 |
|* 10 | TABLE ACCESS FULL | DW_CRM_DAY_USER | 309K| 15M| 3277 (2)| 00:00:40 | | |
| 11 | TABLE ACCESS FULL | DW_CHANNEL | 48425 | 1276K| 168 (1)| 00:00:03 | | |
|* 12 | FILTER | | | | | | | |
|* 13 | VIEW | | 114K| 6791K| 90 (3)| 00:00:02 | | |
| 14 | TABLE ACCESS FULL | SYS_TEMP_0FD9D662E_D625B872 | 114K| 3228K| 90 (3)| 00:00:02 | | |
--------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
4 - filter(TO_CHAR("STAFF_ID") LIKE '36%')
5 - filter( NOT EXISTS (SELECT /*+ */ 0 FROM (SELECT /*+ CACHE_TEMP_TABLE ("T1") */ "C0" "STAFF_ID","C1"
"PARTY_ID","C2" "BO_ID","C3" "PROD_ID","C4" "COMPLETE_DT" FROM "SYS"."SYS_TEMP_0FD9D662E_D625B872" "T1") "D"
WHERE (:B1='新装' OR :B2='移机' OR :B3='资费变更') AND TO_CHAR("D"."STAFF_ID") LIKE '36%' AND "D"."PARTY_ID"=:B4 AND
"D"."BO_ID"<>:B5 AND "D"."PROD_ID"<>:B6 AND "D"."COMPLETE_DT">:B7-INTERVAL'+07 00:00:00' DAY(2) TO SECOND(0)))
6 - access("A"."CHANNEL_ID"="C"."CHANNEL_ID")
7 - access("A"."PROD_ID"="B"."PROD_ID")
9 - filter("A"."PROD_ID" IS NOT NULL AND ("A"."BO_ACTION_NAME"='新装' OR "A"."BO_ACTION_NAME"='移机' OR
"A"."BO_ACTION_NAME"='资费变更') AND TO_CHAR("A"."SO_STAFF_ID") LIKE '36%')
10 - filter("B"."PROD_SPEC"='(新) 全省_紧密融合型E9套餐产品规格' OR "B"."PROD_SPEC"='ADSL' OR "B"."PROD_SPEC"='E6移动版' OR
"B"."PROD_SPEC"='E8 - 2S' OR "B"."PROD_SPEC"='E9版1M(老版)' OR "B"."PROD_SPEC"='LAN' OR "B"."PROD_SPEC"='普通E9' OR
"B"."PROD_SPEC"='普通电话' OR "B"."PROD_SPEC"='普通新版E8' OR "B"."PROD_SPEC"='全省_紧密融合型E9套餐产品规格' OR "B"."PROD_SPEC"='手机'
OR "B"."PROD_SPEC"='新春欢乐送之E6套餐' OR "B"."PROD_SPEC"='新春欢乐送之E8套餐')
12 - filter(:B1='新装' OR :B2='移机' OR :B3='资费变更')
13 - filter(TO_CHAR("D"."STAFF_ID") LIKE '36%' AND "D"."PARTY_ID"=:B1 AND "D"."BO_ID"<>:B2 AND
"D"."PROD_ID"<>:B3 AND "D"."COMPLETE_DT">:B4-INTERVAL'+07 00:00:00' DAY(2) TO SECOND(0))
统计信息
----------------------------------------------------------
2 recursive calls
29 db block gets
110506 consistent gets
22 physical reads
656 redo size
2438096 bytes sent via SQL*Net to client
449 bytes received via SQL*Net from client
11 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
49245 rows processed

问题解决啦，这次过程中也学到了很多东西。谢谢那位哥们。