Oracle存储过程迁移ODPS-03(专有云):ODPS1.0支持exists语法 ...

专有云目前还有不少ODPS1.0版本,主要是应对V3之前的平台版本。这个版本的ODPS不支持exists语法,如何换种写法实现。
-- ODPS1.0 如何实现 exists 语法
-- oracle
select col1,col2,col3,col4
  from (select t1.col1,t1.col2,t2.col3,t2.col4
          from      table1 t1
         INNER JOIN table2 t2
            ON t1.col1 = t2.col1
         WHERE (t2.col3 between '14400000000' and '14499000000')
           and (NOT EXISTS
                (SELECT 1
                    FROM table3 t3
                   WHERE t1.col2 = t3.col2
                     AND GREATEST(t3.col5,t3.col6,t3.col7) <= sysdate()
                     AND GREATEST(t3.col5,t3.col6,t3.col7) >= sysdate() - interval '1095' day))) foo
 group by col1
 order by col1;
-- odps
select col1,col2,col3,col4
  from (select t1.col1,t1.col2,t2.col3,t2.col4
          from            table1 t1
         inner       join table2 t2
            ON t1.col1 = t2.col1
          left outer join(
               SELECT col2
                 FROM table3 t3
                WHERE GREATEST(t3.col5,t3.col6,t3.col7) <= getdate()
                  AND GREATEST(t3.col5,t3.col6,t3.col7) >= getdate() - 1095) t3
            ON t1.col2 = t3.col2
         WHERE (t2.col3 >= '14400000000' and t2.col3 <='14499000000')
           and t3.col2 is null
) foo
 group by col1
 order by col1;
-- 暮角 15901445705 update at 20181224
exists写法在oracle上其实就是可用,可不用的写法。主要是使用exists语法的查询效率高于JOIN,所以,很多人在使用。
在dataworks上使用scala编写spark任务,获取hologres表数据,数据量有1个亿, 现在报错 2025-08-04 10:04:28,985 ERROR org.apache.spark.ContextCleaner - Error cleaning broadcast 7 org.apache.spark.SparkException: Exception thrown in awaitResult: at org.apache.spark.util.ThreadUtils$.awaitResult(ThreadUtils.scala:205) ~[spark-core_2.11-2.3.0-odps0.30.0.jar:?] at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:75) ~[spark-core_2.11-2.3.0-odps0.30.0.jar:?] at org.apache.spark.storage.BlockManagerMaster.removeBroadcast(BlockManagerMaster.scala:155) ~[spark-core_2.11-2.3.0-odps0.30.0.jar:?] at org.apache.spark.broadcast.TorrentBroadcast$.unpersist(TorrentBroadcast.scala:321) ~[spark-core_2.11-2.3.0-odps0.30.0.jar:?] at org.apache.spark.broadcast.TorrentBroadcastFactory.unbroadcast(TorrentBroadcastFactory.scala:45) ~[spark-core_2.11-2.3.0-odps0.30.0.jar:?] at org.apache.spark.broadcast.BroadcastManager.unbroadcast(BroadcastManager.scala:66) ~[spark-core_2.11-2.3.0-odps0.30.0.jar:?] at org.apache.spark.ContextCleaner.doCleanupBroadcast(ContextCleaner.scala:238) [spark-core_2.11-2.3.0-odps0.30.0.jar:?] at org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1$$anonfun$apply$mcV$sp$1.apply(ContextCleaner.scala:194) [spark-core_2.11-2.3.0-odps0.30.0.jar:?] at org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1$$anonfun$apply$mcV$sp$1.apply(ContextCleaner.scala:185) [spark-core_2.11-2.3.0-odps0.30.0.jar:?] at scala.Option.foreach(Option.scala:257) [scala-library-2.11.8.jar:?] at org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1.apply$mcV$sp(ContextCleaner.scala:185) [spark-core_2.11-2.3.0-odps0.30.0.jar:?] at org.apache.spark.util.Utils$.tryOrStopSparkContext(Utils.scala:1319) [spark-core_2.11-2.3.0-odps0.30.0.jar:?] at org.apache.spark.ContextCleaner.org$apache$spark$ContextCleaner$$keepCleaning(ContextCleaner.scala:178) [spark-core_2.11-2.3.0-odps0.30.0.jar:?] at org.apache.spark.ContextCleaner$$anon$1.run(ContextCleaner.scala:73) [spark-core_2.11-2.3.0-odps0.30.0.jar:?] Caused by: java.io.IOException: Failed to send RPC 5204201379953139886 to /25.91.0.79:52768: java.nio.channels.ClosedChannelException at org.apache.spark.network.client.TransportClient.lambda$sendRpc$2(TransportClient.java:237) ~[spark-network-common_2.11-2.3.0-odps0.30.0.jar:?] at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:507) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:481) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:122) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetFailure(AbstractChannel.java:987) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:869) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1316) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:738) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeWrite(AbstractChannelHandlerContext.java:730) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.channel.AbstractChannelHandlerContext.access$1900(AbstractChannelHandlerContext.java:38) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.write(AbstractChannelHandlerContext.java:1081) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.channel.AbstractChannelHandlerContext$WriteAndFlushTask.write(AbstractChannelHandlerContext.java:1128) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.channel.AbstractChannelHandlerContext$AbstractWriteTask.run(AbstractChannelHandlerContext.java:1070) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:403) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:463) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138) ~[netty-all-4.1.17.Final.jar:4.1.17.Final] at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_65-AliJVM] Caused by: java.nio.channels.ClosedChannelException at io.netty.channel.AbstractChannel$AbstractUnsafe.write(...)(Unknown Source) ~[netty-all-4.1.17.Final.jar:4.1.17.Final]
最新发布
08-05
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值