https://blog.youkuaiyun.com/sysocczh-cnhttps://blog.youkuaiyun.com/https://blog.youkuaiyun.com/sysocc/article/details/131073763https://blog.youkuaiyun.com/sysocc/article/details/131073763sysoccWed, 07 Jun 2023 09:53:27 +0800https://blog.youkuaiyun.com/sysocc/article/details/130801146https://blog.youkuaiyun.com/sysocc/article/details/130801146sysoccMon, 22 May 2023 09:59:39 +0800https://blog.youkuaiyun.com/sysocc/article/details/130772015https://blog.youkuaiyun.com/sysocc/article/details/130772015sysoccFri, 19 May 2023 18:27:41 +0800https://blog.youkuaiyun.com/sysocc/article/details/130771938https://blog.youkuaiyun.com/sysocc/article/details/130771938sysoccFri, 19 May 2023 18:12:18 +0800https://blog.youkuaiyun.com/sysocc/article/details/130573806https://blog.youkuaiyun.com/sysocc/article/details/130573806sysoccTue, 09 May 2023 10:18:02 +0800https://blog.youkuaiyun.com/sysocc/article/details/130573233https://blog.youkuaiyun.com/sysocc/article/details/130573233sysoccTue, 09 May 2023 09:42:04 +0800https://blog.youkuaiyun.com/sysocc/article/details/130533019https://blog.youkuaiyun.com/sysocc/article/details/130533019sysoccSat, 06 May 2023 18:12:43 +0800https://blog.youkuaiyun.com/sysocc/article/details/130532839https://blog.youkuaiyun.com/sysocc/article/details/130532839sysoccSat, 06 May 2023 18:02:49 +0800https://blog.youkuaiyun.com/sysocc/article/details/129014879https://blog.youkuaiyun.com/sysocc/article/details/129014879sysoccMon, 13 Feb 2023 18:50:55 +0800https://blog.youkuaiyun.com/sysocc/article/details/129014772https://blog.youkuaiyun.com/sysocc/article/details/129014772sysoccMon, 13 Feb 2023 18:33:52 +0800https://blog.youkuaiyun.com/sysocc/article/details/128982065https://blog.youkuaiyun.com/sysocc/article/details/128982065sysoccSun, 12 Feb 2023 11:49:54 +0800 Shuffle Hash Join --> Sort Merge Join ---> Cartesian Product Join --> Broadcast Nested Loop Join顺序选择Join策略。当然用户也可以手动选择策。1、Shuffle阶段：对两个表参与Join的keys使用相同的分区算法和分区数进行分区，目的就是为了保障相同的key都分配到相同的分区里面。]]>https://blog.youkuaiyun.com/sysocc/article/details/128981306https://blog.youkuaiyun.com/sysocc/article/details/128981306sysoccSun, 12 Feb 2023 11:49:48 +0800https://blog.youkuaiyun.com/sysocc/article/details/128981283https://blog.youkuaiyun.com/sysocc/article/details/128981283sysoccSat, 11 Feb 2023 10:54:28 +0800https://blog.youkuaiyun.com/sysocc/article/details/128981248https://blog.youkuaiyun.com/sysocc/article/details/128981248sysoccSat, 11 Feb 2023 10:52:31 +0800https://blog.youkuaiyun.com/sysocc/article/details/128948392https://blog.youkuaiyun.com/sysocc/article/details/128948392sysoccFri, 10 Feb 2023 10:38:48 +0800https://blog.youkuaiyun.com/sysocc/article/details/128947555https://blog.youkuaiyun.com/sysocc/article/details/128947555sysoccFri, 10 Feb 2023 10:38:16 +0800https://blog.youkuaiyun.com/sysocc/article/details/128947465https://blog.youkuaiyun.com/sysocc/article/details/128947465sysoccThu, 09 Feb 2023 09:48:43 +0800https://blog.youkuaiyun.com/sysocc/article/details/128947344https://blog.youkuaiyun.com/sysocc/article/details/128947344sysoccThu, 09 Feb 2023 09:45:08 +0800https://blog.youkuaiyun.com/sysocc/article/details/128923760https://blog.youkuaiyun.com/sysocc/article/details/128923760sysoccWed, 08 Feb 2023 09:35:13 +0800https://blog.youkuaiyun.com/sysocc/article/details/128923727https://blog.youkuaiyun.com/sysocc/article/details/128923727sysoccWed, 08 Feb 2023 09:35:01 +0800目标分区数T,且是一种极端的合并，比如从1000个分区数进行重新分配到几个分区，那么这个时候可能会造成运行节点IO异常甚至OOM，所以为了避免这种情况发生，可以设置shuffle=true。RDD每次转换都会生成新的RDD，所以RDD之间会形成依赖关系，对于这种关系，RDD会进行存储，这样当发生错误的时候可以从上一个RDD中进行恢复，这样保证了容错，对于RDD的容错又分为了几种情况。分区器的使用只针对于KV类型的RDD，也就是说对于非KV类型的RDD没有具体的分区器。]]>