<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[cingdy0124的博客]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/cingdy0124</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; cingdy0124]]></copyright><item><title><![CDATA[Hadoop安装]]></title><link>https://blog.youkuaiyun.com/cingdy0124/article/details/142420744</link><guid>https://blog.youkuaiyun.com/cingdy0124/article/details/142420744</guid><author>cingdy0124</author><pubDate>Sun, 22 Sep 2024 20:08:28 +0800</pubDate><description><![CDATA[Hadoop的安装方式有三种，分别是单机模式，伪分布式模式，分布式模式。]]></description><category></category></item><item><title><![CDATA[本地安装Flink]]></title><link>https://blog.youkuaiyun.com/cingdy0124/article/details/133756791</link><guid>https://blog.youkuaiyun.com/cingdy0124/article/details/133756791</guid><author>cingdy0124</author><pubDate>Tue, 10 Oct 2023 21:41:41 +0800</pubDate><description><![CDATA[注：运行Flink需要安装JAVA环境（Java 8 or 11）
可以查看环境是否已安装JAVA：
命令：　java -version　下载Flink安装包
D ownload: https://flink.apache.org/downloads/ 将下载的安装包进行解压：$ tar -xzf flink-1.13.6-bin-scala_2.11.tgz
$ cd flink-1.13.6-bin-scala_2.11$ ./bin/start-cluster.sh
Starting cluster.]]></description><category></category></item><item><title><![CDATA[Spark/Hive性能优化建议]]></title><link>https://blog.youkuaiyun.com/cingdy0124/article/details/132946883</link><guid>https://blog.youkuaiyun.com/cingdy0124/article/details/132946883</guid><author>cingdy0124</author><pubDate>Sun, 17 Sep 2023 17:15:26 +0800</pubDate><description><![CDATA[不管是spark还是hive，不要相信自己的代码，一定要去看执行计划和spark的执行ui界面。踩过的坑…一开始还切过hive计算，发现mapjoin不起效果，直接hint方式没用，还需要设置最小的mapjoin数据大小的参数。同时hive中不同的是，如果mapjoin分发内存溢出，那么会有备用执行stage来代替。而spark走了broadcast hash join失败了就是内存溢出就推出了。]]></description><category></category></item><item><title><![CDATA[Spark安装及配置详细步骤]]></title><link>https://blog.youkuaiyun.com/cingdy0124/article/details/132918717</link><guid>https://blog.youkuaiyun.com/cingdy0124/article/details/132918717</guid><author>cingdy0124</author><pubDate>Sat, 16 Sep 2023 14:49:27 +0800</pubDate><description><![CDATA[解压scala、spark：
     tar -zxvf scala-2.11.8.tgz
     tar -zxvf spark-2.4.4-bin-hadoop2.6.tgz     配置环境变量(master、slave1、slave2分别执行)：
     export SCALA_HOME=/usr/local/src/scala-2.11.8
     export SPARK_HOME=/usr/local/src/spark-2.4.4-bin-hadoop2.6
     export]]></description><category></category></item><item><title><![CDATA[Centos7开启访问端口]]></title><link>https://blog.youkuaiyun.com/cingdy0124/article/details/132570234</link><guid>https://blog.youkuaiyun.com/cingdy0124/article/details/132570234</guid><author>cingdy0124</author><pubDate>Tue, 29 Aug 2023 21:26:23 +0800</pubDate><description><![CDATA[参数：--zone    (作用域)--add-port=80/tcp   (添加端口，格式：端口/通讯协议）--permanent   (永久生效，没有此参数重启后失效）]]></description><category></category></item><item><title><![CDATA[虚拟机centos7系统安装后需手动配置ip地址]]></title><link>https://blog.youkuaiyun.com/cingdy0124/article/details/132504673</link><guid>https://blog.youkuaiyun.com/cingdy0124/article/details/132504673</guid><author>cingdy0124</author><pubDate>Sat, 26 Aug 2023 00:36:50 +0800</pubDate><description><![CDATA[centos 7 ip地址配置]]></description><category></category></item><item><title><![CDATA[Python连接Mysql 1044 Error]]></title><link>https://blog.youkuaiyun.com/cingdy0124/article/details/132402986</link><guid>https://blog.youkuaiyun.com/cingdy0124/article/details/132402986</guid><author>cingdy0124</author><pubDate>Mon, 21 Aug 2023 10:24:13 +0800</pubDate><description><![CDATA[python 连接 mysql 报 1044 错误]]></description><category></category></item><item><title><![CDATA[SPARK软件栈]]></title><link>https://blog.youkuaiyun.com/cingdy0124/article/details/132164374</link><guid>https://blog.youkuaiyun.com/cingdy0124/article/details/132164374</guid><author>cingdy0124</author><pubDate>Tue, 08 Aug 2023 14:00:14 +0800</pubDate><description><![CDATA[是一个用来实现快速而通用的。Spark扩展了广泛你爱我的MapReduce计算模型，而且高效地支持更多计算模式，包括交互式查询和流处理。Spark的一个主要特点就是能够在内存中进行计算，因而更快。即使是必须在磁盘上进行的复杂计算，Spark依然比MapReduce更加高效。Spark提供丰富的接口，除了基于Python、Java、Scala和 SQL的简单易用的API以及内建的丰富的程序库外，Spark还能和其他大数据工具密切配合使用。]]></description><category></category></item><item><title><![CDATA[Hive-安装]]></title><link>https://blog.youkuaiyun.com/cingdy0124/article/details/128022893</link><guid>https://blog.youkuaiyun.com/cingdy0124/article/details/128022893</guid><author>cingdy0124</author><pubDate>Thu, 24 Nov 2022 17:24:25 +0800</pubDate><description><![CDATA[HIVE安装]]></description><category></category></item><item><title><![CDATA[hadoop+spark安装]]></title><link>https://blog.youkuaiyun.com/cingdy0124/article/details/127912896</link><guid>https://blog.youkuaiyun.com/cingdy0124/article/details/127912896</guid><author>cingdy0124</author><pubDate>Thu, 17 Nov 2022 22:03:55 +0800</pubDate><description><![CDATA[JAVA+HADOOP+SPARK安装]]></description><category></category></item><item><title><![CDATA[Spark安装及配置详细步骤]]></title><link>https://blog.youkuaiyun.com/cingdy0124/article/details/127912829</link><guid>https://blog.youkuaiyun.com/cingdy0124/article/details/127912829</guid><author>cingdy0124</author><pubDate>Thu, 17 Nov 2022 21:58:50 +0800</pubDate><description><![CDATA[Spark安装及配置]]></description><category></category></item><item><title><![CDATA[配置-Java环境变量]]></title><link>https://blog.youkuaiyun.com/cingdy0124/article/details/127912265</link><guid>https://blog.youkuaiyun.com/cingdy0124/article/details/127912265</guid><author>cingdy0124</author><pubDate>Thu, 17 Nov 2022 21:30:34 +0800</pubDate><description><![CDATA[Java环境变量配置]]></description><category></category></item></channel></rss>