<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[enoughgood的博客]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/enoughgood</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; enoughgood]]></copyright><item><title><![CDATA[Hive中生成连续的天,小时,分钟的HQL写法]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/123515003</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/123515003</guid><author>enoughgood</author><pubDate>Tue, 15 Mar 2022 23:06:20 +0800</pubDate><description><![CDATA[---生成连续的天
select
date_add(start_date,pos) as dt
from
(
   select 
  '2021-10-01' as start_date
  ,'2021-10-15' as end_date
)t
lateral view posexplode(split(repeat(',',datediff(end_date,start_date)),',')) tab as pos,val
;

---生成连续的小时
select
from_unixtime(t1]]></description><category></category></item><item><title><![CDATA[大数据与数仓的学习重点]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/122918915</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/122918915</guid><author>enoughgood</author><pubDate>Mon, 14 Feb 2022 10:11:38 +0800</pubDate><description><![CDATA[大数据学习重点]]></description><category></category></item><item><title><![CDATA[2022最新大数据面试题之华为]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/122886570</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/122886570</guid><author>enoughgood</author><pubDate>Fri, 11 Feb 2022 18:33:47 +0800</pubDate><description><![CDATA[1.spark内存管理
2.hive分区表中，单值分区和范围分区的区别
3.你们公司执行spark任务时，资源怎么设置的（需要直接说出来）
4.介绍一下kafka水位线（其实就是leo和Hw）
5.说几个指标，分别从什么数据层拿取了数据，需要直接说出来
6.数仓采用了什么模型？为什么？
7.hive分区表，单值分区和范围分区的区别
8.spark任务切分，怎么判断有没有执行shuffle
9.你们公司拉链表都有什么字段，拉链表出错怎么办
10.列举几张表的同步策略
11.flink Sql 了解吗

...]]></description><category></category></item><item><title><![CDATA[2022最新大数据面试题之米哈游]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/122886414</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/122886414</guid><author>enoughgood</author><pubDate>Fri, 11 Feb 2022 18:26:00 +0800</pubDate><description><![CDATA[都是从各个地方搜集的实际面试真题.]]></description><category></category></item><item><title><![CDATA[Maven项目中引入Guava依赖]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/121360064</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/121360064</guid><author>enoughgood</author><pubDate>Tue, 16 Nov 2021 16:56:22 +0800</pubDate><description><![CDATA[&lt;!-- https://mvnrepository.com/artifact/com.google.guava/guava --&gt;
        &lt;dependency&gt;
            &lt;groupId&gt;com.google.guava&lt;/groupId&gt;
            &lt;artifactId&gt;guava&lt;/artifactId&gt;
            &lt;version&gt;22.0&l...]]></description><category></category></item><item><title><![CDATA[Hive实现获取指定月的第一天和最后一天]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/121226810</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/121226810</guid><author>enoughgood</author><pubDate>Tue, 09 Nov 2021 14:07:54 +0800</pubDate><description><![CDATA[--指定月的第一天
select trunc('2020-11-10','MM');
2020-11-01
--指定月的最后一天
select last_day('2020-11-10');
2020-11-30


]]></description><category></category></item><item><title><![CDATA[在Hive中生成连续的分钟]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/120543857</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/120543857</guid><author>enoughgood</author><pubDate>Wed, 29 Sep 2021 10:37:19 +0800</pubDate><description><![CDATA[---生成连续的分钟
select
from_unixtime(t1.start_time+tab.pos*60) as continue_time
from
(
    select
    unix_timestamp('2020-07-27 00:00:00') as start_time   ---生成连续分钟的开始时间
		,unix_timestamp('2020-07-27 00:15:00') as end_time   ---生成连续分钟的结束时间
)t1
lateral view pos]]></description><category></category></item><item><title><![CDATA[源码编译Kylin(详细版)]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/120543681</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/120543681</guid><author>enoughgood</author><pubDate>Wed, 29 Sep 2021 10:34:42 +0800</pubDate><description><![CDATA[1. 在阿里云ECS上源码编译Kylin
1.1 编译前准备
  在Kylin源码编译过程中有不少坑,报各种错.为了减少报错信息,需要提前对编译环境做一些准备.

安装Maven(版本在3.5.3及以上)

#1.解压安装包
[root@linux122 software]# tar -zvxf apache-maven-3.6.3-bin.tar.gz -C /opt/servers
#2.配置maven环境变量
[root@linux122 software]# vim /etc/profile
#MA]]></description><category></category></item><item><title><![CDATA[jdk-11.0.9_linux-x64_bin.tar.gz免费下载]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/112078991</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/112078991</guid><author>enoughgood</author><pubDate>Fri, 01 Jan 2021 23:22:00 +0800</pubDate><description><![CDATA[这是从Oracle官方下载的,已在生产使用.免费下载地址:
https://pan.baidu.com/s/1gJw4jmauxoYsHrMtvVyJPw
提取码：0k3e

]]></description><category></category></item><item><title><![CDATA[Linux下查看某个文件夹下的所有文件个数]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/109776107</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/109776107</guid><author>enoughgood</author><pubDate>Wed, 18 Nov 2020 16:06:07 +0800</pubDate><description><![CDATA[ll | grep "^-" | wc -l


]]></description><category></category></item><item><title><![CDATA[在Hive中根据起止日期生成连续的日期]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/109775878</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/109775878</guid><author>enoughgood</author><pubDate>Wed, 18 Nov 2020 16:00:51 +0800</pubDate><description><![CDATA[格式:
with dates as(
    select date_add(start_date, a.pos) as d
    from (select posexplode(split(repeat("m", datediff(end_date, start_date)), "m"))) a
)
select * from dates;

使用方式: 将start_date和end_date替换为自己的日期即可.
案例:
with dates as(
    select date_add("202]]></description><category></category></item><item><title><![CDATA[Apache所有组件下载地址(包括官方地址和国内清华镜像)]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/109405845</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/109405845</guid><author>enoughgood</author><pubDate>Sat, 31 Oct 2020 17:49:15 +0800</pubDate><description><![CDATA[点击链接

]]></description><category></category></item><item><title><![CDATA[CentOS7.x国内镜像下载地址]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/109384796</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/109384796</guid><author>enoughgood</author><pubDate>Fri, 30 Oct 2020 14:36:04 +0800</pubDate><description><![CDATA[清华大学开源镜像

]]></description><category></category></item><item><title><![CDATA[DataGrip所有版本下载地址]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/109317737</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/109317737</guid><author>enoughgood</author><pubDate>Tue, 27 Oct 2020 18:33:02 +0800</pubDate><description><![CDATA[下载地址

]]></description><category></category></item><item><title><![CDATA[将Hive的执行引擎更换为Tez]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/109313914</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/109313914</guid><author>enoughgood</author><pubDate>Tue, 27 Oct 2020 16:50:28 +0800</pubDate><description><![CDATA[将Hive的执行引擎更换为Tez
1.1 集群环境
CentOS7.x
JDK1.8
hadoop2.7.2
hive2.3.7
tez-0.9.2
hiveserver2,metastore和tez都安装在hadoop102上
1.2 操作步骤
1.2.1 下载并解压Tez安装包

下载地址

https://mirrors.bfsu.edu.cn/apache/tez/0.9.2/


解压

#HiveServer2在哪个节点就解压到哪个节点
[root@hadoop102 software]$ ta]]></description><category></category></item><item><title><![CDATA[原生Hadoop环境下安装Hue]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/108434349</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/108434349</guid><author>enoughgood</author><pubDate>Sun, 06 Sep 2020 18:27:31 +0800</pubDate><description><![CDATA[1. 写在前面的话
  从事大数据开发已经有几年了,最近工作遇到瓶颈,时下火热的技术用的不多,想再提升提升.经过对方对比,详细了解了拉勾的课程体系,觉得是比较适合自己的,所以最后选择了拉勾教育大数据高薪训练营.现对学习内容以及结合自身的工作经验做一些总结.
2. Hue的编译安装
2.1 hue相关的网址
Hue官方网站
Hue官方用户手册
官方安装文档
HUE下载地址
2.2 安装说明
  由于官方没有提供编译好的软件包,所以这里需要自己下载源码,安装依赖,编译安装.安装Hue的节点上最好没有安装过MyS]]></description><category></category></item><item><title><![CDATA[HiveSQL常用优化方法经验总结]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/108431757</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/108431757</guid><author>enoughgood</author><pubDate>Sun, 06 Sep 2020 17:43:19 +0800</pubDate><description><![CDATA[1. 写在前面的话
从事大数据开发已经有几年了,最近工作遇到瓶颈,时下火热的技术用的不多,想再提升提升.经过对方对比,详细了解了拉勾的课程体系,觉得是比较适合自己的,所以最后选择了拉勾教育大数据高薪训练营.现对学习内容以及结合自身的工作经验做一些总结,也算是开启自己的博客之路.
2. Hive中解决数据倾斜的场景
2.1 大表Join小表时的数据倾斜(map join)
  在大表Join小表时,解决数据倾斜最好的方式是使用Map Join,避免Shuffle,从而也避免了数据倾斜. map join主要通]]></description><category></category></item><item><title><![CDATA[阿里云ECS服务器下安装MySQL]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/108431233</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/108431233</guid><author>enoughgood</author><pubDate>Sun, 06 Sep 2020 15:03:57 +0800</pubDate><description><![CDATA[1. 写在前面的话
  从事大数据开发已经有几年了,最近工作遇到瓶颈,时下火热的技术用的不多,想再提升提升.经过对方对比,详细了解了拉勾的课程体系,觉得是比较适合自己的,所以最后选择了拉勾教育大数据高薪训练营.现对学习内容以及结合自身的工作经验做一些总结,也算是开启自己的博客之路.
2. 安装MySQL的前置条件
2.1 安装环境说明
  这里安装选用的Linux系统的版本为CentOS7.5,MySQL的版本为MySQL5.7.26,在阿里云ECS服务器上进行安装.
2.2 删除CentOS7.x自带的M]]></description><category></category></item><item><title><![CDATA[原生Hadoop环境下Impala的安装和使用]]></title><link>https://blog.youkuaiyun.com/enoughgood/article/details/108257488</link><guid>https://blog.youkuaiyun.com/enoughgood/article/details/108257488</guid><author>enoughgood</author><pubDate>Sun, 06 Sep 2020 14:17:48 +0800</pubDate><description><![CDATA[1. Impala的基本概念
1.1 什么是Impala
  Impala就是使用SQL语句来操作Hive中的数据库和表,它可以提供低延迟的交互式的SQL查询功能.它与Hive共用表的元数据信息,所以需要使用Impala必须要先有Hive.
1.2 Impala的优缺点以及使用场景
优点:

基于内存计算,不需要把中间结果写入磁盘,省掉了大量的I/O开销.
无需将SQL转换为MapReduce任务,可以直接访问存储在HDFS,HBASE中的数据进行作业调度,可以提高查询速度.
使用了支持数据本地化的I/O调]]></description><category></category></item></channel></rss>