<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[你懂了我的冬天的博客]]></title><description><![CDATA[学习学习学习]]></description><link>https://blog.youkuaiyun.com/u012071918</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; u012071918]]></copyright><item><title><![CDATA[Hive 运行流程总结包含优化]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/114023645</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/114023645</guid><author>u012071918</author><pubDate>Wed, 24 Feb 2021 15:23:07 +0800</pubDate><description><![CDATA[关键词：hive处理小文件，数据倾斜，mr程序流程，SQL解析步骤，hive优化，hive面试，hive控制map数，hive控制reduce数
由于上传图片大小限制，思维导图未展开，请移步展开


]]></description><category></category></item><item><title><![CDATA[Dataflow模型]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/113796746</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/113796746</guid><author>u012071918</author><pubDate>Fri, 12 Feb 2021 20:26:31 +0800</pubDate><description><![CDATA[Dataflow模型
查看交互版本的思维导图请点击交互版连接


]]></description><category></category></item><item><title><![CDATA[特征工程]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/113784888</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/113784888</guid><author>u012071918</author><pubDate>Wed, 10 Feb 2021 19:25:04 +0800</pubDate><description><![CDATA[特征工程
1.特征提取 :计数向量器(Countvectorizer) 、词频-逆向文件频率(TF-IDF)、词转向量(Word2Vec)
2.特征转换 :连续型数据转换成离散数据、字符串和索引相互转换、正则化(Normalizer)、规范化(StandardScaler)、主成分分析 (PCA) 、向量-索引变换(VectorIndexer)、SQL转换器(SQLTransformer)        独热编码(OneHotEncoder)、最大值-最小值缩放(MinMaxScaler)特征向量合并(Ve]]></description><category></category></item><item><title><![CDATA[kafka 知识总结，以及问题总结]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/113780839</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/113780839</guid><author>u012071918</author><pubDate>Wed, 10 Feb 2021 11:56:37 +0800</pubDate><description><![CDATA[kafka 知识总结，问题总结kafka 知识总结，以及问题总结
kafka 知识总结，以及问题总结
kafka总结：消息队列，kafka架构，组件作用，为什么比磁盘快，怎么保证数据不丢失，如何手动设置offset，如何保证kafka消费顺序，kafka事务什么，搭建kafka集群等，内容持续更新。
由于图片大小设置，思维导图未展开，详细请进入链接：详细思维导图


...]]></description><category></category></item><item><title><![CDATA[ETL 血的教训]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/80907830</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/80907830</guid><author>u012071918</author><pubDate>Wed, 04 Jul 2018 09:52:06 +0800</pubDate><description><![CDATA[1.任何 drop 都要加if 判断是否存在（一旦作业中途终止，重新调起的时候会报错） 
2.两表join 先过滤再join]]></description><category></category></item><item><title><![CDATA[大数据项目-----网站点击流]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/80897607</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/80897607</guid><author>u012071918</author><pubDate>Tue, 03 Jul 2018 14:08:05 +0800</pubDate><description><![CDATA[网站点击流数据分析项目

：https://blog.youkuaiyun.com/u014033218/article/details/76847263]]></description><category></category></item><item><title><![CDATA[数据字典生成sql]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/80758292</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/80758292</guid><author>u012071918</author><pubDate>Thu, 21 Jun 2018 11:47:37 +0800</pubDate><description><![CDATA[SELECT  
    表名       = case when a.colorder=1 then d.name else ” end, 
    表说明     = case when a.colorder=1 then isnull(f.value,”) else ” end, 
    字段序号   = a.colorder, 
    字段名     = a.name, 
    标识...]]></description><category></category></item><item><title><![CDATA[SCD 缓慢变化维 按月保存 lag lead 实现]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/80735313</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/80735313</guid><author>u012071918</author><pubDate>Tue, 19 Jun 2018 16:06:59 +0800</pubDate><description><![CDATA[需求 
将上表变为下表


*year_month nestle_outlet_code  UserCode*
201801  I00002N15052359710625   TSRB41CGAB013
201802  I00002N15052359710625   TSRB41CGAB013
201803  I00002N15052359710625   TSRB41CGAB013
201804...]]></description><category></category></item><item><title><![CDATA[SQL Server的Merge —— 一步实现 insert,update,delete]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/80690817</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/80690817</guid><author>u012071918</author><pubDate>Thu, 14 Jun 2018 11:50:16 +0800</pubDate><description><![CDATA[USE tempdb
GO
IF OBJECT_ID('SourceTable') IS NOT NULL
DROP TABLE SourceTable
IF OBJECT_ID('TargetTable') IS NOT NULL
DROP TABLE TargetTable

--源
CREATE TABLE SourceTable(id INT,[desc] NVARCHAR(50))
--...]]></description><category></category></item><item><title><![CDATA[完全关闭及再次启动cdh集群]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/80599928</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/80599928</guid><author>u012071918</author><pubDate>Wed, 06 Jun 2018 19:40:07 +0800</pubDate><description><![CDATA[关闭集群 
-　关闭集群所有组件 


关闭Cloudera Management Service 
 
关闭cdh所有客户端节点

sudo /opt/cloudera-manager/cm-5.11.1/etc/init.d/cloudera-scm-agent hard_stop_confirmed 
使用stop不会停止pid相关的进程，会造成在客户端机器重启后再次启动cloudera-s...]]></description><category></category></item><item><title><![CDATA[merger  实现缓慢变化维带时间戳type2的方式]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/80568457</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/80568457</guid><author>u012071918</author><pubDate>Mon, 04 Jun 2018 15:55:00 +0800</pubDate><description><![CDATA[原文地址：http://www.cnblogs.com/biwork/p/3371338.html



USE BIWORK_SSIS
GO

IF OBJECT_ID('Customer') IS NOT NULL
DROP TABLE Customer 
GO

IF OBJECT_ID('DimCustomer') IS NOT NULL
DROP TABLE DimCustomer 
G...]]></description><category></category></item><item><title><![CDATA[Flume(NG)架构设计要点及配置实践]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/80309907</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/80309907</guid><author>u012071918</author><pubDate>Mon, 14 May 2018 15:09:20 +0800</pubDate><description><![CDATA[看到一个特别好的博客，对于flume有较深的理解，现转载如下：

Flume(NG)架构设计要点及配置实践 
原文链接 ： http://shiyanjun.cn/archives/915.html

Flume日志收集分层架构应用实践

原文链接： http://shiyanjun.cn/archives/category/opensource/flume...]]></description><category></category></item><item><title><![CDATA[sql server 性能优化日志]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/80248495</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/80248495</guid><author>u012071918</author><pubDate>Wed, 09 May 2018 08:56:43 +0800</pubDate><description><![CDATA[sql server 性能优化日志


尽量减少行及操作，例如：在逻辑可以替代的情况下使用group by 而非 over函数
当某个表数据是缓慢流入，建议 加 时间字段上加聚集索引
当操作读取大表，且此大表全量更新，建议先drop table  ，  然后 select into  ，最后加索引 
不使用 select  *   
...]]></description><category></category></item><item><title><![CDATA[Python中threading的join和setDaemon的区别及用法［例子］]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/78924915</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/78924915</guid><author>u012071918</author><pubDate>Thu, 28 Dec 2017 18:02:03 +0800</pubDate><description><![CDATA[Python多线程编程时，经常会用到join()和setDaemon()方法，今天特地研究了一下两者的区别。







1、join ()方法：主线程A中，创建了子线程B，并且在主线程A中调用了B.join()，那么，主线程A会在调用的地方等待，直到子线程B完成操作后，才可以接着往下执行，那么在调用这个线程时可以使用被调用线程的join方法。




原型：join(]]></description><category></category></item><item><title><![CDATA[javascript来实现无缝文字的滚屏]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/78874761</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/78874761</guid><author>u012071918</author><pubDate>Fri, 22 Dec 2017 16:38:27 +0800</pubDate><description><![CDATA[js文字向左无缝滚动



.qimo8{ overflow:hidden; width:815px;}
.qimo8 .qimo {/*width:99999999px;*/width:8000%; height:30px;}
.qimo8 .qimo div{ float:left;}
.qimo8 .qimo ul{float:left; height:30px; overfl]]></description><category></category></item><item><title><![CDATA[牛牛的Gate One的安装指南]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/78843490</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/78843490</guid><author>u012071918</author><pubDate>Tue, 19 Dec 2017 15:45:24 +0800</pubDate><description><![CDATA[原文地址：http://www.cnblogs.com/lienhua34/p/4876300.html

LinuxMint系统下Gate One的安装指南


1. Gate One简介
前面有两个随笔介绍过开源软件tty.js和wetty在Linux的安装。Tty.js和wetty都是采用Node.js实现的开源Web-based ssh。今天来介绍另一个开源Web-based ]]></description><category></category></item><item><title><![CDATA[安装Anaconda 在Linux上]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/78825015</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/78825015</guid><author>u012071918</author><pubDate>Sun, 17 Dec 2017 14:24:55 +0800</pubDate><description><![CDATA[How To Install Anaconda / Miniconda / Conda on Linux (Ubuntu, CentOS, Fedora)

June
 15, 2016 by Aun


Anaconda is an enterprise-ready platform for data science analytics. It works on all popula]]></description><category></category></item><item><title><![CDATA[Linux python 重装]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/78817373</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/78817373</guid><author>u012071918</author><pubDate>Fri, 15 Dec 2017 22:38:29 +0800</pubDate><description><![CDATA[这个人写的不错



公司测试机环境不知道给我卸了什么包，导致yum运行报错状况：

系统版本：Red Hat Enterprise Linux Server release 6.2 (Santiago)

内核版本：2.6.32-220.el6.x86_64




报错情况：

There was a problem importing one of the Py]]></description><category></category></item><item><title><![CDATA[完美解决 Linux安装python2.7 方案]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/78817344</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/78817344</guid><author>u012071918</author><pubDate>Fri, 15 Dec 2017 22:32:50 +0800</pubDate><description><![CDATA[Linux下安装python 2.7
原文地址：http://www.jianshu.com/p/6425d18d3e47

安装依赖的库

yum -y install python-devel openssl openssl-devel gcc sqlite sqlite-devel mysql-devel libxml2-devel libxslt-devel


Python]]></description><category></category></item><item><title><![CDATA[centos下安装pip时失败]]></title><link>https://blog.youkuaiyun.com/u012071918/article/details/78817220</link><guid>https://blog.youkuaiyun.com/u012071918/article/details/78817220</guid><author>u012071918</author><pubDate>Fri, 15 Dec 2017 22:10:25 +0800</pubDate><description><![CDATA[centos下安装pip时失败：

[root@wfm ~]# yum -y install pip
Loaded plugins: fastestmirror, refresh-packagekit, security
Loading mirror speeds from cached hostfile
* base: mirrors.tuna.tsinghua.edu.cn
* e]]></description><category></category></item></channel></rss>