<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[lazythinker的专栏]]></title><description><![CDATA[]]></description><link>https://blog.youkuaiyun.com/lazythinker</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; lazythinker]]></copyright><item><title><![CDATA[Linux 删除特殊符合的文件]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/81236656</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/81236656</guid><author>lazythinker</author><pubDate>Fri, 27 Jul 2018 14:29:51 +0800</pubDate><description><![CDATA[服务器空间清理的时候，发现了一些 ？、空格等，不知道如何产生的文件。不方便直接删除，可以用如下方法：

1) 使用 ls -i 找到文件的 inode 号如122562389
2) 使用find命令删除 
find ./ -inum 122562389 -exec rm '{}' \;

find ./ -inum 122562389 |xargs rm...]]></description><category></category></item><item><title><![CDATA[Hive 添加第三方包的方式]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/79878311</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/79878311</guid><author>lazythinker</author><pubDate>Tue, 10 Apr 2018 11:25:31 +0800</pubDate><description><![CDATA[1、Add jar    对当前shell有效，简单明了；2、hive-env.sh   HIVE.AUX.JARS.PATH对服务器无效，仅对当前hive shell有效，不同的hive shell相互不影响，每个hive shell都需要配置，可以配置成文件夹形式。    注意：1，根据hive启动脚本，此属性应该也只支持file://也就是本地文件，暂不支持HDFS，没有做验证。2，此配置可...]]></description><category></category></item><item><title><![CDATA[HIVE参数调优（汇总）]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/75497774</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/75497774</guid><author>lazythinker</author><pubDate>Thu, 20 Jul 2017 10:51:04 +0800</pubDate><description><![CDATA[//设置自动mapjoin 为false 
     set hive.auto.convert.join=false; 
     set hive.ignore.mapjoin.hint=false//关闭严格模式 
     set hive.exec.dynamic.partition.mode=nonstrict; 
     set hive.mapred.mode=nonstrict;]]></description><category></category></item><item><title><![CDATA[ElasticSearch批量更新索引的优化]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/60869903</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/60869903</guid><author>lazythinker</author><pubDate>Wed, 08 Mar 2017 14:02:40 +0800</pubDate><description><![CDATA[1. 多线程程序插入可以根据服务器情况开启多个线程index，速度可以提高n倍, n>=2 2. 取消replias如果有多台机器，可以以每台设置n个shards的方式，根据业务情况，可以考虑取消replias  
curl -XPUT ‘http://127.0.0.1:9200/dg-test/’ -d ‘{  
    “settings” : {  
        “number_of_s]]></description><category></category></item><item><title><![CDATA[CentOS下安装MySQLdb]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/60573860</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/60573860</guid><author>lazythinker</author><pubDate>Mon, 06 Mar 2017 11:58:42 +0800</pubDate><description><![CDATA[目的好久不用Python写代码了，最近需要写一个简单的小程序，整理了一下安装MySQLdb遇到的问题。测试是否安装MySQLdb>>> import MySQLdb
ImportError: No module named MySQLdb
原来机器上没有安装这个包；
下载安装MySQLdb在这个页面下载
http://sourceforge.net/projects/mysql-python/fil]]></description><category></category></item><item><title><![CDATA[HIVE 语句执行 中报错：Java heap space]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/52872171</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/52872171</guid><author>lazythinker</author><pubDate>Thu, 20 Oct 2016 14:17:07 +0800</pubDate><description><![CDATA[1、报错信息
ask with the most failures(4):
-----
Task ID:
  task_1472873234260_9322_m_000004


URL:
  http://hadoop-namenode1:8088/taskdetails.jsp?jobid=job_1472873234260_9322&tipid=task_1472873234]]></description><category></category></item><item><title><![CDATA[HiverServer严格模式配置]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/52512140</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/52512140</guid><author>lazythinker</author><pubDate>Mon, 12 Sep 2016 14:26:19 +0800</pubDate><description><![CDATA[因为在系统配置了HiveServer服务开放给需要进行统计的同事，但因为hql的原因还是会给集群造成一些影响。我决定给HiveServer服务增加严格模式：

hiveserver2 start --hiveconf hive.mapred.mode=strict  
具体严格模式下的注意事项，参考 http://blog.youkuaiyun.com/wisgood/article/details/1]]></description><category></category></item><item><title><![CDATA[TOMCAT运行时OOM解决]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/52458981</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/52458981</guid><author>lazythinker</author><pubDate>Wed, 07 Sep 2016 18:31:13 +0800</pubDate><description><![CDATA[统计分析系统因为内部系统，访问量不是太高，一直都是完成功能，没有对TOMCAT的配置关注过，直到最近发生了如下的报错：


Exception in thread "http-bio-8080-Acceptor-0" java.lang.OutOfMemoryError: Java heap space
Exception in thread "ContainerBackgroundPro]]></description><category></category></item><item><title><![CDATA[Hadoop 任务运行中 java.net.ConnectException: to 0.0.0.0:10020 failed]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/52064165</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/52064165</guid><author>lazythinker</author><pubDate>Fri, 29 Jul 2016 13:55:16 +0800</pubDate><description><![CDATA[10020 failed java.lang.OutOfMemoryError GC overhead limit exceeded Java heap space]]></description><category></category></item><item><title><![CDATA[修改NodeManager的内存]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/49337553</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/49337553</guid><author>lazythinker</author><pubDate>Thu, 22 Oct 2015 19:20:34 +0800</pubDate><description><![CDATA[1、查看nodemanager节点状态
bin/yarn node -list
2、停止nodemanager
sbin/yarn-daemon.sh stop nodemanager



3、修改etc/hoadoop/yarn-site.xml 

        
                yarn.nodemanager.resource.memory-mb
]]></description><category></category></item><item><title><![CDATA[hadoop集群操作常用命令]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/47836097</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/47836097</guid><author>lazythinker</author><pubDate>Fri, 21 Aug 2015 16:15:57 +0800</pubDate><description><![CDATA[一、HDFS相关
1、启动NameNode

sbin/hadoop-daemon.sh start namenode
2、启动DataNode
sbin/hadoop-daemon.sh start datanode
二、YARN相关
1、启动Resource Manager
sbin/yarn-daemon.sh start resourcemanager


2、启动N]]></description><category></category></item><item><title><![CDATA[hadoop Unhealthy Nodes问题解决]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/47832117</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/47832117</guid><author>lazythinker</author><pubDate>Fri, 21 Aug 2015 11:58:11 +0800</pubDate><description><![CDATA[1、问题来源
前几天因为一个hive SQL的问题，导致其中一台机器的磁盘空间不足，删除临时文件解决了空间不足的问题；
查看http://hadoop/cluster/nodes/unhealthy 发现出现了一个unhealty节点，错误信息如下
1/1 local-dirs turned bad: /usr/local/goldmine/hadoop/tmp/nm-local-dir;1]]></description><category></category></item><item><title><![CDATA[Java报错&解决汇总]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/40109669</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/40109669</guid><author>lazythinker</author><pubDate>Wed, 15 Oct 2014 13:37:35 +0800</pubDate><description><![CDATA[1、The stack size specified is too small, Specify at least 228k
一个服务应为压力比较大，打算在多部署一台服务器，部署阶段很顺利，到了服务启动阶段jiu's]]></description><category></category></item><item><title><![CDATA[CentOS下mysql的安装]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/36188005</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/36188005</guid><author>lazythinker</author><pubDate>Fri, 04 Jul 2014 14:27:17 +0800</pubDate><description><![CDATA[项目里需要用到mysql数据库，自己下载了最新版的mysql打算安装]]></description><category></category></item><item><title><![CDATA[电影推荐-基于 Apache Mahout 构建社会化推荐引擎]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/25382047</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/25382047</guid><author>lazythinker</author><pubDate>Fri, 09 May 2014 12:07:56 +0800</pubDate><description><![CDATA[最近开始研究推荐系统，]]></description><category></category></item><item><title><![CDATA[Java开源推荐引擎]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/25110961</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/25110961</guid><author>lazythinker</author><pubDate>Tue, 06 May 2014 11:46:44 +0800</pubDate><description><![CDATA[收集整理了目前互联网上能找到的和推荐系统相关的开源项目(Open Source Software | Recommendation)，罗列如下，希望对本领域感兴趣的朋友有帮助（文/陈运文）]]></description><category></category></item><item><title><![CDATA[Maven依赖jar包的查询]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/9180875</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/9180875</guid><author>lazythinker</author><pubDate>Fri, 25 Apr 2014 16:45:08 +0800</pubDate><description><![CDATA[1、查询地址1
       http://search.maven.org/#browse
2、查询地址2
       http://mvnrepository.com/]]></description><category></category></item><item><title><![CDATA[CentOS 安装rz和sz命令]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/23469123</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/23469123</guid><author>lazythinker</author><pubDate>Fri, 11 Apr 2014 19:36:35 +0800</pubDate><description><![CDATA[最近要测试hadoop2.4的ha，在虚机装了Cen]]></description><category></category></item><item><title><![CDATA[Mac下安装maven]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/21399507</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/21399507</guid><author>lazythinker</author><pubDate>Mon, 17 Mar 2014 17:58:52 +0800</pubDate><description><![CDATA[1、下载maven安装包
http://maven.apache.org/download.html
http://apache.fayea.com/apache-mirror/maven/maven-3/3.2.1/binaries/apache-maven-3.2.1-bin.tar.gz
2、解压到目录
tar -zxvf

apache-maven-3.2.1-bin.tar.]]></description><category></category></item><item><title><![CDATA[sudo命令]]></title><link>https://blog.youkuaiyun.com/lazythinker/article/details/21240893</link><guid>https://blog.youkuaiyun.com/lazythinker/article/details/21240893</guid><author>lazythinker</author><pubDate>Fri, 14 Mar 2014 17:45:54 +0800</pubDate><description><![CDATA[1、sudo
Sudo（superuser do）用于类Unix操作系统、BSD、Mac OS X及GNU/Linux，允许使用者透过安全方式使用特殊权限执行程序（通常为系统超级使用者）；


2、Mac系统中使用sudo
a、一般情况不用开启root帐户，除非你需要要su root操作。
b、只有在sudoers文件中被给予权限的用户才能用sudo。
查看/etc/sudoers文]]></description><category></category></item></channel></rss>