<?xml version="1.0" encoding="utf-8" ?><rss version="2.0"><channel><title><![CDATA[George的博客]]></title><description><![CDATA[学而不思则罔，思而不学则殆]]></description><link>https://blog.youkuaiyun.com/yo8237233</link><language>zh-cn</language><generator>https://blog.youkuaiyun.com/</generator><copyright><![CDATA[Copyright &copy; yo8237233]]></copyright><item><title><![CDATA[YARN中自己总结的几个关键点]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84830523</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84830523</guid><author>yo8237233</author><pubDate>Sun, 11 Dec 2016 15:43:43 +0800</pubDate><description><![CDATA[以前在Hadoop 1.0中JobTracker主要完成两项功能：资源的管理和作业控制。在集群规模过大的场景下，JobTracker
存在以下不足：
1）JobTracker 单点故障。
2）JobTracker 承受的访问压力大，影响系统的扩展性。
3）不支持MapReduce之外的计算框架，比如Storm、Spa rk、Flink

因此在YARN的设计中，资源的管理和作业控制是分...]]></description><category></category></item><item><title><![CDATA[将博客搬至cnblogs]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84830522</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84830522</guid><author>yo8237233</author><pubDate>Sun, 11 Dec 2016 15:36:29 +0800</pubDate><description><![CDATA[地址：http://www.cnblogs.com/qiaoyihang/]]></description><category></category></item><item><title><![CDATA[Flume应用场景及架构原理]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84830484</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84830484</guid><author>yo8237233</author><pubDate>Sat, 10 Dec 2016 18:53:58 +0800</pubDate><description><![CDATA[Flume概念

Flume是一个分布式、可靠、和高可用的海量日志聚合的系统，支持在系统中定制[size=large]各类数据发送方[/size]，用于收集数据；同时，Flume提供对数据进行简单处理，并写到[size=large]各种数据接受方[/size]（可定制）的能力。

flume 特点：



1、可靠性
当节点出现故障时，日志能够被传送到其他节点上而不会丢失。Flu...]]></description><category></category></item><item><title><![CDATA[遍历Map的四种方法]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84830359</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84830359</guid><author>yo8237233</author><pubDate>Fri, 09 Dec 2016 15:38:29 +0800</pubDate><description><![CDATA[[code=&quot;java&quot;]
public static void main(String[] args) {


  Map map = new HashMap();
  map.put(&quot;1&quot;, &quot;value1&quot;);
  map.put(&quot;2&quot;, &quot;value2&quot;);
  map.put(&quot;3&quot;, &quot;value3&quot;);
  
  //第一种：普遍使用]]></description><category></category></item><item><title><![CDATA[zookeeper集群某个follower启动失败]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84830006</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84830006</guid><author>yo8237233</author><pubDate>Wed, 07 Dec 2016 20:28:23 +0800</pubDate><description><![CDATA[配置完成zookeeper集群，发现有一个节点，进程正常但是状态异常

[img]http://dl2.iteye.com/upload/attachment/0121/8256/5d31e28b-19b6-3638-aa19-337d43671f1a.jpg[/img]


查看日志一开始进入歧途了，查看的是这个目录

[img]http://dl2.iteye.com/uploa...]]></description><category></category></item><item><title><![CDATA[HDFS 和YARN HA 简介]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84829942</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84829942</guid><author>yo8237233</author><pubDate>Wed, 07 Dec 2016 13:34:14 +0800</pubDate><description><![CDATA[HDFS：
[img]http://dl2.iteye.com/upload/attachment/0121/8139/4bc1aec8-749d-35b2-87b4-83a9572d1c58.png[/img]

基础架构
1、NameNode（Master）

1)命名空间管理：命名空间支持对HDFS中的目录、文件和块做类似文件系统的创建、修改、删除、列表文件和目录等基本操作。
...]]></description><category></category></item><item><title><![CDATA[cdh集群数据恢复]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84829209</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84829209</guid><author>yo8237233</author><pubDate>Thu, 01 Dec 2016 22:03:23 +0800</pubDate><description><![CDATA[CDH 数据库 磁盘坏了  所有集群配置 都没了    而且 还没备份  ....
   
元数据 还在  cdh  软件配置 和  安装软件 不能用了

下载 apache hadoop   重新配置  namenode datenode   journode  加载以前的数据  读出来了  
..
强制了  namenode 主   把数据迁移到 其他节点   

因为 重新 ...]]></description><category></category></item><item><title><![CDATA[mapreduce流程中的几个关键点]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84826296</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84826296</guid><author>yo8237233</author><pubDate>Sat, 12 Nov 2016 20:46:04 +0800</pubDate><description><![CDATA[MapReduce中数据流动
   （1）最简单的过程：  map - reduce
   （2）定制了partitioner以将map的结果送往指定reducer的过程：　map - partition - reduce
   （3）增加了在本地先进性一次reduce（优化）过程：　map - combin(本地reduce) - partition -reduce

map函数的处理...]]></description><category></category></item><item><title><![CDATA[MapReduce二次排序]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84825337</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84825337</guid><author>yo8237233</author><pubDate>Mon, 07 Nov 2016 14:51:05 +0800</pubDate><description><![CDATA[     默认情况下，Map 输出的结果会对 Key 进行默认的排序，但是有时候需要对 Key 排序的同时再对 Value 进行排序，这时候就要用到二次排序了。下面让我们来介绍一下什么是二次排序。

二次排序原理
        我们把二次排序主要分为以下几个阶段。
Map 起始阶段
        在Map阶段，使用 job.setInputFormatClass() 定义的 Inpu...]]></description><category></category></item><item><title><![CDATA[Hadoop自定义分组Group]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84825305</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84825305</guid><author>yo8237233</author><pubDate>Mon, 07 Nov 2016 12:35:48 +0800</pubDate><description><![CDATA[matadata：
[code=&quot;java&quot;]
hadoop  a  
spark   a  
hive    a  
hbase   a  
tachyon a  
storm   a  
redis   a  
[/code]

自定义分组
[code=&quot;java&quot;]
import org.apache.hadoop.conf.Configuration;
impo...]]></description><category></category></item><item><title><![CDATA[编译hadoop2.6.0]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84824000</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84824000</guid><author>yo8237233</author><pubDate>Sat, 29 Oct 2016 23:24:00 +0800</pubDate><description><![CDATA[具体情况比较曲折：hadoop2.6.0编译不过
错误如下：
[img]http://dl2.iteye.com/upload/attachment/0120/9044/7f4f4d5e-4cc6-375a-8820-57bfb914e901.png[/img]


这个kms模块始终编译不过，最后得出结论国内的aliyun maven仓库有问题，

在编译hadoop2.2.0 可...]]></description><category></category></item><item><title><![CDATA[ERROR [org.apache.hadoop.security.UserGroupInformation] - PriviledgedActionExcep]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84822290</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84822290</guid><author>yo8237233</author><pubDate>Thu, 20 Oct 2016 10:23:33 +0800</pubDate><description><![CDATA[换了个环境，出现此异常
016-10-18 23:54:01,334 WARN [org.apache.hadoop.util.NativeCodeLoader] - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2016-10-18 2...]]></description><category></category></item><item><title><![CDATA[Java集合分组]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84821754</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84821754</guid><author>yo8237233</author><pubDate>Mon, 17 Oct 2016 16:25:25 +0800</pubDate><description><![CDATA[[code=&quot;java&quot;]
public class Data {

	private Long id ;
	private Long courseId ;
	private String content ;

	public Long getId() {
		return id;
	}

	public Data setId(Long id) {
		this.id = ...]]></description><category></category></item><item><title><![CDATA[Mysql  hql字符串字段中是否包含某个字符串，用 find_in_set]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84819201</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84819201</guid><author>yo8237233</author><pubDate>Tue, 27 Sep 2016 17:25:05 +0800</pubDate><description><![CDATA[有这样一个需求，在Mysql数据库字符串字段（权限）中，有范围在 1 到 N 之间代表不同权限的值，分别被‘,’分开，现在要取出具有某权限的所有成员列表。

创建表：

[code=&quot;sql&quot;]
1	CREATE TABLE users(id int(6) NOT NULL AUTO_INCREMENT,PRIMARY KEY (id),name VARCHAR(20) NOT NULL...]]></description><category></category></item><item><title><![CDATA[org.apache.hadoop.ipc.RemoteException(java.io.IOException)]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84818475</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84818475</guid><author>yo8237233</author><pubDate>Thu, 22 Sep 2016 15:38:28 +0800</pubDate><description><![CDATA[昨晚突然之间mr跑步起来了
jps查看 进程都在的，但是在reduce任务跑了85%的时候会抛异常
异常情况如下：

[code=&quot;java&quot;]
2016-09-21 21:32:28,538 INFO [org.apache.hadoop.mapreduce.Job] -  map 100% reduce 84%
2016-09-21 21:32:30,623 INFO [org.a...]]></description><category></category></item><item><title><![CDATA[jenkins打包成功，部署失败]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84815472</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84815472</guid><author>yo8237233</author><pubDate>Thu, 01 Sep 2016 17:41:34 +0800</pubDate><description><![CDATA[环境一直正常，更新了tomcat版本后自动部署报错
[code=&quot;java&quot;]
ERROR: Publisher hudson.plugins.deploy.DeployPublisher aborted due to exception
org.codehaus.cargo.container.ContainerException: Failed to deploy [/var/lib/j...]]></description><category></category></item><item><title><![CDATA[druid连接池异常]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84815243</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84815243</guid><author>yo8237233</author><pubDate>Wed, 31 Aug 2016 14:24:55 +0800</pubDate><description><![CDATA[[size=large]
在从excel导入10W条数据到mysql中时，运行一段时间就会抛这个异常，连接池问题
[/size]
[code=&quot;java&quot;]
org.springframework.transaction.CannotCreateTransactionException: Could not open Hibernate Session for transaction; n...]]></description><category></category></item><item><title><![CDATA[Lock wait timeout exceeded; try restarting transaction ]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84814551</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84814551</guid><author>yo8237233</author><pubDate>Wed, 24 Aug 2016 15:36:18 +0800</pubDate><description><![CDATA[查看innodb的事务表INNODB_TRX，看下里面是否有正在锁定的事务线程，看看ID是否在show full processlist里面的sleep线程中，如果是，就证明这个线程事务一直没有commit或者rollback而是卡住了，我们需要手动kill掉。

mysql&gt; SELECT * FROM information_schema.INNODB_TRX\G;

trx_mysq...]]></description><category></category></item><item><title><![CDATA[MySQL substring：字符串截取 (转载)]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84814500</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84814500</guid><author>yo8237233</author><pubDate>Wed, 24 Aug 2016 10:10:41 +0800</pubDate><description><![CDATA[MySQL 字符串截取函数：left(), right(), substring(), substring_index()。还有 mid(), substr()。其中，mid(), substr() 等价于 substring() 函数，substring() 的功能非常强大和灵活。

1. 字符串截取：left(str, length)

mysql&gt; select left('exam...]]></description><category></category></item><item><title><![CDATA[Maven项目无法引入 Maven Dependencies Libraries 问题]]></title><link>https://blog.youkuaiyun.com/yo8237233/article/details/84814387</link><guid>https://blog.youkuaiyun.com/yo8237233/article/details/84814387</guid><author>yo8237233</author><pubDate>Tue, 23 Aug 2016 10:53:10 +0800</pubDate><description><![CDATA[昨天在check下来maven项目之后一些配置好了，就是下载不是maven 依赖库，后面再网上找到如下解决方案。
在.classpath文件中加入如下代码就好了.

[code=&quot;xml&quot;]


     
	
	
     

[/code]...]]></description><category></category></item></channel></rss>