莱布尼兹

  看了一堆不太相关的东西... 其实只要解压运行即可,形如hadoop-0.20.xx,不过要注意jar的位置.   hadoop jar share/hadoop/hadoop-mapreduce/hadoop-example-xx.jar wordcount input output   接下来将进行cluster布署,加深了解新架构的动作流程.   12/06/17 11:...

  I dont want to restruct wheels of open sources,in contrast, just wonder the implied features and use cases as possible.so i will write somethings to summary or memo. Agenda  1.what is   2.how t...

 note,this article is focus on hadoop-2.5.1,it maybe a little different from hadoop-0.23.x. Agenda:  I:hdfs federation abstract  II:implement  III:other distributed namespaces scalability  ----...

  below are the outlines of this article ,using by hadoop-2.5.1 : 1.abstraction 1.1 what is HDFS HA 1.2 how to 1.3 other HAs 2.installation 2.1 manual failover 2.2 auto failover 3.conc...

 this article is the another part follow to hadoop 2.x-HDFS HA --Part I: abstraction ,and here will talk about these topics:   2.installation HA 2.1 manual failover 2.2 auto failover 3.conc...

  I. installation mode   same as hadoop 1.x ,there are several mode to install hadoop: 1.standalone   just run it on one machine,includes running of mapreduce. 2.pseudo   setup it with hdfs mo...

  same as compile hbase ,it 's necessary for u to compile your own dist of hadoop to get best performance specified to your hardwares.   bellow are some steps needed by this theme: 1.install some ...

1.what is   so as the named 'unmanaged application master/UnmanagedAM' ,this masters are not controlled by RM,but client it self !yep,it's easy to test this kind of AMs in local ,and may be no any ...

     there are something differences between [2] which is pressed in the early 2.2 release,specially in step 5 shown in above.      ref: hadoop 2-state machine-(1) [2] 《Hadoop技术内幕：深入解析YARN架构设计...

1.what     in many scenarios,we see some figures like this:some state node,some events triggered state changes,and certain directed-arcs.it look likes a running-machine with misc commands ,say like...

  this figure demonstrates the flow of how to submit a AM to yarn.    so u see this is a very flexible running-mode

  this application is introduced to run shell command in distributed nodes(containers) as it named,so it's is ealy and let's to go ahead.   1.run 'ls' command in containers 2.which path does that...

  we know,hadoop will show only abstract  result info about mapreduce(represent in http://xxx:19888/clsuter by default),so it's unconvenient to track the number of mappers,where mappers run,how to t...

一.dfs 1.旧的dfs方案   可以看到block管理与NN是一对一的,即整个集群中只有一个'block pool',隔离性差; 另外物理是它是置于NN端的,逻辑上把它归入到bock storage上了,这是需要注意的;   2.dfs federation   新存储架构采用了multi-namespaces机制,安全/故障隔离性好; 每个Ns都有一个自己的Pool...

以下大部分内容来自网络,这里主要是进行学习,比较   1、Hadoop 0.20.*的局限性 HDFS单NameNode的不足 1）扩展性问题。可以随着数据量进行水平扩展，而元数据服务器不能扩展。      2）随着文件数目的增长，元数据服务器的压力变大。据统计，2.5亿个文件在NameNode中Namespace占据  的大概64GB的内存空间。     3）...

1. submitting a MR job       2.flow from nodemanager to resourcemanager:       so from this figure,we know that the yarn use WritableRPCEnginge.java as rpc engine by default,but here of cou...