
Hadoop
kaloKu
要与不要
展开
-
yarn的调度模式详解
Yarn的调度yarn的默认调度器1:Apache hadoop2.x的默认调度器是Capacity Scheduler(计算调度器)2:CDH的默认调度器是Fair Scheduler(公平调度器)Yarn三种调度策略对比在Yarn中有三种调度器可以选择:FIFO Scheduler ,Capacity Scheduler,FairScheduler。1:FIFO Scheduler...原创 2020-05-15 09:35:27 · 1669 阅读 · 0 评论 -
生产恢复HDFS block块小记
测试文件 test.md上传:-bash-4.2$ hdfs dfs -mkdir /blockrecover-bash-4.2$ echo "test" > test.md-bash-4.2$ hdfs dfs -put test.md /blockrecover-bash-4.2$ hdfs dfs -ls /blockrecoverFound 1 items-rw-r-...原创 2019-12-02 14:51:42 · 160 阅读 · 0 评论 -
hadoop 常用linux命令
单独启动和关闭hadoop服务启动名称节点 :hadoop-daemon.sh start namenode启动数据节点:hadoop-daemons.sh start datanode slave启动secondarynamenode:hadoop-daemon.sh start secondarynamenode启动resourcemanager:yarn-daemon.sh st...原创 2019-06-25 18:16:48 · 475 阅读 · 1 评论 -
HDFS架构原理思想及各模块功能简述
1:架构原理2 block(块)解析块的大小128M 块的副本数3 <property> <name>dfs.blocksize</name> <value>134217728</value></property><property> <n原创 2019-02-23 16:53:02 · 2045 阅读 · 0 评论 -
hadoop hdfs单节点简易部署(一)
1.创建用户和上传hadoop软件:useradd hadoopsu - hadoop[hadoop@hadoop002 ~]$ mkdir app[hadoop@hadoop002 ~]$ cd app/[hadoop@hadoop002 ~]$ wget http://archive.cloudera.com/cdh5/cdh/5/hadoop-2.6.0-cdh5.7.0.tar....原创 2019-02-23 16:30:06 · 615 阅读 · 0 评论