hadoop2.6.0实践：004 启动伪分布式hadoop的进程

最新推荐文章于 2022-05-23 21:44:56 发布

转载最新推荐文章于 2022-05-23 21:44:56 发布 · 137 阅读

0 ·

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/lexiaofei/p/6384402.html

文章标签：

#大数据

本文详细记录了Hadoop集群的搭建过程，包括启动NameNode、DataNode、ResourceManager等组件，并通过实例展示了如何使用HDFS进行文件操作及YARN的任务调度。

[hadoop@LexiaofeiMaster hadoop-2.6.0]$ start-dfs.sh
Starting namenodes on [localhost]
localhost: starting namenode, logging to /opt/modules/hadoop-2.6.0/logs/hadoop-hadoop-namenode-LexiaofeiMaster.out
LexiaofeiMaster: starting datanode, logging to /opt/modules/hadoop-2.6.0/logs/hadoop-hadoop-datanode-LexiaofeiMaster.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /opt/modules/hadoop-2.6.0/logs/hadoop-hadoop-secondarynamenode-LexiaofeiMaster.out
[hadoop@LexiaofeiMaster hadoop-2.6.0]$ jps
4421 SecondaryNameNode
4238 DataNode
4530 Jps
4153 NameNode
[hadoop@LexiaofeiMaster hadoop-2.6.0]$ hdfs dfs -ls /
[hadoop@LexiaofeiMaster hadoop-2.6.0]$ hdfs dfs -mkdir -p /input/grep
[hadoop@LexiaofeiMaster hadoop-2.6.0]$ start-yarn.sh
starting yarn daemons
starting resourcemanager, logging to /opt/modules/hadoop-2.6.0/logs/yarn-hadoop-resourcemanager-LexiaofeiMaster.out
LexiaofeiMaster: starting nodemanager, logging to /opt/modules/hadoop-2.6.0/logs/yarn-hadoop-nodemanager-LexiaofeiMaster.out
[hadoop@LexiaofeiMaster hadoop-2.6.0]$ jps
4421 SecondaryNameNode
4744 NodeManager
4238 DataNode
4801 Jps
4653 ResourceManager
4153 NameNode
[hadoop@LexiaofeiMaster hadoop-2.6.0]$ hdfs dfs -put /opt/data/input/* /input/grep
[hadoop@LexiaofeiMaster hadoop-2.6.0]$ hdfs dfs -ls /input/grep
Found 8 items
-rw-r--r-- 1 hadoop supergroup 4436 2017-02-09 20:01 /input/grep/capacity-scheduler.xml
-rw-r--r-- 1 hadoop supergroup 774 2017-02-09 20:01 /input/grep/core-site.xml
-rw-r--r-- 1 hadoop supergroup 9683 2017-02-09 20:01 /input/grep/hadoop-policy.xml
-rw-r--r-- 1 hadoop supergroup 775 2017-02-09 20:01 /input/grep/hdfs-site.xml
-rw-r--r-- 1 hadoop supergroup 620 2017-02-09 20:01 /input/grep/httpfs-site.xml
-rw-r--r-- 1 hadoop supergroup 3523 2017-02-09 20:01 /input/grep/kms-acls.xml
-rw-r--r-- 1 hadoop supergroup 5511 2017-02-09 20:01 /input/grep/kms-site.xml
-rw-r--r-- 1 hadoop supergroup 690 2017-02-09 20:01 /input/grep/yarn-site.xml
[hadoop@LexiaofeiMaster hadoop-2.6.0]$ ./sbin/mr-jobhistory-daemon.sh start historyserver
starting historyserver, logging to /opt/modules/hadoop-2.6.0/logs/mapred-hadoop-historyserver-LexiaofeiMaster.out
[hadoop@LexiaofeiMaster hadoop-2.6.0]$ jps
5179 JobHistoryServer
4421 SecondaryNameNode
4744 NodeManager
4238 DataNode
5210 Jps
4653 ResourceManager
4153 NameNode
[hadoop@LexiaofeiMaster hadoop-2.6.0]$