hadoop2.5的第一个HelloWorld程序—单词计数（WordCount.）

最新推荐文章于 2021-11-19 09:55:21 发布

原创最新推荐文章于 2021-11-19 09:55:21 发布 · 841 阅读

0 ·

CC 4.0 BY-SA版权

hadoop 专栏收录该内容

5 篇文章

订阅专栏

本文介绍了如何使用Hadoop进行WordCount实例操作，包括启动Hadoop服务、上传文本文件到HDFS、运行WordCount程序并查看结果等步骤。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

首先启动hadoop服务.

在文件系统上创建一个文件夹data:

[root@Master hadoop-2.5.0]# hdfs dfs -mkdir /data
Java HotSpot(TM) Client VM warning: You have loaded library /home/jzz/hadoop-2.5.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
14/12/21 22:36:36 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[root@Master hadoop-2.5.0]# hdfs dfs -put -f example/file1.txt example/file2.txt  /data
Java HotSpot(TM) Client VM warning: You have loaded library /home/jzz/hadoop-2.5.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
14/12/21 22:37:49 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

此时访问文件系统，

http://192.168.122.130:50070

可以查看创建的文件夹。

在本地创建单词文件：

[root@Master hadoop-2.5.0]# cat example/words.txt 
hello world
hello hadoop
hello mapreduce

将单词文件放入文件系统中：

[root@Master hadoop-2.5.0]# hdfs dfs -put -f example/words.txt /data
Java HotSpot(TM) Client VM warning: You have loaded library /home/jzz/hadoop-2.5.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
14/12/21 23:09:23 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

运行WordCount.java:

[root@Master hadoop-2.5.0]# hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.5.0-sources.jar org.apache.hadoop.examples.WordCount /data /outputs
Java HotSpot(TM) Client VM warning: You have loaded library /home/jzz/hadoop-2.5.0/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
14/12/21 23:12:02 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/12/21 23:12:02 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
14/12/21 23:12:03 INFO input.FileInputFormat: Total input paths to process : 3
14/12/21 23:12:04 INFO mapreduce.JobSubmitter: number of splits:3

......

查看输出结果：

[root@Master hadoop-2.5.0]# hdfs dfs -cat /outputs/part-r-00000

hadoop	1
hello	3
mapreduce	1
world	1

至于文件名，可以查询文件系统。