Hadoop学习1_在使用命令行运行WordCount时，遇到的jar命令说明

1. jar cf WordCount.jar WordCount*.class

使用情况：

编译WordCount.java文件，编译java文件的命令为javac，截图如下：

编译WordCount.java

此时，在workspace文件夹下将会出现生成三个class文件，

编译后生成class文件

编译成功后，即可将三个class文件打包成jar文件，

打包class成jar文件

执行成功后，在workspace文件下生成了WordCount.jar文件，

打包jar完成

jar cf WordCount.jar WordCount*.class

-c：创建新的jar文件包；

-f：指定jar文件名；

WordCount.jar：[jar-文件] 即需要生成、查看、更新或者解开的 JAR 文件包，它是 -f 参数的附属参数 ；

WordCount*.class：名字可以简写为WordCount的.class文件；

2.bin/hadoop jar workspace/WordCount.jar WordCount input output

使用情况：

在/usr/local/hadoop文件夹下新建一个input文件夹，用于存放数据，

创建input文件夹

接着cd 到input文件下，执行以下命令，就是将’Hello World Bye World’写进file01文件，将’Hello Hadoop Goodbye Hadoop’ 写进file02文件

创建输入数据

最后运行程序，

运行程序

bin/hadoop jar workspace/WordCount.jar WordCount input output

类似的：

hadoop jar WordCount.jar WordCount input output

hadoop jar WordCount.jar WordCount /tmp/input /tmp/output

/usr/local/hadoop/bin/hadoop jar WordCount.jar WordCount input output

/usr/local/hadoop/bin/hadoop jar WordCount.jarorg.apache.hadoop.examples.WordCount input output

（因为某些程序中声明了 package ，所以在命令中也要 org.apache.hadoop.examples 写完整，这些程序的第一行代码就是：package org.apache.hadoop.examples）

bin/hadoop：/usr/local/hadoop/bin/hadoop，这是一个hadoop文件的位置，不是文件夹，是对java命令的又一层封装，可以认为是hadoop在shell端的脚本；

jar：执行一个作业任务，其数据在jar中；

workspace/WordCount.jar：WordCount.jar的详细位置，结合前面的参数/usr/local/hadoop，详细位置为/usr/local/hadoop/workspace/WordCount.jar；

WordCount：

input：在hdfs中的数据输入目录；

output：在hdfs中的数据输出目录；

代码示例：

1. /**
2. * Licensed to the Apache Software Foundation (ASF) under one
3. * or more contributor license agreements. See the NOTICE file
4. * distributed with this work for additional information
5. * regarding copyright ownership. The ASF licenses this file
6. * to you under the Apache License, Version 2.0 (the
7. * "License"); you may not use this file except in compliance
8. * with the License. You may obtain a copy of the License at
9. *
10. * http://www.apache.org/licenses/LICENSE-2.0
11. *
12. * Unless required by applicable law or agreed to in writing, software
13. * distributed under the License is distributed on an "AS IS" BASIS,
14. * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15. * See the License for the specific language governing permissions and
16. * limitations under the License.
17. */
18. package org.apache.hadoop.examples;
20. import java.io.IOException;
21. import java.util.StringTokenizer;
23. import org.apache.hadoop.conf.Configuration;
24. import org.apache.hadoop.fs.Path;
25. import org.apache.hadoop.io.IntWritable;
26. import org.apache.hadoop.io.Text;
27. import org.apache.hadoop.mapreduce.Job;
28. import org.apache.hadoop.mapreduce.Mapper;
29. import org.apache.hadoop.mapreduce.Reducer;
30. import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
31. import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
32. import org.apache.hadoop.util.GenericOptionsParser;
34. publicclassWordCount{
36. publicstaticclassTokenizerMapper
37. extendsMapper<Object,Text,Text,IntWritable>{
39. privatefinalstaticIntWritable one=newIntWritable(1);
40. privateText word=newText();
42. publicvoid map(Object key,Text value,Context context
43. )throwsIOException,InterruptedException{
44. StringTokenizer itr =newStringTokenizer(value.toString());
45. while(itr.hasMoreTokens()){
46. word.set(itr.nextToken());
47. context.write(word, one);
48. }
49. }
50. }
52. publicstaticclassIntSumReducer
53. extendsReducer<Text,IntWritable,Text,IntWritable>{
54. privateIntWritable result=newIntWritable();
56. publicvoid reduce(Text key,Iterable<IntWritable> values,
57. Context context
58. )throwsIOException,InterruptedException{
59. int sum =0;
60. for(IntWritable val: values){
61. sum+= val.get();
62. }
63. result.set(sum);
64. context.write(key, result);
65. }
66. }
68. publicstaticvoid main(String[] args)throwsException{
69. Configuration conf =newConfiguration();
70. String[] otherArgs =newGenericOptionsParser(conf, args).getRemainingArgs();
71. if(otherArgs.length!=2){
72. System.err.println("Usage: wordcount <in> <out>");
73. System.exit(2);
74. }
75. Job job =newJob(conf,"word count");
76. job.setJarByClass(WordCount.class);
77. job.setMapperClass(TokenizerMapper.class);
78. job.setCombinerClass(IntSumReducer.class);
79. job.setReducerClass(IntSumReducer.class);
80. job.setOutputKeyClass(Text.class);
81. job.setOutputValueClass(IntWritable.class);
82. FileInputFormat.addInputPath(job,newPath(otherArgs[0]));
83. FileOutputFormat.setOutputPath(job,newPath(otherArgs[1]));
84. System.exit(job.waitForCompletion(true)?0:1);
85. }
86. }

参考资料：

http://blog.youkuaiyun.com/wang_zhenwei/article/details/47403825

http://dblab.xmu.edu.cn/blog/hadoop-build-project-by-shell/