报错代码如下:
[hadoop@mini1 mypro]$ hadoop jar flow.jar /flumcount/input /flumcount/output
17/03/31 16:29:37 INFO client.RMProxy: Connecting to ResourceManager at mini1/192.168.40.201:803217/03/31 16:29:38 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
17/03/31 16:29:39 INFO input.FileInputFormat: Total input paths to process : 1
17/03/31 16:29:39 INFO mapreduce.JobSubmitter: number of splits:1
17/03/31 16:29:40 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1490947205776_0003
17/03/31 16:29:41 INFO impl.YarnClientImpl: Submitted application application_1490947205776_0003
17/03/31 16:29:41 INFO mapreduce.Job: The url to track the job: http://mini1:8088/proxy/application_1490947205776_0003/
17/03/31 16:29:41 INFO mapreduce.Job: Running job: job_1490947205776_0003
17/03/31 16:29:59 INFO mapreduce.Job: Job job_1490947205776_0003 running in uber mode : false
17/03/31 16:29:59 INFO mapreduce.Job: map 0% reduce 0%
17/03/31 16:30:30 INFO mapreduce.Job: Task Id : attempt_1490947205776_0003_m_000000_0, Status : FAILED
Error: java.io.IOException: Unable to initialize any output collector (无法初始化任何输出收集器)
at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:412)
at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:695)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
看错误信息报的是IO流错误,查看自己的代码:
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = Job.getInstance(conf);
job.setJarByClass(FlowClient.class);
job.setMapperClass(FLowMapper.class);
job.setReducerClass(FlowReducer.class);
job.setMapOutputKeyClass(Text.class); // (这里下面的Text类应该都是org.apache.hadoop.io.Text)
job.setMapOutputValueClass(FlowBean.class);
job.setOutputKeyClass(Text.class);
//( 在输出这里引用的不是import org.apache.hadoop.io.Text;所以会出错)
job.setOutputValueClass(FlowBean.class);
FileInputFormat.setInputPaths(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
boolean rs = job.waitForCompletion(true);
System.exit(rs ? 0 : 1);
}