Filesystem closed

最新推荐文章于 2023-05-21 16:37:59 发布

posa88

最新推荐文章于 2023-05-21 16:37:59 发布

阅读量3.8k

点赞数

分类专栏： Hadoop BASIC 文章标签： hadoop

本文链接：https://blog.youkuaiyun.com/posa88/article/details/41018031

版权

在执行Hadoop MapReduce任务时遇到'Filesystem closed'异常，原因是方法内关闭了FileSystem，而默认配置fs.hdfs.impl.disable.cache为false，导致FileSystem缓存被关闭。解决方案是将该配置设置为true，避免在后续操作中使用到已关闭的FileSystem。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

java.io.IOException: Filesystem closed
	at org.apache.hadoop.hdfs.DFSClient.checkOpen(DFSClient.java:232)
	at org.apache.hadoop.hdfs.DFSClient.access$600(DFSClient.java:70)
	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1937)
	at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.read(DFSClient.java:1876)
	at java.io.DataInputStream.readInt(DataInputStream.java:370)
	at org.apache.hadoop.hive.ql.io.RCFile$Reader.readRecordLength(RCFile.java:1397)
	at org.apache.hadoop.hive.ql.io.RCFile$Reader.nextKeyBuffer(RCFile.java:1436)
	at org.apache.hadoop.hive.ql.io.RCFile$Reader.next(RCFile.java:1602)
	at com.sohu.adrd.targeting.hadoop.mapreduce.RCFileRecordReader.nextKeyValue(RCFileRecordReader.java:82)
	at org.apache.hadoop.mapreduce.lib.input.DelegatingRecordReader.nextKeyValue(DelegatingRecordReader.java:85)
	at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:456)
	at org.apache.hadoop.ma