spark读取HDFS文件java.net.ConnectException: Connection refused异常解决方法

最新推荐文章于 2025-05-29 15:51:31 发布

原创

最新推荐文章于 2025-05-29 15:51:31 发布 · 1w 阅读

4 ·

CC 4.0 BY-SA版权

在尝试使用Spark读取HDFS文件时遇到`java.net.ConnectException: Connection refused`错误，原因是代码中使用的路径为相对路径，而HDFS配置的端口非默认的8020，而是9000。解决方法是将读取路径改为绝对路径。

问题：java.net.ConnectException: Call From Master/xxx.xxx.x.xxx to user:8020 failed on connection exception: java.net.ConnectException: Connection refused

scala> disfile.collect().foreach(println)
java.net.ConnectException: Call From Master/xxx.xxx.x.xxx to user:8020 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
  at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
  at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
  at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
  at org.apache.hadoop.ipc.Client.call(Client.java:14