spark连接问题

关注

原创最新推荐文章于 2026-07-19 14:37:52 发布 6.3k 阅读 AI 写同款· GEO 优化 ›

搭建Spark on YARN报错 Hadoop 2.7.3 Spark 2.1.1 JDK 1.8 报错 17/05/27 17:14:00 ERROR YarnClientSchedulerBackend: Yarn application has already exited with state FAILED! 17/05/27 17:14:00 ERROR TransportClient: Failed to send R... 阅读详情

18/04/19 23:27:58 INFO master.Master: Registering worker 192.168.72.143:41196 with 4 cores, 2.7 GB RAM
18/04/19 23:28:00 INFO master.Master: Registering worker 192.168.72.141:38108 with 4 cores, 2.7 GB RAM
18/04/19 23:28:06 INFO master.Master: Registering worker 192.168.72.141:38108 with 4 cores, 2.7 GB RAM
18/04/19 23:28:06 INFO master.Master: Registering worker 192.168.72.143:41196 with 4 cores, 2.7 GB RAM
18/04/19 23:28:11 INFO master.Master: Registering worker 192.168.72.142:42803 with 4 cores, 2.7 GB RAM
18/04/19 23:28:12 INFO master.Master: Registering worker 192.168.72.141:38108 with 4 cores, 2.7 GB RAM
18/04/19 23:28:15 INFO master.ZooKeeperLeaderElectionAgent: We have gained leadership
18/04/19 23:28:15 INFO master.Master: I have been elected leader! New state: RECOVERING
18/04/19 23:28:15 INFO master.Master: Trying to recover worker: worker-20180419223807-192.168.72.141-43776
18/04/19 23:28:15 INFO master.Master: Trying to recover worker: worker-20180419223753-192.168.72.143-38431
18/04/19 23:28:15 WARN netty.OneWayOutboxMessage: Failed to send one-way RPC.
java.io.IOException: Failed to connect to /192.168.72.143:38431
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:216)
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:167)
at org.apache.spark.rpc.netty.NettyRpcEnv.createClient(NettyRpcEnv.scala:200)
at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:187)
at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:183)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: 拒绝连接: /192.168.72.143:38431
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
... 1 more
18/04/19 23:28:15 WARN netty.OneWayOutboxMessage: Failed to send one-way RPC.
java.io.IOException: Failed to connect to /192.168.72.141:43776
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:216)
at org.apache.spark.network.client.TransportClientFactory.createClient(TransportClientFactory.java:167)
at org.apache.spark.rpc.netty.NettyRpcEnv.createClient(NettyRpcEnv.scala:200)
at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:187)
at org.apache.spark.rpc.netty.Outbox$$anon$1.call(Outbox.scala:183)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.ConnectException: 拒绝连接: /192.168.72.141:43776
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:224)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:289)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
... 1 more
18/04/19 23:28:15 INFO master.Master: Registering worker 192.168.72.143:41196 with 4 cores, 2.7 GB RAM
18/04/19 23:28:18 INFO master.Master: Registering worker 192.168.72.141:38108 with 4 cores, 2.7 GB RAM
18/04/19 23:28:25 INFO master.Master: Registering worker 192.168.72.142:42803 with 4 cores, 2.7 GB RAM
18/04/19 23:29:15 INFO master.Master: Removing worker worker-20180419223753-192.168.72.143-38431 on 192.168.72.143:38431
18/04/19 23:29:15 INFO master.Master: Removing worker worker-20180419223807-192.168.72.141-43776 on 192.168.72.141:43776
18/04/19 23:29:15 INFO master.Master: Recovery complete - resuming operations!

收录于