Spark应用程序无法写入在docker中运行的elasticsearch解决办法

最新推荐文章于 2024-10-12 23:00:16 发布

原创最新推荐文章于 2024-10-12 23:00:16 发布 · 297 阅读

0 ·

CC 4.0 BY-SA版权

Docker 同时被 3 个专栏收录

75 篇文章

订阅专栏

Spark

17 篇文章

订阅专栏

ElasticSearch

7 篇文章

订阅专栏

这篇博客主要记录了在使用Spark写入Elasticsearch时遇到的连接超时错误，通过设置`es.nodes.wan.only`为`true`来解决此问题，确保数据能够成功写入。

报错如下：

20/12/15 16:23:44 INFO EsRDDWriter: Writing to [spark/docs]
20/12/15 16:24:05 INFO HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
20/12/15 16:24:05 INFO HttpMethodDirector: Retrying request
20/12/15 16:24:05 INFO HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
20/12/15 16:24:05 INFO HttpMethodDirector: Retrying request
20/12/15 16:24:26 INFO HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
20/12/15 16:24:26 INFO HttpMethodDirector: Retrying request
20/12/15 16:24:26 INFO HttpMethodDirector: I/O exception (java.net.ConnectException) caught when processing request: Connection timed out: connect
20/12/15 16:24:26 INFO HttpMethodDirector: Retrying request

解决办法：