单机运行 spark-shell错误

最新推荐文章于 2025-06-20 15:10:53 发布

取个昵称好难啊Elaine

最新推荐文章于 2025-06-20 15:10:53 发布

阅读量3.3k

点赞数 1

CC 4.0 BY-SA版权

分类专栏： spark

本文链接：https://blog.youkuaiyun.com/dickysun1987/article/details/78829830

spark 专栏收录该内容

6 篇文章

订阅专栏

本文记录了在安装Spark过程中遇到的问题及解决方案，特别是针对spark-shell启动时出现的警告和错误信息。通过检查和修改/etc/hosts文件中的主机名绑定地址，最终成功解决了Spark无法绑定随机空闲端口的问题。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

安装spark，spark-shell遇到错误

Using Spark’s default log4j profile: org/apache/spark/log4j-defaults.properties
Setting default log level to “WARN”.
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel).
17/12/18 08:40:12 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 WARN Utils: Service ‘sparkDriver’ could not bind on a random free port. You may check whether configuring an appropriate binding address.
17/12/18 08:40:13 ERROR SparkContext: Error initializing SparkContext.
java.net.BindException: Cannot assign requested address: Service ‘sparkDriver’ failed after 16 retries (on a random free port)! Consider explicitly setting the appropriate binding address for the service ‘sparkDriver’ (for example spark.driver.bindAddress for SparkDriver) to the correct binding address.
.
.
.
Welcome to
__
/ / _ _/ /__
\ \/ \/ _ `/ _/ ‘/
// ./_,// //_\ version 2.2.1
/_/
Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_151)
Type in expressions to have them evaluated.
Type :help for more information
scala>

原因：
host文件绑定地址错误
解决：
修改 /etc/hosts文件
添加或配置正确地址
原hosts文件

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6

查看本机hostname及地址
修改hosts文件

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
10.64.42.108 ls-dj-test-3

重新运行 spark-shell 成功