《Spark环境部署》spark-shell环境下配置本地读入文本信息时出现报错： console>:17: error: not found: value sc

最新推荐文章于 2022-02-21 10:31:52 发布

原创

最新推荐文章于 2022-02-21 10:31:52 发布 · 2.6k 阅读

3 ·

CC 4.0 BY-SA版权

文章标签：

#hadoop #spark

在Spark环境部署中，尝试使用`spark-shell`读取本地文本时遇到报错，原因是Hadoop处于安全模式。错误信息提示不能创建目录，名节点在安全模式下。解决方案是通过`hadoop dfsadmin -safemode leave`命令退出Hadoop的安全模式。

Spark环境部署,配置本地读入文本信息时出现报错：

Caused by: org.apache.hadoop.ipc.RemoteException: Cannot create directory /tmp/hive/root/648867dc-0554-406f-8a02-233cef6b6cf2. Name node is in safe mode.
The reported blocks 71 needs additional 2 blocks to reach the threshold 0.9990 of total blocks 73.
The number of live datanodes 1 has reached the minimum number 0. Safe mode will be turned off automatically once the thresholds have been reached.
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkNameNodeSafeMode(FSNamesystem.java:1335)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.mkdirs(FSNamesystem.java:3874)
at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.mkdirs(NameNodeRpcServer.java:984)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.mkdirs(ClientNamenodeProtocolServerSideTranslatorPB.java:634)
at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol $2 . c a l l B l o c k i n g M e t h o d (C l i e n t N a m e n o d e P r o t o c o l P r o t o s . j a v a) a t o r g . a p a c h e . h a d o o p . i p c . P r o t o b u f R p c E n g i n e$ Server