java.lang.UnsupportedOperationException: getSubject is supported only if a security manager is allow

背景

macOS arm架构上start-dfs.sh时报这个错误java.lang.UnsupportedOperationException: getSubject is supported only if a security manager is allowed

解决方法

在hadoop的hadoop-env.sh中添加export HADOOP_OPTS="$HADOOP_OPTS -Djava.security.manager=allow"

java.lang.UnsupportedOperationException: getSubject is supported only if a security manager is allowed at java.base/javax.security.auth.Subject.getSubject(Subject.java:347) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:568) at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3487) at org.apache.hadoop.fs.FileSystem$Cache$Key.<init>(FileSystem.java:3477) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3319) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479) at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:217) at org.apache.hadoop.fs.FileSystem$1.run(FileSystem.java:214) at java.base/jdk.internal.vm.ScopedValueContainer.callWithoutScope(ScopedValueContainer.java:162) at java.base/jdk.internal.vm.ScopedValueContainer.call(ScopedValueContainer.java:147) at java.base/java.lang.ScopedValue$Carrier.call(ScopedValue.java:420) at java.base/java.lang.ScopedValue.callWhere(ScopedValue.java:568) at java.base/javax.security.auth.Subject.callAs(Subject.java:439) at java.base/javax.security.auth.Subject.doAs(Subject.java:614) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:214) at com.yx.hdfs.HdfsClient.init(HdfsClient.java:36) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) a
04-25
WARNING: Using incubator modules: jdk.incubator.vector WARNING: package sun.security.action not in java.base 25/08/13 23:33:32 WARN Shell: Did not find winutils.exe: java.io.FileNotFoundException: java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset. -see https://cwiki.apache.org/confluence/display/HADOOP2/WindowsProblems Using Spark's default log4j profile: org/apache/spark/log4j2-defaults.properties Setting default log level to "WARN". To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). WARNING: A restricted method in java.lang.System has been called WARNING: java.lang.System::loadLibrary has been called by org.apache.hadoop.util.NativeCodeLoader in an unnamed module (file:/D:/%e4%bb%a3%e7%a0%81/pycharmgongcheng/.venv/Lib/site-packages/pyspark/jars/hadoop-client-api-3.4.1.jar) WARNING: Use --enable-native-access=ALL-UNNAMED to avoid a warning for callers in this module WARNING: Restricted methods will be blocked in a future release unless native access is enabled 25/08/13 23:33:33 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Traceback (most recent call last): File "D:\代码\pycharmgongcheng\9527.py", line 8, in <module> sc = SparkContext(conf=conf) File "D:\代码\pycharmgongcheng\.venv\Lib\site-packages\pyspark\core\context.py", line 207, in __init__ self._do_init( ~~~~~~~~~~~~~^ master, ^^^^^^^ ...<10 lines>... memory_profiler_cls, ^^^^^^^^^^^^^^^^^^^^ ) ^ File "D:\代码\pycharmgongcheng\.venv\Lib\site-packages\pyspark\core\context.py", line 300, in _do_init self._jsc = jsc or self._initialize_context(self._conf._jconf) ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^ File "D:\代码\pycharmgongcheng\.venv\Lib\site-packages\pyspark\core\context.py", line 429, in _initialize_context return self._jvm.JavaSparkContext(jconf) ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^ File "D:\代码\pycharmgongcheng\.venv\Lib\site-packages\py4j\java_gateway.py", line 1627, in __call__ return_value = get_return_value( answer, self._gateway_client, None, self._fqn) File "D:\代码\pycharmgongcheng\.venv\Lib\site-packages\py4j\protocol.py", line 327, in get_return_value raise Py4JJavaError( "An error occurred while calling {0}{1}{2}.\n". format(target_id, ".", name), value) py4j.protocol.Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext. : java.lang.UnsupportedOperationException: getSubject is not supported at java.base/javax.security.auth.Subject.getSubject(Subject.java:277) at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:588) at org.apache.spark.util.Utils$.$anonfun$getCurrentUserName$1(Utils.scala:2446) at scala.Option.getOrElse(Option.scala:201) at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2446) at org.apache.spark.SparkContext.<init>(SparkContext.scala:339) at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:59) at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:374) at py4j.Gateway.invoke(Gateway.java:238) at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) at py4j.ClientServerConnection.waitForCommands(ClientServerConnection.java:184) at py4j.ClientServerConnection.run(ClientServerConnection.java:108) at java.base/java.lang.Thread.run(Thread.java:1447)
最新发布
08-14
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值