resourcemanager java.lang.ClassNotFoundException: javax.activation.DataSource

hadoop无法启动resourcemanager和nodeManager ,
查看log发现java.lang.ClassNotFoundException: javax.activation.DataSource错误。

我出现这种情况是因为使用了jdk9,jdk9默认禁用访问许多javax. * API,可以更换为jdk8

还有一种方案不用修改jdk9(这种方法不行)

修改$HADOOP_HOME/etc/hadoop/yarn-env.sh文件

export YARN_RESOURCEMANAGER_OPTS="--add-modules=ALL-SYSTEM"
export YARN_NODEMANAGER_OPTS="--add-modules=ALL-SYSTEM

参考: https://issues.apache.org/jira/browse/HADOOP-14978

第二天。。。又是一段惨痛的经历

使用jdk9之后修改了yarn-env.sh后resourcemanager和nodemanager。虽然服务成功启动了,但是在wordcount测试 中这两货还是执行失败了,由于菜鸟一枚,几番周折才发现错误原因,结果还是由于java.lang.ClassNotFoundException: javax.activation.DataSource的错误造成的。因此我还是用回jdk8,避免各种乱七八糟的原因

以下是resourcemanager中的日志输出

[2017-12-21 20:59:13.006]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/home/hadoop/hadoop/hadoop-3.0.0/share/hadoop/yarn/lib/guice-4.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider class
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a root resource class
Dec 21, 2017 8:59:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.


[2017-12-21 20:59:13.006]Container exited with a non-zero exit code 1. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$2 (file:/home/hadoop/hadoop/hadoop-3.0.0/share/hadoop/yarn/lib/guice-4.0.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider class
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register
INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a root resource class
Dec 21, 2017 8:59:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19 02/11/2015 03:25 AM'
Dec 21, 2017 8:59:11 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider
INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope "Singleton"
log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.


For more detailed output, check the application tracking page: http://master:8088/cluster/app/application_1513860808177_0001 Then click on links to logs of each attempt.
. Failing the application.
2017-12-21 20:59:13,063 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1513860808177_0001 State change from FINAL_SAVING to FAILED on event = APP_UPDATE_SAVED
2017-12-21 20:59:13,063 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.ParentQueue: Application removed - appId: application_1513860808177_0001 user: hadoop leaf-queue of parent: root #applications: 0
2017-12-21 20:59:13,072 WARN org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hadoop   OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE  DESCRIPTION=App failed with state: FAILED   PERMISSIONS=Application application_1513860808177_0001 failed 2 times due to AM Container for appattempt_1513860808177_0001_000002 exited with  exitCode: 1
Failing this attempt.Diagnostics: [2017-12-21 20:59:12.978]Exception from container-launch.
Container id: container_1513860808177_0001_02_000001
Exit code: 1

最后在logs/userlogs/好几级目录下找到了找到了application_1513860808177_0001日志,里面出现了
java.lang.ClassNotFoundException: javax.activation.DataSource错误

### Java环境中`ClassNotFoundException`异常解决方案 对于Java应用程序中的`java.lang.ClassNotFoundException`,该异常表明JVM尝试加载某个类而未能找到相应的.class文件。通常情况下,这可能是由于缺少必要的库或者配置不正确引起的。 针对特定于Jakarta Servlet Filter的情况,在Web应用中遇到此类问题可能是因为Servlet API版本兼容性问题或者是项目依赖管理不当造成的[^1]。为了处理这个问题: - **确认依赖项**:确保项目的构建工具(Maven, Gradle等)已经包含了正确的Servlet API依赖,并且其范围设置为provided,因为容器本身会提供这些API。 对于Maven而言,可以在pom.xml中加入如下片段来声明Servlet API作为编译期而非运行时期依赖: ```xml <dependency> <groupId>jakarta.servlet</groupId> <artifactId>jakarta.servlet-api</artifactId> <version>6.0.0</version> <scope>provided</scope> </dependency> ``` - **检查ClassPath路径**:验证所有的jar包都位于classpath下;如果是在IDE内部开发,则需保证IDE能够识别到所需的外部资源。 关于Spark集群环境下的`java.lang.ClassNotFoundException: org.slf4j.Logger`错误,此情况往往发生在日志框架slf4j未被正确引入至SPARK_CLASSPATH变量之中时发生[^3]。对此采取措施可以是修改spark-env.sh脚本以包含Hadoop Classpath: ```bash export SPARK_DIST_CLASSPATH=$(/usr/local/hadoop/bin/hadoop classpath) ``` 此外,解决任何类型的`ClassNotFoundException`的一般方法还包括但不限于以下几个方面[^2]: - 验证所需Jar文件确实存在于系统的CLASSPATH环境变量里; - 如果使用的是打包后的WAR/EAR部署单元,请确认它们被打包进了最终制品当中; - 当涉及第三方组件或插件时,查阅官方文档获取最新的安装指南以及支持的平台列表; - 使用命令行参数指定额外的classpaths,比如通过 `-cp` 或者 `-classpath` 参数传递给java执行器。 #### 关联端口(port=8088) 需要注意的是,虽然上述讨论主要围绕着如何修复`ClassNotFoundException`,但是提到port=8088这个细节暗示了可能存在与网络服务有关的问题。然而,除非YARN ResourceManager Web UI或其他相关服务监听在此端口上出现问题影响到了ClassLoader的行为模式,否则端口号本身不太可能导致这类异常的发生。
评论 3
成就一亿技术人!
拼手气红包6.0元
还能输入1000个字符
 
红包 添加红包
表情包 插入表情
 条评论被折叠 查看
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值