Zeppelin-Interperter遇到的logging问题-以IoTDB为例

一次debug记录，以IoTDB的Zeppelin连接器中遇到的logging为例。

直接解决log问题

有cli，去掉cli，可以编译，但是报错：

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/Log
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.<init>(RemoteInterpreterServer.java:153)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.<init>(RemoteInterpreterServer.java:161)
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.main(RemoteInterpreterServer.java:356)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.Log
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
	... 3 more

查阅stack 添加commons-logging依赖，但仍然报错，这个包不对。

看zeppelin的其他解决方案

看zeppelin的github 中Cassandra的pom，发现zeppelin最近一次commit中修改了log的引用：

Remove commons-logging and switch to slf4j and jcl-over-slf4j

Iotdb恰好也做了了，所以也没有什么用。

brute-force：从maven依赖逐一排除

依赖cli显然是没有道理的，从zeppelin-interpreter的代码中来看，仅应该依赖iotdb-session（service-rpc也用到了，但是session中已经依赖了rpc)，应该是cli中用到的一些log包恰好解决了zeppelin的日志问题，开始排查：

1. 去掉cli依赖，审视cli的各种包，加入iotdb-jdbc，成功运行；
2. 继续审视jdbc，排除tsfile并测试了其他package，锁定到org.ops4j.pax.jdbc，将其加入到zeppelin的pom中，成功运行；
3. 从org.ops4j.pax.jdbc的github-pom文件中检查有关log的依赖，排除了log4j-slf4j-impl之后，将pax-logging-api加入，成功运行
4. 继续检查pax-logging-api的pom.xml 。这个pax-logging项目是关于OSGi日志记录框架实现。支持SLF4J，LOG4J，JCL等，所以把上述各种log都引用了。排除掉iotdb已经包含的各种log，以及jboss、tomcat等无关依赖，锁定到org.apache.logging.log4j的log4j-api ，但仅引用这个包却失败了，有点搞；
5. 重新审视pax-logging-api的依赖，所有logging相关的package全加进来，还是不行，难道解决这个问题靠的是pax-logging-api本身？
6. 可能是logging-api的父项目，将org.ops4j.pax.logging/pom.xml 中日志相关的依赖引入，修改一些冲突版本，编译成功，运行成功；
7. 去掉tomcat、jboss、avalon-logkit，仍然成功，现在只剩下logback、log4j和slf4j等官方package了
8. 去掉commons-logging、slf4j-jcl、jcl-over-slf4j等已经引用的package。测试未通过；
9. 直接取反，把剩下的包都注释掉，放出来这些包，结果运行成功
10. 仔细检查，去掉版本完全相同（即确定重复引用）的包commons-logging、logback-classic，仍然成功，现在
11. zeppelin的slf4j版本是1.7.30，领先于iotdb的1.7.25，改成版本一直，仍成功，范围缩减到slf4j-simple、nop和jcl：
    1. simple: Binding for Simple implementation, which outputs all events to System.err. Only messages of level INFO and higher are printed. This binding may be useful in the context of small applications.
    2. nop: Binding for NOP(no operation), silently discarding all logging.
    3. jcl: Binding for Jakarta Commons Logging. This binding will delegate all SLF4J logging to JCL.
12. 留下jcl，成功。

小结

新加package: slf4j-jcl

<!--<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-simple</artifactId>
  <version>${org.slf4j.version}</version>
</dependency>
<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-nop</artifactId>
  <version>${org.slf4j.version}</version>
</dependency>-->
<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-jcl</artifactId>
  <version>${org.slf4j.version}</version>
</dependency>

附-warning引发的maven error

Mvn编译报错，发现了package dependency的版本冲突warning。解决warning，error消除。

遇到编译问题，使用-e编译：

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce (enforce-version-convergence) on project zeppelin: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M2:enforce (enforce-version-convergence) on project zeppelin: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed.
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:213)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:154)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:146)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:117)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:81)
	at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:309)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:194)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:107)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:993)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:345)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:191)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: Some Enforcer rules have failed. Look above for specific messages explaining why the rule failed.
	at org.apache.maven.plugins.enforcer.EnforceMojo.execute(EnforceMojo.java:235)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
	... 20 more

看到一个warning，版本号重复定义了，应该是basic中已经有了core？去掉：

Dependency convergence error for ch.qos.logback:logback-core:1.1.11 paths to dependency are:
+-org.apache.iotdb:zeppelin:0.12.0-SNAPSHOT
  +-org.apache.iotdb:iotdb-session:0.12.0-SNAPSHOT
    +-org.apache.iotdb:tsfile:0.12.0-SNAPSHOT
      +-ch.qos.logback:logback-classic:1.1.11
        +-ch.qos.logback:logback-core:1.1.11
and
+-org.apache.iotdb:zeppelin:0.12.0-SNAPSHOT
  +-ch.qos.logback:logback-core:1.2.3

编译成功。