spark，poi，jar包冲突(commons.io)

最新推荐文章于 2024-11-18 07:30:00 发布

青云游子

最新推荐文章于 2024-11-18 07:30:00 发布

阅读量513

点赞数 2

文章标签： spark jar 大数据

本文链接：https://blog.youkuaiyun.com/qq_40382400/article/details/142533341

版权

1.查看报错类是属于哪个jar包

System.out.println("ZipArchiveInputStream类是：" + ZipArchiveInputStream.class.getProtectionDomain().getCodeSource().getLocation());
System.out.println("UnsynchronizedByteArrayOutputStream 类是：" + UnsynchronizedByteArrayOutputStream.class.getProtectionDomain().getCodeSource().getLocation());
System.out.println("IOUtils 类是：" + IOUtils.class.getProtectionDomain().getCodeSource().getLocation());

2.使用shade来打包并隔离依赖，就是重命名包名

<plugin>
	<groupId>org.apache.maven.plugins</groupId>
	<artifactId>maven-shade-plugin</artifactId>
	<version>3.2.4</version>
	<executions>
		<execution>
			<phase>package</phase>
			<goals>
				<goal>shade</goal>
			</goals>
			<configuration>
				<relocations>
					<relocation>
						<pattern>org.apache.commons.io</pattern>
						<shadedPattern>myshade.org.apache.commons.io</shadedPattern>
					</relocation>
				</relocations>
			</configuration>
		</execution>
	</executions>
</plugin>

3.排除jar包

<dependency>
	<groupId>org.apache.spark</groupId>
	<artifactId>spark-core_${scala.version}</artifactId>
	<version>${spark.version}</version>
	<exclusions>
		<exclusion>
			<artifactId>jackson-annotations</artifactId>
			<groupId>com.fasterxml.jackson.core</groupId>
		</exclusion>
		<exclusion>
			<artifactId>jackson-core</artifactId>
			<groupId>com.fasterxml.jackson.core</groupId>
		</exclusion>
		<exclusion>
			<artifactId>jackson-databind</artifactId>
			<groupId>com.fasterxml.jackson.core</groupId>
		</exclusion>
		<exclusion>
			<artifactId>jackson-module-scala_2.11</artifactId>
			<groupId>com.fasterxml.jackson.module</groupId>
		</exclusion>
		<exclusion>
			<artifactId>commons-compress</artifactId>
			<groupId>org.apache.commons</groupId>
		</exclusion>
	</exclusions>
<!--            <scope>provided</scope>-->
</dependency>

4.引入jar包

<dependency>
	<groupId>commons-io</groupId>
	<artifactId>commons-io</artifactId>
	<version>2.17.0</version>
</dependency>
<dependency>
	<groupId>org.apache.commons</groupId>
	<artifactId>commons-compress</artifactId>
	<version>1.18</version>
</dependency>