hive整合springcloud框架

最新推荐文章于 2024-02-22 14:11:51 发布

原创最新推荐文章于 2024-02-22 14:11:51 发布 · 774 阅读

0 ·

CC 4.0 BY-SA版权

java 专栏收录该内容

2 篇文章

订阅专栏

本文解决Hive中数据插入速度慢的问题，通过调整Maven依赖排除日志和Tomcat冲突，并提供了解决Hive权限异常的方法，包括关闭Hadoop安全模式和更改目录权限。

hive插入表数据非常慢

1.maven解决日志冲突和Tomcat冲突

<dependency>
    <groupId>org.apache.hive</groupId>
    <artifactId>hive-jdbc</artifactId>
    <version>2.3.2</version>
    <exclusions>
        <exclusion>
            <groupId>org.mortbay.jetty</groupId>
            <artifactId>*</artifactId>
        </exclusion>
        <exclusion>
            <groupId>javax.servlet</groupId>
            <artifactId>*</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.eclipse.jetty.orbit</groupId>
            <artifactId>*</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.eclipse.jetty.aggregate</groupId>
            <artifactId>*</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.slf4j</groupId>
            <artifactId>*</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.apache.logging.log4j</groupId>
            <artifactId>*</artifactId>
        </exclusion>
        <exclusion>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
        </exclusion>
        <exclusion>
            <groupId>org.apache.hive</groupId>
            <artifactId>hive-shims</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework.data</groupId>
    <artifactId>spring-data-hadoop</artifactId>
    <version>2.5.0.RELEASE</version>
    <exclusions>
        <exclusion>
            <groupId>javax.servlet</groupId>
            <artifactId>servlet-api</artifactId>
        </exclusion>
    </exclusions>
</dependency>

2.插入语句出现异常:

Permission denied: user=anonymous, access=WRITE, inode="/":root:supergroup:drwxr-xr-x

hadoop dfsadmin -safemode leave 关闭安全模式

hdfs dfs -chmod 777 /

重启hadoop和hive