hive启用LAST_ACCESS_TIME(访问时间)

原创已于 2023-06-06 20:41:58 修改

· 627 阅读

1 ·

版权

文章标签：

#hive #hadoop #大数据

于 2023-06-06 19:16:31 首次发布

Hive 专栏收录该内容

19 篇文章

订阅专栏

文章讲述了如何在Hive的hive-site.xml配置文件中添加安全阀参数，用于允许特定的hook在执行阶段运行。示例中添加了hive.exec.pre.hooks参数，指定了org.apache.hadoop.hive.ql.hooks.UpdateInputAccessTimeHook$PreExec预执行钩子。之后，展示了查询表扩展信息和通过JavaAPI获取表的lastAccessTime的方法。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

1.hive-site.xml 的 Hive 服务高级配置代码段（安全阀）中添加2个参数：

hive.security.authorization.sqlstd.confwhitelist=hive.exec.pre.hooks

hive.exec.pre.hooks=org.apache.hadoop.hive.ql.hooks.UpdateInputAccessTimeHook$PreExec

2.重启hive即可

<property>
	<name>hive.security.authorization.sqlstd.confwhitelist</name>
	<value>hive.exec.pre.hooks</value>
</property>

<property>
	<name>hive.exec.pre.hooks</name>
	<value>org.apache.hadoop.hive.ql.hooks.UpdateInputAccessTimeHook$PreExec</value>
</property>

SELECT *
FROM TBLS
WHERE TBL_NAME = 'student3';

在这里插入图片描述

show table extended like student3;

hive (default)>
              >
              >
              >
              >
              > show table extended like student3;
OK
tab_name
tableName:student3
owner:atguigu
location:hdfs://hadoop102:8020/user/hive/warehouse/student3
inputformat:org.apache.hadoop.mapred.TextInputFormat
outputformat:org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
columns:struct columns { i32 id, string name}
partitioned:false
partitionColumns:
totalNumberFiles:1
totalFileSize:7
maxFileSize:7
minFileSize:7
lastAccessTime:1686048455210
lastUpdateTime:1685972721193

Time taken: 0.216 seconds, Fetched: 15 row(s)

import org.apache.hadoop.hive.conf.HiveConf;
import org.apache.hadoop.hive.metastore.HiveMetaStoreClient;
import org.apache.hadoop.hive.metastore.api.Table;
import org.apache.hadoop.hive.metastore.conf.MetastoreConf;

/**

 */
public class Test2 {

    public static void main(String[] args) throws Exception {

        HiveConf hiveConf = new HiveConf();
        MetastoreConf.setVar(hiveConf,MetastoreConf.ConfVars.THRIFT_URIS ,"thrift://hadoop102:9083" );

        HiveMetaStoreClient hiveClient = new HiveMetaStoreClient(hiveConf);

        Table aaa = hiveClient.getTable("default", "student3");
        int lastAccessTime = aaa.getLastAccessTime();
        System.out.println("lastAccessTime = " + lastAccessTime);

        System.out.println(aaa.getTableName());

//        String schema = "blending";
//        HCatClient client = HCatalogClient.getClient(null);
//        List<String> tables = client.listTableNamesByPattern(schema, "*");
//        for(String table : tables) {
//            HCatTable hTable = client.getTable(schema, table);
//            System.out.println("Table name is :" +hTable.getTableName());
//            System.out.println("Input file format is:"+hTable.getInputFileFormat());
//            System.out.println("Output file format is:"+hTable.getOutputFileFormat());
//        }
//
//        HCatCli hCatCli = new HCatCli();



    }


}

结果：

lastAccessTime = 1686049690

查询分区信息：

show table extended like dim_user_zip PARTITION(dt='9999-12-31');

在这里插入图片描述

查询总大小：