Updating the Schema Cache

本文介绍了在Active Directory中遇到的SchemaCache未及时更新的问题及解决方案。作者在导入自定义Class的Schema后无法立即实例化,原因是SchemaCache未能同步最新更改。通过在LDIF文件中添加特定属性,可以即时触发SchemaCache更新。

昨天发现的“导入自定义的class的Schema后,不能实例化”的问题,原来是由于Schema Cache导致的。

今天看了MSDN中的相关文章后,才恍然大悟,导致这个问题的原因是我在编写ldif文件时,没有加上让Schema Cache 及时更新的语句。

将MSDN中的说明转载于此,便于日后查阅。

 

All information that is written to an Active Directory server is validated against the schema. The schema is held in memory on directory servers (domain controllers) for performance reasons. The in-memory version is updated automatically after the on-disk version has been updated. The automatic update occurs five minutes after the last change was applied; applying another change to the schema in the 5-minute window resets the timer for another 5 minutes. This behavior keeps the cache consistent, but can be confusing, because changes do not appear in the schema until the cache is updated, even though they were applied on disk.

To update the Active Directory schema cache after a schema update, or if you want to use the schema update for non-schema operations immediately, add the schemaUpdateNow attribute (it is an operational attribute) to the root DSE (blank DN) with value 1. A schema cache update will start immediately. The call is blocking. If the call returns with no error, the cache is updated and all schema updates are ready to be used. An error return indicates the cache update was unsuccessful. Applications that must use this feature should be designed to accommodate the blocking write, particularly in giving the user feedback, if the program or script executes interactively.

The following code example is a sample LDIFDE script that shows how to trigger a cache reload.

dn:
changetype: modify
add: schemaUpdateNow
schemaUpdateNow: 1
-

 

转载于:https://www.cnblogs.com/oneiter/articles/1516021.html

[root@hadoop01 apache-hive-3.1.3-bin]# $HIVE_HOME/bin/hive which: no hbase in (/export/servers/hadoop-3.3.5/bin::/export/servers/apache-hive-3.1.3-bin/bin:/export/servers/flume-1.9.0/bin::/export/servers/apache-hive-3.1.3-bin/bin:/export/servers/flume-1.9.0/bin:/export/servers/flume-1.9.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/export/servers/jdk1.8.0_161/bin:/export/servers/hadoop-3.3.5/bin:/export/servers/hadoop-3.3.5/sbin:/export/servers/scala-2.12.10/bin:/root/bin:/export/servers/jdk1.8.0_161/bin:/export/servers/hadoop-3.3.5/bin:/export/servers/hadoop-3.3.5/sbin:/export/servers/scala-2.12.10/bin:/export/servers/jdk1.8.0_161/bin:/export/servers/hadoop-3.3.5/bin:/export/servers/hadoop-3.3.5/sbin:/export/servers/scala-2.12.10/bin:/root/bin) 2025-06-17 19:30:31,773 INFO conf.HiveConf: Found configuration file file:/export/servers/apache-hive-3.1.3-bin/conf/hive-site.xml Hive Session ID = b6ee71f0-5d43-4149-99c7-808d6c553bb8 2025-06-17 19:30:36,006 INFO SessionState: Hive Session ID = b6ee71f0-5d43-4149-99c7-808d6c553bb8 Logging initialized using configuration in jar:file:/export/servers/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true 2025-06-17 19:30:36,195 INFO SessionState: Logging initialized using configuration in jar:file:/export/servers/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true 2025-06-17 19:30:40,759 INFO session.SessionState: Created HDFS directory: /tmp/hive/root/b6ee71f0-5d43-4149-99c7-808d6c553bb8 2025-06-17 19:30:40,863 INFO session.SessionState: Created local directory: /tmp/root/b6ee71f0-5d43-4149-99c7-808d6c553bb8 2025-06-17 19:30:40,874 INFO session.SessionState: Created HDFS directory: /tmp/hive/root/b6ee71f0-5d43-4149-99c7-808d6c553bb8/_tmp_space.db 2025-06-17 19:30:40,916 INFO conf.HiveConf: Using the default value passed in for log id: b6ee71f0-5d43-4149-99c7-808d6c553bb8 2025-06-17 19:30:40,916 INFO session.SessionState: Updating thread name to b6ee71f0-5d43-4149-99c7-808d6c553bb8 main 2025-06-17 19:30:43,385 INFO metastore.HiveMetaStore: 0: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore 2025-06-17 19:30:43,505 WARN metastore.ObjectStore: datanucleus.autoStartMechanismMode is set to unsupported value null . Setting it to value: ignored 2025-06-17 19:30:43,521 INFO metastore.ObjectStore: ObjectStore, initialize called 2025-06-17 19:30:43,523 INFO conf.MetastoreConf: Found configuration file file:/export/servers/apache-hive-3.1.3-bin/conf/hive-site.xml 2025-06-17 19:30:43,526 INFO conf.MetastoreConf: Unable to find config file hivemetastore-site.xml 2025-06-17 19:30:43,526 INFO conf.MetastoreConf: Found configuration file null 2025-06-17 19:30:43,528 INFO conf.MetastoreConf: Unable to find config file metastore-site.xml 2025-06-17 19:30:43,528 INFO conf.MetastoreConf: Found configuration file null 2025-06-17 19:30:44,060 INFO DataNucleus.Persistence: Property datanucleus.cache.level2 unknown - will be ignored 2025-06-17 19:30:44,701 INFO hikari.HikariDataSource: HikariPool-1 - Starting... 2025-06-17 19:30:45,564 INFO hikari.HikariDataSource: HikariPool-1 - Start completed. 2025-06-17 19:30:45,707 INFO hikari.HikariDataSource: HikariPool-2 - Starting... 2025-06-17 19:30:45,741 INFO hikari.HikariDataSource: HikariPool-2 - Start completed. 2025-06-17 19:30:46,209 INFO metastore.ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" 2025-06-17 19:30:46,656 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is MYSQL 2025-06-17 19:30:46,662 INFO metastore.ObjectStore: Initialized ObjectStore 2025-06-17 19:30:47,806 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:30:47,807 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:30:47,808 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:30:47,809 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:30:47,809 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:30:47,810 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:30:51,421 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:30:51,422 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:30:51,422 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:30:51,422 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:30:51,423 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:30:51,423 WARN DataNucleus.MetaData: Metadata has jdbc-type of null yet this is not valid. Ignored 2025-06-17 19:30:56,388 WARN metastore.ObjectStore: Version information not found in metastore. metastore.schema.verification is not enabled so recording the schema version 3.1.0 2025-06-17 19:30:56,388 WARN metastore.ObjectStore: setMetaStoreSchemaVersion called but recording version is disabled: version = 3.1.0, comment = Set by MetaStore root@192.168.245.131 2025-06-17 19:30:57,110 INFO metastore.HiveMetaStore: Added admin role in metastore 2025-06-17 19:30:57,116 INFO metastore.HiveMetaStore: Added public role in metastore 2025-06-17 19:30:57,279 INFO metastore.HiveMetaStore: No user is added in admin role, since config is empty 2025-06-17 19:30:57,779 INFO metastore.RetryingMetaStoreClient: RetryingMetaStoreClient proxy=class org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=root (auth:SIMPLE) retries=1 delay=1 lifetime=0 2025-06-17 19:30:57,891 INFO metastore.HiveMetaStore: 0: get_all_functions 2025-06-17 19:30:57,923 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_all_functions Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. 2025-06-17 19:30:58,126 INFO CliDriver: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. Hive Session ID = 06c85488-8547-4e38-a0d1-5e386cd373f1 2025-06-17 19:30:58,131 INFO SessionState: Hive Session ID = 06c85488-8547-4e38-a0d1-5e386cd373f1 2025-06-17 19:30:58,173 INFO session.SessionState: Created HDFS directory: /tmp/hive/root/06c85488-8547-4e38-a0d1-5e386cd373f1 2025-06-17 19:30:58,195 INFO session.SessionState: Created local directory: /tmp/root/06c85488-8547-4e38-a0d1-5e386cd373f1 2025-06-17 19:30:58,203 INFO session.SessionState: Created HDFS directory: /tmp/hive/root/06c85488-8547-4e38-a0d1-5e386cd373f1/_tmp_space.db 2025-06-17 19:30:58,210 INFO metastore.HiveMetaStore: 1: get_databases: @hive# 2025-06-17 19:30:58,211 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_databases: @hive# 2025-06-17 19:30:58,213 INFO metastore.HiveMetaStore: 1: Opening raw store with implementation class:org.apache.hadoop.hive.metastore.ObjectStore 2025-06-17 19:30:58,217 INFO metastore.ObjectStore: ObjectStore, initialize called 2025-06-17 19:30:58,253 INFO metastore.MetaStoreDirectSql: Using direct SQL, underlying DB is MYSQL 2025-06-17 19:30:58,255 INFO metastore.ObjectStore: Initialized ObjectStore 2025-06-17 19:30:58,272 INFO metastore.HiveMetaStore: 1: get_tables_by_type: db=@hive#db_hive1 pat=.*,type=MATERIALIZED_VIEW 2025-06-17 19:30:58,272 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_tables_by_type: db=@hive#db_hive1 pat=.*,type=MATERIALIZED_VIEW 2025-06-17 19:30:58,305 INFO metastore.HiveMetaStore: 1: get_multi_table : db=db_hive1 tbls= 2025-06-17 19:30:58,305 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=db_hive1 tbls= 2025-06-17 19:30:58,310 INFO metastore.HiveMetaStore: 1: get_tables_by_type: db=@hive#default pat=.*,type=MATERIALIZED_VIEW 2025-06-17 19:30:58,310 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_tables_by_type: db=@hive#default pat=.*,type=MATERIALIZED_VIEW 2025-06-17 19:30:58,327 INFO metastore.HiveMetaStore: 1: get_multi_table : db=default tbls= 2025-06-17 19:30:58,327 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=default tbls= 2025-06-17 19:30:58,327 INFO metastore.HiveMetaStore: 1: get_tables_by_type: db=@hive#itcast_ods pat=.*,type=MATERIALIZED_VIEW 2025-06-17 19:30:58,327 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_tables_by_type: db=@hive#itcast_ods pat=.*,type=MATERIALIZED_VIEW 2025-06-17 19:30:58,332 INFO metastore.HiveMetaStore: 1: get_multi_table : db=itcast_ods tbls= 2025-06-17 19:30:58,332 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_multi_table : db=itcast_ods tbls= 2025-06-17 19:30:58,332 INFO metadata.HiveMaterializedViewsRegistry: Materialized views registry has been initialized hive> SHOW DATABASES; 2025-06-17 19:31:11,344 INFO conf.HiveConf: Using the default value passed in for log id: b6ee71f0-5d43-4149-99c7-808d6c553bb8 2025-06-17 19:31:11,624 INFO ql.Driver: Compiling command(queryId=root_20250617193111_702231fb-8543-45f5-b003-86a49e9c4298): SHOW DATABASES 2025-06-17 19:31:12,626 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager 2025-06-17 19:31:12,673 INFO ql.Driver: Semantic Analysis Completed (retrial = false) 2025-06-17 19:31:12,819 INFO ql.Driver: Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null) 2025-06-17 19:31:12,982 INFO exec.ListSinkOperator: Initializing operator LIST_SINK[0] 2025-06-17 19:31:13,009 INFO ql.Driver: Completed compiling command(queryId=root_20250617193111_702231fb-8543-45f5-b003-86a49e9c4298); Time taken: 1.484 seconds 2025-06-17 19:31:13,011 INFO reexec.ReExecDriver: Execution #1 of query 2025-06-17 19:31:13,011 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager 2025-06-17 19:31:13,012 INFO ql.Driver: Executing command(queryId=root_20250617193111_702231fb-8543-45f5-b003-86a49e9c4298): SHOW DATABASES 2025-06-17 19:31:13,034 INFO ql.Driver: Starting task [Stage-0:DDL] in serial mode 2025-06-17 19:31:13,041 INFO metastore.HiveMetaStore: 0: get_databases: @hive# 2025-06-17 19:31:13,041 INFO HiveMetaStore.audit: ugi=root ip=unknown-ip-addr cmd=get_databases: @hive# 2025-06-17 19:31:13,057 INFO exec.DDLTask: results : 3 2025-06-17 19:31:13,244 INFO ql.Driver: Completed executing command(queryId=root_20250617193111_702231fb-8543-45f5-b003-86a49e9c4298); Time taken: 0.233 seconds OK 2025-06-17 19:31:13,245 INFO ql.Driver: OK 2025-06-17 19:31:13,245 INFO ql.Driver: Concurrency mode is disabled, not creating a lock manager 2025-06-17 19:31:13,276 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir 2025-06-17 19:31:13,381 INFO mapred.FileInputFormat: Total input files to process : 1 2025-06-17 19:31:13,458 INFO exec.ListSinkOperator: RECORDS_OUT_INTERMEDIATE:0, RECORDS_OUT_OPERATOR_LIST_SINK_0:3, db_hive1 default itcast_ods Time taken: 1.725 seconds, Fetched: 3 row(s) 2025-06-17 19:31:13,479 INFO CliDriver: Time taken: 1.725 seconds, Fetched: 3 row(s) 2025-06-17 19:31:13,480 INFO conf.HiveConf: Using the default value passed in for log id: b6ee71f0-5d43-4149-99c7-808d6c553bb8 2025-06-17 19:31:13,480 INFO session.SessionState: Resetting thread name to main hive>
06-18
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值