11.深入理解juc-并发容器-COW类

最新推荐文章于 2023-10-07 23:37:44 发布

原创最新推荐文章于 2023-10-07 23:37:44 发布 · 3.1k 阅读

0 ·

CC 4.0 BY-SA版权

java并发专栏收录该内容

12 篇文章

订阅专栏

本文详细记录了Hive中的多个常见问题及其解决方案，包括倾斜表中创建索引失败、数据库删除异常、ORC文件大小写敏感、内存溢出、查询性能问题、SASL配置错误、统计收集失败、Hive版本兼容性问题、表名大小写冲突、查询分区失败、JDO与PostgreSQL不兼容、Hive客户端缓存问题、Tez版本更新、错误的统计计算、类型转换错误、SQL语法错误、并行处理问题、并发控制问题、错误的分区处理、数据源不兼容、性能优化建议、异常处理机制改进、以及特定数据库版本的兼容性调整。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Bug

[HIVE-5631] - Index creation on a skew table fails
- 描述：在倾斜表中创建index失败。
[HIVE-5664] - Drop cascade database fails when the db has any tables with indexes
- 描述：删除数据库时，如果该数据库中的表建了索引，会报出表找不到异常。因为删除表的时候获取的是正常表和index表列表，但是删除正常表的同时也会删除对应的index表，删除index表的时候会因为找不到index表而报错。
[HIVE-6198] - ORC file and struct column names are case sensitive
- 描述：ORC 文件和结构列大小写敏感。
[HIVE-6468] - HS2 & Metastore using SASL out of memory error when curl sends a get request
[HIVE-7270] - SerDe Properties are not considered by show create table Command
[HIVE-8099] - IN operator for partition column fails when the partition column type is DATE
0.13.0
[HIVE-8295] - Add batch retrieve partition objects for metastore direct sql
[HIVE-8374] - schematool fails on Postgres versions < 9.2
- 描述：schematool命令在 Postgres 版本<9.2会失败。
[HIVE-8485] - HMS on Oracle incompatibility
- 描述：参数 hive.metastore.orm.retrieveMapNullsAsEmptyStrings默认值为false。对于oracle等把空转化成null的数据库需要把该值设为true。
[HIVE-8706] - Table statistic collection on counter failed due to table name character case.
- 描述：表名大小写导致表统计失败。hive会忽略表名的大小写，在hive的metastore中会转化成小写，但是Counters/TezCounters是大小写敏感的，当Hive collection table使用Counters，而hive使用table name 作为counter组名时这可能导致表的统计汇总失败。
- 影响版本：0.13.1
[HIVE-8715] - Hive 14 upgrade scripts can fail for statistics if database was created using auto-create
- 描述：当数据库是通过auto-create，hive不能使用schemeTool升级
- 影响版本：
[HIVE-8739] - handle Derby and Oracle errors with joins and filters in Direct SQL in a invalid-DB-specific path
- 描述：
[HIVE-8784] - Querying partition does not work with JDO enabled against PostgreSQL
- 描述：通过PostgreSQL 使用 JDO 查询一个分区会失败。
- 影响版本： 0.14.1
[HIVE-8805] - CBO skipped due to SemanticException: Line 0:-1 Both left and right aliases encountered in JOIN 'avg_cs_ext_discount_amt'
[HIVE-8807] - Obsolete default values in webhcat-default.xml
[HIVE-8811] - Dynamic partition pruning can result in NPE during query compilation
[HIVE-8827] - Remove SSLv2Hello from list of disabled protocols
- 描述：
[HIVE-8830] - hcatalog process don't exit because of non daemon thread
- 描述：HiveClientCache 清理线程而不是进程，会导致hcatalogclient挂起。
- 影响版本： 0.14.0
[HIVE-8845] - Switch to Tez 0.5.2
[HIVE-8866] - Vectorization on partitioned table throws ArrayIndexOutOfBoundsException when partitions are not of same #of columns
[HIVE-8870] - errors when selecting a struct field within an array from ORC based tables
0.13.0
[HIVE-8873] - Switch to calcite 0.9.2
[HIVE-8876] - incorrect upgrade script for Oracle (13->14)
- 描述：升级脚本对oracle13-14不适用。
[HIVE-8880] - non-synchronized access to split list in OrcInputFormat
[HIVE-8886] - Some Vectorized String CONCAT expressions result in runtime error Vectorization: Unsuported vector output type: StringGroup
[HIVE-8888] - Mapjoin with LateralViewJoin generates wrong plan in Tez
[HIVE-8947] - HIVE-8876 also affects Postgres < 9.2
- 描述：升级hive时的脚本对postgre数据库低于9.2的版本是错误的。
- 影响版本： 0.14.0
[HIVE-8966] - Delta files created by hive hcatalog streaming cannot be compacted
- ？
[HIVE-9003] - Vectorized IF expr broken for the scalar and scalar case
- 描述：if表达式被分成scalar和scalar case？
- 影响版本：0.14.0
[HIVE-9025] - join38.q (without map join) produces incorrect result when testing with multiple reducers
- 描述：当设置mapred.reduce.tasks大于1时候，做join会产生错误的结果。
- 影响版本： 0.14.0
[HIVE-9038] - Join tests fail on Tez
- 描述：
[HIVE-9051] - TezJobMonitor in-place updates logs too often to logfile
- 描述：0.15版本UI监控日志太频繁，改成与老版本一样。
- 影响版本：0.15
[HIVE-9053] - select constant in union all followed by group by gives wrong result
0.13.0
[HIVE-9067] - OrcFileMergeOperator may create merge file that does not match properties of input files
0.14.0
[HIVE-9090] - Rename "Tez File Merge Work" to smaller name
- 描述：这是一个小改动， Tez File Merge Work这个名字过长改成更短。
[HIVE-9108] - Fix for HIVE-8735 is incorrect (stats with long paths)
[HIVE-9111] - Potential NPE in OrcStruct for list and map types
- 描述：对 OrcStruct中list，map的检查中没有做null判断，可能导致空指针异常。
[HIVE-9112] - Query may generate different results depending on the number of reducers
- 描述：设置不同的reducer可能导致不同的结果，甚至出错。
[HIVE-9114] - union all query in cbo test has undefined ordering
- 描述：union all在其他平台上测试结果不一致
[HIVE-9126] - Backport HIVE-8827 (Remove SSLv2Hello from list of disabled protocols) to 0.14 branch
[HIVE-9141] - HiveOnTez: mix of union all, distinct, group by generates error
- 描述：当设置 hive.execution.engine为tez时，一条sql语句中使用union all，distinct和group by会产生错误。
[HIVE-9155] - HIVE_LOCKS uses int instead of bigint hive-txn-schema-0.14.0.mssql.sql
- 描述： HL_LAST_HEARTBEAT and HL_ACQUIRED_AT使用了bigint来代替int以容纳更大的数据。
[HIVE-9162] - stats19 test is environment-dependant
[HIVE-9166] - Place an upper bound for SARG CNF conversion
[HIVE-9168] - Vectorized Coalesce for strings is broken
[HIVE-9205] - Change default tez install directory to use /tmp instead of /user and create the directory if it does not exist
- 描述：如果用户使用tez 启动hive shell，会在/usr/下面建目录，权限不足会抛出异常。现在使用/tmp目录，以满足权限问题。
[HIVE-9234] - HiveServer2 leaks FileSystem objects in FileSystem.CACHE
0.12.0
[HIVE-9249] - java.lang.ClassCastException: org.apache.hadoop.hive.serde2.io.HiveVarcharWritable cannot be cast to org.apache.hadoop.hive.common.type.HiveVarchar when joining tables
- 描述：类型转化错误，HiveVarcharWritable 不能转化成HiveVarchar。
- 影响版本： 0.14.0
[HIVE-9278] - Cached expression feature broken in one case
- 描述： hive.cache.expr.evaluation设置为true时，查询结果无输出，并且如果udf函数嵌套其他udf函数会导致错误结果。
[HIVE-9317] - move Microsoft copyright to NOTICE file
- 版权问题
[HIVE-9359] - Export of a large table causes OOM in Metastore and Client
- 描述：导出大表导致Metastore 和client OOM问题。
[HIVE-9361] - Intermittent NPE in SessionHiveMetaStoreClient.alterTempTable
[HIVE-9390] - Enhance retry logic wrt DB access in TxnHandler
[HIVE-9401] - Backport: Fastpath for limited fetches from unpartitioned tables
0.14.0
[HIVE-9404] - NPE in org.apache.hadoop.hive.metastore.txn.TxnHandler.determineDatabaseProduct()
- 描述：org.apache.hadoop.hive.metastore.txn.TxnHandler.determineDatabaseProduct()方法抛出空指针异常。
- 影响版本： 0.15.0
[HIVE-9436] - RetryingMetaStoreClient does not retry JDOExceptions
- 描述：RetryingMetaStoreClient 代码一个正则匹配错误，有些情况会匹配不到导致不会抛出异常。
- 影响版本：0.14.0, 0.13.1
[HIVE-9473] - sql std auth should disallow built-in udfs that allow any java methods to be called
- 描述：标准sql授权不应该允许能调用任何java方法的内置udfs运行。
[HIVE-9514] - schematool is broken in hive 1.0.0
- 描述：使用bin/schematool时产生的错误
[HIVE-9632] - inconsistent results between year(), month(), day(), and the actual values in formulas
- 描述：时间的返回不一致问题。例如：2015-01-09这个时间，year(2015-01-09) month(2015-01-09) day(2015-01-09)会返回2015,1,9year(2015-01-09)*10000会返回20150000,month(2015-01-09)*100会返回100,day(2015-01-09)会返回9但是 year(2015-01-09)*10000+month(2015-01-09*100+day(2015-01-09) 会返回 20353515
- 影响版本：0.14.0
[HIVE-9684] - Incorrect disk range computation in ORC because of optional stream kind