You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hive.apache.org by "Naresh P R (Jira)" <ji...@apache.org> on 2020/11/26 01:53:00 UTC
[jira] [Assigned] (HIVE-24433) AutoCompaction is not getting
triggered for CamelCase Partition Values
[ https://issues.apache.org/jira/browse/HIVE-24433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Naresh P R reassigned HIVE-24433:
---------------------------------
> AutoCompaction is not getting triggered for CamelCase Partition Values
> ----------------------------------------------------------------------
>
> Key: HIVE-24433
> URL: https://issues.apache.org/jira/browse/HIVE-24433
> Project: Hive
> Issue Type: Bug
> Reporter: Naresh P R
> Assignee: Naresh P R
> Priority: Major
>
> partionKey=paritionValue is getting converted into lowerCase in below 2 places.
> [https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L2728]
> https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L2851
> Because of which TXN_COMPONENTS & HIVE_LOCKS tables are not having entries from proper partition values.
> When query completes, the entry moves from TXN_COMPONENTS to COMPLETED_TXN_COMPONENTS. Hive AutoCompaction will not recognize the partition & considers it as invalid partition
> create table abc(name string) partitioned by(city string) stored as orc tblproperties('transactional'='true');
> insert into abc partition(city='Bangalore') values('aaa');
> Example entry in COMPLETED_TXN_COMPONENTS
>
> {noformat}
> +-----------+--------------+--------------------+-------------------+---------------------+-------------+-------------------+
> | CTC_TXNID | CTC_DATABASE | CTC_TABLE | CTC_PARTITION | CTC_TIMESTAMP | CTC_WRITEID | CTC_UPDATE_DELETE |
> +-----------+--------------+--------------------+-------------------+---------------------+-------------+-------------------+
> | 2 | default | abc | city=bangalore | 2020-11-25 09:26:59 | 1 | N |
> +-----------+--------------+--------------------+-------------------+---------------------+-------------+-------------------+
> {noformat}
>
> AutoCompaction fails to get triggered with below error
> 2020-11-25T09:35:10,364 INFO [Thread-9]: compactor.Initiator (Initiator.java:run(98)) - Checking to see if we should compact default.abc.city=bangalore
> 2020-11-25T09:35:10,380 INFO [Thread-9]: compactor.Initiator (Initiator.java:run(155)) - Can't find partition default.compaction_test.city=bhubaneshwar, assuming it has been dropped and moving on
--
This message was sent by Atlassian Jira
(v8.3.4#803005)