[jira] [Commented] (HIVE-22654) ACID: Allow TxnHandler::checkLock to chunk partitions by 1000

["Peter Vary (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HIVE-22654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17008904#comment-17008904 ] 

Peter Vary commented on HIVE-22654:
-----------------------------------

[~rajesh.balamohan]: Any specific reason behind not using "buildQueryWithINClause" for this case as it is done in most cases in TxnHandler? (I personally prefer the batched way used in MetastoreDirectSql, but I really do not like to use different approach for solving the same issue in the same class)

Thanks,
Peter

> ACID: Allow TxnHandler::checkLock to chunk partitions by 1000 
> --------------------------------------------------------------
>
>                 Key: HIVE-22654
>                 URL: https://issues.apache.org/jira/browse/HIVE-22654
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Gopal Vijayaraghavan
>            Assignee: Rajesh Balamohan
>            Priority: Major
>         Attachments: HIVE-22654.1.patch, HIVE-22654.2.patch
>
>
> The following loop can end up with too many entries within the IN clause throwing.
> https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L4428
> {code:java}
>         // If any of the partition requests are null, then I need to pull all
>         // partition locks for this table.
>         sawNull = false;
>         strings.clear();
>         for (LockInfo info : locksBeingChecked) {
>           if (info.partition == null) {
>             sawNull = true;
>             break;
>           } else {
>             strings.add(info.partition);
>           }
>         } 
> {code}
> {code}
> 2019-12-17T04:28:57,991 ERROR [pool-8-thread-143]: metastore.RetryingHMSHandler (RetryingHMSHandler.java:invokeInternal(201)) - MetaException(message:Unable to update transaction database java.sql.SQLSyntaxErrorException: ORA-01795: maximum number of expressions in a list is 1000
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)