[jira] [Work logged] (HIVE-25936) ValidWriteIdList & table id are sometimes missing when requesting partitions by name via HS2

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-25936?focusedWorklogId=722968&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-722968 ]

ASF GitHub Bot logged work on HIVE-25936:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 08/Feb/22 16:26
            Start Date: 08/Feb/22 16:26
    Worklog Time Spent: 10m 
      Work Description: zabetak opened a new pull request #3007:
URL: https://github.com/apache/hive/pull/3007


   ### What changes were proposed in this pull request?
   1. Refactor code handling remaining batches to use `GetPartitionsByNamesRequest` and `getPartitionsByNames(GetPartitionsByNamesRequest,Table)` method
   2. Use method parameter to determine if column stats must be fetched instead of harcoding the value to false when sending the request to HMS.
   3. Remove redundant exception wrapping
   4. Unify batch creation/execution logic using Guava's Lists#partition method.
   5. Drop unused requests from TestHiveMetaStoreClientApiArgumentsChecker
   6. Add tests for getPartitionsByName with non-empty partition name lists
   
   ### Why are the changes needed?
   1. The write id list & table id are not set when the number of names is not an exact multiple of the batch size which can have an impact in the consistency of the cache.
   2. Column stats for partitions are always missing since the request was hardcoded to false.
   3. Improve code readability and remove duplicated fragments.
   
   ### Does this PR introduce _any_ user-facing change?
   Fixes the bugs.
   
   ### How was this patch tested?
   `mvn test -Dtest=TestHiveMetaStoreClientApiArgumentsChecker`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: gitbox-unsubscribe@hive.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 722968)
    Remaining Estimate: 0h
            Time Spent: 10m

> ValidWriteIdList & table id are sometimes missing when requesting partitions by name via HS2
> --------------------------------------------------------------------------------------------
>
>                 Key: HIVE-25936
>                 URL: https://issues.apache.org/jira/browse/HIVE-25936
>             Project: Hive
>          Issue Type: Sub-task
>            Reporter: Stamatis Zampetakis
>            Assignee: Stamatis Zampetakis
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> According to HIVE-24743 the table id and {{ValidWriteIdList}} are important for keeping HMS remote metadata cache consistent. Although HIVE-24743 attempted to pass the write id list and table id in every call to HMS it failed to do so completely. For those partitions not handled in the batch logic, the [metastore call|https://github.com/apache/hive/blob/4b7a948e45fd88372fef573be321cda40d189cc7/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java#L4161] in {{Hive#getPartitionsByName}} method does not pass the table id and write id list.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)