[jira] [Work logged] (HIVE-24343) Table partition operations (create, drop, select) fail when the number of partitions is greater than 32767 (signed int)

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HIVE-24343?focusedWorklogId=506829&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506829 ]

ASF GitHub Bot logged work on HIVE-24343:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 03/Nov/20 13:49
            Start Date: 03/Nov/20 13:49
    Worklog Time Spent: 10m 
      Work Description: vnhive opened a new pull request #1640:
URL: https://github.com/apache/hive/pull/1640


   …en the number of partitions is greater than 32767 (signed int)
   
   <!--
   Thanks for sending a pull request!  Here are some tips for you:
     1. If this is your first time, please read our contributor guidelines: https://cwiki.apache.org/confluence/display/Hive/HowToContribute
     2. Ensure that you have created an issue on the Hive project JIRA: https://issues.apache.org/jira/projects/HIVE/summary
     3. Ensure you have added or run the appropriate tests for your PR: 
     4. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP]HIVE-XXXXX:  Your PR title ...'.
     5. Be sure to keep the PR description updated to reflect all changes.
     6. Please write your PR title to summarize what this PR proposes.
     7. If possible, provide a concise example to reproduce the issue for a faster review.
   
   -->
   
   ### What changes were proposed in this pull request?
   The implementation of getPartitionIdsViaSqlFilter should be extended to account for the limit of 32767
   on the number of parameters to the underlying JDBC driver.
   
   
   ### Why are the changes needed?
   The table partition operations - create, drop, select access the underlying relation database using JDO,
   which internally routes the operations through the JDBC driver. Most of the underlying JDBC driver
   implementations place a limit on the number of parameters that can be passed through a statement
   implementation. These limits should be taken into account when querying the underlying metastore.
   
   
   
   ### Does this PR introduce _any_ user-facing change?
   The patch will ensure that if more than 32767 operations are being operated on (added, dropped or queried)
   at the same time, it will not fail.
   
   
   ### How was this patch tested?
   1. The patch was verified by attaching the debugger and verifying the code flow
   2. The existing metastore tests passed.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

            Worklog Id:     (was: 506829)
    Remaining Estimate: 0h
            Time Spent: 10m

> Table partition operations (create, drop, select) fail when the number of partitions is greater than 32767 (signed int)
> -----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-24343
>                 URL: https://issues.apache.org/jira/browse/HIVE-24343
>             Project: Hive
>          Issue Type: Bug
>          Components: Metastore
>            Reporter: Narayanan Venkateswaran
>            Assignee: Narayanan Venkateswaran
>            Priority: Minor
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> The table partition operations - create, drop, select access the underlying relation database using JDO, which internally routes the operations through the JDBC driver. Most of the underlying JDBC driver implementations place a limit on the number of parameters that can be passed through a statement implementation. The limitations are as follows,
> postgreSQL - 32767
> (https://www.postgresql.org/message-id/16832734.post%40talk.nabble.com)
> MySQL - 32767 - 2 Byte Integer - num of params
> (https://dev.mysql.com/doc/internals/en/com-stmt-prepare-response.html#packet-COM_STMT_PREPARE_OK)
> Oracle - 32767 -
> https://www.jooq.org/doc/3.12/manual/sql-building/dsl-context/custom-settings/settings-inline-threshold/
> Derby - 32767 - stored in an unsinged integer - Note the Prepared
> Statement implementation here -
> [https://svn.apache.org/repos/asf/db/derby/code/branches/10.1/java/client/org/apache/derby/client/am/PreparedStatement.java]
>  
> These limits should be taken into account when querying the underlying metastore.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)