You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2022/06/28 13:26:00 UTC

[jira] [Created] (IMPALA-11402) Add flag to limit the number of partitions in a single getPartialCatalogObject request

|  ![](cid:jira-generated-image-avatar-4aa426ff-5ed4-44ae-88b8-1235c8e34dd1) |
[Quanlong
Huang](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=stigahuang)
**created** an issue  
---|---  
|  
---  
|  [IMPALA](https://issues.apache.org/jira/browse/IMPALA) /
[![Improvement](cid:jira-generated-image-
avatar-23752a48-e010-4494-9b8e-7ca4351b8b1f)](https://issues.apache.org/jira/browse/IMPALA-11402)
[IMPALA-11402](https://issues.apache.org/jira/browse/IMPALA-11402)  
---  
[Add flag to limit the number of partitions in a single
getPartialCatalogObject
request](https://issues.apache.org/jira/browse/IMPALA-11402)  
| Issue Type: |  ![Improvement](cid:jira-generated-image-
avatar-23752a48-e010-4494-9b8e-7ca4351b8b1f) Improvement  
---|---  
Assignee: |  [Quanlong
Huang](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=stigahuang)  
Components: |  Catalog  
Created: |  28/Jun/22 13:25  
Priority: |  ![Major](cid:jira-generated-image-static-
major-56a3b6c1-a371-48b3-afaa-ecde5f61ecab) Major  
Reporter: |  [Quanlong
Huang](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=stigahuang)  
|

The response size of getPartialCatalogObject depends on the number of
partitions in the request. Even with the optimization of
[~~IMPALA-7501~~](https://issues.apache.org/jira/browse/IMPALA-7501 "Slim down
metastore Partition objects in LocalCatalog cache"), the response size could
still exceeds the 2GB byte array limit if requesting all partitions of a huge
table.

We should add flag to limit the number of partitions in a single
getPartialCatalogObject request. When more partitions are required, fetch them
in different batches.  
  
---  
|  |  [ ![Add Comment](cid:jira-generated-image-static-comment-
icon-89e82ba6-0566-49b6-9d51-3c77a5b8c515)
](https://issues.apache.org/jira/browse/IMPALA-11402#add-comment "Add
Comment") |  [Add
Comment](https://issues.apache.org/jira/browse/IMPALA-11402#add-comment "Add
Comment")  
---|---  
  
|  This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9) |  |
![Atlassian logo](https://issues.apache.org/jira/images/mail/atlassian-email-
logo.png)  
---