You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2022/06/28 13:26:00 UTC
[jira] [Created] (IMPALA-11402) Add flag to limit the number of partitions in a single getPartialCatalogObject request
| ![](cid:jira-generated-image-avatar-f6e60285-364c-484f-95d9-acb56f65dbe5) |
[Quanlong
Huang](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=stigahuang)
**created** an issue
---|---
|
---
| [IMPALA](https://issues.apache.org/jira/browse/IMPALA) /
[![Improvement](cid:jira-generated-image-avatar-
ac1e40df-4800-4d16-b10d-89e26c1f6538)](https://issues.apache.org/jira/browse/IMPALA-11402)
[IMPALA-11402](https://issues.apache.org/jira/browse/IMPALA-11402)
---
[Add flag to limit the number of partitions in a single
getPartialCatalogObject
request](https://issues.apache.org/jira/browse/IMPALA-11402)
| Issue Type: | ![Improvement](cid:jira-generated-image-avatar-
ac1e40df-4800-4d16-b10d-89e26c1f6538) Improvement
---|---
Assignee: | [Quanlong
Huang](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=stigahuang)
Components: | Catalog
Created: | 28/Jun/22 13:25
Priority: | ![Major](cid:jira-generated-image-static-major-
dfdb0af7-1b6e-492c-9c66-971cd7e8a55b) Major
Reporter: | [Quanlong
Huang](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=stigahuang)
|
The response size of getPartialCatalogObject depends on the number of
partitions in the request. Even with the optimization of
[~~IMPALA-7501~~](https://issues.apache.org/jira/browse/IMPALA-7501 "Slim down
metastore Partition objects in LocalCatalog cache"), the response size could
still exceeds the 2GB byte array limit if requesting all partitions of a huge
table.
We should add flag to limit the number of partitions in a single
getPartialCatalogObject request. When more partitions are required, fetch them
in different batches.
---
| | [ ![Add Comment](cid:jira-generated-image-static-comment-icon-
da64aa65-0e6a-4ef3-b1e0-8377f795792f)
](https://issues.apache.org/jira/browse/IMPALA-11402#add-comment "Add
Comment") | [Add
Comment](https://issues.apache.org/jira/browse/IMPALA-11402#add-comment "Add
Comment")
---|---
| This message was sent by Atlassian Jira (v8.20.10#820010-sha1:ace47f9) | |
![Atlassian logo](https://issues.apache.org/jira/images/mail/atlassian-email-
logo.png)
---