You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "bharath v (JIRA)" <ji...@apache.org> on 2019/01/29 23:54:00 UTC
[jira] [Commented] (IMPALA-7034) Increase scalability of metadata
handling
[ https://issues.apache.org/jira/browse/IMPALA-7034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16755488#comment-16755488 ]
bharath v commented on IMPALA-7034:
-----------------------------------
[~thundergun] This should be substantially improved with IMPALA-7127 and IMPALA-2649. Can we resolve this?
> Increase scalability of metadata handling
> -----------------------------------------
>
> Key: IMPALA-7034
> URL: https://issues.apache.org/jira/browse/IMPALA-7034
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Affects Versions: Impala 2.13.0
> Reporter: Vincent Tran
> Priority: Major
>
> Currently the practical limit for catalog topic update is in the neighborhood of 4GB - the fundamental limit of max thrift message size. This is an architectural limitation and not a resource limitation
> Larger enterprise clusters with high file counts can easily surpass this with normal usage.
> The high level ask here is for a more scalable implementation for metadata handling. The amount metadata that a cluster can handle should be proportional to the amount of hardware resource that an user is willing to allocate to it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org