[jira] [Work started] (IMPALA-3127) Decouple partitions from tables

["Quanlong Huang (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IMPALA-3127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Work on IMPALA-3127 started by Quanlong Huang.
----------------------------------------------
> Decouple partitions from tables
> -------------------------------
>
>                 Key: IMPALA-3127
>                 URL: https://issues.apache.org/jira/browse/IMPALA-3127
>             Project: IMPALA
>          Issue Type: Improvement
>          Components: Catalog
>    Affects Versions: Impala 2.2.4
>            Reporter: Dimitris Tsirogiannis
>            Assignee: Quanlong Huang
>            Priority: Major
>              Labels: catalog-server, performance
>
> Currently, partitions are tightly integrated into the HdfsTable objects, making incremental metadata updates difficult to perform. Furthermore, the catalog transmits entire table metadata even when only few partitions change, introducing significant latencies, wasting network bandwidth and CPU cycles while updating table metadata at the receiving impalads. As a first step, we should decouple partitions from tables and add them as a separate level in the hierarchy of catalog entities (server-db-table-partition). Subsequently, the catalog should transmit only entities that have changed after DDL/DML statements.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org