You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues-all@impala.apache.org by "Quanlong Huang (Jira)" <ji...@apache.org> on 2020/06/22 00:59:00 UTC
[jira] [Updated] (IMPALA-9778) Refactor HdfsPartition to be
immutable
[ https://issues.apache.org/jira/browse/IMPALA-9778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Quanlong Huang updated IMPALA-9778:
-----------------------------------
Description:
Making HdfsPartition immutable has several benefits:
* HdfsPartition meta can be shared across table versions, which means in the legacy catalog update mode, catalog update can ignore unchanged partitions (IMPALA-3234) and send the update in partition granularity.
* Aborted DDL/DMLs won’t leave partition metadata in illegal states (IMPALA-8406), which requires invalidation to recover.
* LocalCatalog coordinators can cache partition meta using the partition id as the key. When table version updates, only metadata of changed partitions needs to be reloaded (IMPALA-7533).
* In the work of IMPALA-3127, we don't need to assign a catalog version to partitions since the partition ids already identify the partitions.
was:
Making HdfsPartition immutable has several benefits:
* HdfsPartition meta can be shared across table versions, which means in the legacy catalog update mode, catalog update can ignore unchanged partitions (IMPALA-3234) and send the update in partition granularity.
* Aborted DDL/DMLs won’t leave partition metadata in illegal states, which requires invalidation to recover.
* LocalCatalog coordinators can cache partition meta using the partition id as the key. When table version updates, only metadata of changed partitions needs to be reloaded (IMPALA-7533).
* In the work of IMPALA-3127, we don't need to assign a catalog version to partitions since the partition ids already identify the partitions.
> Refactor HdfsPartition to be immutable
> --------------------------------------
>
> Key: IMPALA-9778
> URL: https://issues.apache.org/jira/browse/IMPALA-9778
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Quanlong Huang
> Assignee: Quanlong Huang
> Priority: Critical
> Fix For: Impala 4.0
>
>
> Making HdfsPartition immutable has several benefits:
> * HdfsPartition meta can be shared across table versions, which means in the legacy catalog update mode, catalog update can ignore unchanged partitions (IMPALA-3234) and send the update in partition granularity.
> * Aborted DDL/DMLs won’t leave partition metadata in illegal states (IMPALA-8406), which requires invalidation to recover.
> * LocalCatalog coordinators can cache partition meta using the partition id as the key. When table version updates, only metadata of changed partitions needs to be reloaded (IMPALA-7533).
> * In the work of IMPALA-3127, we don't need to assign a catalog version to partitions since the partition ids already identify the partitions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-all-unsubscribe@impala.apache.org
For additional commands, e-mail: issues-all-help@impala.apache.org