[Impala-ASF-CR] IMPALA-9778: Refactor HdfsPartition to be immutable

["Quanlong Huang (Code Review)" <ge...@cloudera.org>]

Quanlong Huang has uploaded a new patch set (#9). ( http://gerrit.cloudera.org:8080/15985 )

Change subject: IMPALA-9778: Refactor HdfsPartition to be immutable
......................................................................

IMPALA-9778: Refactor HdfsPartition to be immutable

Making HdfsPartition immutable has several benefits:
 - HdfsPartition can be shared across table versions. In full catalog
   update mode, catalog update can ignore unchanged partitions
   (IMPALA-3234) and send the update in partition granularity.
 - Aborted DDL/DMLs won’t leave partition metadata in a bad shape, which
   usually requires invalidation to recover.
 - Fetch-on-demand coordinators can cache partition meta using the
   partition id as the key. When table version updates, only metadata of
   changed partitions needs to be reloaded (IMPALA-7533).
 - In the work of decoupling partitions from tables (IMPALA-3127), we
   don't need to assign a catalog version to partitions since the
   partition ids already identify the partitions.

Changes:
 - Make all fields of HdfsPartition immutable (final). Move
   HdfsPartition constructor logics and all its update methods into
   HdfsPartition.Builder.
 - Move the “dirty” marker of HdfsPartition into a map of HdfsTable. It
   maps from the old partition id to the in-progress partition builder.
 - Original code paths that modifying HdfsPartition are changed to
   modify and pass through the HdfsPartition.Builder. The HdfsPartition
   is only replaced at the end of the modifications.

Tests:
 - Run CORE tests

Change-Id: Ib52e5810d01d5e0c910daacb9c98977426d3914c
---
M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java
M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java
M fe/src/main/java/org/apache/impala/catalog/FileMetadataLoader.java
M fe/src/main/java/org/apache/impala/catalog/HdfsPartition.java
M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java
M fe/src/main/java/org/apache/impala/catalog/PartitionStatsUtil.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/main/java/org/apache/impala/util/HdfsCachingUtil.java
M fe/src/test/java/org/apache/impala/catalog/CatalogObjectToFromThriftTest.java
M fe/src/test/java/org/apache/impala/catalog/CatalogTest.java
11 files changed, 754 insertions(+), 491 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/15985/9
-- 
To view, visit http://gerrit.cloudera.org:8080/15985
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib52e5810d01d5e0c910daacb9c98977426d3914c
Gerrit-Change-Number: 15985
Gerrit-PatchSet: 9
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>