[jira] [Created] (IMPALA-9779) Unnecessarily reloading file metadata in some DDLs

["Quanlong Huang (Jira)" <ji...@apache.org>]

Quanlong Huang created IMPALA-9779:
--------------------------------------

             Summary: Unnecessarily reloading file metadata in some DDLs
                 Key: IMPALA-9779
                 URL: https://issues.apache.org/jira/browse/IMPALA-9779
             Project: IMPALA
          Issue Type: Improvement
          Components: Catalog
    Affects Versions: Impala 3.4.0, Impala 3.3.0, Impala 3.2.0, Impala 3.1.0, Impala 2.12.0, Impala 3.0, Impala 2.11.0, Impala 2.10.0, Impala 2.9.0, Impala 2.8.0
            Reporter: Quanlong Huang


Some DDLs are not modifying the actual table data. We don't need to reload file meta for them. These DDLs include:
* Compute (incremental) stats
* Drop stats
* Alter table set row format
* Alter table set file format

Code paths of them both call CatalogOpExecutor.bulkAlterPartitions(). The related partitions are marked as "dirty" anyway. Dirty partitions will be dropped and reloaded at the end of CatalogOpExecutor.alterTable(TAlterTableParams, TDdlExecResponse). See the details in HdfsTable.updatePartitionsFromHms().

We can consider not marking related partitions as "dirty" in these DDLs.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)