You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@impala.apache.org by "Noemi Pap-Takacs (Jira)" <ji...@apache.org> on 2023/08/30 11:02:00 UTC

[jira] [Created] (IMPALA-12412) Support partition evolution in OPTIMIZE statement

Noemi Pap-Takacs created IMPALA-12412:
-----------------------------------------

             Summary: Support partition evolution in OPTIMIZE statement
                 Key: IMPALA-12412
                 URL: https://issues.apache.org/jira/browse/IMPALA-12412
             Project: IMPALA
          Issue Type: Sub-task
          Components: Frontend
            Reporter: Noemi Pap-Takacs


OPTIMIZE TABLE statement currently uses INSERT OVERWRITE to rewrite Iceberg tables. Therefore it inherits its limitations as well, such as the inability to rewrite tables with partition evolution.

This change aims to increase the support OPTIMIZE TABLE provides, by making it independent from INSERT statment. After the refactoring, OPTIMIZE TABLEĀ  statement will be able to:
 * rewrite all files in Iceberg tables according to the latest partition spec
 * compact tables with partition evolution

This change also serves as a base for further improvements.
{code:java}
Syntax: OPTIMIZE TABLE <table_name>;{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)