[jira] [Created] (IMPALA-13173) Redundant Catalog Update Check in Coordinator.

["Noemi Pap-Takacs (Jira)" <ji...@apache.org>]

Noemi Pap-Takacs created IMPALA-13173:
-----------------------------------------

             Summary: Redundant Catalog Update Check in Coordinator.
                 Key: IMPALA-13173
                 URL: https://issues.apache.org/jira/browse/IMPALA-13173
             Project: IMPALA
          Issue Type: Bug
          Components: Backend, be
            Reporter: Noemi Pap-Takacs
            Assignee: Noemi Pap-Takacs


In case of DML operations, the Coordinator sends an update to the Catalog about the files changed in the table. Before sending the update, we check if any file was created. If no files were added or deleted, we skip the catalog update. See the logic in _'DmlExecState::PrepareCatalogUpdate'._

However, in case of unpartitioned Iceberg tables, the check in _'DmlExecState::PrepareCatalogUpdate'_ always returns true, and updates the Catalog even if no files were added. Currently, this does not cause incorrect behavior because it is double-checked later in client-request-state.cc.

On the other hand, there are cases, when not writing any files does not equal a NO-OP. For example overwriting a table with empty content or an OPTIMIZE TABLE that merges delete files. The Catalog needs to be informed about the changes in such cases.

We should filter NO-OP DMLs correctly in the Coordinator, eliminating false positive and false negative updates as well.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)