[jira] [Created] (IMPALA-12599) Support equality delete files that don't contain the partition values

["Gabor Kaszab (Jira)" <ji...@apache.org>]

Gabor Kaszab created IMPALA-12599:
-------------------------------------

             Summary: Support equality delete files that don't contain the partition values
                 Key: IMPALA-12599
                 URL: https://issues.apache.org/jira/browse/IMPALA-12599
             Project: IMPALA
          Issue Type: Sub-task
          Components: Frontend
            Reporter: Gabor Kaszab


When you write equality delete files with Flink the partition columns have to also be part of the primary key. As a result the partition values will be added into the equality delete files. However, the Iceberg spec is more flexible than that and it's also a valid case when the partition values aren't written into the eq-delete files.

To be able to read such tables Impala should also check if the partition spec and the partition values match between the data and delete files when applying the delete rows. This could be achieved by adding a virtual columns and conjuncts for the partition spec IDs and also for the partition values. These virtual columns already exist, but have to be added to the scan nodes, and the conjuncts have to be created for the ANTI JOIN node.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)