[jira] [Created] (IMPALA-15041) Push down Deletion Vector evaluation to the scanners

["Zoltán Borók-Nagy (Jira)" <ji...@apache.org>]

Zoltán Borók-Nagy created IMPALA-15041:
------------------------------------------

             Summary: Push down Deletion Vector evaluation to the scanners
                 Key: IMPALA-15041
                 URL: https://issues.apache.org/jira/browse/IMPALA-15041
             Project: IMPALA
          Issue Type: Improvement
            Reporter: Zoltán Borók-Nagy


This is an alternative to IMPALA-15040.

To process Iceberg V3 tables we execute the following plan:
{noformat}
       UNION ALL
      /        \
     /          \
    /            \
   SCAN all  IcebergDeleteNode
   datafiles  /      \
   without   /        \
   deletes  SCAN      SCAN
            datafiles deletes
            with
            deletes
{noformat}

IcebergDeleteNode deals with position delete records and Deletion Vectors as well. Since position delete files are being deprecated by Iceberg V3, in most cases "SCAN deletes" will be empty, and IcebergDeleteNode only need to deal with Deletion Vectors.

DV evaluation could be pushed down to "SCAN datafiles with deletes". The would be especially beneficial in the context of late materialization, in which case we can skip materializing column values of inactive records.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)