[Impala-ASF-CR] IMPALA-10453: Support file pruning via runtime filters on Iceberg

["Tamas Mate (Code Review)" <ge...@cloudera.org>]

Tamas Mate has posted comments on this change. ( http://gerrit.cloudera.org:8080/18531 )

Change subject: IMPALA-10453: Support file pruning via runtime filters on Iceberg
......................................................................


Patch Set 4:

(2 comments)

Thank you for the reviews Csaba/Zoltan.
Updated the change, modified the filter logic a bit, it is now skipping the filters that would check slots that were not written during template tuple creation.

http://gerrit.cloudera.org:8080/#/c/18531/4/fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java
File fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java:

http://gerrit.cloudera.org:8080/#/c/18531/4/fe/src/main/java/org/apache/impala/catalog/FeIcebergTable.java@268
PS4, Line 268: partField.getTransformType() == TIcebergPartitionTransformType.IDENTITY)
> We cannot create template tuple from the other partition transforms. E.g. i
Thank you for the additional explanation.
Yeah, that makes sense, I was thinking about checking the result of the CreateTemplateTuple but keeping the logic in FE seemed more descriptive. At this point it makes more sense to handle in the BE side.


http://gerrit.cloudera.org:8080/#/c/18531/5/tests/query_test/test_iceberg.py
File tests/query_test/test_iceberg.py:

http://gerrit.cloudera.org:8080/#/c/18531/5/tests/query_test/test_iceberg.py@617
PS5, Line 617:     self.execute_query("SET RUNTIME_FILTER_MODE=OFF")
> Why are we disabling it in this case? Would it make the tests meaningless? 
During an earlier verify job the runtime filter rejected a file in this test, this test checks the number of rowgroups read, which could become flaky without disabling it here.



-- 
To view, visit http://gerrit.cloudera.org:8080/18531
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I7762e1238bdf236b85d2728881a402a2bb41f36a
Gerrit-Change-Number: 18531
Gerrit-PatchSet: 4
Gerrit-Owner: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Gergely Fürnstáhl <gf...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Tamas Mate <tm...@apache.org>
Gerrit-Reviewer: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 Jul 2022 16:00:39 +0000
Gerrit-HasComments: Yes