You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "vishnuprakaz (via GitHub)" <gi...@apache.org> on 2026/07/30 10:25:55 UTC
[PR] Parquet: Guard ordered predicates on variant/nested types in metrics row group filter [iceberg]
vishnuprakaz opened a new pull request, #17431:
URL: https://github.com/apache/iceberg/pull/17431
`ParquetMetricsRowGroupFilter` defers variant and nested types to post-scan evaluation in
`eq`/`in`/`notNull` #14081, #14279 those columns have no primitive row-group stats keyed
by the field id, so they can't be evaluated against stats. The ordered predicates
`lt`/`ltEq`/`gt`/`gtEq` were missed: a variant/nested reference falls through to
`valueCount == null -> ROWS_CANNOT_MATCH` and the row group is skipped, which would
incorrectly drop matching rows.
This applies the same guard to the four ordered comparisons, mirroring #14279.
Tested with `testVariantFieldLt/LtEq/Gt/GtEq`, which mirror the existing variant `eq`/`in`
tests.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Parquet: Guard ordered predicates on variant/nested types in metrics row group filter [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #17431: Parquet: Guard ordered predicates on variant/nested types in metrics row group filter
URL: https://github.com/apache/iceberg/pull/17431
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Parquet: Guard ordered predicates on variant/nested types in metrics row group filter [iceberg]
Posted by "nssalian (via GitHub)" <gi...@apache.org>.
nssalian commented on code in PR #17431:
URL: https://github.com/apache/iceberg/pull/17431#discussion_r3793921196
##########
data/src/test/java/org/apache/iceberg/data/TestMetricsRowGroupFilter.java:
##########
@@ -1259,6 +1259,62 @@ public void testVariantFieldIn() throws IOException {
.isTrue();
}
+ @TestTemplate
+ public void testVariantFieldLt() throws IOException {
+ assumeThat(format).isEqualTo(FileFormat.PARQUET);
Review Comment:
assumeThat(format).isEqualTo(FileFormat.PARQUET) seems redundant here and other places.
##########
parquet/src/main/java/org/apache/iceberg/parquet/ParquetMetricsRowGroupFilter.java:
##########
@@ -237,6 +237,13 @@ public <T> Boolean notNaN(BoundReference<T> ref) {
public <T> Boolean lt(BoundReference<T> ref, Literal<T> lit) {
int id = ref.fieldId();
+ // Leave all nested column type and variant type filters to be
Review Comment:
This could be made into a helper something like isNestedOrVariant(int id) since it is repeated for notNull, eq, in, lt, ltEq, gt, gtEq
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Parquet: Guard ordered predicates on variant/nested types in metrics row group filter [iceberg]
Posted by "vishnuprakaz (via GitHub)" <gi...@apache.org>.
vishnuprakaz commented on PR #17431:
URL: https://github.com/apache/iceberg/pull/17431#issuecomment-5315650749
Thanks for the review @nssalian !
Rebased on main noticed an existing helper from #17377, and dropped the `assumeThat` since `shouldReadVariant` already has it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Parquet: Guard ordered predicates on variant/nested types in metrics row group filter [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #17431:
URL: https://github.com/apache/iceberg/pull/17431#issuecomment-6257067710
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Parquet: Guard ordered predicates on variant/nested types in metrics row group filter [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #17431:
URL: https://github.com/apache/iceberg/pull/17431#issuecomment-6103790510
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org