You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "lookingUpAtTheSky (via GitHub)" <gi...@apache.org> on 2025/08/13 09:12:50 UTC
[I] Does Spark partition transform expression support pushdown? [iceberg]
lookingUpAtTheSky opened a new issue, #13798:
URL: https://github.com/apache/iceberg/issues/13798
### Query engine
Spark
### Question
Does Spark partition transform expression support pushdown?
Such as, select * from iceberg_table where year(ts) = 2025, the condition 'year(ts) = 2025' can pushdown to iceberg to do file pruning?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Does Spark partition transform expression support pushdown? [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on issue #13798:
URL: https://github.com/apache/iceberg/issues/13798#issuecomment-3881478433
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Does Spark partition transform expression support pushdown? [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on issue #13798:
URL: https://github.com/apache/iceberg/issues/13798#issuecomment-3188026658
I think it's been always supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Does Spark partition transform expression support pushdown? [iceberg]
Posted by "manuzhang (via GitHub)" <gi...@apache.org>.
manuzhang commented on issue #13798:
URL: https://github.com/apache/iceberg/issues/13798#issuecomment-3187833208
This is pruned at manifests level during scan planning. Only files matching `year(ts)=2025` are scanned.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Does Spark partition transform expression support pushdown? [iceberg]
Posted by "lookingUpAtTheSky (via GitHub)" <gi...@apache.org>.
lookingUpAtTheSky commented on issue #13798:
URL: https://github.com/apache/iceberg/issues/13798#issuecomment-3188009358
From which version is it supported, or has it been supported all along?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [I] Does Spark partition transform expression support pushdown? [iceberg]
Posted by "RussellSpitzer (via GitHub)" <gi...@apache.org>.
RussellSpitzer commented on issue #13798:
URL: https://github.com/apache/iceberg/issues/13798#issuecomment-3190051106
It is supported both when pruning manifests and datafiles,
Manifests - Checking Summary
https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/ManifestGroup.java#L251-L257
Datafiles - Checking actual tuple
https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/ManifestFilterManager.java#L584-L608
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org