You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "anoopj (via GitHub)" <gi...@apache.org> on 2026/07/17 01:04:34 UTC
[PR] API, Core: Only fall back to unknown partition type for source-dependent transforms [iceberg]
anoopj opened a new pull request, #17262:
URL: https://github.com/apache/iceberg/pull/17262
`PartitionSpec.partitionType()` degrades a partition field's type to `unknown` whenever the source column has been dropped from the schema, even for transforms like bucket and the date/time transforms whose result type does not depend on the source type. Substitute unknown as the source type instead and let the transform derive its result type, matching the existing behavior of javaClasses(). Identity and truncate still fall back to unknown because their result type is the source type.
Follow-up to the discussion in https://github.com/apache/iceberg/pull/16958#discussion_r3555857111
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] API, Core: Only fall back to unknown partition type for source-dependent transforms [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on code in PR #17262:
URL: https://github.com/apache/iceberg/pull/17262#discussion_r3606431876
##########
api/src/main/java/org/apache/iceberg/PartitionSpec.java:
##########
@@ -202,6 +188,18 @@ public Class<?>[] javaClasses() {
return lazyJavaClasses;
}
+ private Type resultType(PartitionField field) {
+ Type sourceType = schema.findType(field.sourceId());
+ if (sourceType == null) {
+ // when the source field has been dropped, substitute unknown and let the transform derive
+ // its result type; transforms that return the source type (identity, truncate, void) yield
+ // unknown, while transforms with a fixed result type still resolve
Review Comment:
I think this is a better solution to the problem. This comment seems a bit long though. What about something shorter like this?
```
// When the source field has been dropped, the source type has been lost
// Transforms with a fixed result type still work, so use unknown for source
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] API, Core: Only fall back to unknown partition type for source-dependent transforms [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue merged PR #17262:
URL: https://github.com/apache/iceberg/pull/17262
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] API, Core: Only fall back to unknown partition type for source-dependent transforms [iceberg]
Posted by "rdblue (via GitHub)" <gi...@apache.org>.
rdblue commented on PR #17262:
URL: https://github.com/apache/iceberg/pull/17262#issuecomment-5007936723
Thanks, @anoopj!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] API, Core: Only fall back to unknown partition type for source-dependent transforms [iceberg]
Posted by "anoopj (via GitHub)" <gi...@apache.org>.
anoopj commented on code in PR #17262:
URL: https://github.com/apache/iceberg/pull/17262#discussion_r3615384046
##########
api/src/main/java/org/apache/iceberg/PartitionSpec.java:
##########
@@ -202,6 +188,18 @@ public Class<?>[] javaClasses() {
return lazyJavaClasses;
}
+ private Type resultType(PartitionField field) {
+ Type sourceType = schema.findType(field.sourceId());
+ if (sourceType == null) {
+ // when the source field has been dropped, substitute unknown and let the transform derive
+ // its result type; transforms that return the source type (identity, truncate, void) yield
+ // unknown, while transforms with a fixed result type still resolve
Review Comment:
Thanks for review/merge. Fixing this in https://github.com/apache/iceberg/pull/17303
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org