You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@iceberg.apache.org by "Abhishek-Gawande (via GitHub)" <gi...@apache.org> on 2026/08/15 18:12:54 UTC
[PR] Core: Handle UnknownType in SingleValueParser [iceberg]
Abhishek-Gawande opened a new pull request, #17672:
URL: https://github.com/apache/iceberg/pull/17672
Fixes #17324
### What changes were proposed in this pull request?
This PR adds support for `TypeID.UNKNOWN` in `SingleValueParser`, fixing an issue where the REST catalog's `/plan` endpoint can fail with a truncated `HTTP 200 OK` response.
### Why are the changes needed?
When a partition field references a column that is subsequently dropped from the table schema, `PartitionSpec.resultType()` falls back to `Types.UnknownType.get()`.
However, when `ContentFileParser` serializes the data file's partition values, `SingleValueParser.toJson()` does not handle `UNKNOWN` and throws an `UnsupportedOperationException`.
This can occur while the JSON response is being streamed, resulting in a partially written response when serialization fails.
### How was this patch tested?
* Added `TestSingleValueParser.unknownTypeRoundTrip` to verify that `UnknownType` correctly handles nulls, integer values serialized as strings, and string values.
* Added `TestContentFileParser.partitionSerializationWithDroppedSourceColumn` to reproduce the issue by simulating a dropped source column, re-binding the historical partition spec, and verifying that the partition data can be serialized and deserialized successfully.
* Verified locally with `./gradlew :iceberg-core:test`.
* Ran `./gradlew spotlessApply`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Handle UnknownType in SingleValueParser [iceberg]
Posted by "anoopj (via GitHub)" <gi...@apache.org>.
anoopj commented on code in PR #17672:
URL: https://github.com/apache/iceberg/pull/17672#discussion_r3797697126
##########
core/src/main/java/org/apache/iceberg/SingleValueParser.java:
##########
@@ -410,6 +414,9 @@ public static void toJson(Type type, Object defaultValue, JsonGenerator generato
}
generator.writeEndObject();
break;
+ case UNKNOWN:
+ generator.writeString(String.valueOf(defaultValue));
Review Comment:
I thought the spec treats `unknown` as `null`. Should this be `generator.writeNull()`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Handle UnknownType in SingleValueParser [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #17672:
URL: https://github.com/apache/iceberg/pull/17672#issuecomment-5737822452
This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@iceberg.apache.org list. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Handle UnknownType in SingleValueParser [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #17672:
URL: https://github.com/apache/iceberg/pull/17672#issuecomment-5851283931
This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Handle UnknownType in SingleValueParser [iceberg]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #17672: Core: Handle UnknownType in SingleValueParser
URL: https://github.com/apache/iceberg/pull/17672
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org
Re: [PR] Core: Handle UnknownType in SingleValueParser [iceberg]
Posted by "Abhishek-Gawande (via GitHub)" <gi...@apache.org>.
Abhishek-Gawande commented on code in PR #17672:
URL: https://github.com/apache/iceberg/pull/17672#discussion_r3813317969
##########
core/src/main/java/org/apache/iceberg/SingleValueParser.java:
##########
@@ -410,6 +414,9 @@ public static void toJson(Type type, Object defaultValue, JsonGenerator generato
}
generator.writeEndObject();
break;
+ case UNKNOWN:
+ generator.writeString(String.valueOf(defaultValue));
Review Comment:
Thanks for the review @anoopj
So the null case is actually already handled.if defaultValue is null, we write null at the top of toJson() (before the switch) and return early. The UNKNOWN case only runs for non-null values.
The scenario where this matters is partition values for dropped columns. When a source column gets dropped, PartitionSpec.resultType() falls back to UnknownType, but the existing data files still have the original partition values (like 4888 from a date column). If we unconditionally wrote null here, those values would get lost during REST serialization and break the /plan response.
Since unknown isn't listed in the JSON single-value serialization table (Appendix D), there's no spec-defined format for it . I went with string as a lossless fallback. The fromJson side parses it back as a string too, so round-tripping works.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@iceberg.apache.org
For additional commands, e-mail: issues-help@iceberg.apache.org