You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@xtable.apache.org by "brishi19791 (via GitHub)" <gi...@apache.org> on 2026/06/01 16:42:53 UTC
[PR] Fix map key path handling in Delta schema extractors [incubator-xtable]
brishi19791 opened a new pull request, #826:
URL: https://github.com/apache/incubator-xtable/pull/826
[#852](https://github.com/apache/incubator-xtable/issues/825)
Both DeltaSchemaExtractor and DeltaKernelSchemaExtractor were passing MAP_VALUE_FIELD_NAME when recursing into a map key, so nested fields under a complex (struct/array/map) map key were qualified with "<map>._one_field_value.<child>" instead of"<map>._one_field_key.<child>". This collides with the value-side fully-qualified paths and breaks any downstream lookup keyed off InternalField.parentPath (column stats, field resolution, etc.).
Changes:
- DeltaSchemaExtractor: pass MAP_KEY_FIELD_NAME when building the key
schema's parent path.
- DeltaKernelSchemaExtractor: same fix; additionally promote
trimmedTypeName/type from singleton instance fields to method-local
variables (the new test exposed cross-test state corruption via the
shared mutable fields) and throw NotSupportedException on unknown
DataType to keep the locals definitely-assigned.
- TestDeltaSchemaExtractor / TestDeltaKernelSchemaExtractor: add
testMapWithStructKey covering a map whose key is a 2-field struct
(nullable + non-nullable) and whose value is also a struct, asserting
both _one_field_key and _one_field_value nested parentPath
propagation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix map key path handling in Delta schema extractors [incubator-xtable]
Posted by "brishi19791 (via GitHub)" <gi...@apache.org>.
brishi19791 commented on code in PR #826:
URL: https://github.com/apache/incubator-xtable/pull/826#discussion_r3348864853
##########
xtable-core/src/main/java/org/apache/xtable/kernel/DeltaKernelSchemaExtractor.java:
##########
@@ -219,6 +219,8 @@ private InternalSchema toInternalSchema(
type = InternalType.MAP;
fields = Arrays.asList(keyField, valueField);
trimmedTypeName = "map";
+ } else {
Review Comment:
Yes, intentional. Without it, unhandled DataTypes (e.g. ByteType, ShortType, VariantType) would emit [name=""], [dataType=null] — better to fail at conversion than carry a malformed schema downstream.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix map key path handling in Delta schema extractors [incubator-xtable]
Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.
vinishjail97 commented on PR #826:
URL: https://github.com/apache/incubator-xtable/pull/826#issuecomment-4634183130
Changed the PR description to reflect the current state of the code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix map key path handling in Delta schema extractors [incubator-xtable]
Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.
vinishjail97 commented on code in PR #826:
URL: https://github.com/apache/incubator-xtable/pull/826#discussion_r3337263082
##########
xtable-core/src/main/java/org/apache/xtable/kernel/DeltaKernelSchemaExtractor.java:
##########
@@ -192,7 +192,7 @@ private InternalSchema toInternalSchema(
toInternalSchema(
Review Comment:
This map-key recursion passes `null` for `originalMetadata`. If a map key is a `BinaryType`, the binary branch above (`originalMetadata.contains(InternalSchema.XTABLE_LOGICAL_TYPE)`) has no null guard and will NPE. The Spark sibling `DeltaSchemaExtractor` guards `originalMetadata != null` before `.contains(...)`. Since this PR specifically enables nested/complex map-key handling, a binary-typed map key (or binary array element — also passes null) now becomes more reachable. Worth adding the same null guard here?
##########
xtable-core/src/main/java/org/apache/xtable/kernel/DeltaKernelSchemaExtractor.java:
##########
@@ -219,6 +219,8 @@ private InternalSchema toInternalSchema(
type = InternalType.MAP;
fields = Arrays.asList(keyField, valueField);
trimmedTypeName = "map";
+ } else {
Review Comment:
Good — the `else throw NotSupportedException` is required now that `type`/`trimmedTypeName` are method locals (definite-assignment). One thing to double-check: were any DataTypes previously falling through to the old `null` instance-field defaults and silently producing a schema with `name=""`/`type=null` (e.g. `VariantType`, `NullType`)? If so, this now correctly fails fast rather than emitting a malformed schema — just confirm that's the intended behavior change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix map key path handling in Delta schema extractors [incubator-xtable]
Posted by "brishi19791 (via GitHub)" <gi...@apache.org>.
brishi19791 commented on code in PR #826:
URL: https://github.com/apache/incubator-xtable/pull/826#discussion_r3338448842
##########
xtable-core/src/main/java/org/apache/xtable/kernel/DeltaKernelSchemaExtractor.java:
##########
@@ -192,7 +192,7 @@ private InternalSchema toInternalSchema(
toInternalSchema(
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Fix map key path handling in Delta schema extractors [incubator-xtable]
Posted by "vinishjail97 (via GitHub)" <gi...@apache.org>.
vinishjail97 merged PR #826:
URL: https://github.com/apache/incubator-xtable/pull/826
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org