You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@xtable.apache.org by "Mola-maker (via GitHub)" <gi...@apache.org> on 2026/10/20 18:04:30 UTC

[PR] fix(core): resolve Delta partition values by storage name when column mapping is enabled [incubator-xtable]

Mola-maker opened a new pull request, #973:
URL: https://github.com/apache/incubator-xtable/pull/973

   Fixes #972
   
   ## What is the purpose of the pull request
   
   With `delta.columnMapping.mode=name`, Delta writes the `partitionValues` of add/remove actions keyed by each column's physical name (`col-<uuid>`), while XTable's internal partition fields carry the logical name. `DeltaKernelPartitionExtractor` and `DeltaPartitionExtractor` looked values up by the logical name only, so every identity partition value came out `null` — the sync reported SUCCESS while the converted Iceberg table had NULL partition columns (e.g. `WHERE country='USA'` matched 0 rows instead of 2).
   
   Two gaps combined into the bug on the Kernel path: `DeltaKernelSchemaExtractor` computed each field's physical (storage) name from `delta.columnMapping.physicalName` but never stored it on the built `InternalField` (the Spark-based `DeltaSchemaExtractor` does set it), and neither partition extractor consulted `InternalField.getStorageName()` when resolving partition values.
   
   ## Brief change log
   
   - `DeltaKernelSchemaExtractor`: set the already-computed storage name on the built `InternalField`, mirroring `DeltaSchemaExtractor`.
   - `DeltaKernelPartitionExtractor` / `DeltaPartitionExtractor`: in `getSerializedPartitionValue`, resolve identity partition values by the field's storage name first when one is present, falling back to the logical name.
   - Tests: added `testPartitionValueExtractionWithColumnMappingPhysicalName` to `TestDeltaKernelPartitionExtractor` and `TestDeltaPartitionExtractor`, feeding a partition-values map keyed by the physical name.
   
   ## Verify this pull request
   
   This change added tests and can be verified as follows:
   
   - Before the fix, the new extraction test fails: expected partition range `USA`, actual `null`. Verified in CI (fork run of this branch without the fix): xtable-core ran 550 tests with this single failure.
   - After the fix, the same CI build (`Maven CI Build` on the fork PR) passes: xtable-core `Tests run: 550, Failures: 0, Errors: 0, Skipped: 3`, BUILD SUCCESS. Red/green runs: https://github.com/Mola-maker/incubator-xtable/pull/1
   - `./mvnw -pl xtable-core -Dtest=TestDeltaKernelPartitionExtractor,TestDeltaPartitionExtractor test`


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@xtable.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org