You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yihua (via GitHub)" <gi...@apache.org> on 2026/07/07 19:19:02 UTC
[PR] test(spark): add unit coverage for Spark row, schema and sort utilities [hudi]
yihua opened a new pull request, #19219:
URL: https://github.com/apache/hudi/pull/19219
### Describe the issue this Pull Request addresses
The `hudi-spark-client` row, schema, and sort utility classes have thin unit-test coverage. This PR raises coverage of the space-curve sorting helper and the Spark value-metadata conversion helper using only a local `SparkSession`, with no cluster or table services.
### Summary and Changelog
Adds pure local-Spark unit tests for two `hudi-spark-client` utilities:
- `SpaceCurveSortingHelper.orderDataFrameByMappingValues` (extends the existing `TestRangeSampleSort`): asserts value-by-value that ZORDER and HILBERT ordering preserve the original schema (the internal `Index` column is dropped) and keep every row, that rows come out non-decreasing under an independently recomputed Z-curve ordinal, that all-null rows sort last, that the Hilbert origin sorts first, that a single ordering column short-circuits to ascending linear order, and that an unknown order column returns the untouched input frame.
- `SparkValueMetadataUtils` (new `TestSparkValueMetadataUtils`): a parameterized matrix asserting each Spark data type maps to the expected `ValueType`, decimal precision/scale carry-through, the below-V2 index version returning the shared V1 empty metadata, null-type handling, and the `convertSparkToJava` / `convertJavaTypeToSparkType` round-trips for primitives, decimal, bytes, date (epoch-day), and timestamp (micros) including the legacy vs java8 date-time API branches.
No production code was changed. No code was copied.
### Impact
None. Test-only change; no public API or user-facing behavior is affected.
### Risk Level
low
Test-only additions that run against a local `SparkSession` with unique in-process fixtures and standard teardown, so there is no shared state or resource leakage.
### Documentation Update
none
### Contributor's checklist
- [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Enough context is provided in the sections above
- [ ] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(spark): add unit coverage for Spark row, schema and sort utilities [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19219:
URL: https://github.com/apache/hudi/pull/19219#issuecomment-4909950785
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8882606a10fd62dda1edc4f20b4567a7ea2638d1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15370",
"triggerID" : "8882606a10fd62dda1edc4f20b4567a7ea2638d1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8882606a10fd62dda1edc4f20b4567a7ea2638d1 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15370)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(spark): add unit coverage for Spark row, schema and sort utilities [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous merged PR #19219:
URL: https://github.com/apache/hudi/pull/19219
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(spark): add unit coverage for Spark row, schema and sort utilities [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19219:
URL: https://github.com/apache/hudi/pull/19219#issuecomment-4907912750
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8882606a10fd62dda1edc4f20b4567a7ea2638d1",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "8882606a10fd62dda1edc4f20b4567a7ea2638d1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8882606a10fd62dda1edc4f20b4567a7ea2638d1 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(spark): add unit coverage for Spark row, schema and sort utilities [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19219:
URL: https://github.com/apache/hudi/pull/19219#issuecomment-5022438195
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8882606a10fd62dda1edc4f20b4567a7ea2638d1",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15370",
"triggerID" : "8882606a10fd62dda1edc4f20b4567a7ea2638d1",
"triggerType" : "PUSH"
}, {
"hash" : "bf92199036810f6ae4ec0df63e6632bbc09ffb71",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "bf92199036810f6ae4ec0df63e6632bbc09ffb71",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8882606a10fd62dda1edc4f20b4567a7ea2638d1 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15370)
* bf92199036810f6ae4ec0df63e6632bbc09ffb71 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] test(spark): add unit coverage for Spark row, schema and sort utilities [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19219:
URL: https://github.com/apache/hudi/pull/19219#issuecomment-4907961367
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "8882606a10fd62dda1edc4f20b4567a7ea2638d1",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15370",
"triggerID" : "8882606a10fd62dda1edc4f20b4567a7ea2638d1",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 8882606a10fd62dda1edc4f20b4567a7ea2638d1 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15370)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org