You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yihua (via GitHub)" <gi...@apache.org> on 2026/08/08 00:22:27 UTC
[PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]
yihua opened a new pull request, #19565:
URL: https://github.com/apache/hudi/pull/19565
### Describe the issue this Pull Request addresses
Closes #19561
### Summary and Changelog
Table creation passed `TIMELINE_HISTORY_PATH.defaultValue()` (`history`) into `setArchiveLogFolder(...)` regardless of the table version being created.
A table below version 8 uses timeline layout version 1, and `TimelinePathProviderV1.getTimelineHistoryPath()` resolves the archived timeline through `hoodie.archivelog.folder`. Forcing that config to the layout version 2 value meant such a table archived into `.hoodie/archived`'s sibling `.hoodie/history`, diverging from the conventional location.
Changes:
- Remove the hardcoded value from the table creation paths so `TableBuilder.build()` supplies `ARCHIVELOG_FOLDER.defaultValue()`, and pass through only a value the user configured. Five call sites: `StreamSync`, `BootstrapExecutor`, `BootstrapExecutorUtils`, `HoodieSparkSqlWriter` (both table creation branches), and the Flink `StreamerUtil`.
- `HoodieSparkSqlWriter` switches from `getStringOrDefault` to `getString` so an unset config reaches the builder as null rather than as the layout version 2 default.
- Add `TestStreamSync` coverage across table versions 6, 7, 8 and 9, asserting both the persisted config and the resolved archive path, plus a case pinning that an explicitly configured `hoodie.timeline.history.path` is still honoured.
The metadata table already handled this correctly through `HoodieBackedTableMetadataWriterTableVersionSix`, which returns `ARCHIVELOG_FOLDER.defaultValue()`; it is unchanged.
The active timeline is unaffected: `TimelinePathProviderV1` hardcodes `.hoodie` and never consults `hoodie.timeline.path`.
### Impact
Newly created tables below version 8 archive into `.hoodie/archived`, matching the conventional location and the metadata table.
Tables at version 8 and above now record `hoodie.archivelog.folder=archived` where these paths previously wrote `history`. That config has exactly one reader in the codebase, `TimelinePathProviderV1`, which is unreachable at layout version 2, so it is inert for those tables. It becomes meaningful again after a downgrade to version 7 or below, where the new value is the correct one: `EightToSevenDowngradeHandler` writes the converted archived timeline to a hardcoded `.hoodie/archived`, so a table carrying `history` previously ended up pointing at an empty directory after downgrade.
Existing tables are not modified. Any created below version 8 keep `history` in their properties and continue to resolve consistently, since the persisted value is what is read.
No public API change.
### Risk Level
low
The changed config has a single production reader, selected only for timeline layout version 1, so behaviour at layout version 2 is unchanged in effect. Verified with unit tests across four table versions, and by confirming that reverting the change reproduces the original symptom (`expected: <archived> but was: <history>` at table version 6).
### Documentation Update
none
No config is added and no default value changes; this removes a hardcoded override that bypassed an existing default.
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19565:
URL: https://github.com/apache/hudi/pull/19565#issuecomment-5223646372
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "f0aa5cfac5c891b4f9698917477eac799580899a",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16197",
"triggerID" : "f0aa5cfac5c891b4f9698917477eac799580899a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f0aa5cfac5c891b4f9698917477eac799580899a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16197)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #19565:
URL: https://github.com/apache/hudi/pull/19565#issuecomment-5223588693
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/19565?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 76.84%. Comparing base ([`65cf7e8`](https://app.codecov.io/gh/apache/hudi/commit/65cf7e8ede7d1a08f3b3f13b4a45c6229043cc3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f0aa5cf`](https://app.codecov.io/gh/apache/hudi/commit/f0aa5cfac5c891b4f9698917477eac799580899a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #19565 +/- ##
============================================
- Coverage 76.84% 76.84% -0.01%
Complexity 32379 32379
============================================
Files 2522 2522
Lines 139106 139098 -8
Branches 16713 16713
============================================
- Hits 106892 106884 -8
Misses 24621 24621
Partials 7593 7593
```
| [Components](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hudi-common](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.26% <ø> (ø)` | |
| [hudi-client](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.99% <ø> (ø)` | |
| [hudi-flink](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `84.70% <ø> (-0.01%)` | :arrow_down: |
| [hudi-spark-datasource](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.59% <ø> (-0.01%)` | :arrow_down: |
| [hudi-utilities](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `73.62% <ø> (-0.01%)` | :arrow_down: |
| [hudi-cli](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `15.32% <ø> (ø)` | |
| [hudi-hadoop](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `63.50% <ø> (ø)` | |
| [hudi-sync](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `75.11% <ø> (ø)` | |
| [hudi-io](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `79.46% <ø> (ø)` | |
| [hudi-timeline-service](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.44% <ø> (ø)` | |
| [hudi-cloud](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `64.06% <ø> (ø)` | |
| [hudi-kafka-connect](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.20% <ø> (ø)` | |
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/19565/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/19565/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.73% <ø> (+0.01%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/19565?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...c/main/java/org/apache/hudi/util/StreamerUtil.java](https://app.codecov.io/gh/apache/hudi/pull/19565?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futil%2FStreamerUtil.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbC9TdHJlYW1lclV0aWwuamF2YQ==) | `72.42% <ø> (-0.08%)` | :arrow_down: |
| [...n/scala/org/apache/hudi/HoodieSparkSqlWriter.scala](https://app.codecov.io/gh/apache/hudi/pull/19565?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark-common%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fhudi%2FHoodieSparkSqlWriter.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmstY29tbW9uL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvaHVkaS9Ib29kaWVTcGFya1NxbFdyaXRlci5zY2FsYQ==) | `78.37% <ø> (-0.08%)` | :arrow_down: |
| [...va/org/apache/hudi/cli/BootstrapExecutorUtils.java](https://app.codecov.io/gh/apache/hudi/pull/19565?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcli%2FBootstrapExecutorUtils.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpL0Jvb3RzdHJhcEV4ZWN1dG9yVXRpbHMuamF2YQ==) | `74.46% <ø> (-0.36%)` | :arrow_down: |
| [...che/hudi/utilities/streamer/BootstrapExecutor.java](https://app.codecov.io/gh/apache/hudi/pull/19565?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2Fstreamer%2FBootstrapExecutor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3N0cmVhbWVyL0Jvb3RzdHJhcEV4ZWN1dG9yLmphdmE=) | `73.86% <ø> (-0.59%)` | :arrow_down: |
| [...org/apache/hudi/utilities/streamer/StreamSync.java](https://app.codecov.io/gh/apache/hudi/pull/19565?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2Fstreamer%2FStreamSync.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3N0cmVhbWVyL1N0cmVhbVN5bmMuamF2YQ==) | `77.86% <ø> (-0.04%)` | :arrow_down: |
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19565:
URL: https://github.com/apache/hudi/pull/19565#issuecomment-5224113445
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "f0aa5cfac5c891b4f9698917477eac799580899a",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16197",
"triggerID" : "f0aa5cfac5c891b4f9698917477eac799580899a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f0aa5cfac5c891b4f9698917477eac799580899a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16197)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]
Posted by "voonhous (via GitHub) via commits" <co...@hudi.apache.org>.
voonhous commented on PR #19565:
URL: https://github.com/apache/hudi/pull/19565#issuecomment-6701276382
Temporarily closing this PR while we debug a GitHub settings error on master (#20471); it will be reopened shortly, no action needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19565:
URL: https://github.com/apache/hudi/pull/19565#issuecomment-5223612823
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "f0aa5cfac5c891b4f9698917477eac799580899a",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f0aa5cfac5c891b4f9698917477eac799580899a",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f0aa5cfac5c891b4f9698917477eac799580899a UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]
Posted by "voonhous (via GitHub) via commits" <co...@hudi.apache.org>.
voonhous closed pull request #19565: fix(core): stop forcing the timeline history folder onto pre-v8 tables
URL: https://github.com/apache/hudi/pull/19565
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org