You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yihua (via GitHub)" <gi...@apache.org> on 2026/08/08 00:22:27 UTC

[PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]

yihua opened a new pull request, #19565:
URL: https://github.com/apache/hudi/pull/19565

   ### Describe the issue this Pull Request addresses
   
   Closes #19561
   
   ### Summary and Changelog
   
   Table creation passed `TIMELINE_HISTORY_PATH.defaultValue()` (`history`) into `setArchiveLogFolder(...)` regardless of the table version being created.
   
   A table below version 8 uses timeline layout version 1, and `TimelinePathProviderV1.getTimelineHistoryPath()` resolves the archived timeline through `hoodie.archivelog.folder`. Forcing that config to the layout version 2 value meant such a table archived into `.hoodie/archived`'s sibling `.hoodie/history`, diverging from the conventional location.
   
   Changes:
   
   - Remove the hardcoded value from the table creation paths so `TableBuilder.build()` supplies `ARCHIVELOG_FOLDER.defaultValue()`, and pass through only a value the user configured. Five call sites: `StreamSync`, `BootstrapExecutor`, `BootstrapExecutorUtils`, `HoodieSparkSqlWriter` (both table creation branches), and the Flink `StreamerUtil`.
   - `HoodieSparkSqlWriter` switches from `getStringOrDefault` to `getString` so an unset config reaches the builder as null rather than as the layout version 2 default.
   - Add `TestStreamSync` coverage across table versions 6, 7, 8 and 9, asserting both the persisted config and the resolved archive path, plus a case pinning that an explicitly configured `hoodie.timeline.history.path` is still honoured.
   
   The metadata table already handled this correctly through `HoodieBackedTableMetadataWriterTableVersionSix`, which returns `ARCHIVELOG_FOLDER.defaultValue()`; it is unchanged.
   
   The active timeline is unaffected: `TimelinePathProviderV1` hardcodes `.hoodie` and never consults `hoodie.timeline.path`.
   
   ### Impact
   
   Newly created tables below version 8 archive into `.hoodie/archived`, matching the conventional location and the metadata table.
   
   Tables at version 8 and above now record `hoodie.archivelog.folder=archived` where these paths previously wrote `history`. That config has exactly one reader in the codebase, `TimelinePathProviderV1`, which is unreachable at layout version 2, so it is inert for those tables. It becomes meaningful again after a downgrade to version 7 or below, where the new value is the correct one: `EightToSevenDowngradeHandler` writes the converted archived timeline to a hardcoded `.hoodie/archived`, so a table carrying `history` previously ended up pointing at an empty directory after downgrade.
   
   Existing tables are not modified. Any created below version 8 keep `history` in their properties and continue to resolve consistently, since the persisted value is what is read.
   
   No public API change.
   
   ### Risk Level
   
   low
   
   The changed config has a single production reader, selected only for timeline layout version 1, so behaviour at layout version 2 is unchanged in effect. Verified with unit tests across four table versions, and by confirming that reverting the change reproduces the original symptom (`expected: <archived> but was: <history>` at table version 6).
   
   ### Documentation Update
   
   none
   
   No config is added and no default value changes; this removes a hardcoded override that bypassed an existing default.
   
   ### Contributor's checklist
   
   - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19565:
URL: https://github.com/apache/hudi/pull/19565#issuecomment-5223646372

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "f0aa5cfac5c891b4f9698917477eac799580899a",
       "status" : "PENDING",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16197",
       "triggerID" : "f0aa5cfac5c891b4f9698917477eac799580899a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f0aa5cfac5c891b4f9698917477eac799580899a Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16197) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]

Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #19565:
URL: https://github.com/apache/hudi/pull/19565#issuecomment-5223588693

   ## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/19565?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
   :white_check_mark: All modified and coverable lines are covered by tests.
   :white_check_mark: Project coverage is 76.84%. Comparing base ([`65cf7e8`](https://app.codecov.io/gh/apache/hudi/commit/65cf7e8ede7d1a08f3b3f13b4a45c6229043cc3e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`f0aa5cf`](https://app.codecov.io/gh/apache/hudi/commit/f0aa5cfac5c891b4f9698917477eac799580899a?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
   
   <details><summary>Additional details and impacted files</summary>
   
   
   
   ```diff
   @@             Coverage Diff              @@
   ##             master   #19565      +/-   ##
   ============================================
   - Coverage     76.84%   76.84%   -0.01%     
     Complexity    32379    32379              
   ============================================
     Files          2522     2522              
     Lines        139106   139098       -8     
     Branches      16713    16713              
   ============================================
   - Hits         106892   106884       -8     
     Misses        24621    24621              
     Partials       7593     7593              
   ```
   
   | [Components](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hudi-common](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.26% <ø> (ø)` | |
   | [hudi-client](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.99% <ø> (ø)` | |
   | [hudi-flink](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `84.70% <ø> (-0.01%)` | :arrow_down: |
   | [hudi-spark-datasource](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.59% <ø> (-0.01%)` | :arrow_down: |
   | [hudi-utilities](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `73.62% <ø> (-0.01%)` | :arrow_down: |
   | [hudi-cli](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `15.32% <ø> (ø)` | |
   | [hudi-hadoop](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `63.50% <ø> (ø)` | |
   | [hudi-sync](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `75.11% <ø> (ø)` | |
   | [hudi-io](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `79.46% <ø> (ø)` | |
   | [hudi-timeline-service](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.44% <ø> (ø)` | |
   | [hudi-cloud](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `64.06% <ø> (ø)` | |
   | [hudi-kafka-connect](https://app.codecov.io/gh/apache/hudi/pull/19565/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.20% <ø> (ø)` | |
   
   | [Flag](https://app.codecov.io/gh/apache/hudi/pull/19565/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/19565/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.73% <ø> (+0.01%)` | :arrow_up: |
   
   Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
   
   | [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/19565?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
   |---|---|---|
   | [...c/main/java/org/apache/hudi/util/StreamerUtil.java](https://app.codecov.io/gh/apache/hudi/pull/19565?src=pr&el=tree&filepath=hudi-flink-datasource%2Fhudi-flink%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futil%2FStreamerUtil.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1mbGluay1kYXRhc291cmNlL2h1ZGktZmxpbmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbC9TdHJlYW1lclV0aWwuamF2YQ==) | `72.42% <ø> (-0.08%)` | :arrow_down: |
   | [...n/scala/org/apache/hudi/HoodieSparkSqlWriter.scala](https://app.codecov.io/gh/apache/hudi/pull/19565?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark-common%2Fsrc%2Fmain%2Fscala%2Forg%2Fapache%2Fhudi%2FHoodieSparkSqlWriter.scala&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3BhcmstY29tbW9uL3NyYy9tYWluL3NjYWxhL29yZy9hcGFjaGUvaHVkaS9Ib29kaWVTcGFya1NxbFdyaXRlci5zY2FsYQ==) | `78.37% <ø> (-0.08%)` | :arrow_down: |
   | [...va/org/apache/hudi/cli/BootstrapExecutorUtils.java](https://app.codecov.io/gh/apache/hudi/pull/19565?src=pr&el=tree&filepath=hudi-spark-datasource%2Fhudi-spark%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fcli%2FBootstrapExecutorUtils.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zcGFyay1kYXRhc291cmNlL2h1ZGktc3Bhcmsvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvY2xpL0Jvb3RzdHJhcEV4ZWN1dG9yVXRpbHMuamF2YQ==) | `74.46% <ø> (-0.36%)` | :arrow_down: |
   | [...che/hudi/utilities/streamer/BootstrapExecutor.java](https://app.codecov.io/gh/apache/hudi/pull/19565?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2Fstreamer%2FBootstrapExecutor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3N0cmVhbWVyL0Jvb3RzdHJhcEV4ZWN1dG9yLmphdmE=) | `73.86% <ø> (-0.59%)` | :arrow_down: |
   | [...org/apache/hudi/utilities/streamer/StreamSync.java](https://app.codecov.io/gh/apache/hudi/pull/19565?src=pr&el=tree&filepath=hudi-utilities%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Futilities%2Fstreamer%2FStreamSync.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS11dGlsaXRpZXMvc3JjL21haW4vamF2YS9vcmcvYXBhY2hlL2h1ZGkvdXRpbGl0aWVzL3N0cmVhbWVyL1N0cmVhbVN5bmMuamF2YQ==) | `77.86% <ø> (-0.04%)` | :arrow_down: |
   </details>
   <details><summary> :rocket: New features to boost your workflow: </summary>
   
   - :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
   - :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19565:
URL: https://github.com/apache/hudi/pull/19565#issuecomment-5224113445

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "f0aa5cfac5c891b4f9698917477eac799580899a",
       "status" : "SUCCESS",
       "url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16197",
       "triggerID" : "f0aa5cfac5c891b4f9698917477eac799580899a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f0aa5cfac5c891b4f9698917477eac799580899a Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=16197) 
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]

Posted by "voonhous (via GitHub) via commits" <co...@hudi.apache.org>.
voonhous commented on PR #19565:
URL: https://github.com/apache/hudi/pull/19565#issuecomment-6701276382

   Temporarily closing this PR while we debug a GitHub settings error on master (#20471); it will be reopened shortly, no action needed.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]

Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19565:
URL: https://github.com/apache/hudi/pull/19565#issuecomment-5223612823

   <!--
   Meta data
   {
     "version" : 1,
     "metaDataEntries" : [ {
       "hash" : "f0aa5cfac5c891b4f9698917477eac799580899a",
       "status" : "UNKNOWN",
       "url" : "TBD",
       "triggerID" : "f0aa5cfac5c891b4f9698917477eac799580899a",
       "triggerType" : "PUSH"
     } ]
   }-->
   ## CI report:
   
   * f0aa5cfac5c891b4f9698917477eac799580899a UNKNOWN
   
   <details>
   <summary>Bot commands</summary>
     @hudi-bot supports the following commands:
   
    - `@hudi-bot run azure` re-run the last Azure build
   </details>


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] fix(core): stop forcing the timeline history folder onto pre-v8 tables [hudi]

Posted by "voonhous (via GitHub) via commits" <co...@hudi.apache.org>.
voonhous closed pull request #19565: fix(core): stop forcing the timeline history folder onto pre-v8 tables
URL: https://github.com/apache/hudi/pull/19565


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org