You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "wombatu-kun (via GitHub)" <gi...@apache.org> on 2026/10/15 10:29:39 UTC
[PR] fix(core): refuse to downgrade a table ordered on several fields below version 9 [hudi]
wombatu-kun opened a new pull request, #20348:
URL: https://github.com/apache/hudi/pull/20348
### Describe the issue this Pull Request addresses
Closes #20347
The 9 to 8 downgrade copied several ordering fields as they are into `hoodie.table.precombine.field`, which Hudi 1.0.x and 0.x read as a single field name, so the release it targets could not upsert into the downgraded table.
### Summary and Changelog
- `UpgradeDowngrade.run` refuses a downgrade from version 9 or above to below 9 of a table ordered on several fields, before the rollback, the compaction and the metadata table downgrade. The message names the fields, says why 1.0.x and 0.x cannot read them, and points to keeping the table at version 9 or recreating it with a single ordering field.
- `NineToEightDowngradeHandler` holds the check and applies it itself too. It checks the fields the downgrade would write, so `MySqlDebeziumAvroPayload` tables still go back to their single `_event_seq`, and it now writes the parsed field, so a stray separator as in `ts,` no longer lands in the legacy key.
- `TestNineToEightDowngradeHandler` covers the refusal, a single field, `ts,` and the MySQL Debezium mapping. `TestUpgradeDowngradeOrchestration` refuses a 10 to 8 downgrade before any step and lets 10 to 9 with two fields and 10 to 8 for MySQL Debezium proceed. `TestUpgradeDowngrade` refuses the version 9 MOR fixture ordered on `ts,name` at versions 8 and 6 and checks that the data and metadata tables keep their version, timeline, ordering fields and log files.
### Impact
A downgrade below version 9 of a table ordered on several fields now fails with `Cannot downgrade to table version 8: hoodie.table.ordering.fields=ts,name orders records on several fields, which Hudi 1.0.x and 0.x cannot read. ...` and leaves the table as it was. Before, it reported success and left `hoodie.table.precombine.field=ts,name`, a field that 1.0.2 upserts and MOR merges look for and do not find. Every other downgrade behaves as before. The `downgrade_table` procedure turns the failure into a `false` result and logs the message.
### Risk Level
low
The check only rejects downgrades whose result was unusable, and it runs before anything changes. Verified: the refusal tests fail on master with `Expected org.apache.hudi.exception.HoodieUpgradeDowngradeException to be thrown, but nothing was thrown.`, and the downgrade on master writes `hoodie.table.precombine.field=ts,name`, which Hudi 1.0.2 and 0.15.0 resolve as their precombine field.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): refuse to downgrade a table ordered on several fields below version 9 [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #20348:
URL: https://github.com/apache/hudi/pull/20348#issuecomment-6191201046
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/20348?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:x: Patch coverage is `0%` with `15 lines` in your changes missing coverage. Please review.
:white_check_mark: Project coverage is 80.71%. Comparing base ([`616f9ae`](https://app.codecov.io/gh/apache/hudi/commit/616f9ae867654d70b58674600887e4ff1205ccf2?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`9cf3fe6`](https://app.codecov.io/gh/apache/hudi/commit/9cf3fe6956e38213f7a199d05eaf86daf9028cb6?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
:warning: Report is 2 commits behind head on master.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/20348?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines |
|---|---|---|
| [...udi/table/upgrade/NineToEightDowngradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/20348?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FNineToEightDowngradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvTmluZVRvRWlnaHREb3duZ3JhZGVIYW5kbGVyLmphdmE=) | 0.00% | [13 Missing :warning: ](https://app.codecov.io/gh/apache/hudi/pull/20348?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
| [...rg/apache/hudi/table/upgrade/UpgradeDowngrade.java](https://app.codecov.io/gh/apache/hudi/pull/20348?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FUpgradeDowngrade.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvVXBncmFkZURvd25ncmFkZS5qYXZh) | 0.00% | [1 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/hudi/pull/20348?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) |
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #20348 +/- ##
============================================
- Coverage 80.73% 80.71% -0.02%
- Complexity 35169 35181 +12
============================================
Files 2553 2553
Lines 143582 143650 +68
Branches 17486 17500 +14
============================================
+ Hits 115920 115948 +28
- Misses 19701 19738 +37
- Partials 7961 7964 +3
```
| [Components](https://app.codecov.io/gh/apache/hudi/pull/20348/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hudi-common](https://app.codecov.io/gh/apache/hudi/pull/20348/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `84.13% <ø> (-0.05%)` | :arrow_down: |
| [hudi-client](https://app.codecov.io/gh/apache/hudi/pull/20348/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.91% <0.00%> (-0.05%)` | :arrow_down: |
| [hudi-flink](https://app.codecov.io/gh/apache/hudi/pull/20348/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `86.08% <ø> (ø)` | |
| [hudi-spark-datasource](https://app.codecov.io/gh/apache/hudi/pull/20348/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `74.20% <ø> (ø)` | |
| [hudi-utilities](https://app.codecov.io/gh/apache/hudi/pull/20348/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `78.14% <ø> (ø)` | |
| [hudi-cli](https://app.codecov.io/gh/apache/hudi/pull/20348/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.80% <ø> (ø)` | |
| [hudi-hadoop](https://app.codecov.io/gh/apache/hudi/pull/20348/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `71.17% <ø> (ø)` | |
| [hudi-sync](https://app.codecov.io/gh/apache/hudi/pull/20348/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `76.58% <ø> (ø)` | |
| [hudi-io](https://app.codecov.io/gh/apache/hudi/pull/20348/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.52% <ø> (ø)` | |
| [hudi-timeline-service](https://app.codecov.io/gh/apache/hudi/pull/20348/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.41% <ø> (ø)` | |
| [hudi-cloud](https://app.codecov.io/gh/apache/hudi/pull/20348/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.00% <ø> (ø)` | |
| [hudi-kafka-connect](https://app.codecov.io/gh/apache/hudi/pull/20348/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `57.90% <ø> (ø)` | |
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/20348/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/20348/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.94% <0.00%> (-0.01%)` | :arrow_down: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/20348?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...rg/apache/hudi/table/upgrade/UpgradeDowngrade.java](https://app.codecov.io/gh/apache/hudi/pull/20348?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FUpgradeDowngrade.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvVXBncmFkZURvd25ncmFkZS5qYXZh) | `81.28% <0.00%> (-0.81%)` | :arrow_down: |
| [...udi/table/upgrade/NineToEightDowngradeHandler.java](https://app.codecov.io/gh/apache/hudi/pull/20348?src=pr&el=tree&filepath=hudi-client%2Fhudi-client-common%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Ftable%2Fupgrade%2FNineToEightDowngradeHandler.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1jbGllbnQvaHVkaS1jbGllbnQtY29tbW9uL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL3RhYmxlL3VwZ3JhZGUvTmluZVRvRWlnaHREb3duZ3JhZGVIYW5kbGVyLmphdmE=) | `77.96% <0.00%> (-22.04%)` | :arrow_down: |
... and [1 file with indirect coverage changes](https://app.codecov.io/gh/apache/hudi/pull/20348/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): refuse to downgrade a table ordered on several fields below version 9 [hudi]
Posted by "wombatu-kun (via GitHub) via commits" <co...@hudi.apache.org>.
wombatu-kun commented on code in PR #20348:
URL: https://github.com/apache/hudi/pull/20348#discussion_r4305380720
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/NineToEightDowngradeHandler.java:
##########
@@ -138,12 +143,37 @@ private void reconcileMergeConfigs(Map<ConfigProperty, String> propertiesToAdd,
private void reconcileOrderingFieldsConfig(Map<ConfigProperty, String> propertiesToAdd,
Set<ConfigProperty> propertiesToRemove,
HoodieTableConfig tableConfig) {
- Option<String> orderingFieldsOpt = MySqlDebeziumAvroPayload.class.getName().equals(tableConfig.getLegacyPayloadClass())
- ? Option.of(DebeziumConstants.ADDED_SEQ_COL_NAME)
- : tableConfig.getOrderingFieldsStr();
- orderingFieldsOpt.ifPresent(orderingFields -> {
- propertiesToAdd.put(HoodieTableConfig.PRECOMBINE_FIELD, orderingFields);
+ List<String> orderingFields = getOrderingFieldsBelowVersionNine(tableConfig);
+ if (orderingFields.size() == 1) {
+ // the parsed field, so that a stray separator as in `ts,` does not end up in a key read as one field name
Review Comment:
Done 4117a932397f
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/NineToEightDowngradeHandler.java:
##########
@@ -78,6 +82,7 @@ public UpgradeDowngrade.TableConfigChangeSet downgrade(HoodieWriteConfig config,
SupportsUpgradeDowngrade upgradeDowngradeHelper) {
final HoodieTable table = upgradeDowngradeHelper.getTable(config, context);
HoodieTableMetaClient metaClient = table.getMetaClient();
+ validateOrderingFields(metaClient.getTableConfig(), HoodieTableVersion.EIGHT);
Review Comment:
Done 4117a932397f. Only `UpgradeDowngrade.run` checks now, so a 10 to 6 request reports version 6, which `TestUpgradeDowngradeOrchestration` now covers.
##########
hudi-client/hudi-client-common/src/test/java/org/apache/hudi/table/upgrade/TestNineToEightDowngradeHandler.java:
##########
@@ -251,6 +253,42 @@ void testDowngradeForOtherPayloadClass() {
}
}
+ @Test
+ void testDowngradeRefusesSeveralOrderingFields() {
+ try (MockedStatic<UpgradeDowngradeUtils> utilities = Mockito.mockStatic(UpgradeDowngradeUtils.class)) {
+ when(tableConfig.getOrderingFieldsStr()).thenReturn(Option.of("ts,seq"));
+ HoodieUpgradeDowngradeException exception = assertThrows(HoodieUpgradeDowngradeException.class,
+ () -> handler.downgrade(config, context, "anyInstant", upgradeDowngradeHelper));
+ assertEquals("Cannot downgrade to table version 8: hoodie.table.ordering.fields=ts,seq orders records on several fields, "
Review Comment:
Done 4117a932397f. No test asserts the whole message now: this test went away with the handler check, and `TestUpgradeDowngrade` matches the same leading part as the orchestration test, target version included.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): refuse to downgrade a table ordered on several fields below version 9 [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #20348:
URL: https://github.com/apache/hudi/pull/20348#discussion_r4305380720
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/NineToEightDowngradeHandler.java:
##########
@@ -138,12 +143,37 @@ private void reconcileMergeConfigs(Map<ConfigProperty, String> propertiesToAdd,
private void reconcileOrderingFieldsConfig(Map<ConfigProperty, String> propertiesToAdd,
Set<ConfigProperty> propertiesToRemove,
HoodieTableConfig tableConfig) {
- Option<String> orderingFieldsOpt = MySqlDebeziumAvroPayload.class.getName().equals(tableConfig.getLegacyPayloadClass())
- ? Option.of(DebeziumConstants.ADDED_SEQ_COL_NAME)
- : tableConfig.getOrderingFieldsStr();
- orderingFieldsOpt.ifPresent(orderingFields -> {
- propertiesToAdd.put(HoodieTableConfig.PRECOMBINE_FIELD, orderingFields);
+ List<String> orderingFields = getOrderingFieldsBelowVersionNine(tableConfig);
+ if (orderingFields.size() == 1) {
+ // the parsed field, so that a stray separator as in `ts,` does not end up in a key read as one field name
Review Comment:
Done aef75ce3c4e9
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/NineToEightDowngradeHandler.java:
##########
@@ -78,6 +82,7 @@ public UpgradeDowngrade.TableConfigChangeSet downgrade(HoodieWriteConfig config,
SupportsUpgradeDowngrade upgradeDowngradeHelper) {
final HoodieTable table = upgradeDowngradeHelper.getTable(config, context);
HoodieTableMetaClient metaClient = table.getMetaClient();
+ validateOrderingFields(metaClient.getTableConfig(), HoodieTableVersion.EIGHT);
Review Comment:
Done aef75ce3c4e9. Only `UpgradeDowngrade.run` checks now, so a 10 to 6 request reports version 6, which `TestUpgradeDowngradeOrchestration` now covers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): refuse to downgrade a table ordered on several fields below version 9 [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #20348:
URL: https://github.com/apache/hudi/pull/20348#discussion_r4305381927
##########
hudi-client/hudi-client-common/src/test/java/org/apache/hudi/table/upgrade/TestNineToEightDowngradeHandler.java:
##########
@@ -251,6 +253,42 @@ void testDowngradeForOtherPayloadClass() {
}
}
+ @Test
+ void testDowngradeRefusesSeveralOrderingFields() {
+ try (MockedStatic<UpgradeDowngradeUtils> utilities = Mockito.mockStatic(UpgradeDowngradeUtils.class)) {
+ when(tableConfig.getOrderingFieldsStr()).thenReturn(Option.of("ts,seq"));
+ HoodieUpgradeDowngradeException exception = assertThrows(HoodieUpgradeDowngradeException.class,
+ () -> handler.downgrade(config, context, "anyInstant", upgradeDowngradeHelper));
+ assertEquals("Cannot downgrade to table version 8: hoodie.table.ordering.fields=ts,seq orders records on several fields, "
Review Comment:
Done aef75ce3c4e9. No test asserts the whole message now: this test went away with the handler check, and `TestUpgradeDowngrade` matches the same leading part as the orchestration test, target version included.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): refuse to downgrade a table ordered on several fields below version 9 [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on PR #20348:
URL: https://github.com/apache/hudi/pull/20348#issuecomment-6432202688
hi @voonhous could you help to review this fix please
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): refuse to downgrade a table ordered on several fields below version 9 [hudi]
Posted by "voonhous (via GitHub) via commits" <co...@hudi.apache.org>.
voonhous commented on PR #20348:
URL: https://github.com/apache/hudi/pull/20348#issuecomment-6701729760
Temporarily closing this PR while we debug a GitHub settings error on master (#20471); it will be reopened shortly, no action needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): refuse to downgrade a table ordered on several fields below version 9 [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #20348:
URL: https://github.com/apache/hudi/pull/20348#discussion_r4281954686
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/NineToEightDowngradeHandler.java:
##########
@@ -138,12 +143,37 @@ private void reconcileMergeConfigs(Map<ConfigProperty, String> propertiesToAdd,
private void reconcileOrderingFieldsConfig(Map<ConfigProperty, String> propertiesToAdd,
Set<ConfigProperty> propertiesToRemove,
HoodieTableConfig tableConfig) {
- Option<String> orderingFieldsOpt = MySqlDebeziumAvroPayload.class.getName().equals(tableConfig.getLegacyPayloadClass())
- ? Option.of(DebeziumConstants.ADDED_SEQ_COL_NAME)
- : tableConfig.getOrderingFieldsStr();
- orderingFieldsOpt.ifPresent(orderingFields -> {
- propertiesToAdd.put(HoodieTableConfig.PRECOMBINE_FIELD, orderingFields);
+ List<String> orderingFields = getOrderingFieldsBelowVersionNine(tableConfig);
+ if (orderingFields.size() == 1) {
+ // the parsed field, so that a stray separator as in `ts,` does not end up in a key read as one field name
Review Comment:
🤖 nit: could you reword this comment as a full sentence, e.g. "Write the parsed field rather than the raw string, so a stray separator as in `ts,` does not end up in the key"? As written it is a fragment that is hard to follow.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): refuse to downgrade a table ordered on several fields below version 9 [hudi]
Posted by "voonhous (via GitHub) via commits" <co...@hudi.apache.org>.
voonhous closed pull request #20348: fix(core): refuse to downgrade a table ordered on several fields below version 9
URL: https://github.com/apache/hudi/pull/20348
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): refuse to downgrade a table ordered on several fields below version 9 [hudi]
Posted by "shangxinli (via GitHub)" <gi...@apache.org>.
shangxinli commented on code in PR #20348:
URL: https://github.com/apache/hudi/pull/20348#discussion_r4304760757
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/upgrade/NineToEightDowngradeHandler.java:
##########
@@ -78,6 +82,7 @@ public UpgradeDowngrade.TableConfigChangeSet downgrade(HoodieWriteConfig config,
SupportsUpgradeDowngrade upgradeDowngradeHelper) {
final HoodieTable table = upgradeDowngradeHelper.getTable(config, context);
HoodieTableMetaClient metaClient = table.getMetaClient();
+ validateOrderingFields(metaClient.getTableConfig(), HoodieTableVersion.EIGHT);
Review Comment:
nit: UpgradeDowngrade.run already validates this with the real target version before any handler runs, so this call only fires when the handler is driven directly, and it hardcodes 8 even for a 10 to 6 request.
##########
hudi-client/hudi-client-common/src/test/java/org/apache/hudi/table/upgrade/TestNineToEightDowngradeHandler.java:
##########
@@ -251,6 +253,42 @@ void testDowngradeForOtherPayloadClass() {
}
}
+ @Test
+ void testDowngradeRefusesSeveralOrderingFields() {
+ try (MockedStatic<UpgradeDowngradeUtils> utilities = Mockito.mockStatic(UpgradeDowngradeUtils.class)) {
+ when(tableConfig.getOrderingFieldsStr()).thenReturn(Option.of("ts,seq"));
+ HoodieUpgradeDowngradeException exception = assertThrows(HoodieUpgradeDowngradeException.class,
+ () -> handler.downgrade(config, context, "anyInstant", upgradeDowngradeHelper));
+ assertEquals("Cannot downgrade to table version 8: hoodie.table.ordering.fields=ts,seq orders records on several fields, "
Review Comment:
nit: three tests across three modules assert this message verbatim, so any wording tweak breaks all of them - matching on the ordering.fields part like the orchestration test does would be enough.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(core): refuse to downgrade a table ordered on several fields below version 9 [hudi]
Posted by "wombatu-kun (via GitHub) via commits" <co...@hudi.apache.org>.
wombatu-kun closed pull request #20348: fix(core): refuse to downgrade a table ordered on several fields below version 9
URL: https://github.com/apache/hudi/pull/20348
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org