You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "yihua (via GitHub)" <gi...@apache.org> on 2026/07/30 21:23:48 UTC
[PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
yihua opened a new pull request, #19427:
URL: https://github.com/apache/hudi/pull/19427
### Describe the issue this Pull Request addresses
Hudi 1.x flipped the default of `hoodie.meta.sync.sync_snapshot_with_table_name` from `false` to `true` ([HUDI-7415]). Under the default `ALL` hive-sync-table-strategy, `HiveSyncTool.doSync()` now syncs the bare table name of a MERGE_ON_READ table twice in the same run when `hoodie.datasource.hive_sync.skip_ro_suffix=true` is also set: once as the read-optimized (RO) table (since `skip_ro_suffix` redirects the RO sync onto the bare name instead of `<table>_ro`), then again as the real-time (RT) table for the "sync origin table" step, with the RT sync silently winning. This makes `skip_ro_suffix` a no-op, leaves no RO view registered anywhere, and breaks read-optimized queries against the bare table name for engines that can't read `HoodieParquetRealtimeInputFormat` (e.g. Presto/Trino).
### Summary and Changelog
Skips the redundant bare-name RT sync in `HiveSyncTool.doSync()` when `skip_ro_suffix` is set, logging a WARN naming the table and stating which config takes precedence. The real-time view remains available at `<table>_rt`; the bare table name stays registered as the RO view.
No code was copied.
### Impact
No public API change. Behavior change only for MERGE_ON_READ tables with both `hoodie.datasource.hive_sync.skip_ro_suffix=true` and `hoodie.meta.sync.sync_snapshot_with_table_name=true`: the bare table name now stays registered as the read-optimized view instead of being flipped to real-time.
### Risk Level
low
Change is scoped to a single conditional in `HiveSyncTool.doSync()`; all other config combinations (including the previous default of `sync_snapshot_with_table_name=false`) are unaffected. Covered by a new regression test that reproduces the pre-fix flip.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #19427:
URL: https://github.com/apache/hudi/pull/19427#discussion_r3688027184
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java:
##########
@@ -211,7 +211,16 @@ protected void doSync() {
syncHoodieTable(snapshotTableName, true, false);
// sync origin table for MOR
if (config.getBoolean(META_SYNC_SNAPSHOT_WITH_TABLE_NAME)) {
- syncHoodieTable(tableName, true, false);
+ if (config.getBoolean(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE)) {
+ // skip_ro_suffix explicitly claims the bare table name for the RO view;
+ // an implicit sync_snapshot_with_table_name default must not repurpose it as RT.
Review Comment:
The comment looks redundant now. I'll remove it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "voonhous (via GitHub)" <gi...@apache.org>.
voonhous merged PR #19427:
URL: https://github.com/apache/hudi/pull/19427
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #19427:
URL: https://github.com/apache/hudi/pull/19427#discussion_r3688069645
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java:
##########
@@ -211,7 +211,16 @@ protected void doSync() {
syncHoodieTable(snapshotTableName, true, false);
// sync origin table for MOR
if (config.getBoolean(META_SYNC_SNAPSHOT_WITH_TABLE_NAME)) {
- syncHoodieTable(tableName, true, false);
+ if (config.getBoolean(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE)) {
+ // skip_ro_suffix explicitly claims the bare table name for the RO view;
+ // an implicit sync_snapshot_with_table_name default must not repurpose it as RT.
+ log.warn("{}=true claims the base table name '{}' for the read-optimized view; "
Review Comment:
Done, reworded to "bare table name".
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java:
##########
@@ -211,7 +211,16 @@ protected void doSync() {
syncHoodieTable(snapshotTableName, true, false);
// sync origin table for MOR
if (config.getBoolean(META_SYNC_SNAPSHOT_WITH_TABLE_NAME)) {
- syncHoodieTable(tableName, true, false);
+ if (config.getBoolean(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE)) {
+ // skip_ro_suffix explicitly claims the bare table name for the RO view;
+ // an implicit sync_snapshot_with_table_name default must not repurpose it as RT.
Review Comment:
Done, removed.
##########
hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.java:
##########
@@ -1557,6 +1559,43 @@ public void testSyncMergeOnReadWithStrategy(String syncMode, HoodieSyncTableStra
}
}
+ @Test
+ void testSkipRoSuffixTakesPrecedenceOverSnapshotWithTableName() throws Exception {
+ // skip_ro_suffix explicitly claims the bare table name for the RO view; the now-default-true
+ // sync_snapshot_with_table_name must not be allowed to flip it to RT.
+ hiveSyncProps.setProperty(HIVE_SYNC_TABLE_STRATEGY.key(), HoodieSyncTableStrategy.ALL.name());
+ hiveSyncProps.setProperty(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE.key(), "true");
+ hiveSyncProps.setProperty(META_SYNC_SNAPSHOT_WITH_TABLE_NAME.key(), "true");
+ hiveSyncProps.setProperty(HIVE_SYNC_AS_DATA_SOURCE_TABLE.key(), "true");
+
+ String instantTime = "100";
+ String deltaCommitTime = "101";
+ HiveTestUtil.createMORTable(instantTime, deltaCommitTime, 5, true, true);
+
+ reInitHiveSyncClient();
+ reSyncHiveTable();
Review Comment:
Verified: reverted the production fix and ran the test — it fails as expected on the first commit alone (`isAlreadySynced` short-circuits the redundant sync before it can flip anything), and only fails with the right symptom once a second commit + resync round is added, matching what you found locally. Added the second round (`HiveTestUtil.addMORPartitions` + another `reSyncHiveTable()`), confirmed it fails without the fix (bare table flips to `HoodieParquetRealtimeInputFormat`) and passes with it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #19427:
URL: https://github.com/apache/hudi/pull/19427#discussion_r3688047712
##########
hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.java:
##########
@@ -1557,6 +1559,43 @@ public void testSyncMergeOnReadWithStrategy(String syncMode, HoodieSyncTableStra
}
}
+ @Test
+ void testSkipRoSuffixTakesPrecedenceOverSnapshotWithTableName() throws Exception {
+ // skip_ro_suffix explicitly claims the bare table name for the RO view; the now-default-true
+ // sync_snapshot_with_table_name must not be allowed to flip it to RT.
+ hiveSyncProps.setProperty(HIVE_SYNC_TABLE_STRATEGY.key(), HoodieSyncTableStrategy.ALL.name());
+ hiveSyncProps.setProperty(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE.key(), "true");
+ hiveSyncProps.setProperty(META_SYNC_SNAPSHOT_WITH_TABLE_NAME.key(), "true");
+ hiveSyncProps.setProperty(HIVE_SYNC_AS_DATA_SOURCE_TABLE.key(), "true");
+
+ String instantTime = "100";
+ String deltaCommitTime = "101";
+ HiveTestUtil.createMORTable(instantTime, deltaCommitTime, 5, true, true);
+
+ reInitHiveSyncClient();
+ reSyncHiveTable();
Review Comment:
I'll check if this regression test can be improved. Thanks for raising this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19427:
URL: https://github.com/apache/hudi/pull/19427#issuecomment-5137227622
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15871",
"triggerID" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0fabafc3734ddeb93aef0ecce972267e6588d0a9 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15871)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "codecov-commenter (via GitHub)" <gi...@apache.org>.
codecov-commenter commented on PR #19427:
URL: https://github.com/apache/hudi/pull/19427#issuecomment-5136546428
## [Codecov](https://app.codecov.io/gh/apache/hudi/pull/19427?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 75.56%. Comparing base ([`633d142`](https://app.codecov.io/gh/apache/hudi/commit/633d1427f35fd2d4a21c6e02363dc8dd90239237?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)) to head ([`0fabafc`](https://app.codecov.io/gh/apache/hudi/commit/0fabafc3734ddeb93aef0ecce972267e6588d0a9?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)).
<details><summary>Additional details and impacted files</summary>
```diff
@@ Coverage Diff @@
## master #19427 +/- ##
============================================
- Coverage 75.56% 75.56% -0.01%
Complexity 32651 32651
============================================
Files 2574 2574
Lines 142995 142994 -1
Branches 17530 17530
============================================
- Hits 108050 108049 -1
Misses 26908 26908
Partials 8037 8037
```
| [Components](https://app.codecov.io/gh/apache/hudi/pull/19427/components?src=pr&el=components&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hudi-common](https://app.codecov.io/gh/apache/hudi/pull/19427/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `82.27% <ø> (ø)` | |
| [hudi-client](https://app.codecov.io/gh/apache/hudi/pull/19427/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `81.83% <ø> (ø)` | |
| [hudi-flink](https://app.codecov.io/gh/apache/hudi/pull/19427/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `84.03% <ø> (ø)` | |
| [hudi-spark-datasource](https://app.codecov.io/gh/apache/hudi/pull/19427/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `68.33% <ø> (ø)` | |
| [hudi-utilities](https://app.codecov.io/gh/apache/hudi/pull/19427/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `71.19% <ø> (ø)` | |
| [hudi-cli](https://app.codecov.io/gh/apache/hudi/pull/19427/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `15.32% <ø> (ø)` | |
| [hudi-hadoop](https://app.codecov.io/gh/apache/hudi/pull/19427/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `63.49% <ø> (ø)` | |
| [hudi-sync](https://app.codecov.io/gh/apache/hudi/pull/19427/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `70.72% <ø> (-0.01%)` | :arrow_down: |
| [hudi-io](https://app.codecov.io/gh/apache/hudi/pull/19427/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `79.57% <ø> (ø)` | |
| [hudi-timeline-service](https://app.codecov.io/gh/apache/hudi/pull/19427/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `83.44% <ø> (ø)` | |
| [hudi-cloud](https://app.codecov.io/gh/apache/hudi/pull/19427/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `64.00% <ø> (ø)` | |
| [hudi-kafka-connect](https://app.codecov.io/gh/apache/hudi/pull/19427/components?src=pr&el=component&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `53.96% <ø> (ø)` | |
| [Flag](https://app.codecov.io/gh/apache/hudi/pull/19427/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [hadoop-mr-java-client](https://app.codecov.io/gh/apache/hudi/pull/19427/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | `43.39% <ø> (+0.07%)` | :arrow_up: |
Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more.
| [Files with missing lines](https://app.codecov.io/gh/apache/hudi/pull/19427?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Coverage Δ | |
|---|---|---|
| [...c/main/java/org/apache/hudi/hive/HiveSyncTool.java](https://app.codecov.io/gh/apache/hudi/pull/19427?src=pr&el=tree&filepath=hudi-sync%2Fhudi-hive-sync%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fhudi%2Fhive%2FHiveSyncTool.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-aHVkaS1zeW5jL2h1ZGktaGl2ZS1zeW5jL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9odWRpL2hpdmUvSGl2ZVN5bmNUb29sLmphdmE=) | `82.14% <ø> (-0.08%)` | :arrow_down: |
</details>
<details><summary> :rocket: New features to boost your workflow: </summary>
- :snowflake: [Test Analytics](https://docs.codecov.com/docs/test-analytics): Detect flaky tests, report on failures, and find test suite problems.
- :package: [JS Bundle Analysis](https://docs.codecov.com/docs/javascript-bundle-analysis): Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19427:
URL: https://github.com/apache/hudi/pull/19427#issuecomment-5139337468
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15871",
"triggerID" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"triggerType" : "PUSH"
}, {
"hash" : "6bd8413ea897213532e1788e368f8cd25e886bac",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "6bd8413ea897213532e1788e368f8cd25e886bac",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0fabafc3734ddeb93aef0ecce972267e6588d0a9 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15871)
* 6bd8413ea897213532e1788e368f8cd25e886bac UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "hudi-agent (via GitHub)" <gi...@apache.org>.
hudi-agent commented on code in PR #19427:
URL: https://github.com/apache/hudi/pull/19427#discussion_r3686626271
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java:
##########
@@ -211,7 +211,16 @@ protected void doSync() {
syncHoodieTable(snapshotTableName, true, false);
// sync origin table for MOR
if (config.getBoolean(META_SYNC_SNAPSHOT_WITH_TABLE_NAME)) {
- syncHoodieTable(tableName, true, false);
+ if (config.getBoolean(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE)) {
+ // skip_ro_suffix explicitly claims the bare table name for the RO view;
+ // an implicit sync_snapshot_with_table_name default must not repurpose it as RT.
+ log.warn("{}=true claims the base table name '{}' for the read-optimized view; "
Review Comment:
🤖 nit: "base table name" could be confused with Hudi's base-file concept — the test comments consistently use "bare table name" (meaning the unsuffixed name), which is clearer here too.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19427:
URL: https://github.com/apache/hudi/pull/19427#issuecomment-5140998033
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"status" : "DELETED",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15871",
"triggerID" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"triggerType" : "PUSH"
}, {
"hash" : "6bd8413ea897213532e1788e368f8cd25e886bac",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15880",
"triggerID" : "6bd8413ea897213532e1788e368f8cd25e886bac",
"triggerType" : "PUSH"
}, {
"hash" : "bce565b6c68b6f933d6aff0ed9de21be65f5d0a0",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15894",
"triggerID" : "bce565b6c68b6f933d6aff0ed9de21be65f5d0a0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 6bd8413ea897213532e1788e368f8cd25e886bac Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15880)
* bce565b6c68b6f933d6aff0ed9de21be65f5d0a0 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15894)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #19427:
URL: https://github.com/apache/hudi/pull/19427#discussion_r3688023756
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java:
##########
@@ -211,7 +211,16 @@ protected void doSync() {
syncHoodieTable(snapshotTableName, true, false);
// sync origin table for MOR
if (config.getBoolean(META_SYNC_SNAPSHOT_WITH_TABLE_NAME)) {
- syncHoodieTable(tableName, true, false);
+ if (config.getBoolean(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE)) {
Review Comment:
The issue above is different from what this PR fixes. I'll keep that separate.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19427:
URL: https://github.com/apache/hudi/pull/19427#issuecomment-5140362635
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15871",
"triggerID" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"triggerType" : "PUSH"
}, {
"hash" : "6bd8413ea897213532e1788e368f8cd25e886bac",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15880",
"triggerID" : "6bd8413ea897213532e1788e368f8cd25e886bac",
"triggerType" : "PUSH"
}, {
"hash" : "bce565b6c68b6f933d6aff0ed9de21be65f5d0a0",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15894",
"triggerID" : "bce565b6c68b6f933d6aff0ed9de21be65f5d0a0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0fabafc3734ddeb93aef0ecce972267e6588d0a9 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15871)
* 6bd8413ea897213532e1788e368f8cd25e886bac Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15880)
* bce565b6c68b6f933d6aff0ed9de21be65f5d0a0 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15894)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #19427:
URL: https://github.com/apache/hudi/pull/19427#discussion_r3688644650
##########
hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.java:
##########
@@ -1557,6 +1559,54 @@ public void testSyncMergeOnReadWithStrategy(String syncMode, HoodieSyncTableStra
}
}
+ @Test
+ void testSkipRoSuffixTakesPrecedenceOverSnapshotWithTableName() throws Exception {
+ // skip_ro_suffix explicitly claims the bare table name for the RO view; the now-default-true
+ // sync_snapshot_with_table_name must not be allowed to flip it to RT.
+ hiveSyncProps.setProperty(HIVE_SYNC_TABLE_STRATEGY.key(), HoodieSyncTableStrategy.ALL.name());
+ hiveSyncProps.setProperty(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE.key(), "true");
+ hiveSyncProps.setProperty(META_SYNC_SNAPSHOT_WITH_TABLE_NAME.key(), "true");
+ hiveSyncProps.setProperty(HIVE_SYNC_AS_DATA_SOURCE_TABLE.key(), "true");
+
+ String instantTime = "100";
+ String deltaCommitTime = "101";
+ HiveTestUtil.createMORTable(instantTime, deltaCommitTime, 5, true, true);
+
+ reInitHiveSyncClient();
+ reSyncHiveTable();
+
+ // a second sync round with a new commit: the bare table name's read-optimized sync (which runs
+ // immediately before the origin-table sync within the same doSync() call) advances its own
+ // last-commit-time-synced marker, so the origin-table step is the only one still eligible to run
+ // and flip the format if not guarded
Review Comment:
Dropped the explanation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19427:
URL: https://github.com/apache/hudi/pull/19427#issuecomment-5136473947
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0fabafc3734ddeb93aef0ecce972267e6588d0a9 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #19427:
URL: https://github.com/apache/hudi/pull/19427#discussion_r3687627121
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java:
##########
@@ -211,7 +211,16 @@ protected void doSync() {
syncHoodieTable(snapshotTableName, true, false);
// sync origin table for MOR
if (config.getBoolean(META_SYNC_SNAPSHOT_WITH_TABLE_NAME)) {
- syncHoodieTable(tableName, true, false);
+ if (config.getBoolean(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE)) {
+ // skip_ro_suffix explicitly claims the bare table name for the RO view;
+ // an implicit sync_snapshot_with_table_name default must not repurpose it as RT.
+ log.warn("{}=true claims the base table name '{}' for the read-optimized view; "
Review Comment:
+1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #19427:
URL: https://github.com/apache/hudi/pull/19427#discussion_r3688309235
##########
hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.java:
##########
@@ -1557,6 +1559,54 @@ public void testSyncMergeOnReadWithStrategy(String syncMode, HoodieSyncTableStra
}
}
+ @Test
+ void testSkipRoSuffixTakesPrecedenceOverSnapshotWithTableName() throws Exception {
+ // skip_ro_suffix explicitly claims the bare table name for the RO view; the now-default-true
+ // sync_snapshot_with_table_name must not be allowed to flip it to RT.
+ hiveSyncProps.setProperty(HIVE_SYNC_TABLE_STRATEGY.key(), HoodieSyncTableStrategy.ALL.name());
+ hiveSyncProps.setProperty(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE.key(), "true");
+ hiveSyncProps.setProperty(META_SYNC_SNAPSHOT_WITH_TABLE_NAME.key(), "true");
+ hiveSyncProps.setProperty(HIVE_SYNC_AS_DATA_SOURCE_TABLE.key(), "true");
+
+ String instantTime = "100";
+ String deltaCommitTime = "101";
+ HiveTestUtil.createMORTable(instantTime, deltaCommitTime, 5, true, true);
+
+ reInitHiveSyncClient();
+ reSyncHiveTable();
+
+ // a second sync round with a new commit: the bare table name's read-optimized sync (which runs
+ // immediately before the origin-table sync within the same doSync() call) advances its own
+ // last-commit-time-synced marker, so the origin-table step is the only one still eligible to run
+ // and flip the format if not guarded
Review Comment:
This comment says the read-optimized sync advancing its own last-commit-time-synced marker is what leaves the origin-table step eligible, but that advance is exactly why a single round does not reproduce the flip. Not a blocker: what actually keeps the origin step eligible in the second round is HoodieHiveSyncClient serving a per-run cached table snapshot taken before that write, so either name that or drop the explanation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on PR #19427:
URL: https://github.com/apache/hudi/pull/19427#issuecomment-5138684061
The single-round versus two-round difference behind my `TestHiveSyncTool` comment comes from `HoodieHiveSyncClient` caching the metastore table per run and never invalidating it after its own writes, which is also why the Glue client does not hit this path. Filed separately as #19431 since it needs a contract decision, and it does not block this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #19427:
URL: https://github.com/apache/hudi/pull/19427#issuecomment-5139593540
@wombatu-kun comments are addressed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on PR #19427:
URL: https://github.com/apache/hudi/pull/19427#issuecomment-5140218214
Comments addressed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19427:
URL: https://github.com/apache/hudi/pull/19427#issuecomment-5140320471
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15871",
"triggerID" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"triggerType" : "PUSH"
}, {
"hash" : "6bd8413ea897213532e1788e368f8cd25e886bac",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15880",
"triggerID" : "6bd8413ea897213532e1788e368f8cd25e886bac",
"triggerType" : "PUSH"
}, {
"hash" : "bce565b6c68b6f933d6aff0ed9de21be65f5d0a0",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "bce565b6c68b6f933d6aff0ed9de21be65f5d0a0",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0fabafc3734ddeb93aef0ecce972267e6588d0a9 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15871)
* 6bd8413ea897213532e1788e368f8cd25e886bac Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15880)
* bce565b6c68b6f933d6aff0ed9de21be65f5d0a0 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19427:
URL: https://github.com/apache/hudi/pull/19427#issuecomment-5136522801
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15871",
"triggerID" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0fabafc3734ddeb93aef0ecce972267e6588d0a9 Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15871)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #19427:
URL: https://github.com/apache/hudi/pull/19427#issuecomment-5139367837
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"status" : "SUCCESS",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15871",
"triggerID" : "0fabafc3734ddeb93aef0ecce972267e6588d0a9",
"triggerType" : "PUSH"
}, {
"hash" : "6bd8413ea897213532e1788e368f8cd25e886bac",
"status" : "PENDING",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15880",
"triggerID" : "6bd8413ea897213532e1788e368f8cd25e886bac",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* 0fabafc3734ddeb93aef0ecce972267e6588d0a9 Azure: [SUCCESS](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15871)
* 6bd8413ea897213532e1788e368f8cd25e886bac Azure: [PENDING](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=15880)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "yihua (via GitHub)" <gi...@apache.org>.
yihua commented on code in PR #19427:
URL: https://github.com/apache/hudi/pull/19427#discussion_r3688051380
##########
hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.java:
##########
@@ -1557,6 +1559,43 @@ public void testSyncMergeOnReadWithStrategy(String syncMode, HoodieSyncTableStra
}
}
+ @Test
+ void testSkipRoSuffixTakesPrecedenceOverSnapshotWithTableName() throws Exception {
+ // skip_ro_suffix explicitly claims the bare table name for the RO view; the now-default-true
+ // sync_snapshot_with_table_name must not be allowed to flip it to RT.
+ hiveSyncProps.setProperty(HIVE_SYNC_TABLE_STRATEGY.key(), HoodieSyncTableStrategy.ALL.name());
+ hiveSyncProps.setProperty(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE.key(), "true");
+ hiveSyncProps.setProperty(META_SYNC_SNAPSHOT_WITH_TABLE_NAME.key(), "true");
+ hiveSyncProps.setProperty(HIVE_SYNC_AS_DATA_SOURCE_TABLE.key(), "true");
+
+ String instantTime = "100";
+ String deltaCommitTime = "101";
+ HiveTestUtil.createMORTable(instantTime, deltaCommitTime, 5, true, true);
+
+ reInitHiveSyncClient();
+ reSyncHiveTable();
Review Comment:
In some cases `isAlreadySynced` can still retrun false so the bare table name is synced as RT again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(hive-sync): make skip_ro_suffix take precedence over sync_snapshot_with_table_name [hudi]
Posted by "wombatu-kun (via GitHub)" <gi...@apache.org>.
wombatu-kun commented on code in PR #19427:
URL: https://github.com/apache/hudi/pull/19427#discussion_r3687612274
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java:
##########
@@ -211,7 +211,16 @@ protected void doSync() {
syncHoodieTable(snapshotTableName, true, false);
// sync origin table for MOR
if (config.getBoolean(META_SYNC_SNAPSHOT_WITH_TABLE_NAME)) {
- syncHoodieTable(tableName, true, false);
+ if (config.getBoolean(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE)) {
+ // skip_ro_suffix explicitly claims the bare table name for the RO view;
+ // an implicit sync_snapshot_with_table_name default must not repurpose it as RT.
Review Comment:
The comment says an implicit `sync_snapshot_with_table_name` default must not repurpose the name, but this guard drops that config however the user set it - `HoodieSyncConfig` materializes the default into the props, so the distinction is not visible here. Reword the comment to drop the implicit-versus-explicit framing.
##########
hudi-sync/hudi-hive-sync/src/test/java/org/apache/hudi/hive/TestHiveSyncTool.java:
##########
@@ -1557,6 +1559,43 @@ public void testSyncMergeOnReadWithStrategy(String syncMode, HoodieSyncTableStra
}
}
+ @Test
+ void testSkipRoSuffixTakesPrecedenceOverSnapshotWithTableName() throws Exception {
+ // skip_ro_suffix explicitly claims the bare table name for the RO view; the now-default-true
+ // sync_snapshot_with_table_name must not be allowed to flip it to RT.
+ hiveSyncProps.setProperty(HIVE_SYNC_TABLE_STRATEGY.key(), HoodieSyncTableStrategy.ALL.name());
+ hiveSyncProps.setProperty(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE.key(), "true");
+ hiveSyncProps.setProperty(META_SYNC_SNAPSHOT_WITH_TABLE_NAME.key(), "true");
+ hiveSyncProps.setProperty(HIVE_SYNC_AS_DATA_SOURCE_TABLE.key(), "true");
+
+ String instantTime = "100";
+ String deltaCommitTime = "101";
+ HiveTestUtil.createMORTable(instantTime, deltaCommitTime, 5, true, true);
+
+ reInitHiveSyncClient();
+ reSyncHiveTable();
Review Comment:
Verified locally: with a fresh metastore and a single sync round the bare-name RT step is already skipped by the `isAlreadySynced` guard in `syncHoodieTable`, so this test passes on master with the production hunk reverted. Adding a second commit (`HiveTestUtil.addMORPartitions`) plus another `reSyncHiveTable()` round makes it fail as intended, with the bare table left on `HoodieParquetRealtimeInputFormat`.
##########
hudi-sync/hudi-hive-sync/src/main/java/org/apache/hudi/hive/HiveSyncTool.java:
##########
@@ -211,7 +211,16 @@ protected void doSync() {
syncHoodieTable(snapshotTableName, true, false);
// sync origin table for MOR
if (config.getBoolean(META_SYNC_SNAPSHOT_WITH_TABLE_NAME)) {
- syncHoodieTable(tableName, true, false);
+ if (config.getBoolean(HIVE_SKIP_RO_SUFFIX_FOR_READ_OPTIMIZED_TABLE)) {
Review Comment:
This looks like the fix for the still-open regression in #16637 (HUDI-8264, priority:high). Adding a `Fixes #16637` reference to the description would close it on merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org