You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "nada-attia (via GitHub)" <gi...@apache.org> on 2026/02/09 18:31:39 UTC
[PR] fix(metadata-table): Disable archival and clean on MDT if they are disabled on main table [hudi]
nada-attia opened a new pull request, #18145:
URL: https://github.com/apache/hudi/pull/18145
### Describe the issue this Pull Request addresses
When cleans and archivals are disabled on the data table, these operations should also be disabled on the metadata table (MDT). Currently, the MDT does not respect these settings and continues to perform archival and clean operations even when they are disabled on the main table.
### Summary and Changelog
**Summary:** When `hoodie.clean.automatic` or `hoodie.archive.automatic` are set to false on the data table, the metadata table should respect these settings and skip the corresponding operations.
**Changelog:**
- Modified `HoodieBackedTableMetadataWriter.performTableServices()` to check `dataWriteConfig.isAutoClean()` before running clean operations on MDT
- Modified `HoodieBackedTableMetadataWriter.performTableServices()` to check `dataWriteConfig.isAutoArchive()` before running archive operations on MDT
- Added unit test `testArchivalAndCleanRespectDataTableSettings` with parameterized test cases covering all combinations of autoClean/autoArchive settings
### Impact
This change affects metadata table maintenance behavior. When auto clean or auto archive is disabled on the data table, the corresponding operations will now also be skipped on the metadata table. This is the expected behavior since MDT should follow the data table's configuration.
No public API changes.
### Risk Level
low
The change is straightforward conditional checks that respect existing configuration settings. Unit tests have been added to verify the behavior.
### Documentation Update
none
The configs `hoodie.clean.automatic` and `hoodie.archive.automatic` already exist. This PR just ensures MDT respects these settings.
### Contributor's checklist
- [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(metadata-table): Disable archival and clean on MDT if they are disabled on main table [hudi]
Posted by "voonhous (via GitHub) via commits" <co...@hudi.apache.org>.
voonhous closed pull request #18145: fix(metadata-table): Disable archival and clean on MDT if they are disabled on main table
URL: https://github.com/apache/hudi/pull/18145
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(metadata-table): Disable archival and clean on MDT if they are disabled on main table [hudi]
Posted by "nsivabalan (via GitHub)" <gi...@apache.org>.
nsivabalan commented on PR #18145:
URL: https://github.com/apache/hudi/pull/18145#issuecomment-3880925137
I am not sure if we wanted to disable table services in MDT.
For eg, general recommendation w/ multi-writer scenario, is to enable table services (data table) in one writer and keep it disabled for all other data table writers.
but mdt is a single writer table and its automatically guarded by data table's archival IIUC.
so, once data table archival does not make progress, by design mdt archival may not be able to make much progress.
Trying to understand the necessity of this patch.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(metadata-table): Disable archival and clean on MDT if they are disabled on main table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18145:
URL: https://github.com/apache/hudi/pull/18145#issuecomment-3873298225
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "f59bb489198c660724f4dedc8c0f7dd0aa50f649",
"status" : "UNKNOWN",
"url" : "TBD",
"triggerID" : "f59bb489198c660724f4dedc8c0f7dd0aa50f649",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f59bb489198c660724f4dedc8c0f7dd0aa50f649 UNKNOWN
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(metadata-table): Disable archival and clean on MDT if they are disabled on main table [hudi]
Posted by "hudi-bot (via GitHub)" <gi...@apache.org>.
hudi-bot commented on PR #18145:
URL: https://github.com/apache/hudi/pull/18145#issuecomment-3874694512
<!--
Meta data
{
"version" : 1,
"metaDataEntries" : [ {
"hash" : "f59bb489198c660724f4dedc8c0f7dd0aa50f649",
"status" : "FAILURE",
"url" : "https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=11834",
"triggerID" : "f59bb489198c660724f4dedc8c0f7dd0aa50f649",
"triggerType" : "PUSH"
} ]
}-->
## CI report:
* f59bb489198c660724f4dedc8c0f7dd0aa50f649 Azure: [FAILURE](https://dev.azure.com/apachehudi/a1a51da7-8592-47d4-88dc-fd67bed336bb/_build/results?buildId=11834)
<details>
<summary>Bot commands</summary>
@hudi-bot supports the following commands:
- `@hudi-bot run azure` re-run the last Azure build
</details>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] fix(metadata-table): Disable archival and clean on MDT if they are disabled on main table [hudi]
Posted by "voonhous (via GitHub) via commits" <co...@hudi.apache.org>.
voonhous commented on PR #18145:
URL: https://github.com/apache/hudi/pull/18145#issuecomment-6700973744
Temporarily closing this PR while we debug a GitHub settings error on master (#20471); it will be reopened shortly, no action needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org