You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "lucabem (via GitHub)" <gi...@apache.org> on 2026/03/31 12:32:49 UTC
[I] SQL-Procedure -> archive_commits [hudi]
lucabem opened a new issue, #18429:
URL: https://github.com/apache/hudi/issues/18429
### Bug Description
**What happened:**
I was using archive_commits with retain_commits = 200 due to multi-writer approach into one table. I have noticed that this parameter in doc says:
```
Archiving of instants is batched in best-effort manner, to pack more instants into a single archive log. This config controls such archival batch size.
```
But if you check the code, the HoodieWriteConfig is built as
```
HoodieWriteConfig config = HoodieWriteConfig.newBuilder().withPath(basePath)
.withArchivalConfig(HoodieArchivalConfig.newBuilder().archiveCommitsWith(minCommits, maxCommits).build())
.withCleanConfig(HoodieCleanConfig.newBuilder().retainCommits(commitsRetained).build())
.withEmbeddedTimelineServerEnabled(false)
.withMetadataConfig(HoodieMetadataConfig.newBuilder().enable(enableMetadata).build())
.build();
```
I think it must map the conf `COMMITS_ARCHIVAL_BATCH_SIZE`, because current logs shows
```
2026-03-31 11:15:39.232 [Thread-10] WARN org.apache.hudi.config.HoodieWriteConfig - Increase hoodie.keep.min.commits=4 to be greater than hoodie.cleaner.commits.retained=200 (there is risk of incremental pull missing data from few instants based on the current configuration). The Hudi archiver will automatically adjust the configuration regardless.
```
but code must point
```
public static final ConfigProperty<String> COMMITS_ARCHIVAL_BATCH_SIZE = ConfigProperty
.key("hoodie.commits.archival.batch")
.defaultValue(String.valueOf(10))
.markAdvanced()
.withDocumentation("Archiving of instants is batched in best-effort manner, to pack more instants into a single"
+ " archive log. This config controls such archival batch size.");
```
### Environment
**Hudi version:**
**Query engine:** (Spark/Flink/Trino etc)
**Relevant configs:**
### Logs and Stack Trace
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] SQL-Procedure -> archive_commits [hudi]
Posted by "salvador991998 (via GitHub)" <gi...@apache.org>.
salvador991998 commented on issue #18429:
URL: https://github.com/apache/hudi/issues/18429#issuecomment-4162384645
Thanks for reporting this! I'm running into the exact same behavior and it's impacting my current project. Any updates of when this might be adressed would be helpful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] SQL-Procedure -> archive_commits [hudi]
Posted by "deepakpanda93 (via GitHub)" <gi...@apache.org>.
deepakpanda93 closed issue #18429: SQL-Procedure -> archive_commits
URL: https://github.com/apache/hudi/issues/18429
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] SQL-Procedure -> archive_commits [hudi]
Posted by "deepakpanda93 (via GitHub)" <gi...@apache.org>.
deepakpanda93 commented on issue #18429:
URL: https://github.com/apache/hudi/issues/18429#issuecomment-5355696380
"Thanks for reporting this.
This looks addressed by #18437 (*fix(spark): Add options for archive procedure*, merged 2026-05-26).
Note this merged after 1.2.0, so it is on `master` and will ship in the next release rather than being available in 1.2.0.
Closing on that basis - please reopen or comment if you still see the behaviour on a build containing that change."
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] SQL-Procedure -> archive_commits [hudi]
Posted by "fhan688 (via GitHub)" <gi...@apache.org>.
fhan688 commented on issue #18429:
URL: https://github.com/apache/hudi/issues/18429#issuecomment-4169922087
this may help.
https://github.com/apache/hudi/pull/18437
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] SQL-Procedure -> archive_commits [hudi]
Posted by "fhan688 (via GitHub)" <gi...@apache.org>.
fhan688 commented on issue #18429:
URL: https://github.com/apache/hudi/issues/18429#issuecomment-4169916896
https://github.com/apache/hudi/pull/18437
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org