You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2019/12/30 12:44:12 UTC
[GitHub] [incubator-hudi] dengziming opened a new pull request #1158:
[MINOR]: alter some wrong params which bring fatal exception
dengziming opened a new pull request #1158: [MINOR]: alter some wrong params which bring fatal exception
URL: https://github.com/apache/incubator-hudi/pull/1158
## What is the purpose of the pull request
the `HoodieCompactionConfig.build()` method will check arguments to ensure MIN_COMMITS_TO_KEEP_PROP > CLEANER_COMMITS_RETAINED_PROP, and DEFAULT_CLEANER_COMMITS_RETAINED = "10", so we need to set MIN_COMMITS_TO_KEEP_PROP at least 11.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] dengziming commented on a change in pull request
#1158: [MINOR]: alter some wrong params which bring fatal exception
Posted by GitBox <gi...@apache.org>.
dengziming commented on a change in pull request #1158: [MINOR]: alter some wrong params which bring fatal exception
URL: https://github.com/apache/incubator-hudi/pull/1158#discussion_r361977514
##########
File path: hudi-client/src/test/java/HoodieClientExample.java
##########
@@ -95,7 +95,7 @@ public void run() throws Exception {
HoodieWriteConfig cfg = HoodieWriteConfig.newBuilder().withPath(tablePath)
.withSchema(HoodieTestDataGenerator.TRIP_EXAMPLE_SCHEMA).withParallelism(2, 2).forTable(tableName)
.withIndexConfig(HoodieIndexConfig.newBuilder().withIndexType(IndexType.BLOOM).build())
- .withCompactionConfig(HoodieCompactionConfig.newBuilder().archiveCommitsWith(2, 3).build()).build();
+ .withCompactionConfig(HoodieCompactionConfig.newBuilder().archiveCommitsWith(20, 30).build()).build();
Review comment:
if we set minToKeep=2, the app will exit with `java.lang.IllegalArgumentException: Increase hoodie.keep.min.commits=2 to be greater than hoodie.cleaner.commits.retained=10. Otherwise, there is risk of incremental pull missing data from few instants`.
just set to DEFAULT_MIN_COMMITS_TO_KEEP and DEFAULT_MAX_COMMITS_TO_KEEP
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] bvaradar merged pull request #1158: [MINOR]: alter
some wrong params which bring fatal exception
Posted by GitBox <gi...@apache.org>.
bvaradar merged pull request #1158: [MINOR]: alter some wrong params which bring fatal exception
URL: https://github.com/apache/incubator-hudi/pull/1158
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services