You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by "Sathyaprakash Govindasamy (Jira)" <ji...@apache.org> on 2020/05/24 23:08:00 UTC
[jira] [Updated] (HUDI-942) Increase minmum number of delta commits
for inline compaction
[ https://issues.apache.org/jira/browse/HUDI-942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sathyaprakash Govindasamy updated HUDI-942:
-------------------------------------------
Description:
Default value of number of delta commits required to do inline compaction (hoodie.compact.inline.max.delta.commits) is currently 1. Because of this by default every delta commit to MERGE ON READ table is doing inline compaction as well automatically.
I think default value of 1 is little overkill and also it will make MERGE ON READ work like COPY ON WRITE with compaction on every run. We should increate the default value to more than 1.
https://github.com/apache/hudi/blob/f34de3fb2738c8c36c937eba8df2a6848fafa886/hudi-client/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java#L100
was:
inlineCompactionEvery method in HoodieCompactionConfig.Builder is incorrectly setting number of delta commits to INLINE_COMPACT_PROP property which is boolean variable
I think withMaxNumDeltaCommitsBeforeCompaction method is doing the purpose of setting num of delta commits before compaction.
So, I am proposing to remove inlineCompactionEvery method, which is not used anywhere
> Increase minmum number of delta commits for inline compaction
> -------------------------------------------------------------
>
> Key: HUDI-942
> URL: https://issues.apache.org/jira/browse/HUDI-942
> Project: Apache Hudi
> Issue Type: Bug
> Components: Compaction
> Affects Versions: 0.6.0
> Reporter: Sathyaprakash Govindasamy
> Priority: Minor
> Labels: pull-request-available
> Fix For: 0.6.0
>
>
> Default value of number of delta commits required to do inline compaction (hoodie.compact.inline.max.delta.commits) is currently 1. Because of this by default every delta commit to MERGE ON READ table is doing inline compaction as well automatically.
> I think default value of 1 is little overkill and also it will make MERGE ON READ work like COPY ON WRITE with compaction on every run. We should increate the default value to more than 1.
> https://github.com/apache/hudi/blob/f34de3fb2738c8c36c937eba8df2a6848fafa886/hudi-client/src/main/java/org/apache/hudi/config/HoodieCompactionConfig.java#L100
--
This message was sent by Atlassian Jira
(v8.3.4#803005)