[jira] [Updated] (HUDI-4794) add an option of the log file block size

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HUDI-4794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ASF GitHub Bot updated HUDI-4794:
---------------------------------
    Labels: pull-request-available  (was: )

> add an option of the log file block size
> ----------------------------------------
>
>                 Key: HUDI-4794
>                 URL: https://issues.apache.org/jira/browse/HUDI-4794
>             Project: Apache Hudi
>          Issue Type: Improvement
>            Reporter: zhaoyangming
>            Priority: Major
>              Labels: pull-request-available
>
> on each hoodie log append, hdfs used will be increased with the length of the block(512M), not teh actual data length().
> Consider in a scenario,I use many writers to append concurrently to a large number of files(bucket file),but each time I append only 10 bytes.
> dfs used will be increased with the length of the block(512M),this will cause the datanode to report in-sufficient disk space on data write.
> even though it related to HDFS, We should also have the option to modify the configuration.It helps reduce the rate of increase during the du.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)