[jira] [Assigned] (KUDU-1830) Reduce Kudu WAL log disk usage

["Adar Dembo (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KUDU-1830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Adar Dembo reassigned KUDU-1830:
--------------------------------

    Assignee: Adar Dembo

> Reduce Kudu WAL log disk usage
> ------------------------------
>
>                 Key: KUDU-1830
>                 URL: https://issues.apache.org/jira/browse/KUDU-1830
>             Project: Kudu
>          Issue Type: Sub-task
>          Components: consensus, log
>            Reporter: Juan Yu
>            Assignee: Adar Dembo
>              Labels: data-scalability
>
> WAL log can take significent disk space. So far there are some config to limit it. but it can go very high.
> WAL log size = #tablets * log_segment_size_mb  * log segments (1 if there is write ops to this tablet, can go up to log_max_segments_to_retain)
> Logs are retained even if there is no write for a while.
> We could reduce the WAL log usage by
> - reduce min_segments_to_retain to 1 instead of 2, a
> - reduce steady state consumption of idle tablets, roll a WAL if it has had no writes for a few minutes and size more than a MB or two so that "idle" tablets have 0 WAL space consumed



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)