[jira] [Commented] (MESOS-6267) Mesos master should periodically compact the replicated log

["Avinash Sridharan (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MESOS-6267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15538798#comment-15538798 ] 

Avinash Sridharan commented on MESOS-6267:
------------------------------------------

[~devroot] would you know if anyone is actively working on this? Don't see a shepherd or an assignee.

> Mesos master should periodically compact the replicated log
> -----------------------------------------------------------
>
>                 Key: MESOS-6267
>                 URL: https://issues.apache.org/jira/browse/MESOS-6267
>             Project: Mesos
>          Issue Type: Improvement
>          Components: replicated log
>            Reporter: David Robinson
>            Priority: Minor
>
> The mesos master does not periodically compact the replicated log's leveldb, it grows without bound. Compaction only occurs on startup, therefore in large clusters (10k+ agents) where the master is restarted infrequently compaction at startup can take 2+ minutes to complete. At Twitter we've seen the DB grow to 170+ GB in less than a month and compaction take > 2 minutes. Compaction reduces storage usage from 160 GB to roughly 50 MB.
> {noformat}
> I0928 06:06:20.776908 19725 leveldb.cpp:174] Opened db in 763.082271ms
> I0928 06:08:41.636145 19725 leveldb.cpp:181] Compacted db in 2.34765004808333mins
> {noformat}
> The replicated log should not grow without bounds.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)