You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@ignite.apache.org by "Dmitriy Pavlov (JIRA)" <ji...@apache.org> on 2018/10/30 11:04:00 UTC
[jira] [Resolved] (IGNITE-7730) Improve WAL history size
documentation
[ https://issues.apache.org/jira/browse/IGNITE-7730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dmitriy Pavlov resolved IGNITE-7730.
------------------------------------
Resolution: Done
According there is a new approach to specify WAL history size in bytes
IGNITE-6552
there is no need to deep specification calculation algorithm.
The algorithm is specified in wiki https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-Estimatingdiskspace for an advanced user/Ignite developer
> Improve WAL history size documentation
> --------------------------------------
>
> Key: IGNITE-7730
> URL: https://issues.apache.org/jira/browse/IGNITE-7730
> Project: Ignite
> Issue Type: Task
> Components: documentation
> Affects Versions: 2.1
> Reporter: Dmitriy Pavlov
> Assignee: Dmitriy Pavlov
> Priority: Major
> Fix For: 2.7
>
>
> Until IGNITE-6552 is not implemented, we have only ability to configure WAL hist. size in checkpoints.
> It is needed to improve description for this parameter.
> I've added draft notes to wiki [https://cwiki.apache.org/confluence/display/IGNITE/Ignite+Persistent+Store+-+under+the+hood#IgnitePersistentStore-underthehood-Estimatingdiskspace]
> about ways how wer can estimate WAL sizes without exact bytes/time specification:
> {panel}
> WAL Work max used size: walSegmentSize * walSegments = 640Mb (default)
> in case Default WAL mode - this size is used always,
> in case other modes best case is 1 segment * walSegmentSize
> WAL Work+WAL Archive max size may be estimated by
> 1. average load or
> 2. by maximum size.
> 1st way is applicable if checkpoints are triggered mostly by timer trigger.
> Wal size = 2*Average load(bytes/sec) * trigger interval (sec) * walHistSize (number of checkpoints)
> Where 2 multiplier coming from physical & logical WAL Records.
> 2nd way: Checkpoint is triggered by segments max dirty pages percent. Use persisted data regions max sizes:
> sum(Max configured DataRegionConfiguration.maxSize) * 75% - est. maximum data volume to be writen on 1 checkpoint.
> Overall WAL size (before archiving) = 2* est. data volume * walHistSize = 1,5 * sum(DataRegionConfiguration.maxSize) * walHistSize
> Note applying WAL compressor may significiantly reduce archive size.
> {panel}
> One more note from [~ivan.glukos] on dev.list we need to include. It is answer to question how user can determine if segment from archive folder can be safely removed:
> {quote}By the way: WAL compression is already implemented that way. If there
> are any ".zip" segments in archive dir, they are free to delete.
> This can be a safe workaround for users who experience lack of free
> space - just delete compressed segments. We should mention it in
> documentation for 2.4 release.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)