You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@activemq.apache.org by "Alan Protasio (JIRA)" <ji...@apache.org> on 2018/11/07 02:16:01 UTC
[jira] [Comment Edited] (AMQ-7082) KahaDB index, recover free pages
in parallel with start
[ https://issues.apache.org/jira/browse/AMQ-7082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677379#comment-16677379 ]
Alan Protasio edited comment on AMQ-7082 at 11/7/18 2:15 AM:
-------------------------------------------------------------
[~gtully] [~cshannon]
Sorry guys... but there is one more case that this can happen...
Imagine that you have a 1000 page in the file...
T1 -> 1000 occupied pages
T2 -> page 100 is freed (and so added to the freeList)
T3 -> Background thread see the 100 page as free and so, add to the recoveryList
T3 -> We need one more page and we allocate page 100 (as it was marked as free in the freeList).
T4 -> background thread finish and mark 100 as free again...
I did a PR there to prevent this... I did everything in a way that we dont need synchronization...
I think now we pretty much got all the cases...
was (Author: alanprot):
[~gtully] [~cshannon]
Sorry guys... but there is one more case that this can happen...
Imagine that you have a 1000 page in the file...
T1 -> 1000 occupied pages
T2 -> page 100 is freed (and so added to the freeList)
T3 -> Background thread see the 100 page as free and so, add to the recoveryList
T3 -> We need one more page and we allocate page 100 (its not occupied).
T4 -> background thread finish and mark 100 as free again...
I did a PR there to prevent this... I did everything in a way that we dont need synchronization...
> KahaDB index, recover free pages in parallel with start
> -------------------------------------------------------
>
> Key: AMQ-7082
> URL: https://issues.apache.org/jira/browse/AMQ-7082
> Project: ActiveMQ
> Issue Type: Bug
> Components: KahaDB
> Affects Versions: 5.15.0
> Reporter: Gary Tully
> Assignee: Gary Tully
> Priority: Major
> Fix For: 5.16.0, 5.15.8
>
>
> AMQ-6590 fixes free page loss through recovery. The recover process can be timely, which prevents fast failover, doing recovery on shutdown is preferable, but it is still not ideal b/c it will hold onto the kahadb lock. It also can stall shutdown unexpectedly.
> AMQ-7080 is going to tackle checkpointing the free list. This should help avoid the need for recovery but it may still be necessary. If the perf hit is significant this may need to be optional.
> There will still be the need to walk the index to find the free list.
> It is possible to run with no free list and grow, and we can do that while we recover the free list in parallel, then merge the two at a safe point. This we can do at startup.
> In cases where the disk is the bottleneck this won't help much, but it will help failover and it will help shutdown, with a bit of luck the recovery will complete before we stop.
>
> Initially I thought this would be too complex, but if we concede some growth while we recover, ie: start with an empty free list, it is should be straight forward to merge with a recovered one.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)