[jira] [Commented] (BOOKKEEPER-590) Another Scan-And-Compare GC Implementation

["Jiannan Wang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BOOKKEEPER-590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13608785#comment-13608785 ] 

Jiannan Wang commented on BOOKKEEPER-590:
-----------------------------------------

Here is a concern from Sijie: after removing this order guarantee we need a copy of local active ledger list to process new implementation, which uses more memory.

But I think this should be acceptable, since 1m ledgers id only use tens of M bytes memory.
                
> Another Scan-And-Compare GC Implementation
> ------------------------------------------
>
>                 Key: BOOKKEEPER-590
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-590
>             Project: Bookkeeper
>          Issue Type: Improvement
>          Components: bookkeeper-server
>            Reporter: Jiannan Wang
>            Assignee: Jiannan Wang
>
> The idea of Scan-And-Compare GC is as below:
>    * Assume the ledger id list in local bookie server is *LocalLedgers*
>    * At the same time, the ledger id list at metadata storage is *LiveLedgers*
>    * Then the ledgers require garbage collection are *LocalLedgers - LiveLedgers*
> Under current implementation, an ledger id order guarantee is required when obtain *LiveLedgers* from metadata storage. However, this is unnecessary: we get *LocalLedgers* and we can just remove elements that in *LiveLedgers* one by one in any order.
> What's more, without the order requirement when scan all ledger ids, some things become simple:
>    * We even don't need radix tree to maintain 64-bits ledger metadata, a hierarchical hash tree is enough (just as what topic metadata management does).
>    * Easy to handle 64-bit ledger id backward compatibility for MSLedgerManager:
>       ** Currently, for MSLedgerManager, we format ledger id to a fixed length (it's 10 now) digit string to make order scan
>       ** When a 64-bit ledger id is used we need to enlarge the fixed length, then old ledger id backward compatibility turns to be a trouble if we require this order guarantee.
> As above reasons, it would better to remove specific order requirement from current Scan-And-Compare GC implementation.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira