[jira] [Commented] (BOOKKEEPER-553) New LedgerManager for 64 Bits Ledger ID Management in ZooKeeper

["Jiannan Wang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BOOKKEEPER-553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13558190#comment-13558190 ] 

Jiannan Wang commented on BOOKKEEPER-553:
-----------------------------------------

HierarchicalLedgerManager use 2-4-4 style to split current 10 chars ledger id, E.g Ledger 0000000001 is splited into 3 parts 00,0000,0001 and stored in zookeeper path "(ledgersRootPath)/00/0000/L0001". So each znode could have at most 10000 ledgers, which avoids errors during garbage collection due to lists of children that are too long.

After we enlarge the ledger id space to 64 bits, it's a big problem to manage for large ledger id.

My idea is split the ledger id under the radix 2^13=8192 and then construct it in a radix tree. For example, ledger id 2, 5, and 41093(==5x8192+133) then the tree in zookeeper would be:
{quote}
         (ledger id root)
            /      \
        2 (meta)   5 (meta)
                     \
                  133 (meta)
{quote}
Then there will be at most 8192 children under each znode and the depth is (64/13=5) at most.

Note that the inner znode will also record metadata, so if ledger id generation is increasing step by step, then the depth of this radix tree only grows as needed. And I guess it can handle all 2^64 ledger ids ideally.
                
> New LedgerManager for 64 Bits Ledger ID Management in ZooKeeper
> ---------------------------------------------------------------
>
>                 Key: BOOKKEEPER-553
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-553
>             Project: Bookkeeper
>          Issue Type: Sub-task
>          Components: bookkeeper-server, hedwig-server
>            Reporter: Jiannan Wang
>            Assignee: Jiannan Wang
>             Fix For: 4.3.0
>
>
> Once we extend the ledger id space to 64 bits, the 2-4-4 ledger id partition (see HierarchicalLedgerManager) is not suitable any more. So we need a new LedgerManager for 64 bits ledger id management in ZooKeeper.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira