[jira] [Commented] (BOOKKEEPER-864) 128 bit LedgerId

["Sijie Guo (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/BOOKKEEPER-864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14725483#comment-14725483 ] 

Sijie Guo commented on BOOKKEEPER-864:
--------------------------------------

I've been looking around the code base for a while. The current code base is kind of heavily tight with Long (64bits) ledger id.

- the bookie entry format is formed with ledger_id, entry_id & data.
- the bookie storage is using ledger id as the key on referencing entries. especially -1 (INVALID_LEDGER_ID) is used in the server for identifying as a special entry
- the network wire protocol is also tight with ledger id

It isn't a straightforward change to move ledger id from 64 bits to 128 bits. It involves changes from storage, network wire protocol to client interface. so if we are going to support 128 bit ledger id, I would suggest supporting byte[] as the 'ledger id'. the challenging part of this change is how to make backward compatibility. 

my thinking over this is:

- moving the ledger storage to support byte[] as the ledger id first. providing interfaces like addEntry(byte[] ledger_name, long entry_id, ByteBuffer data) and readEntry(byte[] ledger_name, long entry_id, ByteBuffer data).
- the original wire protocol (using long as the ledger id), could covert long ledger id to byte[] and adding/read entries
- adding a new wire protocol (typically a new AddRequest, ReadRequest), which use byte[] as the ledger id
- changing the ledger metadata manager to use byte[] as the ledger id
- for client side, we convert long to bytes[] to support backward clients who use long as the ledger id. we should propose a brand new client which uses byte[] as the ledger id (so which we could propose a clean API to get rid of some drawbacks of client API).

the most critical part of this change is to take care of backward compatibility. 

> 128 bit LedgerId
> ----------------
>
>                 Key: BOOKKEEPER-864
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-864
>             Project: Bookkeeper
>          Issue Type: Improvement
>            Reporter: Venkateswararao Jujjuri
>
> BookKeeper coordinates with ZooKeeper to generate an cluster wide LedgerId. This is a 64 bit number. This method works great because we have ZK acting as a centralized coordinator.  But this method may not scale as the cluster size and number of ledgers increase. 
> GUIDs ( (https://en.wikipedia.org/wiki/Globally_unique_identifier) are preferred way to generate decentralized globally unique IDs and it takes 128 bits ; This method can scale well as it doesn’t need a centralized coordination. 
> Current BK code used ledgerId as a primary object, any change to this (64 to128 bit) warrants across the board code changes. 
> Opening this issue so we can solicit community input and track the progress.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)