[jira] [Updated] (BOOKKEEPER-181) Scale hedwig

["Sijie Guo (Updated) (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/BOOKKEEPER-181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sijie Guo updated BOOKKEEPER-181:
---------------------------------

    Attachment: hedwigscale.pdf

Attach a draft design document of scaling hedwig/bookkeeper for discussion.

And I think this jira is a master task doing this work. It would better to divide it into several subtasks.

* Provide a MetaStore interface, and a possible implementation such as HBase.
* BookKeeper
** Provide a clean LedgerManager interface to move zookeeper-related meta operations to zk-based ledger manager.
** Implement a MetaStore-based ledger manager to leverage scalability of a scalable MetaStore.
* Hedwig
** Provide a meta operation interface HedwigMetaManager over hedwig, just similar as what LedgerManager does in bookkeeper.
** Implement a MetaStore-based Hedwig meta manager.
                
> Scale hedwig
> ------------
>
>                 Key: BOOKKEEPER-181
>                 URL: https://issues.apache.org/jira/browse/BOOKKEEPER-181
>             Project: Bookkeeper
>          Issue Type: Improvement
>          Components: bookkeeper-server, hedwig-server
>            Reporter: Sijie Guo
>            Assignee: Sijie Guo
>         Attachments: hedwigscale.pdf
>
>
> Current implementation of Hedwig and BookKeeper is designed to scale to hundreds of thousands of topics, but now we are looking at scaling them to tens to hundreds of millions of topics, using a scalable key/value store such as HBase.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira