You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@bookkeeper.apache.org by "Ivan Kelly (Commented) (JIRA)" <ji...@apache.org> on 2011/10/07 12:36:29 UTC
[jira] [Commented] (BOOKKEEPER-70) reduce or demultiplex
subscription connections to a hub server
[ https://issues.apache.org/jira/browse/BOOKKEEPER-70?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122684#comment-13122684 ]
Ivan Kelly commented on BOOKKEEPER-70:
--------------------------------------
Comments which have been made about this before:
Current, in Hedwig, each topic subscription has its own network channel. Thus for every topic present on a hub, it uses
up one port for connecting to the subscriber, and one each for subscribing to that topic in all the other regions. If
we have a 3 region setup, each topic takes up a minimum of 4 ports. Since there are 64K possible ports, a hub can only
handle 16K topics. This number is less in reality because some ports are system reserved, others are need for the
publish calls etc. This number goes down further if we have more regions.
The client should be changed to maintain at most one subscription channel to each hub. Multiple subscriptions can then
be serviced over the same channel. This will allow us to scale independent of the number of topics but in the number of
hosts we have to connect to.
This will require a couple of server-side changes as well:
i) txn-id of original subscribe request should be sent back with every request, so that client can decide which
subscriber the message is for.
ii) Currently the server has a map of channel to subscriber, that should be changed to a multimap.
> reduce or demultiplex subscription connections to a hub server
> --------------------------------------------------------------
>
> Key: BOOKKEEPER-70
> URL: https://issues.apache.org/jira/browse/BOOKKEEPER-70
> Project: Bookkeeper
> Issue Type: Improvement
> Components: hedwig-client, hedwig-server
> Reporter: Sijie Guo
>
> now each subscription will establish a connection to hub server, there will be too many connections on both client/server sides.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira