[jira] [Commented] (AMQ-2875) Create a for ActiveMQ to provide better support for hierarchical broker topologies

["Matt Pavlovich (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/AMQ-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13594778#comment-13594778 ] 

Matt Pavlovich commented on AMQ-2875:
-------------------------------------

Hi Adrian-

I definitely agree that there is room for improvement on the networkConnector/network of brokers front to make it more usable, specifically in the large-scale scenarios you describe. 

A couple of comments:

1. Any improvement or modification to networkConnector-type configuration would be greatly enhanced by having the configuration be dynamically updatable and persisted. (ie.. save JMX/REST config changes to the activemq.xml file)

2. What do you think about supporting the "host" portion of the JMS url for routing? It seems to me that what you are describing might be achieved by supporting that, instead of tracking advisories up and down nodes. Having the ability to support routing a message to "kiosk46" when specifying the destination as "queue://kiosk46/MY_QUEUE" and client sending message to "brokerA" (HQ) might be easier to implement and manage than a number of network/tree connectors. This would also make migrations from commercial platforms that currently support this, such as MQ series, more straightforward. Thoughts?

In my personal experience from implementing ActiveMQ at a global organization with 1000's of nodes is that the number of queues can become problematic in a straight-forward network-of-brokers model. Also, if all the endpoints have unique queues, it pushes the complexity to the application developers which creates a negative developer experience (imho).

[Preferred-- Apps deployed to kiosk46 know to connect to queues and don't need to know anything about its current host]
Host: Kiosk46
  - queue:///ORDERS_Q
  - queue:///TICKETS_Q
  - queue:///INVENTORY_Q
Host: Kiosk48
  - queue:///ORDERS_Q
  - queue:///TICKETS_Q
  - queue:///INVENTORY_Q

[Versus-- Apps need to consider their location when creating queue names]
Host: Kiosk46
  - queue:///K46_ORDERS_Q
  - queue:///K46_TICKETS_Q
  - queue:///K46_INVENTORY_Q

host: Kiosk48
  - queue:///K48_ORDERS_Q
  - queue:///K48_TICKETS_Q
  - queue:///K48_INVENTORY_Q

Thanks, 
Matt Pavlovich
                
> Create a <treeConnector> for ActiveMQ to provide better support for hierarchical broker topologies
> --------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-2875
>                 URL: https://issues.apache.org/jira/browse/AMQ-2875
>             Project: ActiveMQ
>          Issue Type: New Feature
>            Reporter: Adrian Trenaman
>
> The ActiveMQ network connector is excellent at facilitating self-organizing networks of ActiveMQ brokers with dynamically discovered routing, and can be used in situations where network availability is sporadic and not guaranteed. Network connectors can be used along with master-slave pairs to create a distributed 'messaging fabric'. The network connector can also accommodate the creation of hierarchical networks, however, I believe there is a good argument for treating tree topologies as a special case in their own right. In this note, I'm going to describe why trees should be treated as a special case, and describe what a treeConnector might look like.
> Hierarchical (or 'tree') networks are a special case of the abstract notion of a network; they have numerous applications in the real world in cross-geography, wide-area deployments. Consider a retail organization with 1,000's of stores, who wish to send and receive information to and from head-quarters (HQ) and the stores: you can envisage a hierarchy of ActiveMQ brokers: one for HQ, one for each of the regions (e.g. Ireland, UK, France, Germany), and then a broker in each store/outlet in each of the regions. When HQ wants to send a message to a store, it should be able to write a message to the store's queue and have it dynamically routed to the store via the regional broker. Likewise, if a store wants to send a message to HQ, it should be able to write to a local queue in the store and have the message dynamically routed to HQ. 
> Right now in ActiveMQ, there are a number of issues with using a network connector to achieve a tree topology: 
> * Spillage of consumer advisories. When a consumer connects to a store, an advisory message is generated and sent to all brokers within the networkTTL range. These advisories don't just travel up the tree to the top (HQ): they can also spill back down the tree to peers who are simply not interested. We have seen this in a network with 1,000 brokers connected to a regional broker, and a networkTTL of two: the advisory gets sent to the regional broker, HQ, *and* the 999 peers of the broker. 
> * Resource wastage (as a consequence of spillage). When a consumer advisory from a store broker (say, broker 'A') reaches another broker within the networkTTL (say, broker 'B'), then 'B' creates a subscription for the destination (to do forwarding) *and* can result in the allocation of a thread for this destination. So, it's possible that Broker B will allocate a thread for the destination 'BrokerA.Incoming', despite the fact that it will never receive or send to this queue. If there are only a small number of peers at this level then this is not a problem; however, if there are many peers (as in the case of 1000 stores per region) then this will be noticeable.
> * Sensitivity to networkTTL. In order to minimize spillage and resource wastage, you need to se the networkTTL on each broker to the distance between the broker and the root - in this example, 2. However, a later reconfiguration of the network could result in this setting being too low (which means messages won't go the distance) or too high (which means you get inadvertent spillage)
> These issues can be addressed if we create a new tree connector for ActiveMQ. 
> 1) Brokers would identify themselves as 'leaf', 'branch' or 'root' nodes. There should only ever be one root. If a broker has not been identified as either 'leaf' or 'branch' then it should consider itself to be the root.
> 2) Tree connectors would be configured using a 'many to one' approach. Leaf brokers would configure a tree connector to their branch. Branch brokers would configure a tree connector to the root. The root shouldn't need to configure anything, except of course a transport listener. 
> So, a branch broker for the 'UK' region might have the following  (to connect to headquarters): 
> 	<treeConnector nodeType="branch" uri=""tcp://headquarters.myorg.com:61616"/>
> And, a leaf broker at a store or outlet in the UK might have the following (to connect to the 'UK' region'): 
> 	<treeConnector nodeType="leaf" uri=""tcp://uk.myorg.com:61616"/>
> 3) If a consumer connects to a leaf node, then the consumer advisory travels up the tree, all the way to root: the concept of networkTTL is ignored. If this advisory travels through a 'branch' then the branch delegates the advisory upwards. Branches do not pass advisory messages from children to their other child nodes.
> 4) If a consumer connects to a branch node, then the consumer advisory travels up the tree. Additionally, the branch broker will send the advisory downwards to its children.
> 5) If a consumer connects to the root, then the consumer advisory is filtered downwards to all children.
> In this way, we reduce spillage of advisory messages, and ensure that trees can self organize without too much intervention or worrying about the network time-to-live.
> Thoughts?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira