how to probe broker's openwire port from an F5 VIP?

[Adam Whitney <ad...@sony.com>]

I have an HA cluster (currently version 5.13.3) using replicated leveldb and
I want to put the cluster behind an F5 so that clients only need to know the
VIP on the F5, and the F5 forwards the client to the current live "master".
This way, if I change my broker topology (e.g. add another HA cluster and
network the brokers ... or whatever other topology I may choose) my clients
don't need to know ... I can just reconfigure the F5 to handle the new
topology, and clients connect to the new topology automatically via the VIP.

I found this article that basically asks the same question:
http://activemq.2283324.n4.nabble.com/Howto-probe-brokers-from-a-loadBalancer-td2354728.html

As mentioned in the linked article, when the F5 does a simple tcp probe on
port 61616 the broker logs an EOFException in activemq.log ... for me, this
happens once a second (well, once every 2 seconds from both the primary and
backup F5).

Unfortunately, the solution reached by the original poster of that article
isn't exactly what I'm looking for ... while my F5 admins could write an
"irule" (or some such thing - I'm not an F5 admin) to probe one port and
allow traffic on another, they claim that this is more CPU intensive on the
F5 and would prefer to probe on the same port  that the VIP is forwarding
traffic on ... in other words, if the VIP is going to forward traffic on
61616 then my F5 admins want the F5 to probe on 61616.

Does anyone have a suggestion on what the F5 can send to probe the openwire
port without causing an exception on the broker?



--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-probe-broker-s-openwire-port-from-an-F5-VIP-tp4716641.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: how to probe broker's openwire port from an F5 VIP?

[Matt Pavlovich <ma...@gmail.com>]

In my experience, many organizations that put messaging brokers behind 
load a balancer eventually take out the load balancer. Generally, less 
layers is better, and as Tim noted messaging protocols tend to be 
stateful vs stateless and the client -> server uri's already support 
failover and retry.

If you want to have a client -> server uri registry, look at the http 
discovery protocol. You can point your clients to a web page, pass in a 
parameter and get a uri back-- much simpler to manage vs load balancer 
solution.

On 9/20/16 12:32 PM, Adam Whitney wrote:
> Tim,
>
> Thanks for the quick response.
>
> Regarding setting up and starting a full ActiveMQConnection:
> 1) No problem on the links, I appreciate the details. I was hoping to have
> something a little more lightweight than actually creating a connection and
> then closing it ... seems lie creating a new connection every second would
> put undo burden on the broker just to probe if the broker is listening on
> that port? Is there some other "trick" in openwire to make the probe more
> lightweight while still avoiding the error in the broker logs, e.g. just
> send something like a WIREFORMAT_INFO request?
>
> Regarding clients reconnecting after broker failover:
> 2) The F5 would only be used to route the initial connection request ...
> after that, my understanding is that as long as clients use the "failover"
> protocol when connecting to the VIP, e.g. something like
> "failover:(tcp://activemq-vip:61616)", and the broker-side transport
> connector has "updateClusterClients" and "rebalanceClusterClients" set to
> true, then the broker and client would exchange info about the network
> topology and clients would reconnect directly to active brokers should the
> initial connection fail.
>
> adam
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/how-to-probe-broker-s-openwire-port-from-an-F5-VIP-tp4716641p4716662.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: how to probe broker's openwire port from an F5 VIP?

[Adam Whitney <ad...@sony.com>]

Tim,

Thanks for the quick response.

Regarding setting up and starting a full ActiveMQConnection:
1) No problem on the links, I appreciate the details. I was hoping to have
something a little more lightweight than actually creating a connection and
then closing it ... seems lie creating a new connection every second would
put undo burden on the broker just to probe if the broker is listening on
that port? Is there some other "trick" in openwire to make the probe more
lightweight while still avoiding the error in the broker logs, e.g. just
send something like a WIREFORMAT_INFO request?

Regarding clients reconnecting after broker failover:
2) The F5 would only be used to route the initial connection request ...
after that, my understanding is that as long as clients use the "failover"
protocol when connecting to the VIP, e.g. something like
"failover:(tcp://activemq-vip:61616)", and the broker-side transport
connector has "updateClusterClients" and "rebalanceClusterClients" set to
true, then the broker and client would exchange info about the network
topology and clients would reconnect directly to active brokers should the
initial connection fail.

adam



--
View this message in context: http://activemq.2283324.n4.nabble.com/how-to-probe-broker-s-openwire-port-from-an-F5-VIP-tp4716641p4716662.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Re: how to probe broker's openwire port from an F5 VIP?

[Tim Bain <tb...@alumni.duke.edu>]

To do what they want, they will have to send a valid connect/disconnect
sequence in the Openwire protocol.  The code they would have to rewrite
into the F5's scripting language of choice would include the messages and
marshallers here: https://github.com/apache/activemq-openwire, and you'll
probably want to mimic the logic in ActiveMQConnection.start() (
http://grepcode.com/file/repo1.maven.org/maven2/org.apache.activemq/activemq-client/5.11.1/org/apache/activemq/ActiveMQConnection.java#ActiveMQConnection.start%28%29
- sorry, I know I'm providing some Git references and some GrepCode ones).

Also, will you be using the F5 only to route new connections, or are you
trying to use it to shift existing connections when a broker failover
occurs?  ActiveMQ connections are not stateless, so I'm not sure what would
happen if you tried doing the latter.  It might work fine, or it might be
awful, but the safer option is to only do the former.

Tim

On Sep 19, 2016 5:15 PM, "Adam Whitney" <ad...@sony.com> wrote:

> I have an HA cluster (currently version 5.13.3) using replicated leveldb
> and
> I want to put the cluster behind an F5 so that clients only need to know
> the
> VIP on the F5, and the F5 forwards the client to the current live "master".
> This way, if I change my broker topology (e.g. add another HA cluster and
> network the brokers ... or whatever other topology I may choose) my clients
> don't need to know ... I can just reconfigure the F5 to handle the new
> topology, and clients connect to the new topology automatically via the
> VIP.
>
> I found this article that basically asks the same question:
> http://activemq.2283324.n4.nabble.com/Howto-probe-
> brokers-from-a-loadBalancer-td2354728.html
>
> As mentioned in the linked article, when the F5 does a simple tcp probe on
> port 61616 the broker logs an EOFException in activemq.log ... for me, this
> happens once a second (well, once every 2 seconds from both the primary and
> backup F5).
>
> Unfortunately, the solution reached by the original poster of that article
> isn't exactly what I'm looking for ... while my F5 admins could write an
> "irule" (or some such thing - I'm not an F5 admin) to probe one port and
> allow traffic on another, they claim that this is more CPU intensive on the
> F5 and would prefer to probe on the same port  that the VIP is forwarding
> traffic on ... in other words, if the VIP is going to forward traffic on
> 61616 then my F5 admins want the F5 to probe on 61616.
>
> Does anyone have a suggestion on what the F5 can send to probe the openwire
> port without causing an exception on the broker?
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.
> nabble.com/how-to-probe-broker-s-openwire-port-from-
> an-F5-VIP-tp4716641.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>