Using topics for publishing broker metrics

[Raffi <ra...@gmail.com>]

Wanted to run this by the community for some advice.

We have 20 brokers in total (10 master/slave pairs). Operational tooling is
an issue, we need something simple to address short term needs. So our goal
is to create a web page showing unified operational view for the entire
cluster. Simple metrics for each broker (master/slave role, process load,
memory, connections, store/temp usage), is all we need for now. Each broker

Our first thought was to create a JMX spring service configured to run
inside the broker. The service opens a local JMX connection, polls the
desired metrics, and publishes to "topic/metrics" on the local broker as a
simple json payload. The problem with this approach is it doesn't work when
the broker is a slave. 

An alternative option is along the same lines, a JMX service running inside
the broker, but the difference is that, on startup, the services creates an
embedded broker with single network connector configured to talk to all
other brokers in the cluster. Now, regardless of the broker's role,
"topic/metrics" is always a valid topic for publishing and consuming metrics
info for the entire cluster. 

Another option is using advisory messages for carrying metrics info,
although we're not sure if flooding this channel with such data will
negatively impact openwire clients receiving cluster updates with no
interest in metrics.



--
View this message in context: http://activemq.2283324.n4.nabble.com/Using-topics-for-publishing-broker-metrics-tp4705077.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

RE: Using topics for publishing broker metrics

[Tim Bain <tb...@alumni.duke.edu>]

Your assumption is valid, and having both the master and the slave (times N
pairs) in the failover URI guarantees that you'll be able to reach a broker
no matter who's active.
On Dec 17, 2015 7:28 AM, "Basmajian, Raffi" <rb...@ofiglobal.com>
wrote:

> Hi Tim,
>
> We assumed if the broker was a slave it would be unable to publish to
> itself. But you're saying each broker should publish to "failover:(all
> brokers in cluster)" instead of "tcp://localhost:61616"?
>
>
>
> -----Original Message-----
> From: tbain98@gmail.com [mailto:tbain98@gmail.com] On Behalf Of Tim Bain
> Sent: Thursday, December 17, 2015 8:52 AM
> To: ActiveMQ Users
> Subject: Re: Using topics for publishing broker metrics [ EXTERNAL ]
>
> I wouldn't try to make your own advisory topics; just use normal topics as
> you've proposed.
>
> Wouldn't it be simpler to publish to a failover URL containing all the
> brokers rather than starting an embedded broker whose sole purpose is to
> publish stats?  It seems like your proposed approach adds unnecessary
> complexity.
>
> Tim
> On Dec 16, 2015 3:59 PM, "Raffi" <ra...@gmail.com> wrote:
>
> > Wanted to run this by the community for some advice.
> >
> > We have 20 brokers in total (10 master/slave pairs). Operational
> > tooling is an issue, we need something simple to address short term
> > needs. So our goal is to create a web page showing unified operational
> > view for the entire cluster. Simple metrics for each broker
> > (master/slave role, process load, memory, connections, store/temp
> > usage), is all we need for now. Each broker
> >
> > Our first thought was to create a JMX spring service configured to run
> > inside the broker. The service opens a local JMX connection, polls the
> > desired metrics, and publishes to "topic/metrics" on the local broker
> > as a simple json payload. The problem with this approach is it doesn't
> > work when the broker is a slave.
> >
> > An alternative option is along the same lines, a JMX service running
> > inside the broker, but the difference is that, on startup, the
> > services creates an embedded broker with single network connector
> > configured to talk to all other brokers in the cluster. Now,
> > regardless of the broker's role, "topic/metrics" is always a valid
> > topic for publishing and consuming metrics info for the entire
> > cluster.
> >
> > Another option is using advisory messages for carrying metrics info,
> > although we're not sure if flooding this channel with such data will
> > negatively impact openwire clients receiving cluster updates with no
> > interest in metrics.
> >
> >
> >
> > --
> > View this message in context:
> > http://activemq.2283324.n4.nabble.com/Using-topics-for-publishing-brok
> > er-metrics-tp4705077.html Sent from the ActiveMQ - User mailing list
> > archive at Nabble.com.
> >
>
> This e-mail transmission may contain information that is proprietary,
> privileged and/or confidential and is intended exclusively for the
> person(s) to whom it is addressed. Any use, copying, retention or
> disclosure by any person other than the intended recipient or the intended
> recipient's designees is strictly prohibited. If you are not the intended
> recipient or their designee, please notify the sender immediately by return
> e-mail and delete all copies. OppenheimerFunds may, at its sole discretion,
> monitor, review, retain and/or disclose the content of all email
> communications.
>

RE: Using topics for publishing broker metrics

["Basmajian, Raffi" <rb...@ofiglobal.com>]

Hi Tim,

We assumed if the broker was a slave it would be unable to publish to itself. But you're saying each broker should publish to "failover:(all brokers in cluster)" instead of "tcp://localhost:61616"?



-----Original Message-----
From: tbain98@gmail.com [mailto:tbain98@gmail.com] On Behalf Of Tim Bain
Sent: Thursday, December 17, 2015 8:52 AM
To: ActiveMQ Users
Subject: Re: Using topics for publishing broker metrics [ EXTERNAL ]

I wouldn't try to make your own advisory topics; just use normal topics as you've proposed.

Wouldn't it be simpler to publish to a failover URL containing all the brokers rather than starting an embedded broker whose sole purpose is to publish stats?  It seems like your proposed approach adds unnecessary complexity.

Tim
On Dec 16, 2015 3:59 PM, "Raffi" <ra...@gmail.com> wrote:

> Wanted to run this by the community for some advice.
>
> We have 20 brokers in total (10 master/slave pairs). Operational 
> tooling is an issue, we need something simple to address short term 
> needs. So our goal is to create a web page showing unified operational 
> view for the entire cluster. Simple metrics for each broker 
> (master/slave role, process load, memory, connections, store/temp 
> usage), is all we need for now. Each broker
>
> Our first thought was to create a JMX spring service configured to run 
> inside the broker. The service opens a local JMX connection, polls the 
> desired metrics, and publishes to "topic/metrics" on the local broker 
> as a simple json payload. The problem with this approach is it doesn't 
> work when the broker is a slave.
>
> An alternative option is along the same lines, a JMX service running 
> inside the broker, but the difference is that, on startup, the 
> services creates an embedded broker with single network connector 
> configured to talk to all other brokers in the cluster. Now, 
> regardless of the broker's role, "topic/metrics" is always a valid 
> topic for publishing and consuming metrics info for the entire 
> cluster.
>
> Another option is using advisory messages for carrying metrics info, 
> although we're not sure if flooding this channel with such data will 
> negatively impact openwire clients receiving cluster updates with no 
> interest in metrics.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Using-topics-for-publishing-brok
> er-metrics-tp4705077.html Sent from the ActiveMQ - User mailing list 
> archive at Nabble.com.
>

This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.

Re: Using topics for publishing broker metrics

[Tim Bain <tb...@alumni.duke.edu>]

I wouldn't try to make your own advisory topics; just use normal topics as
you've proposed.

Wouldn't it be simpler to publish to a failover URL containing all the
brokers rather than starting an embedded broker whose sole purpose is to
publish stats?  It seems like your proposed approach adds unnecessary
complexity.

Tim
On Dec 16, 2015 3:59 PM, "Raffi" <ra...@gmail.com> wrote:

> Wanted to run this by the community for some advice.
>
> We have 20 brokers in total (10 master/slave pairs). Operational tooling is
> an issue, we need something simple to address short term needs. So our goal
> is to create a web page showing unified operational view for the entire
> cluster. Simple metrics for each broker (master/slave role, process load,
> memory, connections, store/temp usage), is all we need for now. Each broker
>
> Our first thought was to create a JMX spring service configured to run
> inside the broker. The service opens a local JMX connection, polls the
> desired metrics, and publishes to "topic/metrics" on the local broker as a
> simple json payload. The problem with this approach is it doesn't work when
> the broker is a slave.
>
> An alternative option is along the same lines, a JMX service running inside
> the broker, but the difference is that, on startup, the services creates an
> embedded broker with single network connector configured to talk to all
> other brokers in the cluster. Now, regardless of the broker's role,
> "topic/metrics" is always a valid topic for publishing and consuming
> metrics
> info for the entire cluster.
>
> Another option is using advisory messages for carrying metrics info,
> although we're not sure if flooding this channel with such data will
> negatively impact openwire clients receiving cluster updates with no
> interest in metrics.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Using-topics-for-publishing-broker-metrics-tp4705077.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>