Artemis: add FirstMessageAge to queue metrics?

[Brad Harvey <Br...@gbst.com>]

Hello,

Would it be feasible to make the queue attribute FirstMessageAge (as shown in web console/JMX) available to the metrics plugin?  I've found "age of oldest item" to be extremely useful for monitoring in systems that support it - easier to set a good threshold that picks up problems without false positives than queue depth alone.

Thanks
Brad


The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and / or privileged material that may be governed by confidential information provisions contained in the agreement between GBST and your company. Any disclosure, copying, distribution, or other use without the express consent of the sender is prohibited. If you received this in error, please contact the sender and delete the material from any computer. All rights in the information transmitted, including copyright, are reserved. Nothing in this message should be interpreted as a digital signature that can be used to authenticate a document. No warranty is given by the sender that any attachments to this email are free from viruses or other defects.

RE: Artemis: add FirstMessageAge to queue metrics?

[Brad Harvey <Br...@gbst.com>]

Hi Justin,

Short answer: it's for cases where the monitoring infrastructure is basic as it provides an easily digestible measure of whether consumers are reading from the queue and keeping up.  It makes it easy to create an effective alert.

I can understand the trade-offs you need to consider.  Perhaps there's another request here to make the actual metrics exported configurable.  As well as allowing more exotic metrics to be enabled for specific use cases, it could help control costs when using pay per metric services.

We are actually planning to use the prometheus exporter, but in most cases it won't be in conjunction with a Prometheus server.  The Prometheus exporter appears to be a convenient way to get the metrics out of Artemis in a single API call with less fuss than JMX/Jolokia.  (Aside: any chance the Prometheus exporter will become more bundled with Artemis instead of having to build from source?)

Also interested if anyone has advice on using Checkmk with Artemis.  I don't think there's a specific plugin, so the options look like doing something with JMX/Jolokia or the Promtheus exporter.

Cheers
Brad


-----Original Message-----
From: Justin Bertram <jb...@apache.org>
Sent: Friday, 18 February 2022 6:01 AM
To: users@activemq.apache.org
Subject: Re: Artemis: add FirstMessageAge to queue metrics?

What's the specific use-case for needing FirstMessageAge?

If you're wanting to detect, for example, slow (or stopped) consumption on a queue where the message count is low and/or not growing then you can ostensibly use the "messages.acknowledged" metric. Most metric consumers (e.g. Prometheus) can take raw data like this and perform useful functions on it to identify trends over time and send alerts as necessary. In this case, your tool could inspect the delta between successive measurements of "messages.acknowledged" and if there is no change (or perhaps the change falls within some specified limit) then you know consumption on that queue has effectively stalled. This might not fit your use-case at all, but I thought it was worth mentioning.

In general, we want to keep the number of metrics as low as possible while still exporting enough data to provide meaningful intelligence. The fewer metrics-per-queue we export the less work it is on the broker, the metrics'
consumer(s), and the network between them. The larger the deployment (e.g.
thousands of queues) the more impactful this becomes.


Justin

On Thu, Feb 17, 2022 at 4:34 AM Brad Harvey <Br...@gbst.com> wrote:

> Hello,
>
> Would it be feasible to make the queue attribute FirstMessageAge (as
> shown in web console/JMX) available to the metrics plugin?  I've found
> "age of oldest item" to be extremely useful for monitoring in systems
> that support it - easier to set a good threshold that picks up
> problems without false positives than queue depth alone.
>
> Thanks
> Brad
>
>
> The information transmitted is intended only for the person or entity
> to which it is addressed and may contain confidential and / or
> privileged material that may be governed by confidential information
> provisions contained in the agreement between GBST and your company.
> Any disclosure, copying, distribution, or other use without the
> express consent of the sender is prohibited. If you received this in
> error, please contact the sender and delete the material from any
> computer. All rights in the information transmitted, including
> copyright, are reserved. Nothing in this message should be interpreted
> as a digital signature that can be used to authenticate a document. No
> warranty is given by the sender that any attachments to this email are free from viruses or other defects.
>
>
The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and / or privileged material that may be governed by confidential information provisions contained in the agreement between GBST and your company. Any disclosure, copying, distribution, or other use without the express consent of the sender is prohibited. If you received this in error, please contact the sender and delete the material from any computer. All rights in the information transmitted, including copyright, are reserved. Nothing in this message should be interpreted as a digital signature that can be used to authenticate a document. No warranty is given by the sender that any attachments to this email are free from viruses or other defects.

Re: Artemis: add FirstMessageAge to queue metrics?

[Justin Bertram <jb...@apache.org>]

What's the specific use-case for needing FirstMessageAge?

If you're wanting to detect, for example, slow (or stopped) consumption on
a queue where the message count is low and/or not growing then you can
ostensibly use the "messages.acknowledged" metric. Most metric consumers
(e.g. Prometheus) can take raw data like this and perform useful functions
on it to identify trends over time and send alerts as necessary. In this
case, your tool could inspect the delta between successive measurements of
"messages.acknowledged" and if there is no change (or perhaps the change
falls within some specified limit) then you know consumption on that queue
has effectively stalled. This might not fit your use-case at all, but I
thought it was worth mentioning.

In general, we want to keep the number of metrics as low as possible while
still exporting enough data to provide meaningful intelligence. The fewer
metrics-per-queue we export the less work it is on the broker, the metrics'
consumer(s), and the network between them. The larger the deployment (e.g.
thousands of queues) the more impactful this becomes.


Justin

On Thu, Feb 17, 2022 at 4:34 AM Brad Harvey <Br...@gbst.com> wrote:

> Hello,
>
> Would it be feasible to make the queue attribute FirstMessageAge (as shown
> in web console/JMX) available to the metrics plugin?  I've found "age of
> oldest item" to be extremely useful for monitoring in systems that support
> it - easier to set a good threshold that picks up problems without false
> positives than queue depth alone.
>
> Thanks
> Brad
>
>
> The information transmitted is intended only for the person or entity to
> which it is addressed and may contain confidential and / or privileged
> material that may be governed by confidential information provisions
> contained in the agreement between GBST and your company. Any disclosure,
> copying, distribution, or other use without the express consent of the
> sender is prohibited. If you received this in error, please contact the
> sender and delete the material from any computer. All rights in the
> information transmitted, including copyright, are reserved. Nothing in this
> message should be interpreted as a digital signature that can be used to
> authenticate a document. No warranty is given by the sender that any
> attachments to this email are free from viruses or other defects.
>
>