You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@activemq.apache.org by andrea bisogno <bi...@hotmail.it> on 2023/04/27 08:54:30 UTC

Metrics on Artemis for negative use cases

Hi,
I would need support for understanding if the Metrics offered by Artemis (as documented at https://activemq.apache.org/components/artemis/documentation/latest/metrics.html) cover the negative use cases too.
I mean, it would be great to be able to trace:

  *   the number of the MQTT connections failed due to authorization issues
  *   the number of the messages not published due to authorization issues (e.g. no role for that user to send messages on a destination)
  *   the number of the subscriptions failed due to authorization issues (e.g. no role for that user to subscribe to a destination)

If these scenarios are currently not covered by the metrics, is it possible to add these in one of the next releases?
Being able to trace also these negative scenarios could be very useful, for example for comparing how many MQTT connection succeded and how many failed due to authorization issues.

Many thanks in advance,

Andrea

R: Metrics on Artemis for negative use cases

Posted by andrea bisogno <bi...@hotmail.it>.
Hi Justin,
is there some news about the https://issues.apache.org/jira/browse/ARTEMIS-4306 and authn/z metrics?
Is it in plan to address that in one of the next releases?
Many thanks in advance,

Andrea
________________________________
Da: Justin Bertram <jb...@apache.org>
Inviato: giovedì 8 giugno 2023 20:09
A: users@activemq.apache.org <us...@activemq.apache.org>
Oggetto: Re: Metrics on Artemis for negative use cases

Sorry for the late reply. This dropped off my radar.

I opened ARTEMIS-4306 [1] for this work.

The caches I was referring to are indeed the same as those referenced in
the documentation you linked.

And yes, authentication failures would cover things like bad username, bad
password, bad cert, etc., and authorization failures would cover instances
where a client attempted an operation for which it did not have permission
(i.e. configured in security-settings in broker.xml).


Justin

[1] https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.apache.org%2Fjira%2Fbrowse%2FARTEMIS-4306&data=05%7C01%7C%7Cac6fef06836543271fa708db685c4993%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638218517800437131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=OoVuemlT%2F85cZ%2BK19WczMhaI%2BpQjsbJY25USzd1oNyU%3D&reserved=0<https://issues.apache.org/jira/browse/ARTEMIS-4306>

On Mon, May 29, 2023 at 10:08 AM andrea bisogno <bi...@hotmail.it>
wrote:

> Hi Justin,
> do you have any update about my latest mail (see below) ?
> Thanks,
>
> Andrea
>
> ________________________________
> Da: andrea bisogno <bi...@hotmail.it>
> Inviato: giovedì 11 maggio 2023 16:29
> A: users@activemq.apache.org <us...@activemq.apache.org>
> Oggetto: R: Metrics on Artemis for negative use cases
>
> Hi Justin,
> many thanks for your availability.
> Sure, this kind of metrics is protocol independent (I referred to MQTT
> because it is the reference protocol for our scenario).
>
> Below some notes from my side:
>
>   *   I think having success/failure counters for each and every different
> authorization permission (e.g. send, consume, browse, createDurableQueue,
> createAddress, etc.) would be valuable, because it would allow to monitor,
> for example, the percentage of operations failed due to authorization
> issues, category by category. If we can express some sort of
> preference/priority, based on our experience, I would start from:-
>      *   connection (success/failure)
>      *   send (success/failure)
>      *   createAddress (success/failure)
>      *   createDurableQueue (success/failure)
>      *   createNonDurableQueue (success/failure)
>      *   consume (success/failure)
>
>   *   just to confirm my understanding about authentication/authorization
> metrics, in the negative scenario (i.e. failures) the authentication
> metrics should be related only to inconsistencies of the identity
> information provided by the client (e.g. expired client certificates),
> while authorization metrics would be related, for example, to missing
> grants for sending/consuming data... can you confirm that with the new
> metric we will be able to catch all these events?
>
>   *   when you talk about cache, you refer to the
> authentication/authorization cache (as documented in the security guide<
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fsecurity.html&data=05%7C01%7C%7Cac6fef06836543271fa708db685c4993%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638218517800437131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=mi0PPNFiuHYi2AjZdamQT%2BOchCYWp%2FIEUEymEmKkIFU%3D&reserved=0>)<https://activemq.apache.org/components/artemis/documentation/latest/security.html>,
> right? If this is the case, I think these metrics would be useful for
> understanding the performance but maybe it could be more useful to start
> from the metrics highlighted in the first point
>
> Have a nice day,
>
> Andrea
>
> P.S.
> If you will open a Jira Issue or you will create a PR for introducing the
> new metrics, could you please send me the link (just for being able to
> understand when the changes will be introduced)?
> ________________________________
> Da: Justin Bertram <jb...@apache.org>
> Inviato: martedì 9 maggio 2023 18:05
> A: users@activemq.apache.org <us...@activemq.apache.org>
> Oggetto: Re: Metrics on Artemis for negative use cases
>
> I do think metrics like this would be valuable. However, they wouldn't be
> measured or presented specifically in the context of MQTT. They would be
> general metrics for authn & authz for all protocols. Right now I'm thinking
> of adding the following for both authentication and authorization:
>
>   - success count
>   - failure count
>   - cache hit count
>   - cache miss count
>   - cache size
>
> This would be a total of 10 new metrics.
>
> I could also add success/failure counts for each and every different
> authorization permission (e.g. send, consume, browse, createDurableQueue,
> createAddress, etc.), but that would be 20 additional metrics. Do you have
> any thoughts or preferences regarding this?
>
> Keep in mind that notifications [1] are sent for every authn & authz
> failure so you can *already* set up something to monitor and alert if
> necessary.
>
>
> Justin
>
> [1]
>
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmanagement.html%23management-notifications&data=05%7C01%7C%7Cac6fef06836543271fa708db685c4993%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638218517800437131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=8RTPVbNZtU%2BdvXbzcwzdGry8XT4EbAdazGh1sIrURrQ%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/management.html#management-notifications>
> <
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmanagement.html%23management-notifications&data=05%7C01%7C%7Cac6fef06836543271fa708db685c4993%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638218517800437131%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=8RTPVbNZtU%2BdvXbzcwzdGry8XT4EbAdazGh1sIrURrQ%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/management.html#management-notifications>
> ><
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmanagement.html%23management-notifications&data=05%7C01%7C%7Cac6fef06836543271fa708db685c4993%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638218517800593324%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=wMz3neJcW2v4qhLeQ9DNpz8GhIfXioCjVlIbbjWkmiQ%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/management.html#management-notifications>
> >
>
> On Tue, May 9, 2023 at 12:03 PM andrea bisogno <bi...@hotmail.it>
> wrote:
>
> > Hi support,
> > do you have any info to share here?
> > Many thanks in advance,
> >
> > Andrea
> > ________________________________
> > Da: andrea bisogno <bi...@hotmail.it>
> > Inviato: giovedì 27 aprile 2023 08:54
> > A: users@activemq.apache.org <us...@activemq.apache.org>
> > Oggetto: Metrics on Artemis for negative use cases
> >
> > Hi,
> > I would need support for understanding if the Metrics offered by Artemis
> > (as documented at
> >
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cac6fef06836543271fa708db685c4993%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638218517800593324%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=lI9EhKnWlX%2FmE1%2B8xtxpvEtoMlldOQs5XRyrRinAvzQ%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/metrics.html>
> <
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cac6fef06836543271fa708db685c4993%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638218517800593324%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=lI9EhKnWlX%2FmE1%2B8xtxpvEtoMlldOQs5XRyrRinAvzQ%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/metrics.html>
> ><
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cac6fef06836543271fa708db685c4993%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638218517800593324%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=lI9EhKnWlX%2FmE1%2B8xtxpvEtoMlldOQs5XRyrRinAvzQ%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/metrics.html>
> >
> > )<
> >
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cac6fef06836543271fa708db685c4993%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638218517800593324%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=lI9EhKnWlX%2FmE1%2B8xtxpvEtoMlldOQs5XRyrRinAvzQ%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/metrics.html>
> ><
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cac6fef06836543271fa708db685c4993%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638218517800593324%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=lI9EhKnWlX%2FmE1%2B8xtxpvEtoMlldOQs5XRyrRinAvzQ%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/metrics.html>
> ><
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cac6fef06836543271fa708db685c4993%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638218517800593324%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=lI9EhKnWlX%2FmE1%2B8xtxpvEtoMlldOQs5XRyrRinAvzQ%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/metrics.html>
> >
> > cover the negative use cases too.
> > I mean, it would be great to be able to trace:
> >
> >   *   the number of the MQTT connections failed due to authorization
> issues
> >   *   the number of the messages not published due to authorization
> issues
> > (e.g. no role for that user to send messages on a destination)
> >   *   the number of the subscriptions failed due to authorization issues
> > (e.g. no role for that user to subscribe to a destination)
> >
> > If these scenarios are currently not covered by the metrics, is it
> > possible to add these in one of the next releases?
> > Being able to trace also these negative scenarios could be very useful,
> > for example for comparing how many MQTT connection succeded and how many
> > failed due to authorization issues.
> >
> > Many thanks in advance,
> >
> > Andrea
> >
>

Re: Metrics on Artemis for negative use cases

Posted by Justin Bertram <jb...@apache.org>.
Sorry for the late reply. This dropped off my radar.

I opened ARTEMIS-4306 [1] for this work.

The caches I was referring to are indeed the same as those referenced in
the documentation you linked.

And yes, authentication failures would cover things like bad username, bad
password, bad cert, etc., and authorization failures would cover instances
where a client attempted an operation for which it did not have permission
(i.e. configured in security-settings in broker.xml).


Justin

[1] https://issues.apache.org/jira/browse/ARTEMIS-4306

On Mon, May 29, 2023 at 10:08 AM andrea bisogno <bi...@hotmail.it>
wrote:

> Hi Justin,
> do you have any update about my latest mail (see below) ?
> Thanks,
>
> Andrea
>
> ________________________________
> Da: andrea bisogno <bi...@hotmail.it>
> Inviato: giovedì 11 maggio 2023 16:29
> A: users@activemq.apache.org <us...@activemq.apache.org>
> Oggetto: R: Metrics on Artemis for negative use cases
>
> Hi Justin,
> many thanks for your availability.
> Sure, this kind of metrics is protocol independent (I referred to MQTT
> because it is the reference protocol for our scenario).
>
> Below some notes from my side:
>
>   *   I think having success/failure counters for each and every different
> authorization permission (e.g. send, consume, browse, createDurableQueue,
> createAddress, etc.) would be valuable, because it would allow to monitor,
> for example, the percentage of operations failed due to authorization
> issues, category by category. If we can express some sort of
> preference/priority, based on our experience, I would start from:-
>      *   connection (success/failure)
>      *   send (success/failure)
>      *   createAddress (success/failure)
>      *   createDurableQueue (success/failure)
>      *   createNonDurableQueue (success/failure)
>      *   consume (success/failure)
>
>   *   just to confirm my understanding about authentication/authorization
> metrics, in the negative scenario (i.e. failures) the authentication
> metrics should be related only to inconsistencies of the identity
> information provided by the client (e.g. expired client certificates),
> while authorization metrics would be related, for example, to missing
> grants for sending/consuming data... can you confirm that with the new
> metric we will be able to catch all these events?
>
>   *   when you talk about cache, you refer to the
> authentication/authorization cache (as documented in the security guide<
> https://activemq.apache.org/components/artemis/documentation/latest/security.html>),
> right? If this is the case, I think these metrics would be useful for
> understanding the performance but maybe it could be more useful to start
> from the metrics highlighted in the first point
>
> Have a nice day,
>
> Andrea
>
> P.S.
> If you will open a Jira Issue or you will create a PR for introducing the
> new metrics, could you please send me the link (just for being able to
> understand when the changes will be introduced)?
> ________________________________
> Da: Justin Bertram <jb...@apache.org>
> Inviato: martedì 9 maggio 2023 18:05
> A: users@activemq.apache.org <us...@activemq.apache.org>
> Oggetto: Re: Metrics on Artemis for negative use cases
>
> I do think metrics like this would be valuable. However, they wouldn't be
> measured or presented specifically in the context of MQTT. They would be
> general metrics for authn & authz for all protocols. Right now I'm thinking
> of adding the following for both authentication and authorization:
>
>   - success count
>   - failure count
>   - cache hit count
>   - cache miss count
>   - cache size
>
> This would be a total of 10 new metrics.
>
> I could also add success/failure counts for each and every different
> authorization permission (e.g. send, consume, browse, createDurableQueue,
> createAddress, etc.), but that would be 20 additional metrics. Do you have
> any thoughts or preferences regarding this?
>
> Keep in mind that notifications [1] are sent for every authn & authz
> failure so you can *already* set up something to monitor and alert if
> necessary.
>
>
> Justin
>
> [1]
>
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmanagement.html%23management-notifications&data=05%7C01%7C%7Cd393a93dbf7442a7b9c608db523cf1b9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638194193920021051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=eiQv20mvv5pOtWEN%2BUaYGHCzXaa1uu1ShzwqJB%2FRMhc%3D&reserved=0
> <
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmanagement.html%23management-notifications&data=05%7C01%7C%7Cd393a93dbf7442a7b9c608db523cf1b9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638194193920021051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=eiQv20mvv5pOtWEN%2BUaYGHCzXaa1uu1ShzwqJB%2FRMhc%3D&reserved=0
> ><
> https://activemq.apache.org/components/artemis/documentation/latest/management.html#management-notifications
> >
>
> On Tue, May 9, 2023 at 12:03 PM andrea bisogno <bi...@hotmail.it>
> wrote:
>
> > Hi support,
> > do you have any info to share here?
> > Many thanks in advance,
> >
> > Andrea
> > ________________________________
> > Da: andrea bisogno <bi...@hotmail.it>
> > Inviato: giovedì 27 aprile 2023 08:54
> > A: users@activemq.apache.org <us...@activemq.apache.org>
> > Oggetto: Metrics on Artemis for negative use cases
> >
> > Hi,
> > I would need support for understanding if the Metrics offered by Artemis
> > (as documented at
> >
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cd393a93dbf7442a7b9c608db523cf1b9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638194193920021051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RwIHyFVQaKKmqyI%2FerZk3vukmR%2B6QYUkKNdl%2Bv%2FCKRY%3D&reserved=0
> <
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cd393a93dbf7442a7b9c608db523cf1b9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638194193920021051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RwIHyFVQaKKmqyI%2FerZk3vukmR%2B6QYUkKNdl%2Bv%2FCKRY%3D&reserved=0
> ><
> https://activemq.apache.org/components/artemis/documentation/latest/metrics.html
> >
> > )<
> >
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cd393a93dbf7442a7b9c608db523cf1b9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638194193920021051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RwIHyFVQaKKmqyI%2FerZk3vukmR%2B6QYUkKNdl%2Bv%2FCKRY%3D&reserved=0
> ><
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cd393a93dbf7442a7b9c608db523cf1b9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638194193920021051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RwIHyFVQaKKmqyI%2FerZk3vukmR%2B6QYUkKNdl%2Bv%2FCKRY%3D&reserved=0
> ><
> https://activemq.apache.org/components/artemis/documentation/latest/metrics.html
> >
> > cover the negative use cases too.
> > I mean, it would be great to be able to trace:
> >
> >   *   the number of the MQTT connections failed due to authorization
> issues
> >   *   the number of the messages not published due to authorization
> issues
> > (e.g. no role for that user to send messages on a destination)
> >   *   the number of the subscriptions failed due to authorization issues
> > (e.g. no role for that user to subscribe to a destination)
> >
> > If these scenarios are currently not covered by the metrics, is it
> > possible to add these in one of the next releases?
> > Being able to trace also these negative scenarios could be very useful,
> > for example for comparing how many MQTT connection succeded and how many
> > failed due to authorization issues.
> >
> > Many thanks in advance,
> >
> > Andrea
> >
>

R: Metrics on Artemis for negative use cases

Posted by andrea bisogno <bi...@hotmail.it>.
Hi Justin,
do you have any update about my latest mail (see below) ?
Thanks,

Andrea

________________________________
Da: andrea bisogno <bi...@hotmail.it>
Inviato: giovedì 11 maggio 2023 16:29
A: users@activemq.apache.org <us...@activemq.apache.org>
Oggetto: R: Metrics on Artemis for negative use cases

Hi Justin,
many thanks for your availability.
Sure, this kind of metrics is protocol independent (I referred to MQTT because it is the reference protocol for our scenario).

Below some notes from my side:

  *   I think having success/failure counters for each and every different authorization permission (e.g. send, consume, browse, createDurableQueue, createAddress, etc.) would be valuable, because it would allow to monitor, for example, the percentage of operations failed due to authorization issues, category by category. If we can express some sort of preference/priority, based on our experience, I would start from:-
     *   connection (success/failure)
     *   send (success/failure)
     *   createAddress (success/failure)
     *   createDurableQueue (success/failure)
     *   createNonDurableQueue (success/failure)
     *   consume (success/failure)

  *   just to confirm my understanding about authentication/authorization metrics, in the negative scenario (i.e. failures) the authentication metrics should be related only to inconsistencies of the identity information provided by the client (e.g. expired client certificates), while authorization metrics would be related, for example, to missing grants for sending/consuming data... can you confirm that with the new metric we will be able to catch all these events?

  *   when you talk about cache, you refer to the authentication/authorization cache (as documented in the security guide<https://activemq.apache.org/components/artemis/documentation/latest/security.html>), right? If this is the case, I think these metrics would be useful for understanding the performance but maybe it could be more useful to start from the metrics highlighted in the first point

Have a nice day,

Andrea

P.S.
If you will open a Jira Issue or you will create a PR for introducing the new metrics, could you please send me the link (just for being able to understand when the changes will be introduced)?
________________________________
Da: Justin Bertram <jb...@apache.org>
Inviato: martedì 9 maggio 2023 18:05
A: users@activemq.apache.org <us...@activemq.apache.org>
Oggetto: Re: Metrics on Artemis for negative use cases

I do think metrics like this would be valuable. However, they wouldn't be
measured or presented specifically in the context of MQTT. They would be
general metrics for authn & authz for all protocols. Right now I'm thinking
of adding the following for both authentication and authorization:

  - success count
  - failure count
  - cache hit count
  - cache miss count
  - cache size

This would be a total of 10 new metrics.

I could also add success/failure counts for each and every different
authorization permission (e.g. send, consume, browse, createDurableQueue,
createAddress, etc.), but that would be 20 additional metrics. Do you have
any thoughts or preferences regarding this?

Keep in mind that notifications [1] are sent for every authn & authz
failure so you can *already* set up something to monitor and alert if
necessary.


Justin

[1]
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmanagement.html%23management-notifications&data=05%7C01%7C%7Cd393a93dbf7442a7b9c608db523cf1b9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638194193920021051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=eiQv20mvv5pOtWEN%2BUaYGHCzXaa1uu1ShzwqJB%2FRMhc%3D&reserved=0<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmanagement.html%23management-notifications&data=05%7C01%7C%7Cd393a93dbf7442a7b9c608db523cf1b9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638194193920021051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=eiQv20mvv5pOtWEN%2BUaYGHCzXaa1uu1ShzwqJB%2FRMhc%3D&reserved=0><https://activemq.apache.org/components/artemis/documentation/latest/management.html#management-notifications>

On Tue, May 9, 2023 at 12:03 PM andrea bisogno <bi...@hotmail.it> wrote:

> Hi support,
> do you have any info to share here?
> Many thanks in advance,
>
> Andrea
> ________________________________
> Da: andrea bisogno <bi...@hotmail.it>
> Inviato: giovedì 27 aprile 2023 08:54
> A: users@activemq.apache.org <us...@activemq.apache.org>
> Oggetto: Metrics on Artemis for negative use cases
>
> Hi,
> I would need support for understanding if the Metrics offered by Artemis
> (as documented at
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cd393a93dbf7442a7b9c608db523cf1b9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638194193920021051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RwIHyFVQaKKmqyI%2FerZk3vukmR%2B6QYUkKNdl%2Bv%2FCKRY%3D&reserved=0<https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cd393a93dbf7442a7b9c608db523cf1b9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638194193920021051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RwIHyFVQaKKmqyI%2FerZk3vukmR%2B6QYUkKNdl%2Bv%2FCKRY%3D&reserved=0><https://activemq.apache.org/components/artemis/documentation/latest/metrics.html>
> )<
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cd393a93dbf7442a7b9c608db523cf1b9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638194193920021051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RwIHyFVQaKKmqyI%2FerZk3vukmR%2B6QYUkKNdl%2Bv%2FCKRY%3D&reserved=0><https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7Cd393a93dbf7442a7b9c608db523cf1b9%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638194193920021051%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RwIHyFVQaKKmqyI%2FerZk3vukmR%2B6QYUkKNdl%2Bv%2FCKRY%3D&reserved=0><https://activemq.apache.org/components/artemis/documentation/latest/metrics.html>
> cover the negative use cases too.
> I mean, it would be great to be able to trace:
>
>   *   the number of the MQTT connections failed due to authorization issues
>   *   the number of the messages not published due to authorization issues
> (e.g. no role for that user to send messages on a destination)
>   *   the number of the subscriptions failed due to authorization issues
> (e.g. no role for that user to subscribe to a destination)
>
> If these scenarios are currently not covered by the metrics, is it
> possible to add these in one of the next releases?
> Being able to trace also these negative scenarios could be very useful,
> for example for comparing how many MQTT connection succeded and how many
> failed due to authorization issues.
>
> Many thanks in advance,
>
> Andrea
>

R: Metrics on Artemis for negative use cases

Posted by andrea bisogno <bi...@hotmail.it>.
Hi Justin,
many thanks for your availability.
Sure, this kind of metrics is protocol independent (I referred to MQTT because it is the reference protocol for our scenario).

Below some notes from my side:

  *
I think having success/failure counters for each and every different authorization permission (e.g. send, consume, browse, createDurableQueue, createAddress, etc.) would be valuable, because it would allow to monitor, for example, the percentage of operations failed due to authorization issues, category by category. If we can express some sort of preference/priority, based on our experience, I would start from:
     *
connection (success/failure)
     *
send (success/failure)
     *
createAddress (success/failure)
     *
createDurableQueue (success/failure)
     *
createNonDurableQueue (success/failure)
     *
consume (success/failure)
  *
just to confirm my understanding about authentication/authorization metrics, in the negative scenario (i.e. failures) the authentication metrics should be related only to inconsistencies of the identity information provided by the client (e.g. expired client certificates), while authorization metrics would be related, for example, to missing grants for sending/consuming data... can you confirm that with the new metric we will be able to catch all these events?
  *
when you talk about cache, you refer to the authentication/authorization cache (as documented here<https://activemq.apache.org/components/artemis/documentation/latest/security.html>), right? If this is the case, I think these metrics would be useful for understanding the performance but maybe it could be more useful to start from the metrics highlighted in the first point

Have a nice day,

Andrea

P.S.
If you will open a Jira Issue or you will create a PR for introducing the new metrics, could you please send me the link (just for being able to understand when the changes will be introduced)?
________________________________
Da: Justin Bertram <jb...@apache.org>
Inviato: martedì 9 maggio 2023 18:05
A: users@activemq.apache.org <us...@activemq.apache.org>
Oggetto: Re: Metrics on Artemis for negative use cases

I do think metrics like this would be valuable. However, they wouldn't be
measured or presented specifically in the context of MQTT. They would be
general metrics for authn & authz for all protocols. Right now I'm thinking
of adding the following for both authentication and authorization:

  - success count
  - failure count
  - cache hit count
  - cache miss count
  - cache size

This would be a total of 10 new metrics.

I could also add success/failure counts for each and every different
authorization permission (e.g. send, consume, browse, createDurableQueue,
createAddress, etc.), but that would be 20 additional metrics. Do you have
any thoughts or preferences regarding this?

Keep in mind that notifications [1] are sent for every authn & authz
failure so you can *already* set up something to monitor and alert if
necessary.


Justin

[1]
https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmanagement.html%23management-notifications&data=05%7C01%7C%7C8f618a9c8d5b4f9ded6908db50b8a6c8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638192526221485490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=St37c2cyiBhv9Ek3G7Mr1niarxXj307v%2Fi5FF39e4dw%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/management.html#management-notifications>

On Tue, May 9, 2023 at 12:03 PM andrea bisogno <bi...@hotmail.it> wrote:

> Hi support,
> do you have any info to share here?
> Many thanks in advance,
>
> Andrea
> ________________________________
> Da: andrea bisogno <bi...@hotmail.it>
> Inviato: giovedì 27 aprile 2023 08:54
> A: users@activemq.apache.org <us...@activemq.apache.org>
> Oggetto: Metrics on Artemis for negative use cases
>
> Hi,
> I would need support for understanding if the Metrics offered by Artemis
> (as documented at
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7C8f618a9c8d5b4f9ded6908db50b8a6c8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638192526221485490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WDKy%2BA6%2BFFIrqDHjCEB04WMNWEUzXAIPa2cOkXL3GBk%3D&reserved=0<https://activemq.apache.org/components/artemis/documentation/latest/metrics.html>
> )<
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7C8f618a9c8d5b4f9ded6908db50b8a6c8%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638192526221485490%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=WDKy%2BA6%2BFFIrqDHjCEB04WMNWEUzXAIPa2cOkXL3GBk%3D&reserved=0><https://activemq.apache.org/components/artemis/documentation/latest/metrics.html>
> cover the negative use cases too.
> I mean, it would be great to be able to trace:
>
>   *   the number of the MQTT connections failed due to authorization issues
>   *   the number of the messages not published due to authorization issues
> (e.g. no role for that user to send messages on a destination)
>   *   the number of the subscriptions failed due to authorization issues
> (e.g. no role for that user to subscribe to a destination)
>
> If these scenarios are currently not covered by the metrics, is it
> possible to add these in one of the next releases?
> Being able to trace also these negative scenarios could be very useful,
> for example for comparing how many MQTT connection succeded and how many
> failed due to authorization issues.
>
> Many thanks in advance,
>
> Andrea
>

Re: Metrics on Artemis for negative use cases

Posted by Justin Bertram <jb...@apache.org>.
I do think metrics like this would be valuable. However, they wouldn't be
measured or presented specifically in the context of MQTT. They would be
general metrics for authn & authz for all protocols. Right now I'm thinking
of adding the following for both authentication and authorization:

  - success count
  - failure count
  - cache hit count
  - cache miss count
  - cache size

This would be a total of 10 new metrics.

I could also add success/failure counts for each and every different
authorization permission (e.g. send, consume, browse, createDurableQueue,
createAddress, etc.), but that would be 20 additional metrics. Do you have
any thoughts or preferences regarding this?

Keep in mind that notifications [1] are sent for every authn & authz
failure so you can *already* set up something to monitor and alert if
necessary.


Justin

[1]
https://activemq.apache.org/components/artemis/documentation/latest/management.html#management-notifications

On Tue, May 9, 2023 at 12:03 PM andrea bisogno <bi...@hotmail.it> wrote:

> Hi support,
> do you have any info to share here?
> Many thanks in advance,
>
> Andrea
> ________________________________
> Da: andrea bisogno <bi...@hotmail.it>
> Inviato: giovedì 27 aprile 2023 08:54
> A: users@activemq.apache.org <us...@activemq.apache.org>
> Oggetto: Metrics on Artemis for negative use cases
>
> Hi,
> I would need support for understanding if the Metrics offered by Artemis
> (as documented at
> https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7C11071d1eb504475dba7c08db46fd0f55%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638181824912753093%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=36tTJERr10WmiUki8m1A6IvZbQVizRhJb2yyq6l%2BU7U%3D&reserved=0
> )<
> https://activemq.apache.org/components/artemis/documentation/latest/metrics.html>
> cover the negative use cases too.
> I mean, it would be great to be able to trace:
>
>   *   the number of the MQTT connections failed due to authorization issues
>   *   the number of the messages not published due to authorization issues
> (e.g. no role for that user to send messages on a destination)
>   *   the number of the subscriptions failed due to authorization issues
> (e.g. no role for that user to subscribe to a destination)
>
> If these scenarios are currently not covered by the metrics, is it
> possible to add these in one of the next releases?
> Being able to trace also these negative scenarios could be very useful,
> for example for comparing how many MQTT connection succeded and how many
> failed due to authorization issues.
>
> Many thanks in advance,
>
> Andrea
>

R: Metrics on Artemis for negative use cases

Posted by andrea bisogno <bi...@hotmail.it>.
Hi support,
do you have any info to share here?
Many thanks in advance,

Andrea
________________________________
Da: andrea bisogno <bi...@hotmail.it>
Inviato: giovedì 27 aprile 2023 08:54
A: users@activemq.apache.org <us...@activemq.apache.org>
Oggetto: Metrics on Artemis for negative use cases

Hi,
I would need support for understanding if the Metrics offered by Artemis (as documented at https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Factivemq.apache.org%2Fcomponents%2Fartemis%2Fdocumentation%2Flatest%2Fmetrics.html&data=05%7C01%7C%7C11071d1eb504475dba7c08db46fd0f55%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C638181824912753093%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=36tTJERr10WmiUki8m1A6IvZbQVizRhJb2yyq6l%2BU7U%3D&reserved=0)<https://activemq.apache.org/components/artemis/documentation/latest/metrics.html> cover the negative use cases too.
I mean, it would be great to be able to trace:

  *   the number of the MQTT connections failed due to authorization issues
  *   the number of the messages not published due to authorization issues (e.g. no role for that user to send messages on a destination)
  *   the number of the subscriptions failed due to authorization issues (e.g. no role for that user to subscribe to a destination)

If these scenarios are currently not covered by the metrics, is it possible to add these in one of the next releases?
Being able to trace also these negative scenarios could be very useful, for example for comparing how many MQTT connection succeded and how many failed due to authorization issues.

Many thanks in advance,

Andrea