Re: [DISCUSS] PIP-235: Add metric for subscription back size

["rxl@apache.org" <ra...@gmail.com>]

Hello kmter:

For subscription-level indicators, I think we can collect and record them
in Prometheus. The only thing to note here is that the subscription-level
indicators may have a large number of indicators in a specific business
scenario, which will have a certain impact on the indicator collection
itself. load. So the suggested way here is that we add a new dynamic
configuration in the Broker, e.g: subscriptionMetricsEnabled, by default we
set it to false, that is, by default, the metrics at the subscribe level
will not be collected. The business decides whether it needs to be opened
according to its own needs.

--
Thanks
Xiaolong Ran

Girish Sharma <sc...@gmail.com> 于2022年12月31日周六 00:07写道：

>  How about adding a new label subscription in the same metric? It should
> also not break any existing usage of the metric as well for those only
> aggregating on topic or cluster label. The documentation can clearly
> mention that this is an addon on top of existing metrics.
>
> Please pardon my mistake as I am new to this mailing list, but if new
> additions to the same metrics are not considered by default as a general
> guideline in pulsar codebase, then this can go on its own new metric. It is
> just that this new metric would then again need the same labels (cluster,
> broker, topic etc) causing a lot of duplicate data.
> In our in-house usage of pulsar, metrics finally flow into a central
> platform and we have to be careful on the size (number of metrics and
> labels) that we are ingesting into this platform.
>
> Regards
>
> On Fri, Dec 30, 2022 at 8:55 PM 萧 易客 <km...@live.com> wrote:
>
> > Hi pulsar community,
> >
> > Motivation
> > Now we have pulsar_storage_backlog_size for topic backlog size, user can
> > create an alarm rule like pulsar_storage_backlog_size > THRESHOLD​,
> > typically this alarm is going to notify corresponding subscription owner,
> > but it need extra process to identify subscriptions that backlog size
> > exceed the threshold. So we could add a new metric for subscription back
> > size.
> >
> > For more detail, please read PIP-235 issue page [1]
> >
> > It's my first PIP, I'm looking forward to hearing what you think and open
> > for any suggestions.
> >
> > [1]: https://github.com/apache/pulsar/issues/19112
> > [
> >
> https://opengraph.githubassets.com/abebee001c4e8297f9418e3b78bd77a854f26d1c4b73bfea5fb203251968a18e/apache/pulsar/issues/19112
> > ]<https://github.com/apache/pulsar/issues/19112>
> > PIP-235: Add metric for subscription back size · Issue #19112 ·
> > apache/pulsar<https://github.com/apache/pulsar/issues/19112>
> > Motivation Now we have pulsar_storage_backlog_size for topic backlog
> size,
> > user can create an alarm rule like pulsar_storage_backlog_size &gt;
> > THRESHOLD, typically this alarm is going to notify cor...
> > github.com
> > 
> > 
> > 
> > Yike Xiao
> >
>
>
> --
> Girish Sharma
>