Re: [DISCUSS] Remove timestamp from Prometheus metrics

[Asaf Mesika <as...@gmail.com>]

Great addition!
It also reduces the Prometheus response size.

On Wed, Sep 7, 2022 at 7:47 AM Michael Marshall <mm...@apache.org>
wrote:

> Merged. Depending on whether [0] will be cherry picked to release
> branches, I will cherry pick [1] or [2] to all active release
> branches.
>
> Thanks,
> Michael
>
> [0] https://github.com/apache/pulsar/pull/15558
> [1] https://github.com/apache/pulsar/pull/17419
> [2]
> https://github.com/apache/pulsar/commit/b5cb02deb06760a2b6fe7b6c221e08acfabdf830
>
> On Thu, Sep 1, 2022 at 11:05 PM Michael Marshall <mm...@apache.org>
> wrote:
> >
> > Hi Pulsar Community,
> >
> > Recently, we noticed in certain Grafana metrics from the broker that
> > it appeared a topic had metrics reported by two different brokers at
> > the same time.
> >
> > It turns out that the root of the problem is a concept called
> > "staleness" in prometheus and it is directly related to the fact that
> > we export timestamps with our metrics.
> >
> > As such, I wrote a PR to remove these timestamps [0]. In it, I propose
> > that we remove the timestamps and cherry pick this fix to all active
> > branches of Pulsar. The PR has more detail, so please see it if you're
> > interested.
> >
> > If removing these timestamps will break your use case, please let me
> > know. By my reading, we do not qualify as an application that needs to
> > report timestamps. Additionally, I tried to make it configurable, but
> > many of these classes are static, so it would be non-trivial to make
> > the behavior configurable.
> >
> > Thanks,
> > Michael
> >
> > [0] https://github.com/apache/pulsar/pull/17419
>