You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@beam.apache.org by Alex Amato <aj...@google.com> on 2020/09/08 20:08:15 UTC
Re: [Proposal] Apache Beam Fn API - Histogram Style Metrics (Correct
link this time)
Hi Again, Just reviving this thread to mention that I updated the doc with
a few sections:
https://s.apache.org/beam-histogram-metrics
Date
Changes
Sept 8, 2020
-
Added alternative section: “Collect Moment Sketch Variables Instead of
Bucket Counts” (recommend not pursuing, due to opposing trade offs and
significant implementation/maintenance challenge. But may be worth pursuing
in a future MonitoringInfo type).
-
Add distribution variables: min, max, sum, count
-
Added alternative section: “Update all distribution metrics to be
Histograms” (recommend not pursuing, update to histogramDistribution on a
case by case, due to performance concerns).
May 15, 2020
-
Completed review with beam dev list.
PTAL and LMK what you think :)
On Wed, May 6, 2020 at 9:58 AM Luke Cwik <lc...@google.com> wrote:
> Thanks Alex, I had some minor comments.
>
> On Mon, May 4, 2020 at 4:04 PM Alex Amato <aj...@google.com> wrote:
>
>> Thanks Ismaël :). Done
>>
>> On Mon, May 4, 2020 at 3:59 PM Ismaël Mejía <ie...@gmail.com> wrote:
>>
>>> Moving the short link to this thread
>>> https://s.apache.org/beam-histogram-metrics
>>>
>>> Alex can you add this link (and any other of your documents that may
>>> not be there) to
>>> https://cwiki.apache.org/confluence/display/BEAM/Design+Documents
>>>
>>>
>>> On Tue, May 5, 2020 at 12:51 AM Pablo Estrada <pa...@google.com>
>>> wrote:
>>> >
>>> > FYI +Boyuan Zhang worked on implementing a histogram metric that was
>>> performance-optimized into outer space for Python : ) - I don't recall if
>>> she ended up getting it merged, but it's worth looking at the work. I also
>>> remember Scott Wegner wrote the metrics for Java.
>>> >
>>> > Best
>>> > -P.
>>> >
>>> > On Mon, May 4, 2020 at 3:33 PM Alex Amato <aj...@google.com> wrote:
>>> >>
>>> >> Hello,
>>> >>
>>> >> I have created a proposal for Apache Beam FN API to support Histogram
>>> Style Metrics. Which defines a method to collect Histogram style metrics
>>> and pass them over the FN API.
>>> >>
>>> >> I would love to hear your feedback in order to improve this proposal,
>>> please let me know what you think. Thanks for taking a look :)
>>> >> Alex
>>>
>>
Re: [Proposal] Apache Beam Fn API - Histogram Style Metrics (Correct
link this time)
Posted by Robert Bradshaw <ro...@google.com>.
Thanks for the heads up. Though sketches have their advantages, I agree
that it makes sense to stick with bucket counts in the short term for
simplicity and revisit this later.
On Tue, Sep 8, 2020 at 1:08 PM Alex Amato <aj...@google.com> wrote:
> Hi Again, Just reviving this thread to mention that I updated the doc with
> a few sections:
> https://s.apache.org/beam-histogram-metrics
>
>
> Date
>
> Changes
>
> Sept 8, 2020
>
> -
>
> Added alternative section: “Collect Moment Sketch Variables Instead of
> Bucket Counts” (recommend not pursuing, due to opposing trade offs and
> significant implementation/maintenance challenge. But may be worth pursuing
> in a future MonitoringInfo type).
> -
>
> Add distribution variables: min, max, sum, count
> -
>
> Added alternative section: “Update all distribution metrics to be
> Histograms” (recommend not pursuing, update to histogramDistribution on a
> case by case, due to performance concerns).
>
> May 15, 2020
>
> -
>
> Completed review with beam dev list.
>
>
> PTAL and LMK what you think :)
>
> On Wed, May 6, 2020 at 9:58 AM Luke Cwik <lc...@google.com> wrote:
>
>> Thanks Alex, I had some minor comments.
>>
>> On Mon, May 4, 2020 at 4:04 PM Alex Amato <aj...@google.com> wrote:
>>
>>> Thanks Ismaël :). Done
>>>
>>> On Mon, May 4, 2020 at 3:59 PM Ismaël Mejía <ie...@gmail.com> wrote:
>>>
>>>> Moving the short link to this thread
>>>> https://s.apache.org/beam-histogram-metrics
>>>>
>>>> Alex can you add this link (and any other of your documents that may
>>>> not be there) to
>>>> https://cwiki.apache.org/confluence/display/BEAM/Design+Documents
>>>>
>>>>
>>>> On Tue, May 5, 2020 at 12:51 AM Pablo Estrada <pa...@google.com>
>>>> wrote:
>>>> >
>>>> > FYI +Boyuan Zhang worked on implementing a histogram metric that was
>>>> performance-optimized into outer space for Python : ) - I don't recall if
>>>> she ended up getting it merged, but it's worth looking at the work. I also
>>>> remember Scott Wegner wrote the metrics for Java.
>>>> >
>>>> > Best
>>>> > -P.
>>>> >
>>>> > On Mon, May 4, 2020 at 3:33 PM Alex Amato <aj...@google.com> wrote:
>>>> >>
>>>> >> Hello,
>>>> >>
>>>> >> I have created a proposal for Apache Beam FN API to support
>>>> Histogram Style Metrics. Which defines a method to collect Histogram style
>>>> metrics and pass them over the FN API.
>>>> >>
>>>> >> I would love to hear your feedback in order to improve this
>>>> proposal, please let me know what you think. Thanks for taking a look :)
>>>> >> Alex
>>>>
>>>