You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Becket Qin <be...@gmail.com> on 2021/08/02 10:36:34 UTC
Re: [VOTE] FLIP-179: Expose Standardized Operator Metrics
+1 (binding).
Thanks for driving the efforts, Arvid.
Cheers,
Jiangjie (Becket) Qin
On Sat, Jul 31, 2021 at 12:08 PM Steven Wu <st...@gmail.com> wrote:
> +1 (non-binding)
>
> On Fri, Jul 30, 2021 at 3:55 AM Arvid Heise <ar...@apache.org> wrote:
>
> > Dear devs,
> >
> > I'd like to open a vote on FLIP-179: Expose Standardized Operator Metrics
> > [1] which was discussed in this thread [2].
> > The vote will be open for at least 72 hours unless there is an objection
> > or not enough votes.
> >
> > The proposal excludes the implementation for the currentFetchEventTimeLag
> > metric, which caused a bit of discussion without a clear convergence. We
> > will implement that metric in a generic way at a later point and
> encourage
> > sources to implement it themselves in the meantime.
> >
> > Best,
> >
> > Arvid
> >
> > [1]
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-179%3A+Expose+Standardized+Operator+Metrics
> > [2]
> >
> >
> https://lists.apache.org/thread.html/r856920cbfe6a262b521109c5bdb9e904e00a9b3f1825901759c24d85%40%3Cdev.flink.apache.org%3E
> >
>
Re: [VOTE] FLIP-179: Expose Standardized Operator Metrics
Posted by Becket Qin <be...@gmail.com>.
I am with Thomas on the contextual usage of metrics as well. That is the
"global" variable usage I was trying to explain in the discussion thread.
Thanks,
Jiangjie (Becket) Qin
On Thu, Aug 5, 2021 at 1:59 AM Thomas Weise <th...@apache.org> wrote:
> I agree with Becket. Although I find the use of gauge to pass contextual
> information less intuitive, it is acceptable within the metric group
> interface (plus javadoc).
>
> Thanks,
> Thomas
>
> On Tue, Aug 3, 2021 at 10:06 PM Becket Qin <be...@gmail.com> wrote:
>
> > Personally speaking, it is intuitive for me to set a gauge in
> MetricGroup.
> > So I am fine with set*Gauge pattern as long as the method is in
> > *MetricGroup class.
> >
> > Thanks,
> >
> > Jiangjie (Becket) Qin
> >
> > On Tue, Aug 3, 2021 at 7:24 PM Arvid Heise <ar...@apache.org> wrote:
> >
> > > @Becket Qin <be...@gmail.com> @Thomas Weise <th...@apache.org>
> would
> > > you
> > > also agree to @Chesnay Schepler <ch...@apache.org> 's proposal?
> > >
> > > I think the main intention is to only use a Gauge when the exact metric
> > is
> > > exposed. For any partial value that may be used in an
> internal/predefined
> > > metric, we would only use a supplier instead of a Gauge.
> > >
> > > So a connector developer can immediately distinguish the cases: if
> it's a
> > > metric class he would see the exact metric corresponding to the setter.
> > If
> > > it's some Supplier, the developer would expect that the value is used
> in
> > a
> > > differently named metric, which we would describe in the JavaDoc.
> > > Could that also be a solution to the currentEventFetchTimeLag metric?
> > >
> > > On Tue, Aug 3, 2021 at 12:54 PM Thomas Weise <th...@apache.org> wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > On Tue, Aug 3, 2021 at 12:58 AM Chesnay Schepler <chesnay@apache.org
> >
> > > > wrote:
> > > >
> > > > > +1 (binding)
> > > > >
> > > > > Although I still think all the set* methods should accept a
> Supplier
> > > > > instead of a Gauge.
> > > > >
> > > > > On 02/08/2021 12:36, Becket Qin wrote:
> > > > > > +1 (binding).
> > > > > >
> > > > > > Thanks for driving the efforts, Arvid.
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > Jiangjie (Becket) Qin
> > > > > >
> > > > > > On Sat, Jul 31, 2021 at 12:08 PM Steven Wu <stevenz3wu@gmail.com
> >
> > > > wrote:
> > > > > >
> > > > > >> +1 (non-binding)
> > > > > >>
> > > > > >> On Fri, Jul 30, 2021 at 3:55 AM Arvid Heise <ar...@apache.org>
> > > wrote:
> > > > > >>
> > > > > >>> Dear devs,
> > > > > >>>
> > > > > >>> I'd like to open a vote on FLIP-179: Expose Standardized
> Operator
> > > > > Metrics
> > > > > >>> [1] which was discussed in this thread [2].
> > > > > >>> The vote will be open for at least 72 hours unless there is an
> > > > > objection
> > > > > >>> or not enough votes.
> > > > > >>>
> > > > > >>> The proposal excludes the implementation for the
> > > > > currentFetchEventTimeLag
> > > > > >>> metric, which caused a bit of discussion without a clear
> > > convergence.
> > > > > We
> > > > > >>> will implement that metric in a generic way at a later point
> and
> > > > > >> encourage
> > > > > >>> sources to implement it themselves in the meantime.
> > > > > >>>
> > > > > >>> Best,
> > > > > >>>
> > > > > >>> Arvid
> > > > > >>>
> > > > > >>> [1]
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-179%3A+Expose+Standardized+Operator+Metrics
> > > > > >>> [2]
> > > > > >>>
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/r856920cbfe6a262b521109c5bdb9e904e00a9b3f1825901759c24d85%40%3Cdev.flink.apache.org%3E
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] FLIP-179: Expose Standardized Operator Metrics
Posted by Thomas Weise <th...@apache.org>.
I agree with Becket. Although I find the use of gauge to pass contextual
information less intuitive, it is acceptable within the metric group
interface (plus javadoc).
Thanks,
Thomas
On Tue, Aug 3, 2021 at 10:06 PM Becket Qin <be...@gmail.com> wrote:
> Personally speaking, it is intuitive for me to set a gauge in MetricGroup.
> So I am fine with set*Gauge pattern as long as the method is in
> *MetricGroup class.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On Tue, Aug 3, 2021 at 7:24 PM Arvid Heise <ar...@apache.org> wrote:
>
> > @Becket Qin <be...@gmail.com> @Thomas Weise <th...@apache.org> would
> > you
> > also agree to @Chesnay Schepler <ch...@apache.org> 's proposal?
> >
> > I think the main intention is to only use a Gauge when the exact metric
> is
> > exposed. For any partial value that may be used in an internal/predefined
> > metric, we would only use a supplier instead of a Gauge.
> >
> > So a connector developer can immediately distinguish the cases: if it's a
> > metric class he would see the exact metric corresponding to the setter.
> If
> > it's some Supplier, the developer would expect that the value is used in
> a
> > differently named metric, which we would describe in the JavaDoc.
> > Could that also be a solution to the currentEventFetchTimeLag metric?
> >
> > On Tue, Aug 3, 2021 at 12:54 PM Thomas Weise <th...@apache.org> wrote:
> >
> > > +1 (binding)
> > >
> > > On Tue, Aug 3, 2021 at 12:58 AM Chesnay Schepler <ch...@apache.org>
> > > wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Although I still think all the set* methods should accept a Supplier
> > > > instead of a Gauge.
> > > >
> > > > On 02/08/2021 12:36, Becket Qin wrote:
> > > > > +1 (binding).
> > > > >
> > > > > Thanks for driving the efforts, Arvid.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Jiangjie (Becket) Qin
> > > > >
> > > > > On Sat, Jul 31, 2021 at 12:08 PM Steven Wu <st...@gmail.com>
> > > wrote:
> > > > >
> > > > >> +1 (non-binding)
> > > > >>
> > > > >> On Fri, Jul 30, 2021 at 3:55 AM Arvid Heise <ar...@apache.org>
> > wrote:
> > > > >>
> > > > >>> Dear devs,
> > > > >>>
> > > > >>> I'd like to open a vote on FLIP-179: Expose Standardized Operator
> > > > Metrics
> > > > >>> [1] which was discussed in this thread [2].
> > > > >>> The vote will be open for at least 72 hours unless there is an
> > > > objection
> > > > >>> or not enough votes.
> > > > >>>
> > > > >>> The proposal excludes the implementation for the
> > > > currentFetchEventTimeLag
> > > > >>> metric, which caused a bit of discussion without a clear
> > convergence.
> > > > We
> > > > >>> will implement that metric in a generic way at a later point and
> > > > >> encourage
> > > > >>> sources to implement it themselves in the meantime.
> > > > >>>
> > > > >>> Best,
> > > > >>>
> > > > >>> Arvid
> > > > >>>
> > > > >>> [1]
> > > > >>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-179%3A+Expose+Standardized+Operator+Metrics
> > > > >>> [2]
> > > > >>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> https://lists.apache.org/thread.html/r856920cbfe6a262b521109c5bdb9e904e00a9b3f1825901759c24d85%40%3Cdev.flink.apache.org%3E
> > > >
> > > >
> > > >
> > >
> >
>
[RESULT] [VOTE] FLIP-179: Expose Standardized Operator Metrics
Posted by Arvid Heise <ar...@apache.org>.
Dear devs,
I'm happy to announce that we have unanimously approved this FLIP.
There are 4 approving votes of which 3 are binding:
Steven Wu (non-binding)
Jiangjie (Becket) Qin (binding)
Chesnay Schepler (binding)
Thomas Weise (binding)
Best,
Arvid
On Wed, Aug 4, 2021 at 7:06 AM Becket Qin <be...@gmail.com> wrote:
> Personally speaking, it is intuitive for me to set a gauge in MetricGroup.
> So I am fine with set*Gauge pattern as long as the method is in
> *MetricGroup class.
>
> Thanks,
>
> Jiangjie (Becket) Qin
>
> On Tue, Aug 3, 2021 at 7:24 PM Arvid Heise <ar...@apache.org> wrote:
>
> > @Becket Qin <be...@gmail.com> @Thomas Weise <th...@apache.org> would
> > you
> > also agree to @Chesnay Schepler <ch...@apache.org> 's proposal?
> >
> > I think the main intention is to only use a Gauge when the exact metric
> is
> > exposed. For any partial value that may be used in an internal/predefined
> > metric, we would only use a supplier instead of a Gauge.
> >
> > So a connector developer can immediately distinguish the cases: if it's a
> > metric class he would see the exact metric corresponding to the setter.
> If
> > it's some Supplier, the developer would expect that the value is used in
> a
> > differently named metric, which we would describe in the JavaDoc.
> > Could that also be a solution to the currentEventFetchTimeLag metric?
> >
> > On Tue, Aug 3, 2021 at 12:54 PM Thomas Weise <th...@apache.org> wrote:
> >
> > > +1 (binding)
> > >
> > > On Tue, Aug 3, 2021 at 12:58 AM Chesnay Schepler <ch...@apache.org>
> > > wrote:
> > >
> > > > +1 (binding)
> > > >
> > > > Although I still think all the set* methods should accept a Supplier
> > > > instead of a Gauge.
> > > >
> > > > On 02/08/2021 12:36, Becket Qin wrote:
> > > > > +1 (binding).
> > > > >
> > > > > Thanks for driving the efforts, Arvid.
> > > > >
> > > > > Cheers,
> > > > >
> > > > > Jiangjie (Becket) Qin
> > > > >
> > > > > On Sat, Jul 31, 2021 at 12:08 PM Steven Wu <st...@gmail.com>
> > > wrote:
> > > > >
> > > > >> +1 (non-binding)
> > > > >>
> > > > >> On Fri, Jul 30, 2021 at 3:55 AM Arvid Heise <ar...@apache.org>
> > wrote:
> > > > >>
> > > > >>> Dear devs,
> > > > >>>
> > > > >>> I'd like to open a vote on FLIP-179: Expose Standardized Operator
> > > > Metrics
> > > > >>> [1] which was discussed in this thread [2].
> > > > >>> The vote will be open for at least 72 hours unless there is an
> > > > objection
> > > > >>> or not enough votes.
> > > > >>>
> > > > >>> The proposal excludes the implementation for the
> > > > currentFetchEventTimeLag
> > > > >>> metric, which caused a bit of discussion without a clear
> > convergence.
> > > > We
> > > > >>> will implement that metric in a generic way at a later point and
> > > > >> encourage
> > > > >>> sources to implement it themselves in the meantime.
> > > > >>>
> > > > >>> Best,
> > > > >>>
> > > > >>> Arvid
> > > > >>>
> > > > >>> [1]
> > > > >>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-179%3A+Expose+Standardized+Operator+Metrics
> > > > >>> [2]
> > > > >>>
> > > > >>>
> > > > >>
> > > >
> > >
> >
> https://lists.apache.org/thread.html/r856920cbfe6a262b521109c5bdb9e904e00a9b3f1825901759c24d85%40%3Cdev.flink.apache.org%3E
> > > >
> > > >
> > > >
> > >
> >
>
Re: [VOTE] FLIP-179: Expose Standardized Operator Metrics
Posted by Becket Qin <be...@gmail.com>.
Personally speaking, it is intuitive for me to set a gauge in MetricGroup.
So I am fine with set*Gauge pattern as long as the method is in
*MetricGroup class.
Thanks,
Jiangjie (Becket) Qin
On Tue, Aug 3, 2021 at 7:24 PM Arvid Heise <ar...@apache.org> wrote:
> @Becket Qin <be...@gmail.com> @Thomas Weise <th...@apache.org> would
> you
> also agree to @Chesnay Schepler <ch...@apache.org> 's proposal?
>
> I think the main intention is to only use a Gauge when the exact metric is
> exposed. For any partial value that may be used in an internal/predefined
> metric, we would only use a supplier instead of a Gauge.
>
> So a connector developer can immediately distinguish the cases: if it's a
> metric class he would see the exact metric corresponding to the setter. If
> it's some Supplier, the developer would expect that the value is used in a
> differently named metric, which we would describe in the JavaDoc.
> Could that also be a solution to the currentEventFetchTimeLag metric?
>
> On Tue, Aug 3, 2021 at 12:54 PM Thomas Weise <th...@apache.org> wrote:
>
> > +1 (binding)
> >
> > On Tue, Aug 3, 2021 at 12:58 AM Chesnay Schepler <ch...@apache.org>
> > wrote:
> >
> > > +1 (binding)
> > >
> > > Although I still think all the set* methods should accept a Supplier
> > > instead of a Gauge.
> > >
> > > On 02/08/2021 12:36, Becket Qin wrote:
> > > > +1 (binding).
> > > >
> > > > Thanks for driving the efforts, Arvid.
> > > >
> > > > Cheers,
> > > >
> > > > Jiangjie (Becket) Qin
> > > >
> > > > On Sat, Jul 31, 2021 at 12:08 PM Steven Wu <st...@gmail.com>
> > wrote:
> > > >
> > > >> +1 (non-binding)
> > > >>
> > > >> On Fri, Jul 30, 2021 at 3:55 AM Arvid Heise <ar...@apache.org>
> wrote:
> > > >>
> > > >>> Dear devs,
> > > >>>
> > > >>> I'd like to open a vote on FLIP-179: Expose Standardized Operator
> > > Metrics
> > > >>> [1] which was discussed in this thread [2].
> > > >>> The vote will be open for at least 72 hours unless there is an
> > > objection
> > > >>> or not enough votes.
> > > >>>
> > > >>> The proposal excludes the implementation for the
> > > currentFetchEventTimeLag
> > > >>> metric, which caused a bit of discussion without a clear
> convergence.
> > > We
> > > >>> will implement that metric in a generic way at a later point and
> > > >> encourage
> > > >>> sources to implement it themselves in the meantime.
> > > >>>
> > > >>> Best,
> > > >>>
> > > >>> Arvid
> > > >>>
> > > >>> [1]
> > > >>>
> > > >>>
> > > >>
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-179%3A+Expose+Standardized+Operator+Metrics
> > > >>> [2]
> > > >>>
> > > >>>
> > > >>
> > >
> >
> https://lists.apache.org/thread.html/r856920cbfe6a262b521109c5bdb9e904e00a9b3f1825901759c24d85%40%3Cdev.flink.apache.org%3E
> > >
> > >
> > >
> >
>
Re: [VOTE] FLIP-179: Expose Standardized Operator Metrics
Posted by Arvid Heise <ar...@apache.org>.
@Becket Qin <be...@gmail.com> @Thomas Weise <th...@apache.org> would you
also agree to @Chesnay Schepler <ch...@apache.org> 's proposal?
I think the main intention is to only use a Gauge when the exact metric is
exposed. For any partial value that may be used in an internal/predefined
metric, we would only use a supplier instead of a Gauge.
So a connector developer can immediately distinguish the cases: if it's a
metric class he would see the exact metric corresponding to the setter. If
it's some Supplier, the developer would expect that the value is used in a
differently named metric, which we would describe in the JavaDoc.
Could that also be a solution to the currentEventFetchTimeLag metric?
On Tue, Aug 3, 2021 at 12:54 PM Thomas Weise <th...@apache.org> wrote:
> +1 (binding)
>
> On Tue, Aug 3, 2021 at 12:58 AM Chesnay Schepler <ch...@apache.org>
> wrote:
>
> > +1 (binding)
> >
> > Although I still think all the set* methods should accept a Supplier
> > instead of a Gauge.
> >
> > On 02/08/2021 12:36, Becket Qin wrote:
> > > +1 (binding).
> > >
> > > Thanks for driving the efforts, Arvid.
> > >
> > > Cheers,
> > >
> > > Jiangjie (Becket) Qin
> > >
> > > On Sat, Jul 31, 2021 at 12:08 PM Steven Wu <st...@gmail.com>
> wrote:
> > >
> > >> +1 (non-binding)
> > >>
> > >> On Fri, Jul 30, 2021 at 3:55 AM Arvid Heise <ar...@apache.org> wrote:
> > >>
> > >>> Dear devs,
> > >>>
> > >>> I'd like to open a vote on FLIP-179: Expose Standardized Operator
> > Metrics
> > >>> [1] which was discussed in this thread [2].
> > >>> The vote will be open for at least 72 hours unless there is an
> > objection
> > >>> or not enough votes.
> > >>>
> > >>> The proposal excludes the implementation for the
> > currentFetchEventTimeLag
> > >>> metric, which caused a bit of discussion without a clear convergence.
> > We
> > >>> will implement that metric in a generic way at a later point and
> > >> encourage
> > >>> sources to implement it themselves in the meantime.
> > >>>
> > >>> Best,
> > >>>
> > >>> Arvid
> > >>>
> > >>> [1]
> > >>>
> > >>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-179%3A+Expose+Standardized+Operator+Metrics
> > >>> [2]
> > >>>
> > >>>
> > >>
> >
> https://lists.apache.org/thread.html/r856920cbfe6a262b521109c5bdb9e904e00a9b3f1825901759c24d85%40%3Cdev.flink.apache.org%3E
> >
> >
> >
>
Re: [VOTE] FLIP-179: Expose Standardized Operator Metrics
Posted by Thomas Weise <th...@apache.org>.
+1 (binding)
On Tue, Aug 3, 2021 at 12:58 AM Chesnay Schepler <ch...@apache.org> wrote:
> +1 (binding)
>
> Although I still think all the set* methods should accept a Supplier
> instead of a Gauge.
>
> On 02/08/2021 12:36, Becket Qin wrote:
> > +1 (binding).
> >
> > Thanks for driving the efforts, Arvid.
> >
> > Cheers,
> >
> > Jiangjie (Becket) Qin
> >
> > On Sat, Jul 31, 2021 at 12:08 PM Steven Wu <st...@gmail.com> wrote:
> >
> >> +1 (non-binding)
> >>
> >> On Fri, Jul 30, 2021 at 3:55 AM Arvid Heise <ar...@apache.org> wrote:
> >>
> >>> Dear devs,
> >>>
> >>> I'd like to open a vote on FLIP-179: Expose Standardized Operator
> Metrics
> >>> [1] which was discussed in this thread [2].
> >>> The vote will be open for at least 72 hours unless there is an
> objection
> >>> or not enough votes.
> >>>
> >>> The proposal excludes the implementation for the
> currentFetchEventTimeLag
> >>> metric, which caused a bit of discussion without a clear convergence.
> We
> >>> will implement that metric in a generic way at a later point and
> >> encourage
> >>> sources to implement it themselves in the meantime.
> >>>
> >>> Best,
> >>>
> >>> Arvid
> >>>
> >>> [1]
> >>>
> >>>
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-179%3A+Expose+Standardized+Operator+Metrics
> >>> [2]
> >>>
> >>>
> >>
> https://lists.apache.org/thread.html/r856920cbfe6a262b521109c5bdb9e904e00a9b3f1825901759c24d85%40%3Cdev.flink.apache.org%3E
>
>
>
Re: [VOTE] FLIP-179: Expose Standardized Operator Metrics
Posted by Chesnay Schepler <ch...@apache.org>.
+1 (binding)
Although I still think all the set* methods should accept a Supplier
instead of a Gauge.
On 02/08/2021 12:36, Becket Qin wrote:
> +1 (binding).
>
> Thanks for driving the efforts, Arvid.
>
> Cheers,
>
> Jiangjie (Becket) Qin
>
> On Sat, Jul 31, 2021 at 12:08 PM Steven Wu <st...@gmail.com> wrote:
>
>> +1 (non-binding)
>>
>> On Fri, Jul 30, 2021 at 3:55 AM Arvid Heise <ar...@apache.org> wrote:
>>
>>> Dear devs,
>>>
>>> I'd like to open a vote on FLIP-179: Expose Standardized Operator Metrics
>>> [1] which was discussed in this thread [2].
>>> The vote will be open for at least 72 hours unless there is an objection
>>> or not enough votes.
>>>
>>> The proposal excludes the implementation for the currentFetchEventTimeLag
>>> metric, which caused a bit of discussion without a clear convergence. We
>>> will implement that metric in a generic way at a later point and
>> encourage
>>> sources to implement it themselves in the meantime.
>>>
>>> Best,
>>>
>>> Arvid
>>>
>>> [1]
>>>
>>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-179%3A+Expose+Standardized+Operator+Metrics
>>> [2]
>>>
>>>
>> https://lists.apache.org/thread.html/r856920cbfe6a262b521109c5bdb9e904e00a9b3f1825901759c24d85%40%3Cdev.flink.apache.org%3E