You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@flink.apache.org by Yadong Xie <vt...@gmail.com> on 2020/02/20 09:46:14 UTC
[VOTE] FLIP-100: Add Attempt Information
Hi all
I want to start the vote for FLIP-100, which proposes to add attempt
information inside subtask and timeline in web UI.
To help everyone better understand the proposal, we spent some efforts on
making an online POC
Timeline Attempt (click the vertex timeline to see the differences):
previous web:
http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
POC web:
http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
Subtask Attempt (click the vertex and switch to subtask tab to see the
differences):
previous web:
http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
POC web:
http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
The vote will last for at least 72 hours, following the consensus voting
process.
FLIP wiki:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
Discussion thread:
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
Thanks,
Yadong
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Yadong Xie <vt...@gmail.com>.
Hi Gary Kurt, and Jark
I am canceling the vote and restart it since the POC has some changes from
the initial one.
All the changes are following the proposal in this mail thread.
please vote again in the new thread, thanks
Jark Wu <im...@gmail.com> 于2020年3月4日周三 下午12:13写道：
> +1 from my side.
>
> Best,
> Jark
>
> On Wed, 4 Mar 2020 at 11:39, Kurt Young <yk...@gmail.com> wrote:
>
> > LGTM now, +1 from my side.
> >
> > Best,
> > Kurt
> >
> >
> > On Wed, Mar 4, 2020 at 12:27 AM Gary Yao <ga...@apache.org> wrote:
> >
> >> Hi Yadong,
> >>
> >> Thank you for updating the wiki page.
> >>
> >> Only one minor suggestion – I would change:
> >>
> >> > If show-history is true return the information of attempt.
> >>
> >> to
> >>
> >> > If show-history is true, information for all attempts including
> >> previous ones will be returned
> >>
> >> That being said, FLIP-100 looks good to me. From my side there is not
> >> anything
> >> else to discuss.
> >>
> >> @Kurt and @Jark: Can you look into the improvements that have been made
> >> since
> >> the last time you looked at the PoC? If you are happy, we can restart
> the
> >> voting.
> >>
> >> Best,
> >> Gary
> >>
> >> On Tue, Mar 3, 2020 at 2:34 PM Yadong Xie <vt...@gmail.com> wrote:
> >>
> >>> Hi all
> >>>
> >>> The rest API part has been updated with Gary and Till's suggestions
> >>> here is the link:
> >>>
> >>>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> >>>
> >>> Yadong Xie <vt...@gmail.com> 于2020年3月3日周二 下午9:14写道：
> >>>
> >>> > Hi Chesnay
> >>> >
> >>> > most discussions in this vote are about the more feature/demo request
> >>> in
> >>> > POC or discussion about response format, the main proposal the web UI
> >>> part
> >>> > which is not changed
> >>> >
> >>> > and the discussion about the response is converging, the response
> >>> format
> >>> > discussion could happen either here or at the code review stage,
> which
> >>> > would be a minor change from my point of view.
> >>> >
> >>> > Chesnay Schepler <ch...@apache.org> 于2020年3月3日周二 下午8:20写道：
> >>> >
> >>> >> I suggest to cancel this vote.
> >>> >> Several discussion items have been brought up during the vote, some
> of
> >>> >> which are still unresolved, others which resulted in changes to the
> >>> >> proposal.
> >>> >>
> >>> >> My conclusion is that this proposal needs more discussions.
> >>> >>
> >>> >>
> >>> >> On 20/02/2020 10:46, Yadong Xie wrote:
> >>> >> > Hi all
> >>> >> >
> >>> >> > I want to start the vote for FLIP-100, which proposes to add
> attempt
> >>> >> > information inside subtask and timeline in web UI.
> >>> >> >
> >>> >> > To help everyone better understand the proposal, we spent some
> >>> efforts
> >>> >> on
> >>> >> > making an online POC
> >>> >> >
> >>> >> > Timeline Attempt (click the vertex timeline to see the
> differences):
> >>> >> > previous web:
> >>> >> >
> >>> >>
> >>>
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> >>> >> > POC web:
> >>> >> >
> >>> >>
> >>>
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> >>> >> >
> >>> >> > Subtask Attempt (click the vertex and switch to subtask tab to see
> >>> the
> >>> >> > differences):
> >>> >> > previous web:
> >>> >> >
> >>> >>
> >>>
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> >>> >> > POC web:
> >>> >> >
> >>> >>
> >>>
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> >>> >> >
> >>> >> >
> >>> >> > The vote will last for at least 72 hours, following the consensus
> >>> voting
> >>> >> > process.
> >>> >> >
> >>> >> > FLIP wiki:
> >>> >> >
> >>> >>
> >>>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> >>> >> >
> >>> >> > Discussion thread:
> >>> >> >
> >>> >>
> >>>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> >>> >> >
> >>> >> > Thanks,
> >>> >> >
> >>> >> > Yadong
> >>> >> >
> >>> >>
> >>> >>
> >>>
> >>
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Jark Wu <im...@gmail.com>.
+1 from my side.
Best,
Jark
On Wed, 4 Mar 2020 at 11:39, Kurt Young <yk...@gmail.com> wrote:
> LGTM now, +1 from my side.
>
> Best,
> Kurt
>
>
> On Wed, Mar 4, 2020 at 12:27 AM Gary Yao <ga...@apache.org> wrote:
>
>> Hi Yadong,
>>
>> Thank you for updating the wiki page.
>>
>> Only one minor suggestion – I would change:
>>
>> > If show-history is true return the information of attempt.
>>
>> to
>>
>> > If show-history is true, information for all attempts including
>> previous ones will be returned
>>
>> That being said, FLIP-100 looks good to me. From my side there is not
>> anything
>> else to discuss.
>>
>> @Kurt and @Jark: Can you look into the improvements that have been made
>> since
>> the last time you looked at the PoC? If you are happy, we can restart the
>> voting.
>>
>> Best,
>> Gary
>>
>> On Tue, Mar 3, 2020 at 2:34 PM Yadong Xie <vt...@gmail.com> wrote:
>>
>>> Hi all
>>>
>>> The rest API part has been updated with Gary and Till's suggestions
>>> here is the link:
>>>
>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>>>
>>> Yadong Xie <vt...@gmail.com> 于2020年3月3日周二 下午9:14写道：
>>>
>>> > Hi Chesnay
>>> >
>>> > most discussions in this vote are about the more feature/demo request
>>> in
>>> > POC or discussion about response format, the main proposal the web UI
>>> part
>>> > which is not changed
>>> >
>>> > and the discussion about the response is converging, the response
>>> format
>>> > discussion could happen either here or at the code review stage, which
>>> > would be a minor change from my point of view.
>>> >
>>> > Chesnay Schepler <ch...@apache.org> 于2020年3月3日周二 下午8:20写道：
>>> >
>>> >> I suggest to cancel this vote.
>>> >> Several discussion items have been brought up during the vote, some of
>>> >> which are still unresolved, others which resulted in changes to the
>>> >> proposal.
>>> >>
>>> >> My conclusion is that this proposal needs more discussions.
>>> >>
>>> >>
>>> >> On 20/02/2020 10:46, Yadong Xie wrote:
>>> >> > Hi all
>>> >> >
>>> >> > I want to start the vote for FLIP-100, which proposes to add attempt
>>> >> > information inside subtask and timeline in web UI.
>>> >> >
>>> >> > To help everyone better understand the proposal, we spent some
>>> efforts
>>> >> on
>>> >> > making an online POC
>>> >> >
>>> >> > Timeline Attempt (click the vertex timeline to see the differences):
>>> >> > previous web:
>>> >> >
>>> >>
>>> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
>>> >> > POC web:
>>> >> >
>>> >>
>>> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
>>> >> >
>>> >> > Subtask Attempt (click the vertex and switch to subtask tab to see
>>> the
>>> >> > differences):
>>> >> > previous web:
>>> >> >
>>> >>
>>> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
>>> >> > POC web:
>>> >> >
>>> >>
>>> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
>>> >> >
>>> >> >
>>> >> > The vote will last for at least 72 hours, following the consensus
>>> voting
>>> >> > process.
>>> >> >
>>> >> > FLIP wiki:
>>> >> >
>>> >>
>>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>>> >> >
>>> >> > Discussion thread:
>>> >> >
>>> >>
>>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
>>> >> >
>>> >> > Thanks,
>>> >> >
>>> >> > Yadong
>>> >> >
>>> >>
>>> >>
>>>
>>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Kurt Young <yk...@gmail.com>.
LGTM now, +1 from my side.
Best,
Kurt
On Wed, Mar 4, 2020 at 12:27 AM Gary Yao <ga...@apache.org> wrote:
> Hi Yadong,
>
> Thank you for updating the wiki page.
>
> Only one minor suggestion – I would change:
>
> > If show-history is true return the information of attempt.
>
> to
>
> > If show-history is true, information for all attempts including
> previous ones will be returned
>
> That being said, FLIP-100 looks good to me. From my side there is not
> anything
> else to discuss.
>
> @Kurt and @Jark: Can you look into the improvements that have been made
> since
> the last time you looked at the PoC? If you are happy, we can restart the
> voting.
>
> Best,
> Gary
>
> On Tue, Mar 3, 2020 at 2:34 PM Yadong Xie <vt...@gmail.com> wrote:
>
>> Hi all
>>
>> The rest API part has been updated with Gary and Till's suggestions
>> here is the link:
>>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>>
>> Yadong Xie <vt...@gmail.com> 于2020年3月3日周二 下午9:14写道：
>>
>> > Hi Chesnay
>> >
>> > most discussions in this vote are about the more feature/demo request in
>> > POC or discussion about response format, the main proposal the web UI
>> part
>> > which is not changed
>> >
>> > and the discussion about the response is converging, the response format
>> > discussion could happen either here or at the code review stage, which
>> > would be a minor change from my point of view.
>> >
>> > Chesnay Schepler <ch...@apache.org> 于2020年3月3日周二 下午8:20写道：
>> >
>> >> I suggest to cancel this vote.
>> >> Several discussion items have been brought up during the vote, some of
>> >> which are still unresolved, others which resulted in changes to the
>> >> proposal.
>> >>
>> >> My conclusion is that this proposal needs more discussions.
>> >>
>> >>
>> >> On 20/02/2020 10:46, Yadong Xie wrote:
>> >> > Hi all
>> >> >
>> >> > I want to start the vote for FLIP-100, which proposes to add attempt
>> >> > information inside subtask and timeline in web UI.
>> >> >
>> >> > To help everyone better understand the proposal, we spent some
>> efforts
>> >> on
>> >> > making an online POC
>> >> >
>> >> > Timeline Attempt (click the vertex timeline to see the differences):
>> >> > previous web:
>> >> >
>> >>
>> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
>> >> > POC web:
>> >> >
>> >>
>> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
>> >> >
>> >> > Subtask Attempt (click the vertex and switch to subtask tab to see
>> the
>> >> > differences):
>> >> > previous web:
>> >> >
>> >>
>> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
>> >> > POC web:
>> >> >
>> >>
>> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
>> >> >
>> >> >
>> >> > The vote will last for at least 72 hours, following the consensus
>> voting
>> >> > process.
>> >> >
>> >> > FLIP wiki:
>> >> >
>> >>
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>> >> >
>> >> > Discussion thread:
>> >> >
>> >>
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
>> >> >
>> >> > Thanks,
>> >> >
>> >> > Yadong
>> >> >
>> >>
>> >>
>>
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Gary Yao <ga...@apache.org>.
Hi Yadong,
Thank you for updating the wiki page.
Only one minor suggestion – I would change:
> If show-history is true return the information of attempt.
to
> If show-history is true, information for all attempts including
previous ones will be returned
That being said, FLIP-100 looks good to me. From my side there is not
anything
else to discuss.
@Kurt and @Jark: Can you look into the improvements that have been made
since
the last time you looked at the PoC? If you are happy, we can restart the
voting.
Best,
Gary
On Tue, Mar 3, 2020 at 2:34 PM Yadong Xie <vt...@gmail.com> wrote:
> Hi all
>
> The rest API part has been updated with Gary and Till's suggestions
> here is the link:
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>
> Yadong Xie <vt...@gmail.com> 于2020年3月3日周二 下午9:14写道：
>
> > Hi Chesnay
> >
> > most discussions in this vote are about the more feature/demo request in
> > POC or discussion about response format, the main proposal the web UI
> part
> > which is not changed
> >
> > and the discussion about the response is converging, the response format
> > discussion could happen either here or at the code review stage, which
> > would be a minor change from my point of view.
> >
> > Chesnay Schepler <ch...@apache.org> 于2020年3月3日周二 下午8:20写道：
> >
> >> I suggest to cancel this vote.
> >> Several discussion items have been brought up during the vote, some of
> >> which are still unresolved, others which resulted in changes to the
> >> proposal.
> >>
> >> My conclusion is that this proposal needs more discussions.
> >>
> >>
> >> On 20/02/2020 10:46, Yadong Xie wrote:
> >> > Hi all
> >> >
> >> > I want to start the vote for FLIP-100, which proposes to add attempt
> >> > information inside subtask and timeline in web UI.
> >> >
> >> > To help everyone better understand the proposal, we spent some efforts
> >> on
> >> > making an online POC
> >> >
> >> > Timeline Attempt (click the vertex timeline to see the differences):
> >> > previous web:
> >> >
> >>
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> >> > POC web:
> >> >
> >>
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> >> >
> >> > Subtask Attempt (click the vertex and switch to subtask tab to see the
> >> > differences):
> >> > previous web:
> >> >
> >>
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> >> > POC web:
> >> >
> >>
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> >> >
> >> >
> >> > The vote will last for at least 72 hours, following the consensus
> voting
> >> > process.
> >> >
> >> > FLIP wiki:
> >> >
> >>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> >> >
> >> > Discussion thread:
> >> >
> >>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> >> >
> >> > Thanks,
> >> >
> >> > Yadong
> >> >
> >>
> >>
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Yadong Xie <vt...@gmail.com>.
Hi all
The rest API part has been updated with Gary and Till's suggestions
here is the link:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
Yadong Xie <vt...@gmail.com> 于2020年3月3日周二 下午9:14写道：
> Hi Chesnay
>
> most discussions in this vote are about the more feature/demo request in
> POC or discussion about response format, the main proposal the web UI part
> which is not changed
>
> and the discussion about the response is converging, the response format
> discussion could happen either here or at the code review stage, which
> would be a minor change from my point of view.
>
> Chesnay Schepler <ch...@apache.org> 于2020年3月3日周二 下午8:20写道：
>
>> I suggest to cancel this vote.
>> Several discussion items have been brought up during the vote, some of
>> which are still unresolved, others which resulted in changes to the
>> proposal.
>>
>> My conclusion is that this proposal needs more discussions.
>>
>>
>> On 20/02/2020 10:46, Yadong Xie wrote:
>> > Hi all
>> >
>> > I want to start the vote for FLIP-100, which proposes to add attempt
>> > information inside subtask and timeline in web UI.
>> >
>> > To help everyone better understand the proposal, we spent some efforts
>> on
>> > making an online POC
>> >
>> > Timeline Attempt (click the vertex timeline to see the differences):
>> > previous web:
>> >
>> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
>> > POC web:
>> >
>> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
>> >
>> > Subtask Attempt (click the vertex and switch to subtask tab to see the
>> > differences):
>> > previous web:
>> >
>> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
>> > POC web:
>> >
>> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
>> >
>> >
>> > The vote will last for at least 72 hours, following the consensus voting
>> > process.
>> >
>> > FLIP wiki:
>> >
>> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>> >
>> > Discussion thread:
>> >
>> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
>> >
>> > Thanks,
>> >
>> > Yadong
>> >
>>
>>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Yadong Xie <vt...@gmail.com>.
Hi Chesnay
most discussions in this vote are about the more feature/demo request in
POC or discussion about response format, the main proposal the web UI part
which is not changed
and the discussion about the response is converging, the response format
discussion could happen either here or at the code review stage, which
would be a minor change from my point of view.
Chesnay Schepler <ch...@apache.org> 于2020年3月3日周二 下午8:20写道：
> I suggest to cancel this vote.
> Several discussion items have been brought up during the vote, some of
> which are still unresolved, others which resulted in changes to the
> proposal.
>
> My conclusion is that this proposal needs more discussions.
>
>
> On 20/02/2020 10:46, Yadong Xie wrote:
> > Hi all
> >
> > I want to start the vote for FLIP-100, which proposes to add attempt
> > information inside subtask and timeline in web UI.
> >
> > To help everyone better understand the proposal, we spent some efforts on
> > making an online POC
> >
> > Timeline Attempt (click the vertex timeline to see the differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > POC web:
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> >
> > Subtask Attempt (click the vertex and switch to subtask tab to see the
> > differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > POC web:
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> >
> >
> > The vote will last for at least 72 hours, following the consensus voting
> > process.
> >
> > FLIP wiki:
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> >
> > Discussion thread:
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> >
> > Thanks,
> >
> > Yadong
> >
>
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Chesnay Schepler <ch...@apache.org>.
I suggest to cancel this vote.
Several discussion items have been brought up during the vote, some of
which are still unresolved, others which resulted in changes to the
proposal.
My conclusion is that this proposal needs more discussions.
On 20/02/2020 10:46, Yadong Xie wrote:
> Hi all
>
> I want to start the vote for FLIP-100, which proposes to add attempt
> information inside subtask and timeline in web UI.
>
> To help everyone better understand the proposal, we spent some efforts on
> making an online POC
>
> Timeline Attempt (click the vertex timeline to see the differences):
> previous web:
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> POC web:
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
>
> Subtask Attempt (click the vertex and switch to subtask tab to see the
> differences):
> previous web:
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> POC web:
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
>
>
> The vote will last for at least 72 hours, following the consensus voting
> process.
>
> FLIP wiki:
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>
> Discussion thread:
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
>
> Thanks,
>
> Yadong
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Yadong Xie <vt...@gmail.com>.
Hi Jark and Benchao
The link to taks manager log was added to the attempt timeline now, here is
the new POC url
check it here:
http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
thanks for your suggestions!
lining jing <ji...@gmail.com> 于2020年2月24日周一 下午5:16写道：
> I agree with Jark.
> The user can find the previous taskmanager according to the log link in the
> history of the attempt.
>
> The reason for adding attempts' information to the timeline is that for
> some reason, the job failover failed or took a long time. This problem is
> challenging to locate. As the current timeline is a summary of all attempt
> information with the attempted timeline, you could find attempts with an
> abnormal status.
>
>
>
> Benchao Li <li...@gmail.com> 于2020年2月24日周一 上午8:51写道：
>
> > I agree with Jark.
> > It's helpful to add a TM logs link for each attempt. Usually there are
> > multiple TMs per host, so only host name cannot determine the real TM.
> >
> > Jark Wu <im...@gmail.com> 于2020年2月21日周五 下午2:06写道：
> >
> > > Thanks Yadong,
> > >
> > > I think one of the purpose to find the failed attempts is that we want
> to
> > > look into the logs in the failed node (node may change across
> attempts).
> > > However, from the POC, I didn't find a log link on each attempts. Maybe
> > we
> > > can add such feature?
> > >
> > > Best,
> > > Jark
> > >
> > > On Fri, 21 Feb 2020 at 11:51, Kurt Young <yk...@gmail.com> wrote:
> > >
> > > > Hi Yadong,
> > > >
> > > > Thanks for the proposal, it's a useful feature, especially for batch
> > > jobs.
> > > > But according
> > > > to the examples you gave, I can't tell whether i got required
> > information
> > > > from that.
> > > > Can you replace the demo job to a more complex batch job and then we
> > can
> > > > see some
> > > > differences of start/stop time of different tasks and attempts?
> > > >
> > > > Best,
> > > > Kurt
> > > >
> > > >
> > > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <vt...@gmail.com>
> > wrote:
> > > >
> > > > > Hi all
> > > > >
> > > > > I want to start the vote for FLIP-100, which proposes to add
> attempt
> > > > > information inside subtask and timeline in web UI.
> > > > >
> > > > > To help everyone better understand the proposal, we spent some
> > efforts
> > > on
> > > > > making an online POC
> > > > >
> > > > > Timeline Attempt (click the vertex timeline to see the
> differences):
> > > > > previous web:
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > POC web:
> > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > >
> > > > > Subtask Attempt (click the vertex and switch to subtask tab to see
> > the
> > > > > differences):
> > > > > previous web:
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > POC web:
> > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > > >
> > > > >
> > > > > The vote will last for at least 72 hours, following the consensus
> > > voting
> > > > > process.
> > > > >
> > > > > FLIP wiki:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > >
> > > > > Discussion thread:
> > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Yadong
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > Benchao Li
> > School of Electronics Engineering and Computer Science, Peking University
> > Tel:+86-15650713730
> > Email: libenchao@gmail.com; libenchao@pku.edu.cn
> >
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by lining jing <ji...@gmail.com>.
I agree with Jark.
The user can find the previous taskmanager according to the log link in the
history of the attempt.
The reason for adding attempts' information to the timeline is that for
some reason, the job failover failed or took a long time. This problem is
challenging to locate. As the current timeline is a summary of all attempt
information with the attempted timeline, you could find attempts with an
abnormal status.
Benchao Li <li...@gmail.com> 于2020年2月24日周一 上午8:51写道：
> I agree with Jark.
> It's helpful to add a TM logs link for each attempt. Usually there are
> multiple TMs per host, so only host name cannot determine the real TM.
>
> Jark Wu <im...@gmail.com> 于2020年2月21日周五 下午2:06写道：
>
> > Thanks Yadong,
> >
> > I think one of the purpose to find the failed attempts is that we want to
> > look into the logs in the failed node (node may change across attempts).
> > However, from the POC, I didn't find a log link on each attempts. Maybe
> we
> > can add such feature?
> >
> > Best,
> > Jark
> >
> > On Fri, 21 Feb 2020 at 11:51, Kurt Young <yk...@gmail.com> wrote:
> >
> > > Hi Yadong,
> > >
> > > Thanks for the proposal, it's a useful feature, especially for batch
> > jobs.
> > > But according
> > > to the examples you gave, I can't tell whether i got required
> information
> > > from that.
> > > Can you replace the demo job to a more complex batch job and then we
> can
> > > see some
> > > differences of start/stop time of different tasks and attempts?
> > >
> > > Best,
> > > Kurt
> > >
> > >
> > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <vt...@gmail.com>
> wrote:
> > >
> > > > Hi all
> > > >
> > > > I want to start the vote for FLIP-100, which proposes to add attempt
> > > > information inside subtask and timeline in web UI.
> > > >
> > > > To help everyone better understand the proposal, we spent some
> efforts
> > on
> > > > making an online POC
> > > >
> > > > Timeline Attempt (click the vertex timeline to see the differences):
> > > > previous web:
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > POC web:
> > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > >
> > > > Subtask Attempt (click the vertex and switch to subtask tab to see
> the
> > > > differences):
> > > > previous web:
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > POC web:
> > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > >
> > > >
> > > > The vote will last for at least 72 hours, following the consensus
> > voting
> > > > process.
> > > >
> > > > FLIP wiki:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > >
> > > > Discussion thread:
> > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > >
> > > > Thanks,
> > > >
> > > > Yadong
> > > >
> > >
> >
>
>
> --
>
> Benchao Li
> School of Electronics Engineering and Computer Science, Peking University
> Tel:+86-15650713730
> Email: libenchao@gmail.com; libenchao@pku.edu.cn
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Benchao Li <li...@gmail.com>.
I agree with Jark.
It's helpful to add a TM logs link for each attempt. Usually there are
multiple TMs per host, so only host name cannot determine the real TM.
Jark Wu <im...@gmail.com> 于2020年2月21日周五 下午2:06写道：
> Thanks Yadong,
>
> I think one of the purpose to find the failed attempts is that we want to
> look into the logs in the failed node (node may change across attempts).
> However, from the POC, I didn't find a log link on each attempts. Maybe we
> can add such feature?
>
> Best,
> Jark
>
> On Fri, 21 Feb 2020 at 11:51, Kurt Young <yk...@gmail.com> wrote:
>
> > Hi Yadong,
> >
> > Thanks for the proposal, it's a useful feature, especially for batch
> jobs.
> > But according
> > to the examples you gave, I can't tell whether i got required information
> > from that.
> > Can you replace the demo job to a more complex batch job and then we can
> > see some
> > differences of start/stop time of different tasks and attempts?
> >
> > Best,
> > Kurt
> >
> >
> > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <vt...@gmail.com> wrote:
> >
> > > Hi all
> > >
> > > I want to start the vote for FLIP-100, which proposes to add attempt
> > > information inside subtask and timeline in web UI.
> > >
> > > To help everyone better understand the proposal, we spent some efforts
> on
> > > making an online POC
> > >
> > > Timeline Attempt (click the vertex timeline to see the differences):
> > > previous web:
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > POC web:
> > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > >
> > > Subtask Attempt (click the vertex and switch to subtask tab to see the
> > > differences):
> > > previous web:
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > POC web:
> > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > >
> > >
> > > The vote will last for at least 72 hours, following the consensus
> voting
> > > process.
> > >
> > > FLIP wiki:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > >
> > > Discussion thread:
> > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > >
> > > Thanks,
> > >
> > > Yadong
> > >
> >
>
--
Benchao Li
School of Electronics Engineering and Computer Science, Peking University
Tel:+86-15650713730
Email: libenchao@gmail.com; libenchao@pku.edu.cn
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Jark Wu <im...@gmail.com>.
Thanks Yadong,
I think one of the purpose to find the failed attempts is that we want to
look into the logs in the failed node (node may change across attempts).
However, from the POC, I didn't find a log link on each attempts. Maybe we
can add such feature?
Best,
Jark
On Fri, 21 Feb 2020 at 11:51, Kurt Young <yk...@gmail.com> wrote:
> Hi Yadong,
>
> Thanks for the proposal, it's a useful feature, especially for batch jobs.
> But according
> to the examples you gave, I can't tell whether i got required information
> from that.
> Can you replace the demo job to a more complex batch job and then we can
> see some
> differences of start/stop time of different tasks and attempts?
>
> Best,
> Kurt
>
>
> On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <vt...@gmail.com> wrote:
>
> > Hi all
> >
> > I want to start the vote for FLIP-100, which proposes to add attempt
> > information inside subtask and timeline in web UI.
> >
> > To help everyone better understand the proposal, we spent some efforts on
> > making an online POC
> >
> > Timeline Attempt (click the vertex timeline to see the differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > POC web:
> >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> >
> > Subtask Attempt (click the vertex and switch to subtask tab to see the
> > differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > POC web:
> >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> >
> >
> > The vote will last for at least 72 hours, following the consensus voting
> > process.
> >
> > FLIP wiki:
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> >
> > Discussion thread:
> >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> >
> > Thanks,
> >
> > Yadong
> >
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Yadong Xie <vt...@gmail.com>.
Hi everyone
A subtask filter and an attempt id filter was added to the attempt timeline
page now
check the POC here:
http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
FLIP doc was also updated:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
Till Rohrmann <tr...@apache.org> 于2020年2月27日周四 下午9:49写道：
> I think your proposal is better because then we don't have to bump the REST
> API version.
>
> Cheers,
> Till
>
> On Thu, Feb 27, 2020 at 8:02 AM Yadong Xie <vt...@gmail.com> wrote:
>
> > Hi Till
> >
> > We keep response a flattened SubtaskTimeInfo and an array of
> > SubtaskTimeInfo to keep the restAPI backward compatible, since Flink
> users
> > may still need the API('/jobs/{jobId}/vertices/{vertexId}/subtasktimes')
> to
> > get the flattened SubtaskTimeInfo.
> >
> > If we want to get an array of SubtaskTimeInfo in the API, a new URL needs
> > to be created other than reuse the old one.
> >
> > Both solutions are ok for me. What do you think about it?
> >
> > Till Rohrmann <tr...@apache.org> 于2020年2月26日周三 下午10:53写道：
> >
> > > Fair enough. If this should become a problem we could introduce it
> later
> > as
> > > well.
> > >
> > > What about changing the SubtasksTimeInfo response type into an array of
> > > SubtaskTimeInfo? At the moment SubtasksTimeInfo contains a
> > > flattened SubtaskTimeInfo and and array of SubtaskTimeInfo for the
> > previous
> > > attempts.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Wed, Feb 26, 2020 at 1:16 PM Yadong Xie <vt...@gmail.com>
> wrote:
> > >
> > > > Hi Till
> > > >
> > > > Thanks for your comments.
> > > >
> > > > > I have a comment concerning the SubtasksTimesHandler....
> > > >
> > > > It would be much easier for the frontend to handle a large amount of
> > data
> > > > if we have a rest API parameter filter, but in my opinion, the
> attempt
> > > list
> > > > data is not large enough that we have to rely on the rest API
> > parameters
> > > > paging, we still can handle them all in the frontend.
> > > >
> > > > Users can filter the attempt list by the
> > > status(scheduled/created/deploying
> > > > and so on) and other keywords(attempt_id and so on) directly in the
> > > > frontend since all data are listed from the rest API.
> > > > If we move some of the filter parameters to the rest API path
> > parameter,
> > > > all the other filter parameters need to be moved too.
> > > >
> > > > I suggest adding an attempt id filter in the UI to help users filter
> > the
> > > > desired attempt, and all the filtering process is running inside the
> > > > browser side, what do you think about this?
> > > >
> > > >
> > > >
> > > >
> > > > Till Rohrmann <tr...@apache.org> 于2020年2月25日周二 下午11:40写道：
> > > >
> > > > > Hi Yadong,
> > > > >
> > > > > thanks for creating this FLIP. I like the idea to make the web-ui
> > > > > information richer wrt to subtask attempt information.
> > > > >
> > > > > I have a comment concerning the SubtasksTimesHandler: Should we
> > change
> > > > the
> > > > > response type SubtasksTimeInfo so that it simply contains an
> > > > > array of SubtaskTimeInfo? One could add an attempt range path
> > parameter
> > > > to
> > > > > the SubtasksTimesHandler to be able to control which attempts will
> be
> > > > > returned.
> > > > >
> > > > > Cheers,
> > > > > Till
> > > > >
> > > > > On Tue, Feb 25, 2020 at 9:57 AM Benchao Li <li...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hi Yadong,
> > > > > >
> > > > > > Thanks for the updating. LGTM now.
> > > > > >
> > > > > > +1 (non-binding)
> > > > > >
> > > > > > Yadong Xie <vt...@gmail.com> 于2020年2月25日周二 下午4:41写道：
> > > > > >
> > > > > > > Hi Kurt
> > > > > > >
> > > > > > > There will be no differences between batch jobs and stream jobs
> > in
> > > > > > > subtask-attempt level in the UI
> > > > > > > The only differences are in the vertex timeline, I have added a
> > > > > > screenshot
> > > > > > > of the batch job in the FLIP-100 since the batch job will
> > disappear
> > > > > from
> > > > > > > the list after it finished soon.
> > > > > > > here is the link:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > > > >
> > > > > > >
> > > > > > > Kurt Young <yk...@gmail.com> 于2020年2月21日周五 上午11:51写道：
> > > > > > >
> > > > > > > > Hi Yadong,
> > > > > > > >
> > > > > > > > Thanks for the proposal, it's a useful feature, especially
> for
> > > > batch
> > > > > > > jobs.
> > > > > > > > But according
> > > > > > > > to the examples you gave, I can't tell whether i got required
> > > > > > information
> > > > > > > > from that.
> > > > > > > > Can you replace the demo job to a more complex batch job and
> > then
> > > > we
> > > > > > can
> > > > > > > > see some
> > > > > > > > differences of start/stop time of different tasks and
> attempts?
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Kurt
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <
> > vthinkxie@gmail.com>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi all
> > > > > > > > >
> > > > > > > > > I want to start the vote for FLIP-100, which proposes to
> add
> > > > > attempt
> > > > > > > > > information inside subtask and timeline in web UI.
> > > > > > > > >
> > > > > > > > > To help everyone better understand the proposal, we spent
> > some
> > > > > > efforts
> > > > > > > on
> > > > > > > > > making an online POC
> > > > > > > > >
> > > > > > > > > Timeline Attempt (click the vertex timeline to see the
> > > > > differences):
> > > > > > > > > previous web:
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > > > > POC web:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > > > >
> > > > > > > > > Subtask Attempt (click the vertex and switch to subtask tab
> > to
> > > > see
> > > > > > the
> > > > > > > > > differences):
> > > > > > > > > previous web:
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > > > > POC web:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > The vote will last for at least 72 hours, following the
> > > consensus
> > > > > > > voting
> > > > > > > > > process.
> > > > > > > > >
> > > > > > > > > FLIP wiki:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > > > > > >
> > > > > > > > > Discussion thread:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > >
> > > > > > > > > Yadong
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Benchao Li
> > > > > > School of Electronics Engineering and Computer Science, Peking
> > > > University
> > > > > > Tel:+86-15650713730
> > > > > > Email: libenchao@gmail.com; libenchao@pku.edu.cn
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Till Rohrmann <tr...@apache.org>.
I think your proposal is better because then we don't have to bump the REST
API version.
Cheers,
Till
On Thu, Feb 27, 2020 at 8:02 AM Yadong Xie <vt...@gmail.com> wrote:
> Hi Till
>
> We keep response a flattened SubtaskTimeInfo and an array of
> SubtaskTimeInfo to keep the restAPI backward compatible, since Flink users
> may still need the API('/jobs/{jobId}/vertices/{vertexId}/subtasktimes') to
> get the flattened SubtaskTimeInfo.
>
> If we want to get an array of SubtaskTimeInfo in the API, a new URL needs
> to be created other than reuse the old one.
>
> Both solutions are ok for me. What do you think about it?
>
> Till Rohrmann <tr...@apache.org> 于2020年2月26日周三 下午10:53写道：
>
> > Fair enough. If this should become a problem we could introduce it later
> as
> > well.
> >
> > What about changing the SubtasksTimeInfo response type into an array of
> > SubtaskTimeInfo? At the moment SubtasksTimeInfo contains a
> > flattened SubtaskTimeInfo and and array of SubtaskTimeInfo for the
> previous
> > attempts.
> >
> > Cheers,
> > Till
> >
> > On Wed, Feb 26, 2020 at 1:16 PM Yadong Xie <vt...@gmail.com> wrote:
> >
> > > Hi Till
> > >
> > > Thanks for your comments.
> > >
> > > > I have a comment concerning the SubtasksTimesHandler....
> > >
> > > It would be much easier for the frontend to handle a large amount of
> data
> > > if we have a rest API parameter filter, but in my opinion, the attempt
> > list
> > > data is not large enough that we have to rely on the rest API
> parameters
> > > paging, we still can handle them all in the frontend.
> > >
> > > Users can filter the attempt list by the
> > status(scheduled/created/deploying
> > > and so on) and other keywords(attempt_id and so on) directly in the
> > > frontend since all data are listed from the rest API.
> > > If we move some of the filter parameters to the rest API path
> parameter,
> > > all the other filter parameters need to be moved too.
> > >
> > > I suggest adding an attempt id filter in the UI to help users filter
> the
> > > desired attempt, and all the filtering process is running inside the
> > > browser side, what do you think about this?
> > >
> > >
> > >
> > >
> > > Till Rohrmann <tr...@apache.org> 于2020年2月25日周二 下午11:40写道：
> > >
> > > > Hi Yadong,
> > > >
> > > > thanks for creating this FLIP. I like the idea to make the web-ui
> > > > information richer wrt to subtask attempt information.
> > > >
> > > > I have a comment concerning the SubtasksTimesHandler: Should we
> change
> > > the
> > > > response type SubtasksTimeInfo so that it simply contains an
> > > > array of SubtaskTimeInfo? One could add an attempt range path
> parameter
> > > to
> > > > the SubtasksTimesHandler to be able to control which attempts will be
> > > > returned.
> > > >
> > > > Cheers,
> > > > Till
> > > >
> > > > On Tue, Feb 25, 2020 at 9:57 AM Benchao Li <li...@gmail.com>
> > wrote:
> > > >
> > > > > Hi Yadong,
> > > > >
> > > > > Thanks for the updating. LGTM now.
> > > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > Yadong Xie <vt...@gmail.com> 于2020年2月25日周二 下午4:41写道：
> > > > >
> > > > > > Hi Kurt
> > > > > >
> > > > > > There will be no differences between batch jobs and stream jobs
> in
> > > > > > subtask-attempt level in the UI
> > > > > > The only differences are in the vertex timeline, I have added a
> > > > > screenshot
> > > > > > of the batch job in the FLIP-100 since the batch job will
> disappear
> > > > from
> > > > > > the list after it finished soon.
> > > > > > here is the link:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > > >
> > > > > >
> > > > > > Kurt Young <yk...@gmail.com> 于2020年2月21日周五 上午11:51写道：
> > > > > >
> > > > > > > Hi Yadong,
> > > > > > >
> > > > > > > Thanks for the proposal, it's a useful feature, especially for
> > > batch
> > > > > > jobs.
> > > > > > > But according
> > > > > > > to the examples you gave, I can't tell whether i got required
> > > > > information
> > > > > > > from that.
> > > > > > > Can you replace the demo job to a more complex batch job and
> then
> > > we
> > > > > can
> > > > > > > see some
> > > > > > > differences of start/stop time of different tasks and attempts?
> > > > > > >
> > > > > > > Best,
> > > > > > > Kurt
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <
> vthinkxie@gmail.com>
> > > > > wrote:
> > > > > > >
> > > > > > > > Hi all
> > > > > > > >
> > > > > > > > I want to start the vote for FLIP-100, which proposes to add
> > > > attempt
> > > > > > > > information inside subtask and timeline in web UI.
> > > > > > > >
> > > > > > > > To help everyone better understand the proposal, we spent
> some
> > > > > efforts
> > > > > > on
> > > > > > > > making an online POC
> > > > > > > >
> > > > > > > > Timeline Attempt (click the vertex timeline to see the
> > > > differences):
> > > > > > > > previous web:
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > > > POC web:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > > >
> > > > > > > > Subtask Attempt (click the vertex and switch to subtask tab
> to
> > > see
> > > > > the
> > > > > > > > differences):
> > > > > > > > previous web:
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > > > POC web:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > > >
> > > > > > > >
> > > > > > > > The vote will last for at least 72 hours, following the
> > consensus
> > > > > > voting
> > > > > > > > process.
> > > > > > > >
> > > > > > > > FLIP wiki:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > > > > >
> > > > > > > > Discussion thread:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Yadong
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > >
> > > > > Benchao Li
> > > > > School of Electronics Engineering and Computer Science, Peking
> > > University
> > > > > Tel:+86-15650713730
> > > > > Email: libenchao@gmail.com; libenchao@pku.edu.cn
> > > > >
> > > >
> > >
> >
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Yadong Xie <vt...@gmail.com>.
Hi Till
We keep response a flattened SubtaskTimeInfo and an array of
SubtaskTimeInfo to keep the restAPI backward compatible, since Flink users
may still need the API('/jobs/{jobId}/vertices/{vertexId}/subtasktimes') to
get the flattened SubtaskTimeInfo.
If we want to get an array of SubtaskTimeInfo in the API, a new URL needs
to be created other than reuse the old one.
Both solutions are ok for me. What do you think about it?
Till Rohrmann <tr...@apache.org> 于2020年2月26日周三 下午10:53写道：
> Fair enough. If this should become a problem we could introduce it later as
> well.
>
> What about changing the SubtasksTimeInfo response type into an array of
> SubtaskTimeInfo? At the moment SubtasksTimeInfo contains a
> flattened SubtaskTimeInfo and and array of SubtaskTimeInfo for the previous
> attempts.
>
> Cheers,
> Till
>
> On Wed, Feb 26, 2020 at 1:16 PM Yadong Xie <vt...@gmail.com> wrote:
>
> > Hi Till
> >
> > Thanks for your comments.
> >
> > > I have a comment concerning the SubtasksTimesHandler....
> >
> > It would be much easier for the frontend to handle a large amount of data
> > if we have a rest API parameter filter, but in my opinion, the attempt
> list
> > data is not large enough that we have to rely on the rest API parameters
> > paging, we still can handle them all in the frontend.
> >
> > Users can filter the attempt list by the
> status(scheduled/created/deploying
> > and so on) and other keywords(attempt_id and so on) directly in the
> > frontend since all data are listed from the rest API.
> > If we move some of the filter parameters to the rest API path parameter,
> > all the other filter parameters need to be moved too.
> >
> > I suggest adding an attempt id filter in the UI to help users filter the
> > desired attempt, and all the filtering process is running inside the
> > browser side, what do you think about this?
> >
> >
> >
> >
> > Till Rohrmann <tr...@apache.org> 于2020年2月25日周二 下午11:40写道：
> >
> > > Hi Yadong,
> > >
> > > thanks for creating this FLIP. I like the idea to make the web-ui
> > > information richer wrt to subtask attempt information.
> > >
> > > I have a comment concerning the SubtasksTimesHandler: Should we change
> > the
> > > response type SubtasksTimeInfo so that it simply contains an
> > > array of SubtaskTimeInfo? One could add an attempt range path parameter
> > to
> > > the SubtasksTimesHandler to be able to control which attempts will be
> > > returned.
> > >
> > > Cheers,
> > > Till
> > >
> > > On Tue, Feb 25, 2020 at 9:57 AM Benchao Li <li...@gmail.com>
> wrote:
> > >
> > > > Hi Yadong,
> > > >
> > > > Thanks for the updating. LGTM now.
> > > >
> > > > +1 (non-binding)
> > > >
> > > > Yadong Xie <vt...@gmail.com> 于2020年2月25日周二 下午4:41写道：
> > > >
> > > > > Hi Kurt
> > > > >
> > > > > There will be no differences between batch jobs and stream jobs in
> > > > > subtask-attempt level in the UI
> > > > > The only differences are in the vertex timeline, I have added a
> > > > screenshot
> > > > > of the batch job in the FLIP-100 since the batch job will disappear
> > > from
> > > > > the list after it finished soon.
> > > > > here is the link:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > >
> > > > >
> > > > > Kurt Young <yk...@gmail.com> 于2020年2月21日周五 上午11:51写道：
> > > > >
> > > > > > Hi Yadong,
> > > > > >
> > > > > > Thanks for the proposal, it's a useful feature, especially for
> > batch
> > > > > jobs.
> > > > > > But according
> > > > > > to the examples you gave, I can't tell whether i got required
> > > > information
> > > > > > from that.
> > > > > > Can you replace the demo job to a more complex batch job and then
> > we
> > > > can
> > > > > > see some
> > > > > > differences of start/stop time of different tasks and attempts?
> > > > > >
> > > > > > Best,
> > > > > > Kurt
> > > > > >
> > > > > >
> > > > > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <vt...@gmail.com>
> > > > wrote:
> > > > > >
> > > > > > > Hi all
> > > > > > >
> > > > > > > I want to start the vote for FLIP-100, which proposes to add
> > > attempt
> > > > > > > information inside subtask and timeline in web UI.
> > > > > > >
> > > > > > > To help everyone better understand the proposal, we spent some
> > > > efforts
> > > > > on
> > > > > > > making an online POC
> > > > > > >
> > > > > > > Timeline Attempt (click the vertex timeline to see the
> > > differences):
> > > > > > > previous web:
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > > POC web:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > >
> > > > > > > Subtask Attempt (click the vertex and switch to subtask tab to
> > see
> > > > the
> > > > > > > differences):
> > > > > > > previous web:
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > > POC web:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > >
> > > > > > >
> > > > > > > The vote will last for at least 72 hours, following the
> consensus
> > > > > voting
> > > > > > > process.
> > > > > > >
> > > > > > > FLIP wiki:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > > > >
> > > > > > > Discussion thread:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Yadong
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > > --
> > > >
> > > > Benchao Li
> > > > School of Electronics Engineering and Computer Science, Peking
> > University
> > > > Tel:+86-15650713730
> > > > Email: libenchao@gmail.com; libenchao@pku.edu.cn
> > > >
> > >
> >
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Till Rohrmann <tr...@apache.org>.
Fair enough. If this should become a problem we could introduce it later as
well.
What about changing the SubtasksTimeInfo response type into an array of
SubtaskTimeInfo? At the moment SubtasksTimeInfo contains a
flattened SubtaskTimeInfo and and array of SubtaskTimeInfo for the previous
attempts.
Cheers,
Till
On Wed, Feb 26, 2020 at 1:16 PM Yadong Xie <vt...@gmail.com> wrote:
> Hi Till
>
> Thanks for your comments.
>
> > I have a comment concerning the SubtasksTimesHandler....
>
> It would be much easier for the frontend to handle a large amount of data
> if we have a rest API parameter filter, but in my opinion, the attempt list
> data is not large enough that we have to rely on the rest API parameters
> paging, we still can handle them all in the frontend.
>
> Users can filter the attempt list by the status(scheduled/created/deploying
> and so on) and other keywords(attempt_id and so on) directly in the
> frontend since all data are listed from the rest API.
> If we move some of the filter parameters to the rest API path parameter,
> all the other filter parameters need to be moved too.
>
> I suggest adding an attempt id filter in the UI to help users filter the
> desired attempt, and all the filtering process is running inside the
> browser side, what do you think about this?
>
>
>
>
> Till Rohrmann <tr...@apache.org> 于2020年2月25日周二 下午11:40写道：
>
> > Hi Yadong,
> >
> > thanks for creating this FLIP. I like the idea to make the web-ui
> > information richer wrt to subtask attempt information.
> >
> > I have a comment concerning the SubtasksTimesHandler: Should we change
> the
> > response type SubtasksTimeInfo so that it simply contains an
> > array of SubtaskTimeInfo? One could add an attempt range path parameter
> to
> > the SubtasksTimesHandler to be able to control which attempts will be
> > returned.
> >
> > Cheers,
> > Till
> >
> > On Tue, Feb 25, 2020 at 9:57 AM Benchao Li <li...@gmail.com> wrote:
> >
> > > Hi Yadong,
> > >
> > > Thanks for the updating. LGTM now.
> > >
> > > +1 (non-binding)
> > >
> > > Yadong Xie <vt...@gmail.com> 于2020年2月25日周二 下午4:41写道：
> > >
> > > > Hi Kurt
> > > >
> > > > There will be no differences between batch jobs and stream jobs in
> > > > subtask-attempt level in the UI
> > > > The only differences are in the vertex timeline, I have added a
> > > screenshot
> > > > of the batch job in the FLIP-100 since the batch job will disappear
> > from
> > > > the list after it finished soon.
> > > > here is the link:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > >
> > > >
> > > > Kurt Young <yk...@gmail.com> 于2020年2月21日周五 上午11:51写道：
> > > >
> > > > > Hi Yadong,
> > > > >
> > > > > Thanks for the proposal, it's a useful feature, especially for
> batch
> > > > jobs.
> > > > > But according
> > > > > to the examples you gave, I can't tell whether i got required
> > > information
> > > > > from that.
> > > > > Can you replace the demo job to a more complex batch job and then
> we
> > > can
> > > > > see some
> > > > > differences of start/stop time of different tasks and attempts?
> > > > >
> > > > > Best,
> > > > > Kurt
> > > > >
> > > > >
> > > > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <vt...@gmail.com>
> > > wrote:
> > > > >
> > > > > > Hi all
> > > > > >
> > > > > > I want to start the vote for FLIP-100, which proposes to add
> > attempt
> > > > > > information inside subtask and timeline in web UI.
> > > > > >
> > > > > > To help everyone better understand the proposal, we spent some
> > > efforts
> > > > on
> > > > > > making an online POC
> > > > > >
> > > > > > Timeline Attempt (click the vertex timeline to see the
> > differences):
> > > > > > previous web:
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > > POC web:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > >
> > > > > > Subtask Attempt (click the vertex and switch to subtask tab to
> see
> > > the
> > > > > > differences):
> > > > > > previous web:
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > > POC web:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > >
> > > > > >
> > > > > > The vote will last for at least 72 hours, following the consensus
> > > > voting
> > > > > > process.
> > > > > >
> > > > > > FLIP wiki:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > > >
> > > > > > Discussion thread:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Yadong
> > > > > >
> > > > >
> > > >
> > >
> > >
> > > --
> > >
> > > Benchao Li
> > > School of Electronics Engineering and Computer Science, Peking
> University
> > > Tel:+86-15650713730
> > > Email: libenchao@gmail.com; libenchao@pku.edu.cn
> > >
> >
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Yadong Xie <vt...@gmail.com>.
Hi Till
Thanks for your comments.
> I have a comment concerning the SubtasksTimesHandler....
It would be much easier for the frontend to handle a large amount of data
if we have a rest API parameter filter, but in my opinion, the attempt list
data is not large enough that we have to rely on the rest API parameters
paging, we still can handle them all in the frontend.
Users can filter the attempt list by the status(scheduled/created/deploying
and so on) and other keywords(attempt_id and so on) directly in the
frontend since all data are listed from the rest API.
If we move some of the filter parameters to the rest API path parameter,
all the other filter parameters need to be moved too.
I suggest adding an attempt id filter in the UI to help users filter the
desired attempt, and all the filtering process is running inside the
browser side, what do you think about this?
Till Rohrmann <tr...@apache.org> 于2020年2月25日周二 下午11:40写道：
> Hi Yadong,
>
> thanks for creating this FLIP. I like the idea to make the web-ui
> information richer wrt to subtask attempt information.
>
> I have a comment concerning the SubtasksTimesHandler: Should we change the
> response type SubtasksTimeInfo so that it simply contains an
> array of SubtaskTimeInfo? One could add an attempt range path parameter to
> the SubtasksTimesHandler to be able to control which attempts will be
> returned.
>
> Cheers,
> Till
>
> On Tue, Feb 25, 2020 at 9:57 AM Benchao Li <li...@gmail.com> wrote:
>
> > Hi Yadong,
> >
> > Thanks for the updating. LGTM now.
> >
> > +1 (non-binding)
> >
> > Yadong Xie <vt...@gmail.com> 于2020年2月25日周二 下午4:41写道：
> >
> > > Hi Kurt
> > >
> > > There will be no differences between batch jobs and stream jobs in
> > > subtask-attempt level in the UI
> > > The only differences are in the vertex timeline, I have added a
> > screenshot
> > > of the batch job in the FLIP-100 since the batch job will disappear
> from
> > > the list after it finished soon.
> > > here is the link:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > >
> > >
> > > Kurt Young <yk...@gmail.com> 于2020年2月21日周五 上午11:51写道：
> > >
> > > > Hi Yadong,
> > > >
> > > > Thanks for the proposal, it's a useful feature, especially for batch
> > > jobs.
> > > > But according
> > > > to the examples you gave, I can't tell whether i got required
> > information
> > > > from that.
> > > > Can you replace the demo job to a more complex batch job and then we
> > can
> > > > see some
> > > > differences of start/stop time of different tasks and attempts?
> > > >
> > > > Best,
> > > > Kurt
> > > >
> > > >
> > > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <vt...@gmail.com>
> > wrote:
> > > >
> > > > > Hi all
> > > > >
> > > > > I want to start the vote for FLIP-100, which proposes to add
> attempt
> > > > > information inside subtask and timeline in web UI.
> > > > >
> > > > > To help everyone better understand the proposal, we spent some
> > efforts
> > > on
> > > > > making an online POC
> > > > >
> > > > > Timeline Attempt (click the vertex timeline to see the
> differences):
> > > > > previous web:
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > > POC web:
> > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > >
> > > > > Subtask Attempt (click the vertex and switch to subtask tab to see
> > the
> > > > > differences):
> > > > > previous web:
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > > POC web:
> > > > >
> > > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > > >
> > > > >
> > > > > The vote will last for at least 72 hours, following the consensus
> > > voting
> > > > > process.
> > > > >
> > > > > FLIP wiki:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > > >
> > > > > Discussion thread:
> > > > >
> > > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Yadong
> > > > >
> > > >
> > >
> >
> >
> > --
> >
> > Benchao Li
> > School of Electronics Engineering and Computer Science, Peking University
> > Tel:+86-15650713730
> > Email: libenchao@gmail.com; libenchao@pku.edu.cn
> >
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Till Rohrmann <tr...@apache.org>.
Hi Yadong,
thanks for creating this FLIP. I like the idea to make the web-ui
information richer wrt to subtask attempt information.
I have a comment concerning the SubtasksTimesHandler: Should we change the
response type SubtasksTimeInfo so that it simply contains an
array of SubtaskTimeInfo? One could add an attempt range path parameter to
the SubtasksTimesHandler to be able to control which attempts will be
returned.
Cheers,
Till
On Tue, Feb 25, 2020 at 9:57 AM Benchao Li <li...@gmail.com> wrote:
> Hi Yadong,
>
> Thanks for the updating. LGTM now.
>
> +1 (non-binding)
>
> Yadong Xie <vt...@gmail.com> 于2020年2月25日周二 下午4:41写道：
>
> > Hi Kurt
> >
> > There will be no differences between batch jobs and stream jobs in
> > subtask-attempt level in the UI
> > The only differences are in the vertex timeline, I have added a
> screenshot
> > of the batch job in the FLIP-100 since the batch job will disappear from
> > the list after it finished soon.
> > here is the link:
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> >
> >
> > Kurt Young <yk...@gmail.com> 于2020年2月21日周五 上午11:51写道：
> >
> > > Hi Yadong,
> > >
> > > Thanks for the proposal, it's a useful feature, especially for batch
> > jobs.
> > > But according
> > > to the examples you gave, I can't tell whether i got required
> information
> > > from that.
> > > Can you replace the demo job to a more complex batch job and then we
> can
> > > see some
> > > differences of start/stop time of different tasks and attempts?
> > >
> > > Best,
> > > Kurt
> > >
> > >
> > > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <vt...@gmail.com>
> wrote:
> > >
> > > > Hi all
> > > >
> > > > I want to start the vote for FLIP-100, which proposes to add attempt
> > > > information inside subtask and timeline in web UI.
> > > >
> > > > To help everyone better understand the proposal, we spent some
> efforts
> > on
> > > > making an online POC
> > > >
> > > > Timeline Attempt (click the vertex timeline to see the differences):
> > > > previous web:
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > > POC web:
> > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > > >
> > > > Subtask Attempt (click the vertex and switch to subtask tab to see
> the
> > > > differences):
> > > > previous web:
> > > >
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > > POC web:
> > > >
> > > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > > >
> > > >
> > > > The vote will last for at least 72 hours, following the consensus
> > voting
> > > > process.
> > > >
> > > > FLIP wiki:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > > >
> > > > Discussion thread:
> > > >
> > > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > > >
> > > > Thanks,
> > > >
> > > > Yadong
> > > >
> > >
> >
>
>
> --
>
> Benchao Li
> School of Electronics Engineering and Computer Science, Peking University
> Tel:+86-15650713730
> Email: libenchao@gmail.com; libenchao@pku.edu.cn
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Benchao Li <li...@gmail.com>.
Hi Yadong,
Thanks for the updating. LGTM now.
+1 (non-binding)
Yadong Xie <vt...@gmail.com> 于2020年2月25日周二 下午4:41写道：
> Hi Kurt
>
> There will be no differences between batch jobs and stream jobs in
> subtask-attempt level in the UI
> The only differences are in the vertex timeline, I have added a screenshot
> of the batch job in the FLIP-100 since the batch job will disappear from
> the list after it finished soon.
> here is the link:
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>
>
> Kurt Young <yk...@gmail.com> 于2020年2月21日周五 上午11:51写道：
>
> > Hi Yadong,
> >
> > Thanks for the proposal, it's a useful feature, especially for batch
> jobs.
> > But according
> > to the examples you gave, I can't tell whether i got required information
> > from that.
> > Can you replace the demo job to a more complex batch job and then we can
> > see some
> > differences of start/stop time of different tasks and attempts?
> >
> > Best,
> > Kurt
> >
> >
> > On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <vt...@gmail.com> wrote:
> >
> > > Hi all
> > >
> > > I want to start the vote for FLIP-100, which proposes to add attempt
> > > information inside subtask and timeline in web UI.
> > >
> > > To help everyone better understand the proposal, we spent some efforts
> on
> > > making an online POC
> > >
> > > Timeline Attempt (click the vertex timeline to see the differences):
> > > previous web:
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > > POC web:
> > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> > >
> > > Subtask Attempt (click the vertex and switch to subtask tab to see the
> > > differences):
> > > previous web:
> > >
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > > POC web:
> > >
> > >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> > >
> > >
> > > The vote will last for at least 72 hours, following the consensus
> voting
> > > process.
> > >
> > > FLIP wiki:
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> > >
> > > Discussion thread:
> > >
> > >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> > >
> > > Thanks,
> > >
> > > Yadong
> > >
> >
>
--
Benchao Li
School of Electronics Engineering and Computer Science, Peking University
Tel:+86-15650713730
Email: libenchao@gmail.com; libenchao@pku.edu.cn
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Yadong Xie <vt...@gmail.com>.
Hi Kurt
There will be no differences between batch jobs and stream jobs in
subtask-attempt level in the UI
The only differences are in the vertex timeline, I have added a screenshot
of the batch job in the FLIP-100 since the batch job will disappear from
the list after it finished soon.
here is the link:
https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
Kurt Young <yk...@gmail.com> 于2020年2月21日周五 上午11:51写道：
> Hi Yadong,
>
> Thanks for the proposal, it's a useful feature, especially for batch jobs.
> But according
> to the examples you gave, I can't tell whether i got required information
> from that.
> Can you replace the demo job to a more complex batch job and then we can
> see some
> differences of start/stop time of different tasks and attempts?
>
> Best,
> Kurt
>
>
> On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <vt...@gmail.com> wrote:
>
> > Hi all
> >
> > I want to start the vote for FLIP-100, which proposes to add attempt
> > information inside subtask and timeline in web UI.
> >
> > To help everyone better understand the proposal, we spent some efforts on
> > making an online POC
> >
> > Timeline Attempt (click the vertex timeline to see the differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > POC web:
> >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> >
> > Subtask Attempt (click the vertex and switch to subtask tab to see the
> > differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > POC web:
> >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> >
> >
> > The vote will last for at least 72 hours, following the consensus voting
> > process.
> >
> > FLIP wiki:
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> >
> > Discussion thread:
> >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> >
> > Thanks,
> >
> > Yadong
> >
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Kurt Young <yk...@gmail.com>.
Hi Yadong,
Thanks for the proposal, it's a useful feature, especially for batch jobs.
But according
to the examples you gave, I can't tell whether i got required information
from that.
Can you replace the demo job to a more complex batch job and then we can
see some
differences of start/stop time of different tasks and attempts?
Best,
Kurt
On Thu, Feb 20, 2020 at 5:46 PM Yadong Xie <vt...@gmail.com> wrote:
> Hi all
>
> I want to start the vote for FLIP-100, which proposes to add attempt
> information inside subtask and timeline in web UI.
>
> To help everyone better understand the proposal, we spent some efforts on
> making an online POC
>
> Timeline Attempt (click the vertex timeline to see the differences):
> previous web:
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> POC web:
>
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
>
> Subtask Attempt (click the vertex and switch to subtask tab to see the
> differences):
> previous web:
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> POC web:
>
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
>
>
> The vote will last for at least 72 hours, following the consensus voting
> process.
>
> FLIP wiki:
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>
> Discussion thread:
>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
>
> Thanks,
>
> Yadong
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Yadong Xie <vt...@gmail.com>.
> * Why are we duplicating the subtask index in the objects that are
stored in the attempts-time-info array? I thought that all objects in the
same array share the same subtask index.
yes, I agree with you that subtask could be removed in the
`attempts-time-info`
> * Are we confident that the attempts-time-info array does not grow too
large during the lifetime of a job? Should the size of the array be limited?
no, I think it is a similar situation to taskmanager/jobmanager log API,
all data response from the rest API could grow too large, but add query
params in the RestAPI would also bring pressure to the server.
> * Have we considered placing the historic attempts in the same array as
the current attempts, i.e., flatten the arrays? One could toggle the
historic attempts on and off with a query parameter.
it is a good idea considering the previous situation, this would be useful
not blocking the previous feature even if the history becomes too large, we
will implement this and update the POC later.
> * I think 'attempt-history' would be a better name instead of
'attempts-time-info'.
Yes, I agree with you, we will update it later.
Gary Yao <ga...@apache.org> 于2020年3月2日周一 下午10:50写道：
> Hi Yadong,
>
> Thanks for driving this FLIP. I have a few questions/remarks:
>
> * Why are we duplicating the subtask index in the objects that are
> stored in the attempts-time-info array? I thought that all objects in the
> same array share the same subtask index.
> * Are we confident that the attempts-time-info array does not grow too
> large during the lifetime of a job? Should the size of the array be
> limited?
> * Have we considered placing the historic attempts in the same array as
> the current attempts, i.e., flatten the arrays? One could toggle the
> historic attempts on and off with a query parameter.
> * I think 'attempt-history' would be a better name instead of
> 'attempts-time-info'.
>
> Let me know what you think.
>
> Best,
> Gary
>
> On Thu, Feb 20, 2020 at 10:46 AM Yadong Xie <vt...@gmail.com> wrote:
>
> > Hi all
> >
> > I want to start the vote for FLIP-100, which proposes to add attempt
> > information inside subtask and timeline in web UI.
> >
> > To help everyone better understand the proposal, we spent some efforts on
> > making an online POC
> >
> > Timeline Attempt (click the vertex timeline to see the differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> > POC web:
> >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
> >
> > Subtask Attempt (click the vertex and switch to subtask tab to see the
> > differences):
> > previous web:
> >
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> > POC web:
> >
> >
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
> >
> >
> > The vote will last for at least 72 hours, following the consensus voting
> > process.
> >
> > FLIP wiki:
> >
> >
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
> >
> > Discussion thread:
> >
> >
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
> >
> > Thanks,
> >
> > Yadong
> >
>
Re: [VOTE] FLIP-100: Add Attempt Information
Posted by Gary Yao <ga...@apache.org>.
Hi Yadong,
Thanks for driving this FLIP. I have a few questions/remarks:
* Why are we duplicating the subtask index in the objects that are
stored in the attempts-time-info array? I thought that all objects in the
same array share the same subtask index.
* Are we confident that the attempts-time-info array does not grow too
large during the lifetime of a job? Should the size of the array be limited?
* Have we considered placing the historic attempts in the same array as
the current attempts, i.e., flatten the arrays? One could toggle the
historic attempts on and off with a query parameter.
* I think 'attempt-history' would be a better name instead of
'attempts-time-info'.
Let me know what you think.
Best,
Gary
On Thu, Feb 20, 2020 at 10:46 AM Yadong Xie <vt...@gmail.com> wrote:
> Hi all
>
> I want to start the vote for FLIP-100, which proposes to add attempt
> information inside subtask and timeline in web UI.
>
> To help everyone better understand the proposal, we spent some efforts on
> making an online POC
>
> Timeline Attempt (click the vertex timeline to see the differences):
> previous web:
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/timeline
> POC web:
>
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/timeline
>
> Subtask Attempt (click the vertex and switch to subtask tab to see the
> differences):
> previous web:
> http://101.132.122.69:8081/#/job/9d651769488466d33e7a607e85203543/overview
> POC web:
>
> http://101.132.122.69:8081/web/#/job/9d651769488466d33e7a607e85203543/overview
>
>
> The vote will last for at least 72 hours, following the consensus voting
> process.
>
> FLIP wiki:
>
> https://cwiki.apache.org/confluence/display/FLINK/FLIP-100%3A+Add+Attempt+Information
>
> Discussion thread:
>
> http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-75-Flink-Web-UI-Improvement-Proposal-td33540.html
>
> Thanks,
>
> Yadong
>