You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Sangjin Lee <sj...@apache.org> on 2016/07/01 16:00:31 UTC
[VOTE] merge feature branch YARN-2928
Hi folks,
Per earlier discussion [1], I'd like to start a formal vote to merge
feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The vote will
run for 7 days, and will end July 8 9:00 AM PDT.
We have been developing the feature in a feature branch (YARN-2928 [3]) for
a while, and we are reasonably confident that the state of the feature
meets the criteria to be merged onto trunk and we'd love folks to get their
hands on it in a test capacity and provide valuable feedback so that we can
make it production-ready.
In a nutshell, Timeline Service v.2 delivers significant scalability and
usability improvements based on a new architecture. You can browse the
requirements/design doc, the storage schema doc, the new entity/data model,
the YARN documentation, and also discussions on subsequent milestones on
YARN-2928 [2].
What we would like to merge to trunk is termed "alpha 1" (milestone 1). The
feature has a complete end-to-end read/write flow, and you should be able
to start setting it up and testing it. At a high level, the following are
the key features that have been implemented:
- distributed writers (collectors) as NM aux services
- HBase storage
- new entity model that includes flows
- setting the flow context via YARN app tags
- real time metrics aggregation to the application level and the flow level
- rich REST API that supports filters, complex conditionals, limits,
content selection, etc.
- YARN generic events and system metrics
- integration with Distributed Shell and MapReduce
There are a total of 139 subtasks that were completed as part of this
effort.
We paid close attention to ensure that once disabled Timeline Service v.2
does not impact existing functionality when disabled (by default).
Special thanks go to all the folks who worked real hard on this feature: Li
Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena, Vrushali
C, and Joep Rottinghuis!
Regards,
Sangjin
[1]
https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
[2] https://issues.apache.org/jira/browse/YARN-2928
[3] https://github.com/apache/hadoop/commits/YARN-2928
Re: [VOTE] merge feature branch YARN-2928
Posted by 俊平堵 <ju...@apache.org>.
+1 (binding). Thanks for pushing it out, Sangjin!
Thanks,
Junping
2016-07-01 17:00 GMT+01:00 Sangjin Lee <sj...@apache.org>:
> Hi folks,
>
> Per earlier discussion [1], I'd like to start a formal vote to merge
> feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The vote will
> run for 7 days, and will end July 8 9:00 AM PDT.
>
> We have been developing the feature in a feature branch (YARN-2928 [3]) for
> a while, and we are reasonably confident that the state of the feature
> meets the criteria to be merged onto trunk and we'd love folks to get their
> hands on it in a test capacity and provide valuable feedback so that we can
> make it production-ready.
>
> In a nutshell, Timeline Service v.2 delivers significant scalability and
> usability improvements based on a new architecture. You can browse the
> requirements/design doc, the storage schema doc, the new entity/data model,
> the YARN documentation, and also discussions on subsequent milestones on
> YARN-2928 [2].
>
> What we would like to merge to trunk is termed "alpha 1" (milestone 1). The
> feature has a complete end-to-end read/write flow, and you should be able
> to start setting it up and testing it. At a high level, the following are
> the key features that have been implemented:
>
> - distributed writers (collectors) as NM aux services
> - HBase storage
> - new entity model that includes flows
> - setting the flow context via YARN app tags
> - real time metrics aggregation to the application level and the flow level
> - rich REST API that supports filters, complex conditionals, limits,
> content selection, etc.
> - YARN generic events and system metrics
> - integration with Distributed Shell and MapReduce
>
> There are a total of 139 subtasks that were completed as part of this
> effort.
>
> We paid close attention to ensure that once disabled Timeline Service v.2
> does not impact existing functionality when disabled (by default).
>
> Special thanks go to all the folks who worked real hard on this feature: Li
> Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena, Vrushali
> C, and Joep Rottinghuis!
>
> Regards,
> Sangjin
>
> [1]
>
> https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
> [2] https://issues.apache.org/jira/browse/YARN-2928
> [3] https://github.com/apache/hadoop/commits/YARN-2928
>
Re: [VOTE] merge feature branch YARN-2928
Posted by Akira Ajisaka <aa...@apache.org>.
+0
I'm in favor of merging this because the distributed writers/collectors
enables high-availability.
However, I haven't looked at the code or the tests. Therefore I don't
think I can +1 for the vote.
-Akira
On 7/7/16 09:22, Jason Lowe wrote:
> +1 (binding)
> Looking forward to seeing this move forward and mature! I especially appreciate the effort made to ensure it minimizes the impact to users who are not enabling the feature. Thank you to everyone involved!
> Jason
>
> From: Sangjin Lee <sj...@apache.org>
> To: "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>
> Sent: Friday, July 1, 2016 11:00 AM
> Subject: [VOTE] merge feature branch YARN-2928
>
> Hi folks,
>
> Per earlier discussion [1], I'd like to start a formal vote to merge
> feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The vote will
> run for 7 days, and will end July 8 9:00 AM PDT.
>
> We have been developing the feature in a feature branch (YARN-2928 [3]) for
> a while, and we are reasonably confident that the state of the feature
> meets the criteria to be merged onto trunk and we'd love folks to get their
> hands on it in a test capacity and provide valuable feedback so that we can
> make it production-ready.
>
> In a nutshell, Timeline Service v.2 delivers significant scalability and
> usability improvements based on a new architecture. You can browse the
> requirements/design doc, the storage schema doc, the new entity/data model,
> the YARN documentation, and also discussions on subsequent milestones on
> YARN-2928 [2].
>
> What we would like to merge to trunk is termed "alpha 1" (milestone 1). The
> feature has a complete end-to-end read/write flow, and you should be able
> to start setting it up and testing it. At a high level, the following are
> the key features that have been implemented:
>
> - distributed writers (collectors) as NM aux services
> - HBase storage
> - new entity model that includes flows
> - setting the flow context via YARN app tags
> - real time metrics aggregation to the application level and the flow level
> - rich REST API that supports filters, complex conditionals, limits,
> content selection, etc.
> - YARN generic events and system metrics
> - integration with Distributed Shell and MapReduce
>
> There are a total of 139 subtasks that were completed as part of this
> effort.
>
> We paid close attention to ensure that once disabled Timeline Service v.2
> does not impact existing functionality when disabled (by default).
>
> Special thanks go to all the folks who worked real hard on this feature: Li
> Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena, Vrushali
> C, and Joep Rottinghuis!
>
> Regards,
> Sangjin
>
> [1]
> https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
> [2] https://issues.apache.org/jira/browse/YARN-2928
> [3] https://github.com/apache/hadoop/commits/YARN-2928
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] merge feature branch YARN-2928
Posted by Jason Lowe <jl...@yahoo-inc.com.INVALID>.
+1 (binding)
Looking forward to seeing this move forward and mature! I especially appreciate the effort made to ensure it minimizes the impact to users who are not enabling the feature. Thank you to everyone involved!
Jason
From: Sangjin Lee <sj...@apache.org>
To: "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>
Sent: Friday, July 1, 2016 11:00 AM
Subject: [VOTE] merge feature branch YARN-2928
Hi folks,
Per earlier discussion [1], I'd like to start a formal vote to merge
feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The vote will
run for 7 days, and will end July 8 9:00 AM PDT.
We have been developing the feature in a feature branch (YARN-2928 [3]) for
a while, and we are reasonably confident that the state of the feature
meets the criteria to be merged onto trunk and we'd love folks to get their
hands on it in a test capacity and provide valuable feedback so that we can
make it production-ready.
In a nutshell, Timeline Service v.2 delivers significant scalability and
usability improvements based on a new architecture. You can browse the
requirements/design doc, the storage schema doc, the new entity/data model,
the YARN documentation, and also discussions on subsequent milestones on
YARN-2928 [2].
What we would like to merge to trunk is termed "alpha 1" (milestone 1). The
feature has a complete end-to-end read/write flow, and you should be able
to start setting it up and testing it. At a high level, the following are
the key features that have been implemented:
- distributed writers (collectors) as NM aux services
- HBase storage
- new entity model that includes flows
- setting the flow context via YARN app tags
- real time metrics aggregation to the application level and the flow level
- rich REST API that supports filters, complex conditionals, limits,
content selection, etc.
- YARN generic events and system metrics
- integration with Distributed Shell and MapReduce
There are a total of 139 subtasks that were completed as part of this
effort.
We paid close attention to ensure that once disabled Timeline Service v.2
does not impact existing functionality when disabled (by default).
Special thanks go to all the folks who worked real hard on this feature: Li
Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena, Vrushali
C, and Joep Rottinghuis!
Regards,
Sangjin
[1]
https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
[2] https://issues.apache.org/jira/browse/YARN-2928
[3] https://github.com/apache/hadoop/commits/YARN-2928
Re: [VOTE] merge feature branch YARN-2928
Posted by Zhihai Xu <zh...@uber.com>.
+1(binding)
thanks
zhihai
On Mon, Jul 4, 2016 at 7:46 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
> +0
>
> I'm in favor of merging this, especially in time for Hadoop 3 alpha.
>
> However, I haven't looked at the code or tests. So, don't feel qualified
> to vote +1.
>
> Mobile
>
> > On Jul 2, 2016, at 10:17 AM, Arun Suresh <ar...@gmail.com> wrote:
> >
> > +1(binding)
> > Really looking forward to this in trunk..
> >
> > -Arun
> >
> >> On Sat, Jul 2, 2016, 10:15 AM Varun Saxena <vs...@gmail.com>
> wrote:
> >>
> >> +1(binding)
> >>
> >>> On Sat, Jul 2, 2016 at 10:35 PM, Tsuyoshi Ozawa <oz...@apache.org>
> wrote:
> >>>
> >>> +1(binding)
> >>>
> >>> On Sat, Jul 2, 2016 at 9:41 AM, Joep Rottinghuis <
> jrottinghuis@gmail.com
> >>>
> >>> wrote:
> >>>> Perhaps superfluous as a contributor to this branch, but for the
> >> record:
> >>>> +1 (non-binding)
> >>>>
> >>>> Cheers,
> >>>>
> >>>> Joep
> >>>>
> >>>>> On Jul 1, 2016, at 9:00 AM, Sangjin Lee <sj...@apache.org> wrote:
> >>>>>
> >>>>> Hi folks,
> >>>>>
> >>>>> Per earlier discussion [1], I'd like to start a formal vote to merge
> >>>>> feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The
> vote
> >>> will
> >>>>> run for 7 days, and will end July 8 9:00 AM PDT.
> >>>>>
> >>>>> We have been developing the feature in a feature branch (YARN-2928
> >> [3])
> >>> for
> >>>>> a while, and we are reasonably confident that the state of the
> feature
> >>>>> meets the criteria to be merged onto trunk and we'd love folks to get
> >>> their
> >>>>> hands on it in a test capacity and provide valuable feedback so that
> >> we
> >>> can
> >>>>> make it production-ready.
> >>>>>
> >>>>> In a nutshell, Timeline Service v.2 delivers significant scalability
> >> and
> >>>>> usability improvements based on a new architecture. You can browse
> the
> >>>>> requirements/design doc, the storage schema doc, the new entity/data
> >>> model,
> >>>>> the YARN documentation, and also discussions on subsequent milestones
> >> on
> >>>>> YARN-2928 [2].
> >>>>>
> >>>>> What we would like to merge to trunk is termed "alpha 1" (milestone
> >> 1).
> >>> The
> >>>>> feature has a complete end-to-end read/write flow, and you should be
> >>> able
> >>>>> to start setting it up and testing it. At a high level, the following
> >>> are
> >>>>> the key features that have been implemented:
> >>>>>
> >>>>> - distributed writers (collectors) as NM aux services
> >>>>> - HBase storage
> >>>>> - new entity model that includes flows
> >>>>> - setting the flow context via YARN app tags
> >>>>> - real time metrics aggregation to the application level and the flow
> >>> level
> >>>>> - rich REST API that supports filters, complex conditionals, limits,
> >>>>> content selection, etc.
> >>>>> - YARN generic events and system metrics
> >>>>> - integration with Distributed Shell and MapReduce
> >>>>>
> >>>>> There are a total of 139 subtasks that were completed as part of this
> >>>>> effort.
> >>>>>
> >>>>> We paid close attention to ensure that once disabled Timeline Service
> >>> v.2
> >>>>> does not impact existing functionality when disabled (by default).
> >>>>>
> >>>>> Special thanks go to all the folks who worked real hard on this
> >>> feature: Li
> >>>>> Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena,
> >>> Vrushali
> >>>>> C, and Joep Rottinghuis!
> >>>>>
> >>>>> Regards,
> >>>>> Sangjin
> >>>>>
> >>>>> [1]
> >>
> https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
> >>>>> [2] https://issues.apache.org/jira/browse/YARN-2928
> >>>>> [3] https://github.com/apache/hadoop/commits/YARN-2928
> >>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> >>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> >>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
Re: [VOTE] merge feature branch YARN-2928
Posted by Karthik Kambatla <ka...@cloudera.com>.
+0
I'm in favor of merging this, especially in time for Hadoop 3 alpha.
However, I haven't looked at the code or tests. So, don't feel qualified to vote +1.
Mobile
> On Jul 2, 2016, at 10:17 AM, Arun Suresh <ar...@gmail.com> wrote:
>
> +1(binding)
> Really looking forward to this in trunk..
>
> -Arun
>
>> On Sat, Jul 2, 2016, 10:15 AM Varun Saxena <vs...@gmail.com> wrote:
>>
>> +1(binding)
>>
>>> On Sat, Jul 2, 2016 at 10:35 PM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>>>
>>> +1(binding)
>>>
>>> On Sat, Jul 2, 2016 at 9:41 AM, Joep Rottinghuis <jrottinghuis@gmail.com
>>>
>>> wrote:
>>>> Perhaps superfluous as a contributor to this branch, but for the
>> record:
>>>> +1 (non-binding)
>>>>
>>>> Cheers,
>>>>
>>>> Joep
>>>>
>>>>> On Jul 1, 2016, at 9:00 AM, Sangjin Lee <sj...@apache.org> wrote:
>>>>>
>>>>> Hi folks,
>>>>>
>>>>> Per earlier discussion [1], I'd like to start a formal vote to merge
>>>>> feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The vote
>>> will
>>>>> run for 7 days, and will end July 8 9:00 AM PDT.
>>>>>
>>>>> We have been developing the feature in a feature branch (YARN-2928
>> [3])
>>> for
>>>>> a while, and we are reasonably confident that the state of the feature
>>>>> meets the criteria to be merged onto trunk and we'd love folks to get
>>> their
>>>>> hands on it in a test capacity and provide valuable feedback so that
>> we
>>> can
>>>>> make it production-ready.
>>>>>
>>>>> In a nutshell, Timeline Service v.2 delivers significant scalability
>> and
>>>>> usability improvements based on a new architecture. You can browse the
>>>>> requirements/design doc, the storage schema doc, the new entity/data
>>> model,
>>>>> the YARN documentation, and also discussions on subsequent milestones
>> on
>>>>> YARN-2928 [2].
>>>>>
>>>>> What we would like to merge to trunk is termed "alpha 1" (milestone
>> 1).
>>> The
>>>>> feature has a complete end-to-end read/write flow, and you should be
>>> able
>>>>> to start setting it up and testing it. At a high level, the following
>>> are
>>>>> the key features that have been implemented:
>>>>>
>>>>> - distributed writers (collectors) as NM aux services
>>>>> - HBase storage
>>>>> - new entity model that includes flows
>>>>> - setting the flow context via YARN app tags
>>>>> - real time metrics aggregation to the application level and the flow
>>> level
>>>>> - rich REST API that supports filters, complex conditionals, limits,
>>>>> content selection, etc.
>>>>> - YARN generic events and system metrics
>>>>> - integration with Distributed Shell and MapReduce
>>>>>
>>>>> There are a total of 139 subtasks that were completed as part of this
>>>>> effort.
>>>>>
>>>>> We paid close attention to ensure that once disabled Timeline Service
>>> v.2
>>>>> does not impact existing functionality when disabled (by default).
>>>>>
>>>>> Special thanks go to all the folks who worked real hard on this
>>> feature: Li
>>>>> Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena,
>>> Vrushali
>>>>> C, and Joep Rottinghuis!
>>>>>
>>>>> Regards,
>>>>> Sangjin
>>>>>
>>>>> [1]
>> https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
>>>>> [2] https://issues.apache.org/jira/browse/YARN-2928
>>>>> [3] https://github.com/apache/hadoop/commits/YARN-2928
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] merge feature branch YARN-2928
Posted by Arun Suresh <ar...@gmail.com>.
+1(binding)
Really looking forward to this in trunk..
-Arun
On Sat, Jul 2, 2016, 10:15 AM Varun Saxena <vs...@gmail.com> wrote:
> +1(binding)
>
> On Sat, Jul 2, 2016 at 10:35 PM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
> > +1(binding)
> >
> > On Sat, Jul 2, 2016 at 9:41 AM, Joep Rottinghuis <jrottinghuis@gmail.com
> >
> > wrote:
> > > Perhaps superfluous as a contributor to this branch, but for the
> record:
> > > +1 (non-binding)
> > >
> > > Cheers,
> > >
> > > Joep
> > >
> > >> On Jul 1, 2016, at 9:00 AM, Sangjin Lee <sj...@apache.org> wrote:
> > >>
> > >> Hi folks,
> > >>
> > >> Per earlier discussion [1], I'd like to start a formal vote to merge
> > >> feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The vote
> > will
> > >> run for 7 days, and will end July 8 9:00 AM PDT.
> > >>
> > >> We have been developing the feature in a feature branch (YARN-2928
> [3])
> > for
> > >> a while, and we are reasonably confident that the state of the feature
> > >> meets the criteria to be merged onto trunk and we'd love folks to get
> > their
> > >> hands on it in a test capacity and provide valuable feedback so that
> we
> > can
> > >> make it production-ready.
> > >>
> > >> In a nutshell, Timeline Service v.2 delivers significant scalability
> and
> > >> usability improvements based on a new architecture. You can browse the
> > >> requirements/design doc, the storage schema doc, the new entity/data
> > model,
> > >> the YARN documentation, and also discussions on subsequent milestones
> on
> > >> YARN-2928 [2].
> > >>
> > >> What we would like to merge to trunk is termed "alpha 1" (milestone
> 1).
> > The
> > >> feature has a complete end-to-end read/write flow, and you should be
> > able
> > >> to start setting it up and testing it. At a high level, the following
> > are
> > >> the key features that have been implemented:
> > >>
> > >> - distributed writers (collectors) as NM aux services
> > >> - HBase storage
> > >> - new entity model that includes flows
> > >> - setting the flow context via YARN app tags
> > >> - real time metrics aggregation to the application level and the flow
> > level
> > >> - rich REST API that supports filters, complex conditionals, limits,
> > >> content selection, etc.
> > >> - YARN generic events and system metrics
> > >> - integration with Distributed Shell and MapReduce
> > >>
> > >> There are a total of 139 subtasks that were completed as part of this
> > >> effort.
> > >>
> > >> We paid close attention to ensure that once disabled Timeline Service
> > v.2
> > >> does not impact existing functionality when disabled (by default).
> > >>
> > >> Special thanks go to all the folks who worked real hard on this
> > feature: Li
> > >> Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena,
> > Vrushali
> > >> C, and Joep Rottinghuis!
> > >>
> > >> Regards,
> > >> Sangjin
> > >>
> > >> [1]
> > >>
> >
> https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
> > >> [2] https://issues.apache.org/jira/browse/YARN-2928
> > >> [3] https://github.com/apache/hadoop/commits/YARN-2928
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
> >
>
Re: [VOTE] merge feature branch YARN-2928
Posted by Varun Saxena <vs...@gmail.com>.
+1(binding)
On Sat, Jul 2, 2016 at 10:35 PM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> +1(binding)
>
> On Sat, Jul 2, 2016 at 9:41 AM, Joep Rottinghuis <jr...@gmail.com>
> wrote:
> > Perhaps superfluous as a contributor to this branch, but for the record:
> > +1 (non-binding)
> >
> > Cheers,
> >
> > Joep
> >
> >> On Jul 1, 2016, at 9:00 AM, Sangjin Lee <sj...@apache.org> wrote:
> >>
> >> Hi folks,
> >>
> >> Per earlier discussion [1], I'd like to start a formal vote to merge
> >> feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The vote
> will
> >> run for 7 days, and will end July 8 9:00 AM PDT.
> >>
> >> We have been developing the feature in a feature branch (YARN-2928 [3])
> for
> >> a while, and we are reasonably confident that the state of the feature
> >> meets the criteria to be merged onto trunk and we'd love folks to get
> their
> >> hands on it in a test capacity and provide valuable feedback so that we
> can
> >> make it production-ready.
> >>
> >> In a nutshell, Timeline Service v.2 delivers significant scalability and
> >> usability improvements based on a new architecture. You can browse the
> >> requirements/design doc, the storage schema doc, the new entity/data
> model,
> >> the YARN documentation, and also discussions on subsequent milestones on
> >> YARN-2928 [2].
> >>
> >> What we would like to merge to trunk is termed "alpha 1" (milestone 1).
> The
> >> feature has a complete end-to-end read/write flow, and you should be
> able
> >> to start setting it up and testing it. At a high level, the following
> are
> >> the key features that have been implemented:
> >>
> >> - distributed writers (collectors) as NM aux services
> >> - HBase storage
> >> - new entity model that includes flows
> >> - setting the flow context via YARN app tags
> >> - real time metrics aggregation to the application level and the flow
> level
> >> - rich REST API that supports filters, complex conditionals, limits,
> >> content selection, etc.
> >> - YARN generic events and system metrics
> >> - integration with Distributed Shell and MapReduce
> >>
> >> There are a total of 139 subtasks that were completed as part of this
> >> effort.
> >>
> >> We paid close attention to ensure that once disabled Timeline Service
> v.2
> >> does not impact existing functionality when disabled (by default).
> >>
> >> Special thanks go to all the folks who worked real hard on this
> feature: Li
> >> Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena,
> Vrushali
> >> C, and Joep Rottinghuis!
> >>
> >> Regards,
> >> Sangjin
> >>
> >> [1]
> >>
> https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
> >> [2] https://issues.apache.org/jira/browse/YARN-2928
> >> [3] https://github.com/apache/hadoop/commits/YARN-2928
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
Re: [VOTE] merge feature branch YARN-2928
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
+1(binding)
On Sat, Jul 2, 2016 at 9:41 AM, Joep Rottinghuis <jr...@gmail.com> wrote:
> Perhaps superfluous as a contributor to this branch, but for the record:
> +1 (non-binding)
>
> Cheers,
>
> Joep
>
>> On Jul 1, 2016, at 9:00 AM, Sangjin Lee <sj...@apache.org> wrote:
>>
>> Hi folks,
>>
>> Per earlier discussion [1], I'd like to start a formal vote to merge
>> feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The vote will
>> run for 7 days, and will end July 8 9:00 AM PDT.
>>
>> We have been developing the feature in a feature branch (YARN-2928 [3]) for
>> a while, and we are reasonably confident that the state of the feature
>> meets the criteria to be merged onto trunk and we'd love folks to get their
>> hands on it in a test capacity and provide valuable feedback so that we can
>> make it production-ready.
>>
>> In a nutshell, Timeline Service v.2 delivers significant scalability and
>> usability improvements based on a new architecture. You can browse the
>> requirements/design doc, the storage schema doc, the new entity/data model,
>> the YARN documentation, and also discussions on subsequent milestones on
>> YARN-2928 [2].
>>
>> What we would like to merge to trunk is termed "alpha 1" (milestone 1). The
>> feature has a complete end-to-end read/write flow, and you should be able
>> to start setting it up and testing it. At a high level, the following are
>> the key features that have been implemented:
>>
>> - distributed writers (collectors) as NM aux services
>> - HBase storage
>> - new entity model that includes flows
>> - setting the flow context via YARN app tags
>> - real time metrics aggregation to the application level and the flow level
>> - rich REST API that supports filters, complex conditionals, limits,
>> content selection, etc.
>> - YARN generic events and system metrics
>> - integration with Distributed Shell and MapReduce
>>
>> There are a total of 139 subtasks that were completed as part of this
>> effort.
>>
>> We paid close attention to ensure that once disabled Timeline Service v.2
>> does not impact existing functionality when disabled (by default).
>>
>> Special thanks go to all the folks who worked real hard on this feature: Li
>> Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena, Vrushali
>> C, and Joep Rottinghuis!
>>
>> Regards,
>> Sangjin
>>
>> [1]
>> https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
>> [2] https://issues.apache.org/jira/browse/YARN-2928
>> [3] https://github.com/apache/hadoop/commits/YARN-2928
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] merge feature branch YARN-2928
Posted by Joep Rottinghuis <jr...@gmail.com>.
Perhaps superfluous as a contributor to this branch, but for the record:
+1 (non-binding)
Cheers,
Joep
> On Jul 1, 2016, at 9:00 AM, Sangjin Lee <sj...@apache.org> wrote:
>
> Hi folks,
>
> Per earlier discussion [1], I'd like to start a formal vote to merge
> feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The vote will
> run for 7 days, and will end July 8 9:00 AM PDT.
>
> We have been developing the feature in a feature branch (YARN-2928 [3]) for
> a while, and we are reasonably confident that the state of the feature
> meets the criteria to be merged onto trunk and we'd love folks to get their
> hands on it in a test capacity and provide valuable feedback so that we can
> make it production-ready.
>
> In a nutshell, Timeline Service v.2 delivers significant scalability and
> usability improvements based on a new architecture. You can browse the
> requirements/design doc, the storage schema doc, the new entity/data model,
> the YARN documentation, and also discussions on subsequent milestones on
> YARN-2928 [2].
>
> What we would like to merge to trunk is termed "alpha 1" (milestone 1). The
> feature has a complete end-to-end read/write flow, and you should be able
> to start setting it up and testing it. At a high level, the following are
> the key features that have been implemented:
>
> - distributed writers (collectors) as NM aux services
> - HBase storage
> - new entity model that includes flows
> - setting the flow context via YARN app tags
> - real time metrics aggregation to the application level and the flow level
> - rich REST API that supports filters, complex conditionals, limits,
> content selection, etc.
> - YARN generic events and system metrics
> - integration with Distributed Shell and MapReduce
>
> There are a total of 139 subtasks that were completed as part of this
> effort.
>
> We paid close attention to ensure that once disabled Timeline Service v.2
> does not impact existing functionality when disabled (by default).
>
> Special thanks go to all the folks who worked real hard on this feature: Li
> Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena, Vrushali
> C, and Joep Rottinghuis!
>
> Regards,
> Sangjin
>
> [1]
> https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
> [2] https://issues.apache.org/jira/browse/YARN-2928
> [3] https://github.com/apache/hadoop/commits/YARN-2928
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] merge feature branch YARN-2928
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
+1 binding.
Thanks
+Vinod
> On Jul 1, 2016, at 9:00 AM, Sangjin Lee <sj...@apache.org> wrote:
>
> Hi folks,
>
> Per earlier discussion [1], I'd like to start a formal vote to merge
> feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The vote will
> run for 7 days, and will end July 8 9:00 AM PDT.
>
> We have been developing the feature in a feature branch (YARN-2928 [3]) for
> a while, and we are reasonably confident that the state of the feature
> meets the criteria to be merged onto trunk and we'd love folks to get their
> hands on it in a test capacity and provide valuable feedback so that we can
> make it production-ready.
>
> In a nutshell, Timeline Service v.2 delivers significant scalability and
> usability improvements based on a new architecture. You can browse the
> requirements/design doc, the storage schema doc, the new entity/data model,
> the YARN documentation, and also discussions on subsequent milestones on
> YARN-2928 [2].
>
> What we would like to merge to trunk is termed "alpha 1" (milestone 1). The
> feature has a complete end-to-end read/write flow, and you should be able
> to start setting it up and testing it. At a high level, the following are
> the key features that have been implemented:
>
> - distributed writers (collectors) as NM aux services
> - HBase storage
> - new entity model that includes flows
> - setting the flow context via YARN app tags
> - real time metrics aggregation to the application level and the flow level
> - rich REST API that supports filters, complex conditionals, limits,
> content selection, etc.
> - YARN generic events and system metrics
> - integration with Distributed Shell and MapReduce
>
> There are a total of 139 subtasks that were completed as part of this
> effort.
>
> We paid close attention to ensure that once disabled Timeline Service v.2
> does not impact existing functionality when disabled (by default).
>
> Special thanks go to all the folks who worked real hard on this feature: Li
> Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena, Vrushali
> C, and Joep Rottinghuis!
>
> Regards,
> Sangjin
>
> [1]
> https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
> [2] https://issues.apache.org/jira/browse/YARN-2928
> [3] https://github.com/apache/hadoop/commits/YARN-2928
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [VOTE] merge feature branch YARN-2928
Posted by Sangjin Lee <sj...@apache.org>.
I'm adding my +1 (binding) to conclude the vote.
With 13 +1's (11 binding), 3 +0's, 1 -0, and no -1's, the vote passes. I'll
get on merging the branch to trunk soon. Thanks everyone!
Regards,
Sangjin
On Thu, Jul 7, 2016 at 6:33 PM, Subramaniam V K <su...@gmail.com> wrote:
> +0 for same reasons as Karthik/Akira. I skimmed through the API patches as
> I was trying to make sense of integration with Federation (YARN-2915) and
> they seemed reasonable.
>
> Generally excited to see this in YARN.
>
> Also the contributors did a good job of keeping yarn-dev@ in the loop with
> regular updates and early heads up for various milestones during branch
> development
>
> -Subru
>
> On Thu, Jul 7, 2016 at 11:56 AM, Vrushali Channapattan <
> vrushali.c@gmail.com
> > wrote:
>
> > +1 (non-binding)
> >
> > Has been exciting working on this and am looking forward to the merge to
> > trunk.
> >
> > On Fri, Jul 1, 2016 at 9:00 AM, Sangjin Lee <sj...@apache.org> wrote:
> >
> > > Hi folks,
> > >
> > > Per earlier discussion [1], I'd like to start a formal vote to merge
> > > feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The vote
> > will
> > > run for 7 days, and will end July 8 9:00 AM PDT.
> > >
> > > We have been developing the feature in a feature branch (YARN-2928 [3])
> > for
> > > a while, and we are reasonably confident that the state of the feature
> > > meets the criteria to be merged onto trunk and we'd love folks to get
> > their
> > > hands on it in a test capacity and provide valuable feedback so that we
> > can
> > > make it production-ready.
> > >
> > > In a nutshell, Timeline Service v.2 delivers significant scalability
> and
> > > usability improvements based on a new architecture. You can browse the
> > > requirements/design doc, the storage schema doc, the new entity/data
> > model,
> > > the YARN documentation, and also discussions on subsequent milestones
> on
> > > YARN-2928 [2].
> > >
> > > What we would like to merge to trunk is termed "alpha 1" (milestone 1).
> > The
> > > feature has a complete end-to-end read/write flow, and you should be
> able
> > > to start setting it up and testing it. At a high level, the following
> are
> > > the key features that have been implemented:
> > >
> > > - distributed writers (collectors) as NM aux services
> > > - HBase storage
> > > - new entity model that includes flows
> > > - setting the flow context via YARN app tags
> > > - real time metrics aggregation to the application level and the flow
> > level
> > > - rich REST API that supports filters, complex conditionals, limits,
> > > content selection, etc.
> > > - YARN generic events and system metrics
> > > - integration with Distributed Shell and MapReduce
> > >
> > > There are a total of 139 subtasks that were completed as part of this
> > > effort.
> > >
> > > We paid close attention to ensure that once disabled Timeline Service
> v.2
> > > does not impact existing functionality when disabled (by default).
> > >
> > > Special thanks go to all the folks who worked real hard on this
> feature:
> > Li
> > > Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena,
> > Vrushali
> > > C, and Joep Rottinghuis!
> > >
> > > Regards,
> > > Sangjin
> > >
> > > [1]
> > >
> > >
> >
> https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
> > > [2] https://issues.apache.org/jira/browse/YARN-2928
> > > [3] https://github.com/apache/hadoop/commits/YARN-2928
> > >
> >
>
Re: [VOTE] merge feature branch YARN-2928
Posted by Subramaniam V K <su...@gmail.com>.
+0 for same reasons as Karthik/Akira. I skimmed through the API patches as
I was trying to make sense of integration with Federation (YARN-2915) and
they seemed reasonable.
Generally excited to see this in YARN.
Also the contributors did a good job of keeping yarn-dev@ in the loop with
regular updates and early heads up for various milestones during branch
development
-Subru
On Thu, Jul 7, 2016 at 11:56 AM, Vrushali Channapattan <vrushali.c@gmail.com
> wrote:
> +1 (non-binding)
>
> Has been exciting working on this and am looking forward to the merge to
> trunk.
>
> On Fri, Jul 1, 2016 at 9:00 AM, Sangjin Lee <sj...@apache.org> wrote:
>
> > Hi folks,
> >
> > Per earlier discussion [1], I'd like to start a formal vote to merge
> > feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The vote
> will
> > run for 7 days, and will end July 8 9:00 AM PDT.
> >
> > We have been developing the feature in a feature branch (YARN-2928 [3])
> for
> > a while, and we are reasonably confident that the state of the feature
> > meets the criteria to be merged onto trunk and we'd love folks to get
> their
> > hands on it in a test capacity and provide valuable feedback so that we
> can
> > make it production-ready.
> >
> > In a nutshell, Timeline Service v.2 delivers significant scalability and
> > usability improvements based on a new architecture. You can browse the
> > requirements/design doc, the storage schema doc, the new entity/data
> model,
> > the YARN documentation, and also discussions on subsequent milestones on
> > YARN-2928 [2].
> >
> > What we would like to merge to trunk is termed "alpha 1" (milestone 1).
> The
> > feature has a complete end-to-end read/write flow, and you should be able
> > to start setting it up and testing it. At a high level, the following are
> > the key features that have been implemented:
> >
> > - distributed writers (collectors) as NM aux services
> > - HBase storage
> > - new entity model that includes flows
> > - setting the flow context via YARN app tags
> > - real time metrics aggregation to the application level and the flow
> level
> > - rich REST API that supports filters, complex conditionals, limits,
> > content selection, etc.
> > - YARN generic events and system metrics
> > - integration with Distributed Shell and MapReduce
> >
> > There are a total of 139 subtasks that were completed as part of this
> > effort.
> >
> > We paid close attention to ensure that once disabled Timeline Service v.2
> > does not impact existing functionality when disabled (by default).
> >
> > Special thanks go to all the folks who worked real hard on this feature:
> Li
> > Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena,
> Vrushali
> > C, and Joep Rottinghuis!
> >
> > Regards,
> > Sangjin
> >
> > [1]
> >
> >
> https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
> > [2] https://issues.apache.org/jira/browse/YARN-2928
> > [3] https://github.com/apache/hadoop/commits/YARN-2928
> >
>
Re: [VOTE] merge feature branch YARN-2928
Posted by Vrushali Channapattan <vr...@gmail.com>.
+1 (non-binding)
Has been exciting working on this and am looking forward to the merge to
trunk.
On Fri, Jul 1, 2016 at 9:00 AM, Sangjin Lee <sj...@apache.org> wrote:
> Hi folks,
>
> Per earlier discussion [1], I'd like to start a formal vote to merge
> feature branch YARN-2928 [2] (Timeline Service v.2) to trunk. The vote will
> run for 7 days, and will end July 8 9:00 AM PDT.
>
> We have been developing the feature in a feature branch (YARN-2928 [3]) for
> a while, and we are reasonably confident that the state of the feature
> meets the criteria to be merged onto trunk and we'd love folks to get their
> hands on it in a test capacity and provide valuable feedback so that we can
> make it production-ready.
>
> In a nutshell, Timeline Service v.2 delivers significant scalability and
> usability improvements based on a new architecture. You can browse the
> requirements/design doc, the storage schema doc, the new entity/data model,
> the YARN documentation, and also discussions on subsequent milestones on
> YARN-2928 [2].
>
> What we would like to merge to trunk is termed "alpha 1" (milestone 1). The
> feature has a complete end-to-end read/write flow, and you should be able
> to start setting it up and testing it. At a high level, the following are
> the key features that have been implemented:
>
> - distributed writers (collectors) as NM aux services
> - HBase storage
> - new entity model that includes flows
> - setting the flow context via YARN app tags
> - real time metrics aggregation to the application level and the flow level
> - rich REST API that supports filters, complex conditionals, limits,
> content selection, etc.
> - YARN generic events and system metrics
> - integration with Distributed Shell and MapReduce
>
> There are a total of 139 subtasks that were completed as part of this
> effort.
>
> We paid close attention to ensure that once disabled Timeline Service v.2
> does not impact existing functionality when disabled (by default).
>
> Special thanks go to all the folks who worked real hard on this feature: Li
> Lu, Junping Du, Vinod Kumar Vavilapalli, Naga G. R., Varun Saxena, Vrushali
> C, and Joep Rottinghuis!
>
> Regards,
> Sangjin
>
> [1]
>
> https://lists.apache.org/thread.html/314288f399f5ef96b2f4f1fc055ac5d440310c8d98ace9a0045e8729@%3Cyarn-dev.hadoop.apache.org%3E
> [2] https://issues.apache.org/jira/browse/YARN-2928
> [3] https://github.com/apache/hadoop/commits/YARN-2928
>