You are viewing a plain text version of this content. The canonical link for it is here.
Posted to general@hadoop.apache.org by Arun C Murthy <ac...@hortonworks.com> on 2013/01/16 15:52:41 UTC
Re: Heads Up - hadoop-2.0.3 release
Happy new year folks!
I'm glad to see that we are down to the last couple of blockers I hope we can resolve in the next 24-hrs or so.
Once done, I'll create a branch-2.0.3-alpha to unblock branch-2 for commits targeted towards the next release, create the new fix-versions in jira and spin the RC.
Committers - after the branch is created, please use only the new fix-version (2.0.4) and check with me before you commit to branch-2.0.3-alpha. Thanks.
As always, I'd appreciate help to resolve any unexpected surprises and also to help verify the RC.
Hopefully we can start the new year with a great release, there are lots of goodies in 2.0.3-alpha.
thanks,
Arun
On Dec 18, 2012, at 9:00 PM, Arun C Murthy wrote:
> As Sid responded I think we can move off alpha once we fix YARN-142/MAPREDUCE-4067. There are other apis we should clean up, but none as egregious as those two.
>
> Someone on my team is starting on it as we speak and I believe we can get it done sometime in Jan... thus targetting 2.0.4 (as a beta?). By then we'll also have wider rollouts of YARN and would have fixed some more issues we've seen at very high scale deployments at Y!. Sounds like the right time to do a beta release to me.
>
> Arun
>
> On Dec 19, 2012, at 10:13 AM, Aaron T. Myers wrote:
>
>> Hey Arun,
>>
>> Awesome to see we're almost down to zero blockers. What are your thoughts
>> on removing the "alpha" label from the upcoming release? It seems to me
>> from the earlier discussion that most folks feel that we're at the point
>> where the interfaces are sufficiently stable to warrant it.
>>
>> Aaron
>>
>>
>> On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>>
>>> Nearly there:
>>> http://s.apache.org/hadoop-2-blockers
>>>
>>> YARN-217 should be easy, I'd also like to get in YARN-253.
>>>
>>> Arun
>>>
>>> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:
>>>
>>>> Any news on how this is progressing? Some folks in this thread below
>>>> inquired about getting this release out around the New Year timeframe,
>>>> but it looks like YARN-117 subtasks have gone pretty quiet. We all
>>>> know how long lifecycle changes can take to get pushed through ;-)
>>>>
>>>> -Todd
>>>>
>>>> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
>>>> <st...@gmail.com> wrote:
>>>>> I want to make some changes to the lifecycle of a yarn service (in a
>>>>> backwards compatible way).
>>>>>
>>>>> https://issues.apache.org/jira/browse/YARN-117
>>>>>
>>>>>
>>>>> 1. formal state machine model with stop state idempotent and
>>> entry-able
>>>>> from any state
>>>>> 2. waiting/blocked state a service can enter when waiting for
>>> something
>>>>> else
>>>>> 3. an alternate base class that does the state model checks before
>>>>> executing any state change functions -currently its done at
>>>>> end-of-operation in the super() calls.
>>>>> 4. gradual move of services to the stricter base class.
>>>>>
>>>>> With a new base class nothing will break (as the move can be done
>>>>> case-by-case, leaving the heavily subclassed ones alone); the state
>>> model
>>>>> extensions & formalisation would be visible but not used.
>>>>>
>>>>> I don't want to hold anything up, because I need more testing of things
>>>>> before this is ready for review. I just want to get the fixes in before
>>> it
>>>>> ships
>>>>>
>>>>> On 19 November 2012 16:22, Robert Evans <ev...@yahoo-inc.com> wrote:
>>>>>
>>>>>> I am OK with removing the alpha assuming that we think that the APIs
>>> are
>>>>>> stable enough that we are willing to truly start maintaining backwards
>>>>>> compatibility on them within 2.X. From what I have seen I think that
>>> they
>>>>>> are fairly stable and I think there is enough adoption by other
>>> projects
>>>>>> right now that breaking backwards compatibility would be problematic.
>>>>>>
>>>>>> --Bobby Evans
>>>>>>
>>>>>> On 11/16/12 11:34 PM, "Stack" <st...@duboce.net> wrote:
>>>>>>
>>>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <at...@cloudera.com>
>>> wrote:
>>>>>>>> Hi Arun,
>>>>>>>>
>>>>>>>> Given that the 2.0.3 release is intended to reflect the growing
>>>>>>>> stability
>>>>>>>> of YARN, and the QJM work will be included in 2.0.3 which provides a
>>>>>>>> complete HDFS HA solution, I think it's time we consider removing the
>>>>>>>> "-alpha" label from the release version. My preference would be to
>>>>>>>> remove
>>>>>>>> the label entirely, but we could also perhaps call it "-beta" or
>>>>>>>> something.
>>>>>>>>
>>>>>>>> Thoughts?
>>>>>>>>
>>>>>>>
>>>>>>> I think it fine after two minor releases undoing the '-alpha' suffix.
>>>>>>>
>>>>>>> If folks insist we next go to '-beta', I'd hope we'd travel all
>>>>>>> remaining 22 letters of the greek alphabet before we 2.0.x.
>>>>>>>
>>>>>>> St.Ack
>>>>>>
>>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Todd Lipcon
>>>> Software Engineer, Cloudera
>>>
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>>
>>>
>>>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Heads Up - hadoop-2.0.3 release
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
I'd like to get in 2.0.3 the JIRAs that enable pluggable shuffle and
pluggable sort: MAPREDUCE-4049, MAPREDUCE-4809, MAPREDUCE-4807 &
MAPREDUCE-4808
Unless I hear otherwise, I'd be merging those commits in branch-2 tomorrow
mid afternoon PST.
Thx
On Wed, Jan 16, 2013 at 10:38 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Seems reasonable since it's a small patch and has had one round of review.
>
> I spoke to Todd yesterday and he seemed to think it can be pushed in
> fairly soon too. So that's two of you.
>
> Anyway, since I'm still awaiting HADOOP-9215 too, I'd appreciate if we can
> get HDFS-4404 committed asap.
>
> thanks,
> Arun
>
> On Jan 16, 2013, at 10:30 AM, Suresh Srinivas wrote:
>
> > Arun,
> >
> > HDFS-4404 is not marked as a blocker. It is a serious bug and I would
> like
> > to make it a blocker. Let me know if you are okay.
> >
> > If folks do not want to hold the release for that fix, lets at least try
> to
> > capture that in release notes and try get that fix into 2.0.4.
> >
> > Regards,
> > Suresh
> >
> >
> > On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <ac...@hortonworks.com>
> wrote:
> >
> >> Happy new year folks!
> >>
> >> I'm glad to see that we are down to the last couple of blockers I hope
> we
> >> can resolve in the next 24-hrs or so.
> >>
> >> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2 for
> >> commits targeted towards the next release, create the new fix-versions
> in
> >> jira and spin the RC.
> >> Committers - after the branch is created, please use only the new
> >> fix-version (2.0.4) and check with me before you commit to
> >> branch-2.0.3-alpha. Thanks.
> >>
> >> As always, I'd appreciate help to resolve any unexpected surprises and
> >> also to help verify the RC.
> >>
> >> Hopefully we can start the new year with a great release, there are lots
> >> of goodies in 2.0.3-alpha.
> >>
> >> thanks,
> >> Arun
> >>
> >> On Dec 18, 2012, at 9:00 PM, Arun C Murthy wrote:
> >>
> >>> As Sid responded I think we can move off alpha once we fix
> >> YARN-142/MAPREDUCE-4067. There are other apis we should clean up, but
> none
> >> as egregious as those two.
> >>>
> >>> Someone on my team is starting on it as we speak and I believe we can
> >> get it done sometime in Jan... thus targetting 2.0.4 (as a beta?). By
> then
> >> we'll also have wider rollouts of YARN and would have fixed some more
> >> issues we've seen at very high scale deployments at Y!. Sounds like the
> >> right time to do a beta release to me.
> >>>
> >>> Arun
> >>>
> >>> On Dec 19, 2012, at 10:13 AM, Aaron T. Myers wrote:
> >>>
> >>>> Hey Arun,
> >>>>
> >>>> Awesome to see we're almost down to zero blockers. What are your
> >> thoughts
> >>>> on removing the "alpha" label from the upcoming release? It seems to
> me
> >>>> from the earlier discussion that most folks feel that we're at the
> point
> >>>> where the interfaces are sufficiently stable to warrant it.
> >>>>
> >>>> Aaron
> >>>>
> >>>>
> >>>> On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <ac...@hortonworks.com>
> >> wrote:
> >>>>
> >>>>> Nearly there:
> >>>>> http://s.apache.org/hadoop-2-blockers
> >>>>>
> >>>>> YARN-217 should be easy, I'd also like to get in YARN-253.
> >>>>>
> >>>>> Arun
> >>>>>
> >>>>> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:
> >>>>>
> >>>>>> Any news on how this is progressing? Some folks in this thread below
> >>>>>> inquired about getting this release out around the New Year
> timeframe,
> >>>>>> but it looks like YARN-117 subtasks have gone pretty quiet. We all
> >>>>>> know how long lifecycle changes can take to get pushed through ;-)
> >>>>>>
> >>>>>> -Todd
> >>>>>>
> >>>>>> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
> >>>>>> <st...@gmail.com> wrote:
> >>>>>>> I want to make some changes to the lifecycle of a yarn service (in
> a
> >>>>>>> backwards compatible way).
> >>>>>>>
> >>>>>>> https://issues.apache.org/jira/browse/YARN-117
> >>>>>>>
> >>>>>>>
> >>>>>>> 1. formal state machine model with stop state idempotent and
> >>>>> entry-able
> >>>>>>> from any state
> >>>>>>> 2. waiting/blocked state a service can enter when waiting for
> >>>>> something
> >>>>>>> else
> >>>>>>> 3. an alternate base class that does the state model checks before
> >>>>>>> executing any state change functions -currently its done at
> >>>>>>> end-of-operation in the super() calls.
> >>>>>>> 4. gradual move of services to the stricter base class.
> >>>>>>>
> >>>>>>> With a new base class nothing will break (as the move can be done
> >>>>>>> case-by-case, leaving the heavily subclassed ones alone); the state
> >>>>> model
> >>>>>>> extensions & formalisation would be visible but not used.
> >>>>>>>
> >>>>>>> I don't want to hold anything up, because I need more testing of
> >> things
> >>>>>>> before this is ready for review. I just want to get the fixes in
> >> before
> >>>>> it
> >>>>>>> ships
> >>>>>>>
> >>>>>>> On 19 November 2012 16:22, Robert Evans <ev...@yahoo-inc.com>
> wrote:
> >>>>>>>
> >>>>>>>> I am OK with removing the alpha assuming that we think that the
> APIs
> >>>>> are
> >>>>>>>> stable enough that we are willing to truly start maintaining
> >> backwards
> >>>>>>>> compatibility on them within 2.X. From what I have seen I think
> that
> >>>>> they
> >>>>>>>> are fairly stable and I think there is enough adoption by other
> >>>>> projects
> >>>>>>>> right now that breaking backwards compatibility would be
> >> problematic.
> >>>>>>>>
> >>>>>>>> --Bobby Evans
> >>>>>>>>
> >>>>>>>> On 11/16/12 11:34 PM, "Stack" <st...@duboce.net> wrote:
> >>>>>>>>
> >>>>>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <
> atm@cloudera.com>
> >>>>> wrote:
> >>>>>>>>>> Hi Arun,
> >>>>>>>>>>
> >>>>>>>>>> Given that the 2.0.3 release is intended to reflect the growing
> >>>>>>>>>> stability
> >>>>>>>>>> of YARN, and the QJM work will be included in 2.0.3 which
> >> provides a
> >>>>>>>>>> complete HDFS HA solution, I think it's time we consider
> removing
> >> the
> >>>>>>>>>> "-alpha" label from the release version. My preference would be
> to
> >>>>>>>>>> remove
> >>>>>>>>>> the label entirely, but we could also perhaps call it "-beta" or
> >>>>>>>>>> something.
> >>>>>>>>>>
> >>>>>>>>>> Thoughts?
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> I think it fine after two minor releases undoing the '-alpha'
> >> suffix.
> >>>>>>>>>
> >>>>>>>>> If folks insist we next go to '-beta', I'd hope we'd travel all
> >>>>>>>>> remaining 22 letters of the greek alphabet before we 2.0.x.
> >>>>>>>>>
> >>>>>>>>> St.Ack
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Todd Lipcon
> >>>>>> Software Engineer, Cloudera
> >>>>>
> >>>>> --
> >>>>> Arun C. Murthy
> >>>>> Hortonworks Inc.
> >>>>> http://hortonworks.com/
> >>>>>
> >>>>>
> >>>>>
> >>>
> >>> --
> >>> Arun C. Murthy
> >>> Hortonworks Inc.
> >>> http://hortonworks.com/
> >>>
> >>>
> >>
> >> --
> >> Arun C. Murthy
> >> Hortonworks Inc.
> >> http://hortonworks.com/
> >>
> >>
> >>
> >
> >
> > --
> > http://hortonworks.com/download/
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
--
Alejandro
Re: Heads Up - hadoop-2.0.3 release
Posted by Arun C Murthy <ac...@hortonworks.com>.
Seems reasonable since it's a small patch and has had one round of review.
I spoke to Todd yesterday and he seemed to think it can be pushed in fairly soon too. So that's two of you.
Anyway, since I'm still awaiting HADOOP-9215 too, I'd appreciate if we can get HDFS-4404 committed asap.
thanks,
Arun
On Jan 16, 2013, at 10:30 AM, Suresh Srinivas wrote:
> Arun,
>
> HDFS-4404 is not marked as a blocker. It is a serious bug and I would like
> to make it a blocker. Let me know if you are okay.
>
> If folks do not want to hold the release for that fix, lets at least try to
> capture that in release notes and try get that fix into 2.0.4.
>
> Regards,
> Suresh
>
>
> On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
>> Happy new year folks!
>>
>> I'm glad to see that we are down to the last couple of blockers I hope we
>> can resolve in the next 24-hrs or so.
>>
>> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2 for
>> commits targeted towards the next release, create the new fix-versions in
>> jira and spin the RC.
>> Committers - after the branch is created, please use only the new
>> fix-version (2.0.4) and check with me before you commit to
>> branch-2.0.3-alpha. Thanks.
>>
>> As always, I'd appreciate help to resolve any unexpected surprises and
>> also to help verify the RC.
>>
>> Hopefully we can start the new year with a great release, there are lots
>> of goodies in 2.0.3-alpha.
>>
>> thanks,
>> Arun
>>
>> On Dec 18, 2012, at 9:00 PM, Arun C Murthy wrote:
>>
>>> As Sid responded I think we can move off alpha once we fix
>> YARN-142/MAPREDUCE-4067. There are other apis we should clean up, but none
>> as egregious as those two.
>>>
>>> Someone on my team is starting on it as we speak and I believe we can
>> get it done sometime in Jan... thus targetting 2.0.4 (as a beta?). By then
>> we'll also have wider rollouts of YARN and would have fixed some more
>> issues we've seen at very high scale deployments at Y!. Sounds like the
>> right time to do a beta release to me.
>>>
>>> Arun
>>>
>>> On Dec 19, 2012, at 10:13 AM, Aaron T. Myers wrote:
>>>
>>>> Hey Arun,
>>>>
>>>> Awesome to see we're almost down to zero blockers. What are your
>> thoughts
>>>> on removing the "alpha" label from the upcoming release? It seems to me
>>>> from the earlier discussion that most folks feel that we're at the point
>>>> where the interfaces are sufficiently stable to warrant it.
>>>>
>>>> Aaron
>>>>
>>>>
>>>> On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:
>>>>
>>>>> Nearly there:
>>>>> http://s.apache.org/hadoop-2-blockers
>>>>>
>>>>> YARN-217 should be easy, I'd also like to get in YARN-253.
>>>>>
>>>>> Arun
>>>>>
>>>>> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:
>>>>>
>>>>>> Any news on how this is progressing? Some folks in this thread below
>>>>>> inquired about getting this release out around the New Year timeframe,
>>>>>> but it looks like YARN-117 subtasks have gone pretty quiet. We all
>>>>>> know how long lifecycle changes can take to get pushed through ;-)
>>>>>>
>>>>>> -Todd
>>>>>>
>>>>>> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
>>>>>> <st...@gmail.com> wrote:
>>>>>>> I want to make some changes to the lifecycle of a yarn service (in a
>>>>>>> backwards compatible way).
>>>>>>>
>>>>>>> https://issues.apache.org/jira/browse/YARN-117
>>>>>>>
>>>>>>>
>>>>>>> 1. formal state machine model with stop state idempotent and
>>>>> entry-able
>>>>>>> from any state
>>>>>>> 2. waiting/blocked state a service can enter when waiting for
>>>>> something
>>>>>>> else
>>>>>>> 3. an alternate base class that does the state model checks before
>>>>>>> executing any state change functions -currently its done at
>>>>>>> end-of-operation in the super() calls.
>>>>>>> 4. gradual move of services to the stricter base class.
>>>>>>>
>>>>>>> With a new base class nothing will break (as the move can be done
>>>>>>> case-by-case, leaving the heavily subclassed ones alone); the state
>>>>> model
>>>>>>> extensions & formalisation would be visible but not used.
>>>>>>>
>>>>>>> I don't want to hold anything up, because I need more testing of
>> things
>>>>>>> before this is ready for review. I just want to get the fixes in
>> before
>>>>> it
>>>>>>> ships
>>>>>>>
>>>>>>> On 19 November 2012 16:22, Robert Evans <ev...@yahoo-inc.com> wrote:
>>>>>>>
>>>>>>>> I am OK with removing the alpha assuming that we think that the APIs
>>>>> are
>>>>>>>> stable enough that we are willing to truly start maintaining
>> backwards
>>>>>>>> compatibility on them within 2.X. From what I have seen I think that
>>>>> they
>>>>>>>> are fairly stable and I think there is enough adoption by other
>>>>> projects
>>>>>>>> right now that breaking backwards compatibility would be
>> problematic.
>>>>>>>>
>>>>>>>> --Bobby Evans
>>>>>>>>
>>>>>>>> On 11/16/12 11:34 PM, "Stack" <st...@duboce.net> wrote:
>>>>>>>>
>>>>>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <at...@cloudera.com>
>>>>> wrote:
>>>>>>>>>> Hi Arun,
>>>>>>>>>>
>>>>>>>>>> Given that the 2.0.3 release is intended to reflect the growing
>>>>>>>>>> stability
>>>>>>>>>> of YARN, and the QJM work will be included in 2.0.3 which
>> provides a
>>>>>>>>>> complete HDFS HA solution, I think it's time we consider removing
>> the
>>>>>>>>>> "-alpha" label from the release version. My preference would be to
>>>>>>>>>> remove
>>>>>>>>>> the label entirely, but we could also perhaps call it "-beta" or
>>>>>>>>>> something.
>>>>>>>>>>
>>>>>>>>>> Thoughts?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I think it fine after two minor releases undoing the '-alpha'
>> suffix.
>>>>>>>>>
>>>>>>>>> If folks insist we next go to '-beta', I'd hope we'd travel all
>>>>>>>>> remaining 22 letters of the greek alphabet before we 2.0.x.
>>>>>>>>>
>>>>>>>>> St.Ack
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Todd Lipcon
>>>>>> Software Engineer, Cloudera
>>>>>
>>>>> --
>>>>> Arun C. Murthy
>>>>> Hortonworks Inc.
>>>>> http://hortonworks.com/
>>>>>
>>>>>
>>>>>
>>>
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>>
>>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>
>
> --
> http://hortonworks.com/download/
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Heads Up - hadoop-2.0.3 release
Posted by Suresh Srinivas <su...@hortonworks.com>.
Arun,
HDFS-4404 is not marked as a blocker. It is a serious bug and I would like
to make it a blocker. Let me know if you are okay.
If folks do not want to hold the release for that fix, lets at least try to
capture that in release notes and try get that fix into 2.0.4.
Regards,
Suresh
On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Happy new year folks!
>
> I'm glad to see that we are down to the last couple of blockers I hope we
> can resolve in the next 24-hrs or so.
>
> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2 for
> commits targeted towards the next release, create the new fix-versions in
> jira and spin the RC.
> Committers - after the branch is created, please use only the new
> fix-version (2.0.4) and check with me before you commit to
> branch-2.0.3-alpha. Thanks.
>
> As always, I'd appreciate help to resolve any unexpected surprises and
> also to help verify the RC.
>
> Hopefully we can start the new year with a great release, there are lots
> of goodies in 2.0.3-alpha.
>
> thanks,
> Arun
>
> On Dec 18, 2012, at 9:00 PM, Arun C Murthy wrote:
>
> > As Sid responded I think we can move off alpha once we fix
> YARN-142/MAPREDUCE-4067. There are other apis we should clean up, but none
> as egregious as those two.
> >
> > Someone on my team is starting on it as we speak and I believe we can
> get it done sometime in Jan... thus targetting 2.0.4 (as a beta?). By then
> we'll also have wider rollouts of YARN and would have fixed some more
> issues we've seen at very high scale deployments at Y!. Sounds like the
> right time to do a beta release to me.
> >
> > Arun
> >
> > On Dec 19, 2012, at 10:13 AM, Aaron T. Myers wrote:
> >
> >> Hey Arun,
> >>
> >> Awesome to see we're almost down to zero blockers. What are your
> thoughts
> >> on removing the "alpha" label from the upcoming release? It seems to me
> >> from the earlier discussion that most folks feel that we're at the point
> >> where the interfaces are sufficiently stable to warrant it.
> >>
> >> Aaron
> >>
> >>
> >> On Tue, Dec 18, 2012 at 8:15 PM, Arun C Murthy <ac...@hortonworks.com>
> wrote:
> >>
> >>> Nearly there:
> >>> http://s.apache.org/hadoop-2-blockers
> >>>
> >>> YARN-217 should be easy, I'd also like to get in YARN-253.
> >>>
> >>> Arun
> >>>
> >>> On Dec 19, 2012, at 1:15 AM, Todd Lipcon wrote:
> >>>
> >>>> Any news on how this is progressing? Some folks in this thread below
> >>>> inquired about getting this release out around the New Year timeframe,
> >>>> but it looks like YARN-117 subtasks have gone pretty quiet. We all
> >>>> know how long lifecycle changes can take to get pushed through ;-)
> >>>>
> >>>> -Todd
> >>>>
> >>>> On Mon, Nov 19, 2012 at 12:41 PM, Steve Loughran
> >>>> <st...@gmail.com> wrote:
> >>>>> I want to make some changes to the lifecycle of a yarn service (in a
> >>>>> backwards compatible way).
> >>>>>
> >>>>> https://issues.apache.org/jira/browse/YARN-117
> >>>>>
> >>>>>
> >>>>> 1. formal state machine model with stop state idempotent and
> >>> entry-able
> >>>>> from any state
> >>>>> 2. waiting/blocked state a service can enter when waiting for
> >>> something
> >>>>> else
> >>>>> 3. an alternate base class that does the state model checks before
> >>>>> executing any state change functions -currently its done at
> >>>>> end-of-operation in the super() calls.
> >>>>> 4. gradual move of services to the stricter base class.
> >>>>>
> >>>>> With a new base class nothing will break (as the move can be done
> >>>>> case-by-case, leaving the heavily subclassed ones alone); the state
> >>> model
> >>>>> extensions & formalisation would be visible but not used.
> >>>>>
> >>>>> I don't want to hold anything up, because I need more testing of
> things
> >>>>> before this is ready for review. I just want to get the fixes in
> before
> >>> it
> >>>>> ships
> >>>>>
> >>>>> On 19 November 2012 16:22, Robert Evans <ev...@yahoo-inc.com> wrote:
> >>>>>
> >>>>>> I am OK with removing the alpha assuming that we think that the APIs
> >>> are
> >>>>>> stable enough that we are willing to truly start maintaining
> backwards
> >>>>>> compatibility on them within 2.X. From what I have seen I think that
> >>> they
> >>>>>> are fairly stable and I think there is enough adoption by other
> >>> projects
> >>>>>> right now that breaking backwards compatibility would be
> problematic.
> >>>>>>
> >>>>>> --Bobby Evans
> >>>>>>
> >>>>>> On 11/16/12 11:34 PM, "Stack" <st...@duboce.net> wrote:
> >>>>>>
> >>>>>>> On Fri, Nov 16, 2012 at 3:38 PM, Aaron T. Myers <at...@cloudera.com>
> >>> wrote:
> >>>>>>>> Hi Arun,
> >>>>>>>>
> >>>>>>>> Given that the 2.0.3 release is intended to reflect the growing
> >>>>>>>> stability
> >>>>>>>> of YARN, and the QJM work will be included in 2.0.3 which
> provides a
> >>>>>>>> complete HDFS HA solution, I think it's time we consider removing
> the
> >>>>>>>> "-alpha" label from the release version. My preference would be to
> >>>>>>>> remove
> >>>>>>>> the label entirely, but we could also perhaps call it "-beta" or
> >>>>>>>> something.
> >>>>>>>>
> >>>>>>>> Thoughts?
> >>>>>>>>
> >>>>>>>
> >>>>>>> I think it fine after two minor releases undoing the '-alpha'
> suffix.
> >>>>>>>
> >>>>>>> If folks insist we next go to '-beta', I'd hope we'd travel all
> >>>>>>> remaining 22 letters of the greek alphabet before we 2.0.x.
> >>>>>>>
> >>>>>>> St.Ack
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Todd Lipcon
> >>>> Software Engineer, Cloudera
> >>>
> >>> --
> >>> Arun C. Murthy
> >>> Hortonworks Inc.
> >>> http://hortonworks.com/
> >>>
> >>>
> >>>
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
--
http://hortonworks.com/download/
Re: Heads Up - hadoop-2.0.3 release
Posted by Konstantin Boudnik <co...@apache.org>.
On Mon, Jan 28, 2013 at 03:11PM, Roman Shaposhnik wrote:
> Hi Arun,
>
> On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> > Happy new year folks!
> >
> > I'm glad to see that we are down to the last couple of blockers I hope we can resolve in the next 24-hrs or so.
> >
> > Once done, I'll create a branch-2.0.3-alpha to unblock branch-2
> > for commits targeted towards the next release, create the new fix-versions in jira and spin the RC.
>
> Quick question -- when do you think you can have the
> branch-2.0.3-alpha in place?
>
> We'd *really* like to start testing it in Bigtop ASAP (Bigtop 0.6.0 is
> supposed to be
> based on Hadoop 2.0.3) so we can avoid integration issues like we had between
> 2.0.2 and Oozie.
Oozie is hardcoded to be build against 2.0.2-alpha, so there would be
integration issues nonetheless (see BIGTOP-837)
>
> Any help in making it available to us sooner would be greatly appreciated!
>
> Thanks,
> Roman.
Re: Heads Up - hadoop-2.0.3 release
Posted by Konstantin Boudnik <co...@apache.org>.
+1 - that'd be critical.
On Tue, Jan 29, 2013 at 08:51AM, Roman Shaposhnik wrote:
> On Mon, Jan 28, 2013 at 3:11 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> > Hi Arun,
> >
> > On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> >> Happy new year folks!
> >>
> >> I'm glad to see that we are down to the last couple of blockers I hope we can resolve in the next 24-hrs or so.
> >>
> >> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2
> >> for commits targeted towards the next release, create the new fix-versions in jira and spin the RC.
> >
> > Quick question -- when do you think you can have the
> > branch-2.0.3-alpha in place?
> >
> > We'd *really* like to start testing it in Bigtop ASAP (Bigtop 0.6.0 is
> > supposed to be
> > based on Hadoop 2.0.3) so we can avoid integration issues like we had between
> > 2.0.2 and Oozie.
> >
> > Any help in making it available to us sooner would be greatly appreciated!
>
> On top of that I'd like to nominate:
> https://issues.apache.org/jira/browse/MAPREDUCE-4820
> as a blocker. For as long as it is not fixed Oozie is unusable with
> Hadoop 2.0.[23].
>
> Thanks,
> Roman.
Re: Heads Up - hadoop-2.0.3 release
Posted by Roman Shaposhnik <rv...@apache.org>.
On Tue, Jan 29, 2013 at 9:51 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> I believe we can disregard MAPREDUCE-4820 since MAPREDUCE-4549 is already in branch-2. FYI.
Arun, that very well may be the case -- let me take a closer look and
report back (on the JIRA).
Also, any chance you could comment on when the branch can be cut?
It would make verification of things MAPREDUCE-4820 so much easier.
Thanks,
Roman.
Re: Heads Up - hadoop-2.0.3 release
Posted by Arun C Murthy <ac...@hortonworks.com>.
I believe we can disregard MAPREDUCE-4820 since MAPREDUCE-4549 is already in branch-2. FYI.
On Jan 29, 2013, at 8:51 AM, Roman Shaposhnik wrote:
> On Mon, Jan 28, 2013 at 3:11 PM, Roman Shaposhnik <rv...@apache.org> wrote:
>> Hi Arun,
>>
>> On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
>>> Happy new year folks!
>>>
>>> I'm glad to see that we are down to the last couple of blockers I hope we can resolve in the next 24-hrs or so.
>>>
>>> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2
>>> for commits targeted towards the next release, create the new fix-versions in jira and spin the RC.
>>
>> Quick question -- when do you think you can have the
>> branch-2.0.3-alpha in place?
>>
>> We'd *really* like to start testing it in Bigtop ASAP (Bigtop 0.6.0 is
>> supposed to be
>> based on Hadoop 2.0.3) so we can avoid integration issues like we had between
>> 2.0.2 and Oozie.
>>
>> Any help in making it available to us sooner would be greatly appreciated!
>
> On top of that I'd like to nominate:
> https://issues.apache.org/jira/browse/MAPREDUCE-4820
> as a blocker. For as long as it is not fixed Oozie is unusable with
> Hadoop 2.0.[23].
>
> Thanks,
> Roman.
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: Heads Up - hadoop-2.0.3 release
Posted by Roman Shaposhnik <rv...@apache.org>.
On Mon, Jan 28, 2013 at 3:11 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> Hi Arun,
>
> On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
>> Happy new year folks!
>>
>> I'm glad to see that we are down to the last couple of blockers I hope we can resolve in the next 24-hrs or so.
>>
>> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2
>> for commits targeted towards the next release, create the new fix-versions in jira and spin the RC.
>
> Quick question -- when do you think you can have the
> branch-2.0.3-alpha in place?
>
> We'd *really* like to start testing it in Bigtop ASAP (Bigtop 0.6.0 is
> supposed to be
> based on Hadoop 2.0.3) so we can avoid integration issues like we had between
> 2.0.2 and Oozie.
>
> Any help in making it available to us sooner would be greatly appreciated!
On top of that I'd like to nominate:
https://issues.apache.org/jira/browse/MAPREDUCE-4820
as a blocker. For as long as it is not fixed Oozie is unusable with
Hadoop 2.0.[23].
Thanks,
Roman.
Re: Heads Up - hadoop-2.0.3 release
Posted by Roman Shaposhnik <rv...@apache.org>.
On Mon, Jan 28, 2013 at 3:11 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> Hi Arun,
>
> On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
>> Happy new year folks!
>>
>> I'm glad to see that we are down to the last couple of blockers I hope we can resolve in the next 24-hrs or so.
>>
>> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2
>> for commits targeted towards the next release, create the new fix-versions in jira and spin the RC.
>
> Quick question -- when do you think you can have the
> branch-2.0.3-alpha in place?
>
> We'd *really* like to start testing it in Bigtop ASAP (Bigtop 0.6.0 is
> supposed to be
> based on Hadoop 2.0.3) so we can avoid integration issues like we had between
> 2.0.2 and Oozie.
>
> Any help in making it available to us sooner would be greatly appreciated!
On top of that I'd like to nominate:
https://issues.apache.org/jira/browse/MAPREDUCE-4820
as a blocker. For as long as it is not fixed Oozie is unusable with
Hadoop 2.0.[23].
Thanks,
Roman.
Re: Heads Up - hadoop-2.0.3 release
Posted by Roman Shaposhnik <rv...@apache.org>.
Hi Arun,
On Wed, Jan 16, 2013 at 6:52 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Happy new year folks!
>
> I'm glad to see that we are down to the last couple of blockers I hope we can resolve in the next 24-hrs or so.
>
> Once done, I'll create a branch-2.0.3-alpha to unblock branch-2
> for commits targeted towards the next release, create the new fix-versions in jira and spin the RC.
Quick question -- when do you think you can have the
branch-2.0.3-alpha in place?
We'd *really* like to start testing it in Bigtop ASAP (Bigtop 0.6.0 is
supposed to be
based on Hadoop 2.0.3) so we can avoid integration issues like we had between
2.0.2 and Oozie.
Any help in making it available to us sooner would be greatly appreciated!
Thanks,
Roman.