Re: Thinking ahead

[Chris Nauroth <cn...@hortonworks.com>]

+1

The proposed content for 2.5 in the roadmap wiki looks good to me.
On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:

> Gang,
>
>  With hadoop-2.4 out, it's time to think ahead.
>
>  In the short-term hadoop-2.4.1 is in order; particularly with
> https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
> @Private API, unfortunately something Hive is using - sigh!). There are
> some other fixes which testing has uncovered; so it will be nice to pull
> them them in. I'm thinking of an RC by end of the coming week - committers,
> please be *very* conservative when getting stuff into 2.4.1 (i.e. merging
> to branch-2.4).
>
>  Next up, hadoop-2.5.
>
>  I've updated https://wiki.apache.org/hadoop/Roadmap with some candidates
> for consideration - please chime in and say 'aye'/'nay' or add new content.
> IAC, I suspect that list is too large.
>
>  Rather than wait for everything it would be better to plan on releasing
> it on a time-bound manner; particularly around the Hadoop Summit. If that
> makes sense; I think we should target branching for 2.5 by mid-May to get
> it stable and released by early June.
>
>  Thoughts?
>
> thanks,
> Arun
>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Vinod Kumar Vavilapalli <vi...@apache.org>]

I'm still wading through 2.4.1 critical issues (failing tests, blockers
etc) so haven't gotten time to think about the 2.5 list w.r.t YARN. Will do
so in the next week.

BTW, folks, please refrain from adding items to the list that you
yourselves are not volunteering to work on. Anyone keen to see a particular
feature should volunteer to that effect. We don't 'assign' work.

+Vinod

On Sat, Apr 12, 2014 at 10:09 AM, Sandy Ryza <sa...@cloudera.com>wrote:

>
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  I think we
> should probably take YARN-1404 off the list - even with 3 months it's
> unlikely to be complete.
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Zhijie Shen <zs...@hortonworks.com>]

+1 for Timeline Server stability. In addition to the security, we may also
want to deal with scalability, generic and per-framework services
integration and MR integration.

Any plan about YARN long running services?


On Sat, Apr 12, 2014 at 10:09 AM, Sandy Ryza <sa...@cloudera.com>wrote:

> +1 for starting to think about 2.5.  Early June seems a little early to me
> - we had talked about a quarterly release cadence and this would be about
> half that.
>
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  I think we
> should probably take YARN-1404 off the list - even with 3 months it's
> unlikely to be complete.
>
>
> On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cnauroth@hortonworks.com
> >wrote:
>
> > +1
> >
> > The proposed content for 2.5 in the roadmap wiki looks good to me.
> > On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
> >
> > > Gang,
> > >
> > >  With hadoop-2.4 out, it's time to think ahead.
> > >
> > >  In the short-term hadoop-2.4.1 is in order; particularly with
> > > https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
> > > @Private API, unfortunately something Hive is using - sigh!). There are
> > > some other fixes which testing has uncovered; so it will be nice to
> pull
> > > them them in. I'm thinking of an RC by end of the coming week -
> > committers,
> > > please be *very* conservative when getting stuff into 2.4.1 (i.e.
> merging
> > > to branch-2.4).
> > >
> > >  Next up, hadoop-2.5.
> > >
> > >  I've updated https://wiki.apache.org/hadoop/Roadmap with some
> > candidates
> > > for consideration - please chime in and say 'aye'/'nay' or add new
> > content.
> > > IAC, I suspect that list is too large.
> > >
> > >  Rather than wait for everything it would be better to plan on
> releasing
> > > it on a time-bound manner; particularly around the Hadoop Summit. If
> that
> > > makes sense; I think we should target branching for 2.5 by mid-May to
> get
> > > it stable and released by early June.
> > >
> > >  Thoughts?
> > >
> > > thanks,
> > > Arun
> > >
> > >
> > > --
> > > Arun C. Murthy
> > > Hortonworks Inc.
> > > http://hortonworks.com/
> > >
> > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>



-- 
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Zhijie Shen <zs...@hortonworks.com>]

+1 for Timeline Server stability. In addition to the security, we may also
want to deal with scalability, generic and per-framework services
integration and MR integration.

Any plan about YARN long running services?


On Sat, Apr 12, 2014 at 10:09 AM, Sandy Ryza <sa...@cloudera.com>wrote:

> +1 for starting to think about 2.5.  Early June seems a little early to me
> - we had talked about a quarterly release cadence and this would be about
> half that.
>
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  I think we
> should probably take YARN-1404 off the list - even with 3 months it's
> unlikely to be complete.
>
>
> On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cnauroth@hortonworks.com
> >wrote:
>
> > +1
> >
> > The proposed content for 2.5 in the roadmap wiki looks good to me.
> > On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
> >
> > > Gang,
> > >
> > >  With hadoop-2.4 out, it's time to think ahead.
> > >
> > >  In the short-term hadoop-2.4.1 is in order; particularly with
> > > https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
> > > @Private API, unfortunately something Hive is using - sigh!). There are
> > > some other fixes which testing has uncovered; so it will be nice to
> pull
> > > them them in. I'm thinking of an RC by end of the coming week -
> > committers,
> > > please be *very* conservative when getting stuff into 2.4.1 (i.e.
> merging
> > > to branch-2.4).
> > >
> > >  Next up, hadoop-2.5.
> > >
> > >  I've updated https://wiki.apache.org/hadoop/Roadmap with some
> > candidates
> > > for consideration - please chime in and say 'aye'/'nay' or add new
> > content.
> > > IAC, I suspect that list is too large.
> > >
> > >  Rather than wait for everything it would be better to plan on
> releasing
> > > it on a time-bound manner; particularly around the Hadoop Summit. If
> that
> > > makes sense; I think we should target branching for 2.5 by mid-May to
> get
> > > it stable and released by early June.
> > >
> > >  Thoughts?
> > >
> > > thanks,
> > > Arun
> > >
> > >
> > > --
> > > Arun C. Murthy
> > > Hortonworks Inc.
> > > http://hortonworks.com/
> > >
> > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>



-- 
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Vinod Kumar Vavilapalli <vi...@apache.org>]

I'm still wading through 2.4.1 critical issues (failing tests, blockers
etc) so haven't gotten time to think about the 2.5 list w.r.t YARN. Will do
so in the next week.

BTW, folks, please refrain from adding items to the list that you
yourselves are not volunteering to work on. Anyone keen to see a particular
feature should volunteer to that effect. We don't 'assign' work.

+Vinod

On Sat, Apr 12, 2014 at 10:09 AM, Sandy Ryza <sa...@cloudera.com>wrote:

>
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  I think we
> should probably take YARN-1404 off the list - even with 3 months it's
> unlikely to be complete.
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Zhijie Shen <zs...@hortonworks.com>]

+1 for Timeline Server stability. In addition to the security, we may also
want to deal with scalability, generic and per-framework services
integration and MR integration.

Any plan about YARN long running services?


On Sat, Apr 12, 2014 at 10:09 AM, Sandy Ryza <sa...@cloudera.com>wrote:

> +1 for starting to think about 2.5.  Early June seems a little early to me
> - we had talked about a quarterly release cadence and this would be about
> half that.
>
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  I think we
> should probably take YARN-1404 off the list - even with 3 months it's
> unlikely to be complete.
>
>
> On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cnauroth@hortonworks.com
> >wrote:
>
> > +1
> >
> > The proposed content for 2.5 in the roadmap wiki looks good to me.
> > On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
> >
> > > Gang,
> > >
> > >  With hadoop-2.4 out, it's time to think ahead.
> > >
> > >  In the short-term hadoop-2.4.1 is in order; particularly with
> > > https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
> > > @Private API, unfortunately something Hive is using - sigh!). There are
> > > some other fixes which testing has uncovered; so it will be nice to
> pull
> > > them them in. I'm thinking of an RC by end of the coming week -
> > committers,
> > > please be *very* conservative when getting stuff into 2.4.1 (i.e.
> merging
> > > to branch-2.4).
> > >
> > >  Next up, hadoop-2.5.
> > >
> > >  I've updated https://wiki.apache.org/hadoop/Roadmap with some
> > candidates
> > > for consideration - please chime in and say 'aye'/'nay' or add new
> > content.
> > > IAC, I suspect that list is too large.
> > >
> > >  Rather than wait for everything it would be better to plan on
> releasing
> > > it on a time-bound manner; particularly around the Hadoop Summit. If
> that
> > > makes sense; I think we should target branching for 2.5 by mid-May to
> get
> > > it stable and released by early June.
> > >
> > >  Thoughts?
> > >
> > > thanks,
> > > Arun
> > >
> > >
> > > --
> > > Arun C. Murthy
> > > Hortonworks Inc.
> > > http://hortonworks.com/
> > >
> > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>



-- 
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Akira AJISAKA <aj...@oss.nttdata.co.jp>]

+1 (non-binding) for 2.4.1 release.

 > We can debate whether or June is too early for hadoop-2.5 and it's
 > contents, but I recollect there was consensus that releases every 6
 > weeks (or so) makes sense.

Also, +1 (non-binding) for the 6 weeks (or so) timeline releases.
However, I'm thinking it's more important to remove the critical bugs 
(e.g. test failures) from rc than to release on time.

Thanks,
Akira

(2014/04/14 3:30), Arun C. Murthy wrote:
>
>> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
>>
>> +1 for starting to think about 2.5.  Early June seems a little early to me
>> - we had talked about a quarterly release cadence and this would be about
>> half that.
>
> We can debate whether or June is too early for hadoop-2.5 and it's contents, but I recollect there was consensus that releases every 6 weeks (or so) makes sense. Else, with lesser frequency, we'll see a mad rush for everything to get into *the* quarterly release. As an example: hadoop-2.3 helped reduce pressure on contents in hadoop-2.4.
>
> In my experience, even if we do target 6 weeks it will be closer to ~8 weeks before we can *corral* a release. OTOH, a 12 week cycle would result in a release every 4-5 months... this is why the ASF encourages projects/PMCs to make frequent releases.
>
> Arun
>
>>
>> I'm having trouble editing the wiki, but I think Timeline Server stability
>> (e.g. security and locking down APIs) should go on that list.  I think we
>> should probably take YARN-1404 off the list - even with 3 months it's
>> unlikely to be complete.
>>
>>
>> On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
>>
>>> +1
>>>
>>> The proposed content for 2.5 in the roadmap wiki looks good to me.
>>>> On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
>>>>
>>>> Gang,
>>>>
>>>> With hadoop-2.4 out, it's time to think ahead.
>>>>
>>>> In the short-term hadoop-2.4.1 is in order; particularly with
>>>> https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
>>>> @Private API, unfortunately something Hive is using - sigh!). There are
>>>> some other fixes which testing has uncovered; so it will be nice to pull
>>>> them them in. I'm thinking of an RC by end of the coming week -
>>> committers,
>>>> please be *very* conservative when getting stuff into 2.4.1 (i.e. merging
>>>> to branch-2.4).
>>>>
>>>> Next up, hadoop-2.5.
>>>>
>>>> I've updated https://wiki.apache.org/hadoop/Roadmap with some
>>> candidates
>>>> for consideration - please chime in and say 'aye'/'nay' or add new
>>> content.
>>>> IAC, I suspect that list is too large.
>>>>
>>>> Rather than wait for everything it would be better to plan on releasing
>>>> it on a time-bound manner; particularly around the Hadoop Summit. If that
>>>> makes sense; I think we should target branching for 2.5 by mid-May to get
>>>> it stable and released by early June.
>>>>
>>>> Thoughts?
>>>>
>>>> thanks,
>>>> Arun
>>>>
>>>>
>>>> --
>>>> Arun C. Murthy
>>>> Hortonworks Inc.
>>>> http://hortonworks.com/
>>>>
>>>>
>>>>
>>>> --
>>>> CONFIDENTIALITY NOTICE
>>>> NOTICE: This message is intended for the use of the individual or entity
>>> to
>>>> which it is addressed and may contain information that is confidential,
>>>> privileged and exempt from disclosure under applicable law. If the reader
>>>> of this message is not the intended recipient, you are hereby notified
>>> that
>>>> any printing, copying, dissemination, distribution, disclosure or
>>>> forwarding of this communication is strictly prohibited. If you have
>>>> received this communication in error, please contact the sender
>>> immediately
>>>> and delete it from your system. Thank You.
>>>
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>>
>

Re: Thinking ahead

[Akira AJISAKA <aj...@oss.nttdata.co.jp>]

+1 (non-binding) for 2.4.1 release.

 > We can debate whether or June is too early for hadoop-2.5 and it's
 > contents, but I recollect there was consensus that releases every 6
 > weeks (or so) makes sense.

Also, +1 (non-binding) for the 6 weeks (or so) timeline releases.
However, I'm thinking it's more important to remove the critical bugs 
(e.g. test failures) from rc than to release on time.

Thanks,
Akira

(2014/04/14 3:30), Arun C. Murthy wrote:
>
>> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
>>
>> +1 for starting to think about 2.5.  Early June seems a little early to me
>> - we had talked about a quarterly release cadence and this would be about
>> half that.
>
> We can debate whether or June is too early for hadoop-2.5 and it's contents, but I recollect there was consensus that releases every 6 weeks (or so) makes sense. Else, with lesser frequency, we'll see a mad rush for everything to get into *the* quarterly release. As an example: hadoop-2.3 helped reduce pressure on contents in hadoop-2.4.
>
> In my experience, even if we do target 6 weeks it will be closer to ~8 weeks before we can *corral* a release. OTOH, a 12 week cycle would result in a release every 4-5 months... this is why the ASF encourages projects/PMCs to make frequent releases.
>
> Arun
>
>>
>> I'm having trouble editing the wiki, but I think Timeline Server stability
>> (e.g. security and locking down APIs) should go on that list.  I think we
>> should probably take YARN-1404 off the list - even with 3 months it's
>> unlikely to be complete.
>>
>>
>> On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
>>
>>> +1
>>>
>>> The proposed content for 2.5 in the roadmap wiki looks good to me.
>>>> On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
>>>>
>>>> Gang,
>>>>
>>>> With hadoop-2.4 out, it's time to think ahead.
>>>>
>>>> In the short-term hadoop-2.4.1 is in order; particularly with
>>>> https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
>>>> @Private API, unfortunately something Hive is using - sigh!). There are
>>>> some other fixes which testing has uncovered; so it will be nice to pull
>>>> them them in. I'm thinking of an RC by end of the coming week -
>>> committers,
>>>> please be *very* conservative when getting stuff into 2.4.1 (i.e. merging
>>>> to branch-2.4).
>>>>
>>>> Next up, hadoop-2.5.
>>>>
>>>> I've updated https://wiki.apache.org/hadoop/Roadmap with some
>>> candidates
>>>> for consideration - please chime in and say 'aye'/'nay' or add new
>>> content.
>>>> IAC, I suspect that list is too large.
>>>>
>>>> Rather than wait for everything it would be better to plan on releasing
>>>> it on a time-bound manner; particularly around the Hadoop Summit. If that
>>>> makes sense; I think we should target branching for 2.5 by mid-May to get
>>>> it stable and released by early June.
>>>>
>>>> Thoughts?
>>>>
>>>> thanks,
>>>> Arun
>>>>
>>>>
>>>> --
>>>> Arun C. Murthy
>>>> Hortonworks Inc.
>>>> http://hortonworks.com/
>>>>
>>>>
>>>>
>>>> --
>>>> CONFIDENTIALITY NOTICE
>>>> NOTICE: This message is intended for the use of the individual or entity
>>> to
>>>> which it is addressed and may contain information that is confidential,
>>>> privileged and exempt from disclosure under applicable law. If the reader
>>>> of this message is not the intended recipient, you are hereby notified
>>> that
>>>> any printing, copying, dissemination, distribution, disclosure or
>>>> forwarding of this communication is strictly prohibited. If you have
>>>> received this communication in error, please contact the sender
>>> immediately
>>>> and delete it from your system. Thank You.
>>>
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>>
>

Re: Thinking ahead

[Akira AJISAKA <aj...@oss.nttdata.co.jp>]

+1 (non-binding) for 2.4.1 release.

 > We can debate whether or June is too early for hadoop-2.5 and it's
 > contents, but I recollect there was consensus that releases every 6
 > weeks (or so) makes sense.

Also, +1 (non-binding) for the 6 weeks (or so) timeline releases.
However, I'm thinking it's more important to remove the critical bugs 
(e.g. test failures) from rc than to release on time.

Thanks,
Akira

(2014/04/14 3:30), Arun C. Murthy wrote:
>
>> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
>>
>> +1 for starting to think about 2.5.  Early June seems a little early to me
>> - we had talked about a quarterly release cadence and this would be about
>> half that.
>
> We can debate whether or June is too early for hadoop-2.5 and it's contents, but I recollect there was consensus that releases every 6 weeks (or so) makes sense. Else, with lesser frequency, we'll see a mad rush for everything to get into *the* quarterly release. As an example: hadoop-2.3 helped reduce pressure on contents in hadoop-2.4.
>
> In my experience, even if we do target 6 weeks it will be closer to ~8 weeks before we can *corral* a release. OTOH, a 12 week cycle would result in a release every 4-5 months... this is why the ASF encourages projects/PMCs to make frequent releases.
>
> Arun
>
>>
>> I'm having trouble editing the wiki, but I think Timeline Server stability
>> (e.g. security and locking down APIs) should go on that list.  I think we
>> should probably take YARN-1404 off the list - even with 3 months it's
>> unlikely to be complete.
>>
>>
>> On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
>>
>>> +1
>>>
>>> The proposed content for 2.5 in the roadmap wiki looks good to me.
>>>> On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
>>>>
>>>> Gang,
>>>>
>>>> With hadoop-2.4 out, it's time to think ahead.
>>>>
>>>> In the short-term hadoop-2.4.1 is in order; particularly with
>>>> https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
>>>> @Private API, unfortunately something Hive is using - sigh!). There are
>>>> some other fixes which testing has uncovered; so it will be nice to pull
>>>> them them in. I'm thinking of an RC by end of the coming week -
>>> committers,
>>>> please be *very* conservative when getting stuff into 2.4.1 (i.e. merging
>>>> to branch-2.4).
>>>>
>>>> Next up, hadoop-2.5.
>>>>
>>>> I've updated https://wiki.apache.org/hadoop/Roadmap with some
>>> candidates
>>>> for consideration - please chime in and say 'aye'/'nay' or add new
>>> content.
>>>> IAC, I suspect that list is too large.
>>>>
>>>> Rather than wait for everything it would be better to plan on releasing
>>>> it on a time-bound manner; particularly around the Hadoop Summit. If that
>>>> makes sense; I think we should target branching for 2.5 by mid-May to get
>>>> it stable and released by early June.
>>>>
>>>> Thoughts?
>>>>
>>>> thanks,
>>>> Arun
>>>>
>>>>
>>>> --
>>>> Arun C. Murthy
>>>> Hortonworks Inc.
>>>> http://hortonworks.com/
>>>>
>>>>
>>>>
>>>> --
>>>> CONFIDENTIALITY NOTICE
>>>> NOTICE: This message is intended for the use of the individual or entity
>>> to
>>>> which it is addressed and may contain information that is confidential,
>>>> privileged and exempt from disclosure under applicable law. If the reader
>>>> of this message is not the intended recipient, you are hereby notified
>>> that
>>>> any printing, copying, dissemination, distribution, disclosure or
>>>> forwarding of this communication is strictly prohibited. If you have
>>>> received this communication in error, please contact the sender
>>> immediately
>>>> and delete it from your system. Thank You.
>>>
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>>
>

Re: Thinking ahead

[Akira AJISAKA <aj...@oss.nttdata.co.jp>]

+1 (non-binding) for 2.4.1 release.

 > We can debate whether or June is too early for hadoop-2.5 and it's
 > contents, but I recollect there was consensus that releases every 6
 > weeks (or so) makes sense.

Also, +1 (non-binding) for the 6 weeks (or so) timeline releases.
However, I'm thinking it's more important to remove the critical bugs 
(e.g. test failures) from rc than to release on time.

Thanks,
Akira

(2014/04/14 3:30), Arun C. Murthy wrote:
>
>> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
>>
>> +1 for starting to think about 2.5.  Early June seems a little early to me
>> - we had talked about a quarterly release cadence and this would be about
>> half that.
>
> We can debate whether or June is too early for hadoop-2.5 and it's contents, but I recollect there was consensus that releases every 6 weeks (or so) makes sense. Else, with lesser frequency, we'll see a mad rush for everything to get into *the* quarterly release. As an example: hadoop-2.3 helped reduce pressure on contents in hadoop-2.4.
>
> In my experience, even if we do target 6 weeks it will be closer to ~8 weeks before we can *corral* a release. OTOH, a 12 week cycle would result in a release every 4-5 months... this is why the ASF encourages projects/PMCs to make frequent releases.
>
> Arun
>
>>
>> I'm having trouble editing the wiki, but I think Timeline Server stability
>> (e.g. security and locking down APIs) should go on that list.  I think we
>> should probably take YARN-1404 off the list - even with 3 months it's
>> unlikely to be complete.
>>
>>
>> On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
>>
>>> +1
>>>
>>> The proposed content for 2.5 in the roadmap wiki looks good to me.
>>>> On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
>>>>
>>>> Gang,
>>>>
>>>> With hadoop-2.4 out, it's time to think ahead.
>>>>
>>>> In the short-term hadoop-2.4.1 is in order; particularly with
>>>> https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
>>>> @Private API, unfortunately something Hive is using - sigh!). There are
>>>> some other fixes which testing has uncovered; so it will be nice to pull
>>>> them them in. I'm thinking of an RC by end of the coming week -
>>> committers,
>>>> please be *very* conservative when getting stuff into 2.4.1 (i.e. merging
>>>> to branch-2.4).
>>>>
>>>> Next up, hadoop-2.5.
>>>>
>>>> I've updated https://wiki.apache.org/hadoop/Roadmap with some
>>> candidates
>>>> for consideration - please chime in and say 'aye'/'nay' or add new
>>> content.
>>>> IAC, I suspect that list is too large.
>>>>
>>>> Rather than wait for everything it would be better to plan on releasing
>>>> it on a time-bound manner; particularly around the Hadoop Summit. If that
>>>> makes sense; I think we should target branching for 2.5 by mid-May to get
>>>> it stable and released by early June.
>>>>
>>>> Thoughts?
>>>>
>>>> thanks,
>>>> Arun
>>>>
>>>>
>>>> --
>>>> Arun C. Murthy
>>>> Hortonworks Inc.
>>>> http://hortonworks.com/
>>>>
>>>>
>>>>
>>>> --
>>>> CONFIDENTIALITY NOTICE
>>>> NOTICE: This message is intended for the use of the individual or entity
>>> to
>>>> which it is addressed and may contain information that is confidential,
>>>> privileged and exempt from disclosure under applicable law. If the reader
>>>> of this message is not the intended recipient, you are hereby notified
>>> that
>>>> any printing, copying, dissemination, distribution, disclosure or
>>>> forwarding of this communication is strictly prohibited. If you have
>>>> received this communication in error, please contact the sender
>>> immediately
>>>> and delete it from your system. Thank You.
>>>
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender immediately
>>> and delete it from your system. Thank You.
>>>
>

Re: Thinking ahead

["Arun C. Murthy" <ac...@hortonworks.com>]

> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
> 
> +1 for starting to think about 2.5.  Early June seems a little early to me
> - we had talked about a quarterly release cadence and this would be about
> half that.

We can debate whether or June is too early for hadoop-2.5 and it's contents, but I recollect there was consensus that releases every 6 weeks (or so) makes sense. Else, with lesser frequency, we'll see a mad rush for everything to get into *the* quarterly release. As an example: hadoop-2.3 helped reduce pressure on contents in hadoop-2.4.

In my experience, even if we do target 6 weeks it will be closer to ~8 weeks before we can *corral* a release. OTOH, a 12 week cycle would result in a release every 4-5 months... this is why the ASF encourages projects/PMCs to make frequent releases.

Arun

> 
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  I think we
> should probably take YARN-1404 off the list - even with 3 months it's
> unlikely to be complete.
> 
> 
> On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
> 
>> +1
>> 
>> The proposed content for 2.5 in the roadmap wiki looks good to me.
>>> On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
>>> 
>>> Gang,
>>> 
>>> With hadoop-2.4 out, it's time to think ahead.
>>> 
>>> In the short-term hadoop-2.4.1 is in order; particularly with
>>> https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
>>> @Private API, unfortunately something Hive is using - sigh!). There are
>>> some other fixes which testing has uncovered; so it will be nice to pull
>>> them them in. I'm thinking of an RC by end of the coming week -
>> committers,
>>> please be *very* conservative when getting stuff into 2.4.1 (i.e. merging
>>> to branch-2.4).
>>> 
>>> Next up, hadoop-2.5.
>>> 
>>> I've updated https://wiki.apache.org/hadoop/Roadmap with some
>> candidates
>>> for consideration - please chime in and say 'aye'/'nay' or add new
>> content.
>>> IAC, I suspect that list is too large.
>>> 
>>> Rather than wait for everything it would be better to plan on releasing
>>> it on a time-bound manner; particularly around the Hadoop Summit. If that
>>> makes sense; I think we should target branching for 2.5 by mid-May to get
>>> it stable and released by early June.
>>> 
>>> Thoughts?
>>> 
>>> thanks,
>>> Arun
>>> 
>>> 
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>> to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified
>> that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>> immediately
>>> and delete it from your system. Thank You.
>> 
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>> 

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Vinod Kumar Vavilapalli <vi...@apache.org>]

I'm still wading through 2.4.1 critical issues (failing tests, blockers
etc) so haven't gotten time to think about the 2.5 list w.r.t YARN. Will do
so in the next week.

BTW, folks, please refrain from adding items to the list that you
yourselves are not volunteering to work on. Anyone keen to see a particular
feature should volunteer to that effect. We don't 'assign' work.

+Vinod

On Sat, Apr 12, 2014 at 10:09 AM, Sandy Ryza <sa...@cloudera.com>wrote:

>
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  I think we
> should probably take YARN-1404 off the list - even with 3 months it's
> unlikely to be complete.
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Sandy Ryza <sa...@cloudera.com>]

Reference:
http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201401.mbox/%3CCAJS6+b4TKuPVqfaoRf_314XvEiVrYQy-VtRYD8+36Tu_Bc51fw@mail.gmail.com%3E

This is of course open source, and I have neither the ability nor the
desire to compel or pressure anybody to work on anything, but I wanted to
point out that the proposal to merge AHS into trunk stated an intent to tie
ends up on security.


On Sun, Apr 13, 2014 at 7:50 PM, Sandy Ryza <sa...@cloudera.com> wrote:

> Unfortunately I don't have the bandwidth to take on ATS security at this
> time.  My (I now understand mistaken) impression that it was being worked
> on, as we had discussed it under the initial criteria for including the AHS
> in a release.  I brought it up as a feature I felt worth waiting for if it
> meant extending the target release date a couple weeks - I of course had no
> intention to volunteer anybody for it.
>
>
> On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com>wrote:
>
>> Sandy,
>>
>> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
>>
>> I'm having trouble editing the wiki, but I think Timeline Server stability
>> (e.g. security and locking down APIs) should go on that list.
>>
>>
>> I'm very glad to see you are very passionate about security for ATS since
>> you've asked about it a few times around here.
>>
>> I just opened https://issues.apache.org/jira/browse/YARN-1935.
>>
>> Would you be willing to volunteer to take this on? Much appreciated.
>>
>> thanks,
>> Arun
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>
>

Re: Thinking ahead

[Sandy Ryza <sa...@cloudera.com>]

Reference:
http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201401.mbox/%3CCAJS6+b4TKuPVqfaoRf_314XvEiVrYQy-VtRYD8+36Tu_Bc51fw@mail.gmail.com%3E

This is of course open source, and I have neither the ability nor the
desire to compel or pressure anybody to work on anything, but I wanted to
point out that the proposal to merge AHS into trunk stated an intent to tie
ends up on security.


On Sun, Apr 13, 2014 at 7:50 PM, Sandy Ryza <sa...@cloudera.com> wrote:

> Unfortunately I don't have the bandwidth to take on ATS security at this
> time.  My (I now understand mistaken) impression that it was being worked
> on, as we had discussed it under the initial criteria for including the AHS
> in a release.  I brought it up as a feature I felt worth waiting for if it
> meant extending the target release date a couple weeks - I of course had no
> intention to volunteer anybody for it.
>
>
> On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com>wrote:
>
>> Sandy,
>>
>> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
>>
>> I'm having trouble editing the wiki, but I think Timeline Server stability
>> (e.g. security and locking down APIs) should go on that list.
>>
>>
>> I'm very glad to see you are very passionate about security for ATS since
>> you've asked about it a few times around here.
>>
>> I just opened https://issues.apache.org/jira/browse/YARN-1935.
>>
>> Would you be willing to volunteer to take this on? Much appreciated.
>>
>> thanks,
>> Arun
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>
>

Re: Thinking ahead

[Sandy Ryza <sa...@cloudera.com>]

Reference:
http://mail-archives.apache.org/mod_mbox/hadoop-common-dev/201401.mbox/%3CCAJS6+b4TKuPVqfaoRf_314XvEiVrYQy-VtRYD8+36Tu_Bc51fw@mail.gmail.com%3E

This is of course open source, and I have neither the ability nor the
desire to compel or pressure anybody to work on anything, but I wanted to
point out that the proposal to merge AHS into trunk stated an intent to tie
ends up on security.


On Sun, Apr 13, 2014 at 7:50 PM, Sandy Ryza <sa...@cloudera.com> wrote:

> Unfortunately I don't have the bandwidth to take on ATS security at this
> time.  My (I now understand mistaken) impression that it was being worked
> on, as we had discussed it under the initial criteria for including the AHS
> in a release.  I brought it up as a feature I felt worth waiting for if it
> meant extending the target release date a couple weeks - I of course had no
> intention to volunteer anybody for it.
>
>
> On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com>wrote:
>
>> Sandy,
>>
>> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
>>
>> I'm having trouble editing the wiki, but I think Timeline Server stability
>> (e.g. security and locking down APIs) should go on that list.
>>
>>
>> I'm very glad to see you are very passionate about security for ATS since
>> you've asked about it a few times around here.
>>
>> I just opened https://issues.apache.org/jira/browse/YARN-1935.
>>
>> Would you be willing to volunteer to take this on? Much appreciated.
>>
>> thanks,
>> Arun
>>
>>
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>>
>
>

Re: Thinking ahead

[Sandy Ryza <sa...@cloudera.com>]

Unfortunately I don't have the bandwidth to take on ATS security at this
time.  My (I now understand mistaken) impression that it was being worked
on, as we had discussed it under the initial criteria for including the AHS
in a release.  I brought it up as a feature I felt worth waiting for if it
meant extending the target release date a couple weeks - I of course had no
intention to volunteer anybody for it.


On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com> wrote:

> Sandy,
>
> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
>
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.
>
>
> I'm very glad to see you are very passionate about security for ATS since
> you've asked about it a few times around here.
>
> I just opened https://issues.apache.org/jira/browse/YARN-1935.
>
> Would you be willing to volunteer to take this on? Much appreciated.
>
> thanks,
> Arun
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: Thinking ahead

[Sandy Ryza <sa...@cloudera.com>]

Unfortunately I don't have the bandwidth to take on ATS security at this
time.  My (I now understand mistaken) impression that it was being worked
on, as we had discussed it under the initial criteria for including the AHS
in a release.  I brought it up as a feature I felt worth waiting for if it
meant extending the target release date a couple weeks - I of course had no
intention to volunteer anybody for it.


On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com> wrote:

> Sandy,
>
> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
>
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.
>
>
> I'm very glad to see you are very passionate about security for ATS since
> you've asked about it a few times around here.
>
> I just opened https://issues.apache.org/jira/browse/YARN-1935.
>
> Would you be willing to volunteer to take this on? Much appreciated.
>
> thanks,
> Arun
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: Thinking ahead

[Tsuyoshi OZAWA <oz...@gmail.com>]

Hi,

+1 for 2.4.1 release(non-binding).
There are some critical issues including YARN-1861 and YARN-1929 about
RM-HA. IMO, we should include the fixes.

One question is whether we can include RM-HA phase 2(YARN-556) in 2.5
by mid-May. It's still work-in-progress.

Thanks,
- Tsuyoshi


On Mon, Apr 14, 2014 at 4:07 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
> On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>>
>> I just opened https://issues.apache.org/jira/browse/YARN-1935.
>
>
> I've just assigned the ticket to me, and opened/linked a number of related
> tickets. I'll work on the security issues of the timeline server.
>
> Thanks,
> Zhijie
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Re: Thinking ahead

[Tsuyoshi OZAWA <oz...@gmail.com>]

Hi,

+1 for 2.4.1 release(non-binding).
There are some critical issues including YARN-1861 and YARN-1929 about
RM-HA. IMO, we should include the fixes.

One question is whether we can include RM-HA phase 2(YARN-556) in 2.5
by mid-May. It's still work-in-progress.

Thanks,
- Tsuyoshi


On Mon, Apr 14, 2014 at 4:07 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
> On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>>
>> I just opened https://issues.apache.org/jira/browse/YARN-1935.
>
>
> I've just assigned the ticket to me, and opened/linked a number of related
> tickets. I'll work on the security issues of the timeline server.
>
> Thanks,
> Zhijie
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Re: Thinking ahead

[Tsuyoshi OZAWA <oz...@gmail.com>]

Hi,

+1 for 2.4.1 release(non-binding).
There are some critical issues including YARN-1861 and YARN-1929 about
RM-HA. IMO, we should include the fixes.

One question is whether we can include RM-HA phase 2(YARN-556) in 2.5
by mid-May. It's still work-in-progress.

Thanks,
- Tsuyoshi


On Mon, Apr 14, 2014 at 4:07 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
> On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>>
>> I just opened https://issues.apache.org/jira/browse/YARN-1935.
>
>
> I've just assigned the ticket to me, and opened/linked a number of related
> tickets. I'll work on the security issues of the timeline server.
>
> Thanks,
> Zhijie
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Re: Thinking ahead

[Tsuyoshi OZAWA <oz...@gmail.com>]

Hi,

+1 for 2.4.1 release(non-binding).
There are some critical issues including YARN-1861 and YARN-1929 about
RM-HA. IMO, we should include the fixes.

One question is whether we can include RM-HA phase 2(YARN-556) in 2.5
by mid-May. It's still work-in-progress.

Thanks,
- Tsuyoshi


On Mon, Apr 14, 2014 at 4:07 PM, Zhijie Shen <zs...@hortonworks.com> wrote:
> On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>>
>> I just opened https://issues.apache.org/jira/browse/YARN-1935.
>
>
> I've just assigned the ticket to me, and opened/linked a number of related
> tickets. I'll work on the security issues of the timeline server.
>
> Thanks,
> Zhijie
>
> --
> Zhijie Shen
> Hortonworks Inc.
> http://hortonworks.com/
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.

Re: Thinking ahead

[Zhijie Shen <zs...@hortonworks.com>]

On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
> I just opened https://issues.apache.org/jira/browse/YARN-1935.


I've just assigned the ticket to me, and opened/linked a number of related
tickets. I'll work on the security issues of the timeline server.

Thanks,
Zhijie

-- 
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Sandy Ryza <sa...@cloudera.com>]

Unfortunately I don't have the bandwidth to take on ATS security at this
time.  My (I now understand mistaken) impression that it was being worked
on, as we had discussed it under the initial criteria for including the AHS
in a release.  I brought it up as a feature I felt worth waiting for if it
meant extending the target release date a couple weeks - I of course had no
intention to volunteer anybody for it.


On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com> wrote:

> Sandy,
>
> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
>
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.
>
>
> I'm very glad to see you are very passionate about security for ATS since
> you've asked about it a few times around here.
>
> I just opened https://issues.apache.org/jira/browse/YARN-1935.
>
> Would you be willing to volunteer to take this on? Much appreciated.
>
> thanks,
> Arun
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: Thinking ahead

[Zhijie Shen <zs...@hortonworks.com>]

On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
> I just opened https://issues.apache.org/jira/browse/YARN-1935.


I've just assigned the ticket to me, and opened/linked a number of related
tickets. I'll work on the security issues of the timeline server.

Thanks,
Zhijie

-- 
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Zhijie Shen <zs...@hortonworks.com>]

On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
> I just opened https://issues.apache.org/jira/browse/YARN-1935.


I've just assigned the ticket to me, and opened/linked a number of related
tickets. I'll work on the security issues of the timeline server.

Thanks,
Zhijie

-- 
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Zhijie Shen <zs...@hortonworks.com>]

On Sun, Apr 13, 2014 at 5:50 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
> I just opened https://issues.apache.org/jira/browse/YARN-1935.


I've just assigned the ticket to me, and opened/linked a number of related
tickets. I'll work on the security issues of the timeline server.

Thanks,
Zhijie

-- 
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Arun C Murthy <ac...@hortonworks.com>]

Sandy,

On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:

> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  

I'm very glad to see you are very passionate about security for ATS since you've asked about it a few times around here.

I just opened https://issues.apache.org/jira/browse/YARN-1935.

Would you be willing to volunteer to take this on? Much appreciated.

thanks,
Arun


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Arun C Murthy <ac...@hortonworks.com>]

Sandy,

On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:

> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  

I'm very glad to see you are very passionate about security for ATS since you've asked about it a few times around here.

I just opened https://issues.apache.org/jira/browse/YARN-1935.

Would you be willing to volunteer to take this on? Much appreciated.

thanks,
Arun


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Zhijie Shen <zs...@hortonworks.com>]

+1 for Timeline Server stability. In addition to the security, we may also
want to deal with scalability, generic and per-framework services
integration and MR integration.

Any plan about YARN long running services?


On Sat, Apr 12, 2014 at 10:09 AM, Sandy Ryza <sa...@cloudera.com>wrote:

> +1 for starting to think about 2.5.  Early June seems a little early to me
> - we had talked about a quarterly release cadence and this would be about
> half that.
>
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  I think we
> should probably take YARN-1404 off the list - even with 3 months it's
> unlikely to be complete.
>
>
> On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cnauroth@hortonworks.com
> >wrote:
>
> > +1
> >
> > The proposed content for 2.5 in the roadmap wiki looks good to me.
> > On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
> >
> > > Gang,
> > >
> > >  With hadoop-2.4 out, it's time to think ahead.
> > >
> > >  In the short-term hadoop-2.4.1 is in order; particularly with
> > > https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
> > > @Private API, unfortunately something Hive is using - sigh!). There are
> > > some other fixes which testing has uncovered; so it will be nice to
> pull
> > > them them in. I'm thinking of an RC by end of the coming week -
> > committers,
> > > please be *very* conservative when getting stuff into 2.4.1 (i.e.
> merging
> > > to branch-2.4).
> > >
> > >  Next up, hadoop-2.5.
> > >
> > >  I've updated https://wiki.apache.org/hadoop/Roadmap with some
> > candidates
> > > for consideration - please chime in and say 'aye'/'nay' or add new
> > content.
> > > IAC, I suspect that list is too large.
> > >
> > >  Rather than wait for everything it would be better to plan on
> releasing
> > > it on a time-bound manner; particularly around the Hadoop Summit. If
> that
> > > makes sense; I think we should target branching for 2.5 by mid-May to
> get
> > > it stable and released by early June.
> > >
> > >  Thoughts?
> > >
> > > thanks,
> > > Arun
> > >
> > >
> > > --
> > > Arun C. Murthy
> > > Hortonworks Inc.
> > > http://hortonworks.com/
> > >
> > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>



-- 
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Vinod Kumar Vavilapalli <vi...@apache.org>]

I'm still wading through 2.4.1 critical issues (failing tests, blockers
etc) so haven't gotten time to think about the 2.5 list w.r.t YARN. Will do
so in the next week.

BTW, folks, please refrain from adding items to the list that you
yourselves are not volunteering to work on. Anyone keen to see a particular
feature should volunteer to that effect. We don't 'assign' work.

+Vinod

On Sat, Apr 12, 2014 at 10:09 AM, Sandy Ryza <sa...@cloudera.com>wrote:

>
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  I think we
> should probably take YARN-1404 off the list - even with 3 months it's
> unlikely to be complete.
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Arun C Murthy <ac...@hortonworks.com>]

Sandy,

On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:

> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  

I'm very glad to see you are very passionate about security for ATS since you've asked about it a few times around here.

I just opened https://issues.apache.org/jira/browse/YARN-1935.

Would you be willing to volunteer to take this on? Much appreciated.

thanks,
Arun


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Arun C Murthy <ac...@hortonworks.com>]

Sandy,

On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:

> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  

I'm very glad to see you are very passionate about security for ATS since you've asked about it a few times around here.

I just opened https://issues.apache.org/jira/browse/YARN-1935.

Would you be willing to volunteer to take this on? Much appreciated.

thanks,
Arun


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

["Arun C. Murthy" <ac...@hortonworks.com>]

> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
> 
> +1 for starting to think about 2.5.  Early June seems a little early to me
> - we had talked about a quarterly release cadence and this would be about
> half that.

We can debate whether or June is too early for hadoop-2.5 and it's contents, but I recollect there was consensus that releases every 6 weeks (or so) makes sense. Else, with lesser frequency, we'll see a mad rush for everything to get into *the* quarterly release. As an example: hadoop-2.3 helped reduce pressure on contents in hadoop-2.4.

In my experience, even if we do target 6 weeks it will be closer to ~8 weeks before we can *corral* a release. OTOH, a 12 week cycle would result in a release every 4-5 months... this is why the ASF encourages projects/PMCs to make frequent releases.

Arun

> 
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  I think we
> should probably take YARN-1404 off the list - even with 3 months it's
> unlikely to be complete.
> 
> 
> On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
> 
>> +1
>> 
>> The proposed content for 2.5 in the roadmap wiki looks good to me.
>>> On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
>>> 
>>> Gang,
>>> 
>>> With hadoop-2.4 out, it's time to think ahead.
>>> 
>>> In the short-term hadoop-2.4.1 is in order; particularly with
>>> https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
>>> @Private API, unfortunately something Hive is using - sigh!). There are
>>> some other fixes which testing has uncovered; so it will be nice to pull
>>> them them in. I'm thinking of an RC by end of the coming week -
>> committers,
>>> please be *very* conservative when getting stuff into 2.4.1 (i.e. merging
>>> to branch-2.4).
>>> 
>>> Next up, hadoop-2.5.
>>> 
>>> I've updated https://wiki.apache.org/hadoop/Roadmap with some
>> candidates
>>> for consideration - please chime in and say 'aye'/'nay' or add new
>> content.
>>> IAC, I suspect that list is too large.
>>> 
>>> Rather than wait for everything it would be better to plan on releasing
>>> it on a time-bound manner; particularly around the Hadoop Summit. If that
>>> makes sense; I think we should target branching for 2.5 by mid-May to get
>>> it stable and released by early June.
>>> 
>>> Thoughts?
>>> 
>>> thanks,
>>> Arun
>>> 
>>> 
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>> to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified
>> that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>> immediately
>>> and delete it from your system. Thank You.
>> 
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>> 

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

["Arun C. Murthy" <ac...@hortonworks.com>]

> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
> 
> +1 for starting to think about 2.5.  Early June seems a little early to me
> - we had talked about a quarterly release cadence and this would be about
> half that.

We can debate whether or June is too early for hadoop-2.5 and it's contents, but I recollect there was consensus that releases every 6 weeks (or so) makes sense. Else, with lesser frequency, we'll see a mad rush for everything to get into *the* quarterly release. As an example: hadoop-2.3 helped reduce pressure on contents in hadoop-2.4.

In my experience, even if we do target 6 weeks it will be closer to ~8 weeks before we can *corral* a release. OTOH, a 12 week cycle would result in a release every 4-5 months... this is why the ASF encourages projects/PMCs to make frequent releases.

Arun

> 
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  I think we
> should probably take YARN-1404 off the list - even with 3 months it's
> unlikely to be complete.
> 
> 
> On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
> 
>> +1
>> 
>> The proposed content for 2.5 in the roadmap wiki looks good to me.
>>> On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
>>> 
>>> Gang,
>>> 
>>> With hadoop-2.4 out, it's time to think ahead.
>>> 
>>> In the short-term hadoop-2.4.1 is in order; particularly with
>>> https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
>>> @Private API, unfortunately something Hive is using - sigh!). There are
>>> some other fixes which testing has uncovered; so it will be nice to pull
>>> them them in. I'm thinking of an RC by end of the coming week -
>> committers,
>>> please be *very* conservative when getting stuff into 2.4.1 (i.e. merging
>>> to branch-2.4).
>>> 
>>> Next up, hadoop-2.5.
>>> 
>>> I've updated https://wiki.apache.org/hadoop/Roadmap with some
>> candidates
>>> for consideration - please chime in and say 'aye'/'nay' or add new
>> content.
>>> IAC, I suspect that list is too large.
>>> 
>>> Rather than wait for everything it would be better to plan on releasing
>>> it on a time-bound manner; particularly around the Hadoop Summit. If that
>>> makes sense; I think we should target branching for 2.5 by mid-May to get
>>> it stable and released by early June.
>>> 
>>> Thoughts?
>>> 
>>> thanks,
>>> Arun
>>> 
>>> 
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>> to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified
>> that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>> immediately
>>> and delete it from your system. Thank You.
>> 
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>> 

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

["Arun C. Murthy" <ac...@hortonworks.com>]

> On Apr 12, 2014, at 10:09 AM, Sandy Ryza <sa...@cloudera.com> wrote:
> 
> +1 for starting to think about 2.5.  Early June seems a little early to me
> - we had talked about a quarterly release cadence and this would be about
> half that.

We can debate whether or June is too early for hadoop-2.5 and it's contents, but I recollect there was consensus that releases every 6 weeks (or so) makes sense. Else, with lesser frequency, we'll see a mad rush for everything to get into *the* quarterly release. As an example: hadoop-2.3 helped reduce pressure on contents in hadoop-2.4.

In my experience, even if we do target 6 weeks it will be closer to ~8 weeks before we can *corral* a release. OTOH, a 12 week cycle would result in a release every 4-5 months... this is why the ASF encourages projects/PMCs to make frequent releases.

Arun

> 
> I'm having trouble editing the wiki, but I think Timeline Server stability
> (e.g. security and locking down APIs) should go on that list.  I think we
> should probably take YARN-1404 off the list - even with 3 months it's
> unlikely to be complete.
> 
> 
> On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cn...@hortonworks.com>wrote:
> 
>> +1
>> 
>> The proposed content for 2.5 in the roadmap wiki looks good to me.
>>> On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
>>> 
>>> Gang,
>>> 
>>> With hadoop-2.4 out, it's time to think ahead.
>>> 
>>> In the short-term hadoop-2.4.1 is in order; particularly with
>>> https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
>>> @Private API, unfortunately something Hive is using - sigh!). There are
>>> some other fixes which testing has uncovered; so it will be nice to pull
>>> them them in. I'm thinking of an RC by end of the coming week -
>> committers,
>>> please be *very* conservative when getting stuff into 2.4.1 (i.e. merging
>>> to branch-2.4).
>>> 
>>> Next up, hadoop-2.5.
>>> 
>>> I've updated https://wiki.apache.org/hadoop/Roadmap with some
>> candidates
>>> for consideration - please chime in and say 'aye'/'nay' or add new
>> content.
>>> IAC, I suspect that list is too large.
>>> 
>>> Rather than wait for everything it would be better to plan on releasing
>>> it on a time-bound manner; particularly around the Hadoop Summit. If that
>>> makes sense; I think we should target branching for 2.5 by mid-May to get
>>> it stable and released by early June.
>>> 
>>> Thoughts?
>>> 
>>> thanks,
>>> Arun
>>> 
>>> 
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>> 
>>> 
>>> 
>>> --
>>> CONFIDENTIALITY NOTICE
>>> NOTICE: This message is intended for the use of the individual or entity
>> to
>>> which it is addressed and may contain information that is confidential,
>>> privileged and exempt from disclosure under applicable law. If the reader
>>> of this message is not the intended recipient, you are hereby notified
>> that
>>> any printing, copying, dissemination, distribution, disclosure or
>>> forwarding of this communication is strictly prohibited. If you have
>>> received this communication in error, please contact the sender
>> immediately
>>> and delete it from your system. Thank You.
>> 
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>> 

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Thinking ahead

[Sandy Ryza <sa...@cloudera.com>]

+1 for starting to think about 2.5.  Early June seems a little early to me
- we had talked about a quarterly release cadence and this would be about
half that.

I'm having trouble editing the wiki, but I think Timeline Server stability
(e.g. security and locking down APIs) should go on that list.  I think we
should probably take YARN-1404 off the list - even with 3 months it's
unlikely to be complete.


On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cn...@hortonworks.com>wrote:

> +1
>
> The proposed content for 2.5 in the roadmap wiki looks good to me.
> On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
>
> > Gang,
> >
> >  With hadoop-2.4 out, it's time to think ahead.
> >
> >  In the short-term hadoop-2.4.1 is in order; particularly with
> > https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
> > @Private API, unfortunately something Hive is using - sigh!). There are
> > some other fixes which testing has uncovered; so it will be nice to pull
> > them them in. I'm thinking of an RC by end of the coming week -
> committers,
> > please be *very* conservative when getting stuff into 2.4.1 (i.e. merging
> > to branch-2.4).
> >
> >  Next up, hadoop-2.5.
> >
> >  I've updated https://wiki.apache.org/hadoop/Roadmap with some
> candidates
> > for consideration - please chime in and say 'aye'/'nay' or add new
> content.
> > IAC, I suspect that list is too large.
> >
> >  Rather than wait for everything it would be better to plan on releasing
> > it on a time-bound manner; particularly around the Hadoop Summit. If that
> > makes sense; I think we should target branching for 2.5 by mid-May to get
> > it stable and released by early June.
> >
> >  Thoughts?
> >
> > thanks,
> > Arun
> >
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: Thinking ahead

[Sandy Ryza <sa...@cloudera.com>]

+1 for starting to think about 2.5.  Early June seems a little early to me
- we had talked about a quarterly release cadence and this would be about
half that.

I'm having trouble editing the wiki, but I think Timeline Server stability
(e.g. security and locking down APIs) should go on that list.  I think we
should probably take YARN-1404 off the list - even with 3 months it's
unlikely to be complete.


On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cn...@hortonworks.com>wrote:

> +1
>
> The proposed content for 2.5 in the roadmap wiki looks good to me.
> On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
>
> > Gang,
> >
> >  With hadoop-2.4 out, it's time to think ahead.
> >
> >  In the short-term hadoop-2.4.1 is in order; particularly with
> > https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
> > @Private API, unfortunately something Hive is using - sigh!). There are
> > some other fixes which testing has uncovered; so it will be nice to pull
> > them them in. I'm thinking of an RC by end of the coming week -
> committers,
> > please be *very* conservative when getting stuff into 2.4.1 (i.e. merging
> > to branch-2.4).
> >
> >  Next up, hadoop-2.5.
> >
> >  I've updated https://wiki.apache.org/hadoop/Roadmap with some
> candidates
> > for consideration - please chime in and say 'aye'/'nay' or add new
> content.
> > IAC, I suspect that list is too large.
> >
> >  Rather than wait for everything it would be better to plan on releasing
> > it on a time-bound manner; particularly around the Hadoop Summit. If that
> > makes sense; I think we should target branching for 2.5 by mid-May to get
> > it stable and released by early June.
> >
> >  Thoughts?
> >
> > thanks,
> > Arun
> >
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: Thinking ahead

[Sandy Ryza <sa...@cloudera.com>]

+1 for starting to think about 2.5.  Early June seems a little early to me
- we had talked about a quarterly release cadence and this would be about
half that.

I'm having trouble editing the wiki, but I think Timeline Server stability
(e.g. security and locking down APIs) should go on that list.  I think we
should probably take YARN-1404 off the list - even with 3 months it's
unlikely to be complete.


On Sat, Apr 12, 2014 at 9:50 AM, Chris Nauroth <cn...@hortonworks.com>wrote:

> +1
>
> The proposed content for 2.5 in the roadmap wiki looks good to me.
> On Apr 12, 2014 7:26 AM, "Arun C Murthy" <ac...@hortonworks.com> wrote:
>
> > Gang,
> >
> >  With hadoop-2.4 out, it's time to think ahead.
> >
> >  In the short-term hadoop-2.4.1 is in order; particularly with
> > https://issues.apache.org/jira/browse/MAPREDUCE-5830 (it's a break to
> > @Private API, unfortunately something Hive is using - sigh!). There are
> > some other fixes which testing has uncovered; so it will be nice to pull
> > them them in. I'm thinking of an RC by end of the coming week -
> committers,
> > please be *very* conservative when getting stuff into 2.4.1 (i.e. merging
> > to branch-2.4).
> >
> >  Next up, hadoop-2.5.
> >
> >  I've updated https://wiki.apache.org/hadoop/Roadmap with some
> candidates
> > for consideration - please chime in and say 'aye'/'nay' or add new
> content.
> > IAC, I suspect that list is too large.
> >
> >  Rather than wait for everything it would be better to plan on releasing
> > it on a time-bound manner; particularly around the Hadoop Summit. If that
> > makes sense; I think we should target branching for 2.5 by mid-May to get
> > it stable and released by early June.
> >
> >  Thoughts?
> >
> > thanks,
> > Arun
> >
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>