You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Arpit Agarwal <aa...@hortonworks.com> on 2014/01/04 01:13:07 UTC
Re: Next releases
We plan to merge HDFS-2832 to branch-2 next week for inclusion in 2.4.
On Fri, Dec 6, 2013 at 1:53 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Thanks Suresh & Colin.
>
> Please update the Roadmap wiki with your proposals.
>
> As always, we will try our best to get these in - but we can collectively
> decide to slip some of these to subsequent releases based on timelines.
>
> Arun
>
> On Dec 6, 2013, at 10:43 AM, Suresh Srinivas <su...@hortonworks.com>
> wrote:
>
> > Arun,
> >
> > I propose the following changes for 2.3:
> > - There have been a lot of improvements related to supporting http
> policy.
> > - There is a still discussion going on, but I would like to deprecate
> > BackupNode in 2.3 as well.
> > - We are currently working on rolling upgrades related change in HDFS. We
> > might add a couple of changes that enables rolling upgrades from 2.3
> > onwards (hopefully we can this done by December)
> >
> > I propose the following for 2.4 release, if they are tested and stable:
> > - Heterogeneous storage support - HDFS-2832
> > - Datanode cache related change - HDFS-4949
> > - HDFS ACLs - HDFS-4685
> > - Rolling upgrade changes
> >
> > Let me know if you want me to update the wiki.
> >
> > Regards,
> > Suresh
> >
>
> On Dec 6, 2013, at 12:27 PM, Colin McCabe <cm...@alumni.cmu.edu> wrote:
>
> > If 2.4 is released in January, I think it's very unlikely to include
> > symlinks. There is still a lot of work to be done before they're
> > usable. You can look at the progress on HADOOP-10019. For some of
> > the subtasks, it will require some community discussion before any
> > code can be written.
> >
> > For better or worse, symlinks have not been requested by users as
> > often as features like NFS export, HDFS caching, ACLs, etc, so effort
> > has been focused on those instead.
> >
> > For now, I think we should put the symlinks-disabling patches
> > (HADOOP-10020, etc) into branch-2, so that they will be part of the
> > next releases without additional effort.
> >
> > I would like to see HDFS caching make it into 2.4. The APIs and
> > implementation are beginning to stabilize, and around January it
> > should be ok to backport to a stable branch.
> >
> > best,
> > Colin
> >
> >
> > On Thu, Nov 7, 2013 at 6:42 PM, Arun C Murthy <ac...@hortonworks.com>
> wrote:
> >
> >> Gang,
> >>
> >> Thinking through the next couple of releases here, appreciate f/b.
> >>
> >> # hadoop-2.2.1
> >>
> >> I was looking through commit logs and there is a *lot* of content here
> >> (81 commits as on 11/7). Some are features/improvements and some are
> fixes
> >> - it's really hard to distinguish what is important and what isn't.
> >>
> >> I propose we start with a blank slate (i.e. blow away branch-2.2 and
> >> start fresh from a copy of branch-2.2.0) and then be very careful and
> >> meticulous about including only *blocker* fixes in branch-2.2. So, most
> of
> >> the content here comes via the next minor release (i.e. hadoop-2.3)
> >>
> >> In future, we continue to be *very* parsimonious about what gets into a
> >> patch release (major.minor.patch) - in general, these should be only
> >> *blocker* fixes or key operational issues.
> >>
> >> # hadoop-2.3
> >>
> >> I'd like to propose the following features for YARN/MR to make it into
> >> hadoop-2.3 and punt the rest to hadoop-2.4 and beyond:
> >> * Application History Server - This is happening in a branch and is
> >> close; with it we can provide a reasonable experience for new frameworks
> >> being built on top of YARN.
> >> * Bug-fixes in RM Restart
> >> * Minimal support for long-running applications (e.g. security) via
> >> YARN-896
> >> * RM Fail-over via ZKFC
> >> * Anything else?
> >>
> >> HDFS???
> >>
> >> Overall, I feel like we have a decent chance of rolling hadoop-2.3 by
> the
> >> end of the year.
> >>
> >> Thoughts?
> >>
> >> thanks,
> >> Arun
> >>
> >>
> >> --
> >> Arun C. Murthy
> >> Hortonworks Inc.
> >> http://hortonworks.com/
> >>
> >>
> >>
> >> --
> >> CONFIDENTIALITY NOTICE
> >> NOTICE: This message is intended for the use of the individual or
> entity to
> >> which it is addressed and may contain information that is confidential,
> >> privileged and exempt from disclosure under applicable law. If the
> reader
> >> of this message is not the intended recipient, you are hereby notified
> that
> >> any printing, copying, dissemination, distribution, disclosure or
> >> forwarding of this communication is strictly prohibited. If you have
> >> received this communication in error, please contact the sender
> immediately
> >> and delete it from your system. Thank You.
> >>
> >
> >
> >
> > --
> > http://hortonworks.com/download/
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Next releases
Posted by Jun Ping Du <jd...@vmware.com>.
Great news! Thanks Arpit!
I think we should update Roadmap wiki to include this work. :)
Thanks,
Junping
----- Original Message -----
From: "Arpit Agarwal" <aa...@hortonworks.com>
To: common-dev@hadoop.apache.org
Cc: yarn-dev@hadoop.apache.org, hdfs-dev@hadoop.apache.org, mapreduce-dev@hadoop.apache.org
Sent: Tuesday, January 7, 2014 8:47:32 AM
Subject: Re: Next releases
This merge to branch-2 is complete. The changes have been merged to
branch-2 and target version set to 2.4.0 (r1556076).
On Fri, Jan 3, 2014 at 4:13 PM, Arpit Agarwal <aa...@hortonworks.com>wrote:
> We plan to merge HDFS-2832 to branch-2 next week for inclusion in 2.4.
>
>
>
> On Fri, Dec 6, 2013 at 1:53 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
>> Thanks Suresh & Colin.
>>
>> Please update the Roadmap wiki with your proposals.
>>
>> As always, we will try our best to get these in - but we can collectively
>> decide to slip some of these to subsequent releases based on timelines.
>>
>> Arun
>>
>> On Dec 6, 2013, at 10:43 AM, Suresh Srinivas <su...@hortonworks.com>
>> wrote:
>>
>> > Arun,
>> >
>> > I propose the following changes for 2.3:
>> > - There have been a lot of improvements related to supporting http
>> policy.
>> > - There is a still discussion going on, but I would like to deprecate
>> > BackupNode in 2.3 as well.
>> > - We are currently working on rolling upgrades related change in HDFS.
>> We
>> > might add a couple of changes that enables rolling upgrades from 2.3
>> > onwards (hopefully we can this done by December)
>> >
>> > I propose the following for 2.4 release, if they are tested and stable:
>> > - Heterogeneous storage support - HDFS-2832
>> > - Datanode cache related change - HDFS-4949
>> > - HDFS ACLs - HDFS-4685
>> > - Rolling upgrade changes
>> >
>> > Let me know if you want me to update the wiki.
>> >
>> > Regards,
>> > Suresh
>> >
>>
>> On Dec 6, 2013, at 12:27 PM, Colin McCabe <cm...@alumni.cmu.edu> wrote:
>>
>> > If 2.4 is released in January, I think it's very unlikely to include
>> > symlinks. There is still a lot of work to be done before they're
>> > usable. You can look at the progress on HADOOP-10019. For some of
>> > the subtasks, it will require some community discussion before any
>> > code can be written.
>> >
>> > For better or worse, symlinks have not been requested by users as
>> > often as features like NFS export, HDFS caching, ACLs, etc, so effort
>> > has been focused on those instead.
>> >
>> > For now, I think we should put the symlinks-disabling patches
>> > (HADOOP-10020, etc) into branch-2, so that they will be part of the
>> > next releases without additional effort.
>> >
>> > I would like to see HDFS caching make it into 2.4. The APIs and
>> > implementation are beginning to stabilize, and around January it
>> > should be ok to backport to a stable branch.
>> >
>> > best,
>> > Colin
>> >
>> >
>> > On Thu, Nov 7, 2013 at 6:42 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:
>> >
>> >> Gang,
>> >>
>> >> Thinking through the next couple of releases here, appreciate f/b.
>> >>
>> >> # hadoop-2.2.1
>> >>
>> >> I was looking through commit logs and there is a *lot* of content here
>> >> (81 commits as on 11/7). Some are features/improvements and some are
>> fixes
>> >> - it's really hard to distinguish what is important and what isn't.
>> >>
>> >> I propose we start with a blank slate (i.e. blow away branch-2.2 and
>> >> start fresh from a copy of branch-2.2.0) and then be very careful and
>> >> meticulous about including only *blocker* fixes in branch-2.2. So,
>> most of
>> >> the content here comes via the next minor release (i.e. hadoop-2.3)
>> >>
>> >> In future, we continue to be *very* parsimonious about what gets into a
>> >> patch release (major.minor.patch) - in general, these should be only
>> >> *blocker* fixes or key operational issues.
>> >>
>> >> # hadoop-2.3
>> >>
>> >> I'd like to propose the following features for YARN/MR to make it into
>> >> hadoop-2.3 and punt the rest to hadoop-2.4 and beyond:
>> >> * Application History Server - This is happening in a branch and is
>> >> close; with it we can provide a reasonable experience for new
>> frameworks
>> >> being built on top of YARN.
>> >> * Bug-fixes in RM Restart
>> >> * Minimal support for long-running applications (e.g. security) via
>> >> YARN-896
>> >> * RM Fail-over via ZKFC
>> >> * Anything else?
>> >>
>> >> HDFS???
>> >>
>> >> Overall, I feel like we have a decent chance of rolling hadoop-2.3 by
>> the
>> >> end of the year.
>> >>
>> >> Thoughts?
>> >>
>> >> thanks,
>> >> Arun
>> >>
>> >>
>> >> --
>> >> Arun C. Murthy
>> >> Hortonworks Inc.
>> >> https://urldefense.proofpoint.com/v1/url?u=http://hortonworks.com/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=mPOfB21VcWnmM3DU81gzph9Quwrh%2BSSBvfQ2CNr5G0g%3D%0A&s=35573f6f0811fd2b7cc2274469110563eca6b20dce4ed5961ae05ad601d552a0
>> >>
>> >>
>> >>
>> >> --
>> >> CONFIDENTIALITY NOTICE
>> >> NOTICE: This message is intended for the use of the individual or
>> entity to
>> >> which it is addressed and may contain information that is confidential,
>> >> privileged and exempt from disclosure under applicable law. If the
>> reader
>> >> of this message is not the intended recipient, you are hereby notified
>> that
>> >> any printing, copying, dissemination, distribution, disclosure or
>> >> forwarding of this communication is strictly prohibited. If you have
>> >> received this communication in error, please contact the sender
>> immediately
>> >> and delete it from your system. Thank You.
>> >>
>> >
>> >
>> >
>> > --
>> > https://urldefense.proofpoint.com/v1/url?u=http://hortonworks.com/download/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=mPOfB21VcWnmM3DU81gzph9Quwrh%2BSSBvfQ2CNr5G0g%3D%0A&s=36bc2d49d53470c0688729dc109b1385a9fc0559599b8dca8169f00fd7e11fd4
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>> entity to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> https://urldefense.proofpoint.com/v1/url?u=http://hortonworks.com/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=mPOfB21VcWnmM3DU81gzph9Quwrh%2BSSBvfQ2CNr5G0g%3D%0A&s=35573f6f0811fd2b7cc2274469110563eca6b20dce4ed5961ae05ad601d552a0
>>
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You
Re: Next releases
Posted by Jun Ping Du <jd...@vmware.com>.
Great news! Thanks Arpit!
I think we should update Roadmap wiki to include this work. :)
Thanks,
Junping
----- Original Message -----
From: "Arpit Agarwal" <aa...@hortonworks.com>
To: common-dev@hadoop.apache.org
Cc: yarn-dev@hadoop.apache.org, hdfs-dev@hadoop.apache.org, mapreduce-dev@hadoop.apache.org
Sent: Tuesday, January 7, 2014 8:47:32 AM
Subject: Re: Next releases
This merge to branch-2 is complete. The changes have been merged to
branch-2 and target version set to 2.4.0 (r1556076).
On Fri, Jan 3, 2014 at 4:13 PM, Arpit Agarwal <aa...@hortonworks.com>wrote:
> We plan to merge HDFS-2832 to branch-2 next week for inclusion in 2.4.
>
>
>
> On Fri, Dec 6, 2013 at 1:53 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
>> Thanks Suresh & Colin.
>>
>> Please update the Roadmap wiki with your proposals.
>>
>> As always, we will try our best to get these in - but we can collectively
>> decide to slip some of these to subsequent releases based on timelines.
>>
>> Arun
>>
>> On Dec 6, 2013, at 10:43 AM, Suresh Srinivas <su...@hortonworks.com>
>> wrote:
>>
>> > Arun,
>> >
>> > I propose the following changes for 2.3:
>> > - There have been a lot of improvements related to supporting http
>> policy.
>> > - There is a still discussion going on, but I would like to deprecate
>> > BackupNode in 2.3 as well.
>> > - We are currently working on rolling upgrades related change in HDFS.
>> We
>> > might add a couple of changes that enables rolling upgrades from 2.3
>> > onwards (hopefully we can this done by December)
>> >
>> > I propose the following for 2.4 release, if they are tested and stable:
>> > - Heterogeneous storage support - HDFS-2832
>> > - Datanode cache related change - HDFS-4949
>> > - HDFS ACLs - HDFS-4685
>> > - Rolling upgrade changes
>> >
>> > Let me know if you want me to update the wiki.
>> >
>> > Regards,
>> > Suresh
>> >
>>
>> On Dec 6, 2013, at 12:27 PM, Colin McCabe <cm...@alumni.cmu.edu> wrote:
>>
>> > If 2.4 is released in January, I think it's very unlikely to include
>> > symlinks. There is still a lot of work to be done before they're
>> > usable. You can look at the progress on HADOOP-10019. For some of
>> > the subtasks, it will require some community discussion before any
>> > code can be written.
>> >
>> > For better or worse, symlinks have not been requested by users as
>> > often as features like NFS export, HDFS caching, ACLs, etc, so effort
>> > has been focused on those instead.
>> >
>> > For now, I think we should put the symlinks-disabling patches
>> > (HADOOP-10020, etc) into branch-2, so that they will be part of the
>> > next releases without additional effort.
>> >
>> > I would like to see HDFS caching make it into 2.4. The APIs and
>> > implementation are beginning to stabilize, and around January it
>> > should be ok to backport to a stable branch.
>> >
>> > best,
>> > Colin
>> >
>> >
>> > On Thu, Nov 7, 2013 at 6:42 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:
>> >
>> >> Gang,
>> >>
>> >> Thinking through the next couple of releases here, appreciate f/b.
>> >>
>> >> # hadoop-2.2.1
>> >>
>> >> I was looking through commit logs and there is a *lot* of content here
>> >> (81 commits as on 11/7). Some are features/improvements and some are
>> fixes
>> >> - it's really hard to distinguish what is important and what isn't.
>> >>
>> >> I propose we start with a blank slate (i.e. blow away branch-2.2 and
>> >> start fresh from a copy of branch-2.2.0) and then be very careful and
>> >> meticulous about including only *blocker* fixes in branch-2.2. So,
>> most of
>> >> the content here comes via the next minor release (i.e. hadoop-2.3)
>> >>
>> >> In future, we continue to be *very* parsimonious about what gets into a
>> >> patch release (major.minor.patch) - in general, these should be only
>> >> *blocker* fixes or key operational issues.
>> >>
>> >> # hadoop-2.3
>> >>
>> >> I'd like to propose the following features for YARN/MR to make it into
>> >> hadoop-2.3 and punt the rest to hadoop-2.4 and beyond:
>> >> * Application History Server - This is happening in a branch and is
>> >> close; with it we can provide a reasonable experience for new
>> frameworks
>> >> being built on top of YARN.
>> >> * Bug-fixes in RM Restart
>> >> * Minimal support for long-running applications (e.g. security) via
>> >> YARN-896
>> >> * RM Fail-over via ZKFC
>> >> * Anything else?
>> >>
>> >> HDFS???
>> >>
>> >> Overall, I feel like we have a decent chance of rolling hadoop-2.3 by
>> the
>> >> end of the year.
>> >>
>> >> Thoughts?
>> >>
>> >> thanks,
>> >> Arun
>> >>
>> >>
>> >> --
>> >> Arun C. Murthy
>> >> Hortonworks Inc.
>> >> https://urldefense.proofpoint.com/v1/url?u=http://hortonworks.com/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=mPOfB21VcWnmM3DU81gzph9Quwrh%2BSSBvfQ2CNr5G0g%3D%0A&s=35573f6f0811fd2b7cc2274469110563eca6b20dce4ed5961ae05ad601d552a0
>> >>
>> >>
>> >>
>> >> --
>> >> CONFIDENTIALITY NOTICE
>> >> NOTICE: This message is intended for the use of the individual or
>> entity to
>> >> which it is addressed and may contain information that is confidential,
>> >> privileged and exempt from disclosure under applicable law. If the
>> reader
>> >> of this message is not the intended recipient, you are hereby notified
>> that
>> >> any printing, copying, dissemination, distribution, disclosure or
>> >> forwarding of this communication is strictly prohibited. If you have
>> >> received this communication in error, please contact the sender
>> immediately
>> >> and delete it from your system. Thank You.
>> >>
>> >
>> >
>> >
>> > --
>> > https://urldefense.proofpoint.com/v1/url?u=http://hortonworks.com/download/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=mPOfB21VcWnmM3DU81gzph9Quwrh%2BSSBvfQ2CNr5G0g%3D%0A&s=36bc2d49d53470c0688729dc109b1385a9fc0559599b8dca8169f00fd7e11fd4
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>> entity to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> https://urldefense.proofpoint.com/v1/url?u=http://hortonworks.com/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=mPOfB21VcWnmM3DU81gzph9Quwrh%2BSSBvfQ2CNr5G0g%3D%0A&s=35573f6f0811fd2b7cc2274469110563eca6b20dce4ed5961ae05ad601d552a0
>>
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You
Re: Next releases
Posted by Jun Ping Du <jd...@vmware.com>.
Great news! Thanks Arpit!
I think we should update Roadmap wiki to include this work. :)
Thanks,
Junping
----- Original Message -----
From: "Arpit Agarwal" <aa...@hortonworks.com>
To: common-dev@hadoop.apache.org
Cc: yarn-dev@hadoop.apache.org, hdfs-dev@hadoop.apache.org, mapreduce-dev@hadoop.apache.org
Sent: Tuesday, January 7, 2014 8:47:32 AM
Subject: Re: Next releases
This merge to branch-2 is complete. The changes have been merged to
branch-2 and target version set to 2.4.0 (r1556076).
On Fri, Jan 3, 2014 at 4:13 PM, Arpit Agarwal <aa...@hortonworks.com>wrote:
> We plan to merge HDFS-2832 to branch-2 next week for inclusion in 2.4.
>
>
>
> On Fri, Dec 6, 2013 at 1:53 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
>> Thanks Suresh & Colin.
>>
>> Please update the Roadmap wiki with your proposals.
>>
>> As always, we will try our best to get these in - but we can collectively
>> decide to slip some of these to subsequent releases based on timelines.
>>
>> Arun
>>
>> On Dec 6, 2013, at 10:43 AM, Suresh Srinivas <su...@hortonworks.com>
>> wrote:
>>
>> > Arun,
>> >
>> > I propose the following changes for 2.3:
>> > - There have been a lot of improvements related to supporting http
>> policy.
>> > - There is a still discussion going on, but I would like to deprecate
>> > BackupNode in 2.3 as well.
>> > - We are currently working on rolling upgrades related change in HDFS.
>> We
>> > might add a couple of changes that enables rolling upgrades from 2.3
>> > onwards (hopefully we can this done by December)
>> >
>> > I propose the following for 2.4 release, if they are tested and stable:
>> > - Heterogeneous storage support - HDFS-2832
>> > - Datanode cache related change - HDFS-4949
>> > - HDFS ACLs - HDFS-4685
>> > - Rolling upgrade changes
>> >
>> > Let me know if you want me to update the wiki.
>> >
>> > Regards,
>> > Suresh
>> >
>>
>> On Dec 6, 2013, at 12:27 PM, Colin McCabe <cm...@alumni.cmu.edu> wrote:
>>
>> > If 2.4 is released in January, I think it's very unlikely to include
>> > symlinks. There is still a lot of work to be done before they're
>> > usable. You can look at the progress on HADOOP-10019. For some of
>> > the subtasks, it will require some community discussion before any
>> > code can be written.
>> >
>> > For better or worse, symlinks have not been requested by users as
>> > often as features like NFS export, HDFS caching, ACLs, etc, so effort
>> > has been focused on those instead.
>> >
>> > For now, I think we should put the symlinks-disabling patches
>> > (HADOOP-10020, etc) into branch-2, so that they will be part of the
>> > next releases without additional effort.
>> >
>> > I would like to see HDFS caching make it into 2.4. The APIs and
>> > implementation are beginning to stabilize, and around January it
>> > should be ok to backport to a stable branch.
>> >
>> > best,
>> > Colin
>> >
>> >
>> > On Thu, Nov 7, 2013 at 6:42 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:
>> >
>> >> Gang,
>> >>
>> >> Thinking through the next couple of releases here, appreciate f/b.
>> >>
>> >> # hadoop-2.2.1
>> >>
>> >> I was looking through commit logs and there is a *lot* of content here
>> >> (81 commits as on 11/7). Some are features/improvements and some are
>> fixes
>> >> - it's really hard to distinguish what is important and what isn't.
>> >>
>> >> I propose we start with a blank slate (i.e. blow away branch-2.2 and
>> >> start fresh from a copy of branch-2.2.0) and then be very careful and
>> >> meticulous about including only *blocker* fixes in branch-2.2. So,
>> most of
>> >> the content here comes via the next minor release (i.e. hadoop-2.3)
>> >>
>> >> In future, we continue to be *very* parsimonious about what gets into a
>> >> patch release (major.minor.patch) - in general, these should be only
>> >> *blocker* fixes or key operational issues.
>> >>
>> >> # hadoop-2.3
>> >>
>> >> I'd like to propose the following features for YARN/MR to make it into
>> >> hadoop-2.3 and punt the rest to hadoop-2.4 and beyond:
>> >> * Application History Server - This is happening in a branch and is
>> >> close; with it we can provide a reasonable experience for new
>> frameworks
>> >> being built on top of YARN.
>> >> * Bug-fixes in RM Restart
>> >> * Minimal support for long-running applications (e.g. security) via
>> >> YARN-896
>> >> * RM Fail-over via ZKFC
>> >> * Anything else?
>> >>
>> >> HDFS???
>> >>
>> >> Overall, I feel like we have a decent chance of rolling hadoop-2.3 by
>> the
>> >> end of the year.
>> >>
>> >> Thoughts?
>> >>
>> >> thanks,
>> >> Arun
>> >>
>> >>
>> >> --
>> >> Arun C. Murthy
>> >> Hortonworks Inc.
>> >> https://urldefense.proofpoint.com/v1/url?u=http://hortonworks.com/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=mPOfB21VcWnmM3DU81gzph9Quwrh%2BSSBvfQ2CNr5G0g%3D%0A&s=35573f6f0811fd2b7cc2274469110563eca6b20dce4ed5961ae05ad601d552a0
>> >>
>> >>
>> >>
>> >> --
>> >> CONFIDENTIALITY NOTICE
>> >> NOTICE: This message is intended for the use of the individual or
>> entity to
>> >> which it is addressed and may contain information that is confidential,
>> >> privileged and exempt from disclosure under applicable law. If the
>> reader
>> >> of this message is not the intended recipient, you are hereby notified
>> that
>> >> any printing, copying, dissemination, distribution, disclosure or
>> >> forwarding of this communication is strictly prohibited. If you have
>> >> received this communication in error, please contact the sender
>> immediately
>> >> and delete it from your system. Thank You.
>> >>
>> >
>> >
>> >
>> > --
>> > https://urldefense.proofpoint.com/v1/url?u=http://hortonworks.com/download/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=mPOfB21VcWnmM3DU81gzph9Quwrh%2BSSBvfQ2CNr5G0g%3D%0A&s=36bc2d49d53470c0688729dc109b1385a9fc0559599b8dca8169f00fd7e11fd4
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>> entity to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> https://urldefense.proofpoint.com/v1/url?u=http://hortonworks.com/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=mPOfB21VcWnmM3DU81gzph9Quwrh%2BSSBvfQ2CNr5G0g%3D%0A&s=35573f6f0811fd2b7cc2274469110563eca6b20dce4ed5961ae05ad601d552a0
>>
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You
Re: Next releases
Posted by Jun Ping Du <jd...@vmware.com>.
Great news! Thanks Arpit!
I think we should update Roadmap wiki to include this work. :)
Thanks,
Junping
----- Original Message -----
From: "Arpit Agarwal" <aa...@hortonworks.com>
To: common-dev@hadoop.apache.org
Cc: yarn-dev@hadoop.apache.org, hdfs-dev@hadoop.apache.org, mapreduce-dev@hadoop.apache.org
Sent: Tuesday, January 7, 2014 8:47:32 AM
Subject: Re: Next releases
This merge to branch-2 is complete. The changes have been merged to
branch-2 and target version set to 2.4.0 (r1556076).
On Fri, Jan 3, 2014 at 4:13 PM, Arpit Agarwal <aa...@hortonworks.com>wrote:
> We plan to merge HDFS-2832 to branch-2 next week for inclusion in 2.4.
>
>
>
> On Fri, Dec 6, 2013 at 1:53 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
>> Thanks Suresh & Colin.
>>
>> Please update the Roadmap wiki with your proposals.
>>
>> As always, we will try our best to get these in - but we can collectively
>> decide to slip some of these to subsequent releases based on timelines.
>>
>> Arun
>>
>> On Dec 6, 2013, at 10:43 AM, Suresh Srinivas <su...@hortonworks.com>
>> wrote:
>>
>> > Arun,
>> >
>> > I propose the following changes for 2.3:
>> > - There have been a lot of improvements related to supporting http
>> policy.
>> > - There is a still discussion going on, but I would like to deprecate
>> > BackupNode in 2.3 as well.
>> > - We are currently working on rolling upgrades related change in HDFS.
>> We
>> > might add a couple of changes that enables rolling upgrades from 2.3
>> > onwards (hopefully we can this done by December)
>> >
>> > I propose the following for 2.4 release, if they are tested and stable:
>> > - Heterogeneous storage support - HDFS-2832
>> > - Datanode cache related change - HDFS-4949
>> > - HDFS ACLs - HDFS-4685
>> > - Rolling upgrade changes
>> >
>> > Let me know if you want me to update the wiki.
>> >
>> > Regards,
>> > Suresh
>> >
>>
>> On Dec 6, 2013, at 12:27 PM, Colin McCabe <cm...@alumni.cmu.edu> wrote:
>>
>> > If 2.4 is released in January, I think it's very unlikely to include
>> > symlinks. There is still a lot of work to be done before they're
>> > usable. You can look at the progress on HADOOP-10019. For some of
>> > the subtasks, it will require some community discussion before any
>> > code can be written.
>> >
>> > For better or worse, symlinks have not been requested by users as
>> > often as features like NFS export, HDFS caching, ACLs, etc, so effort
>> > has been focused on those instead.
>> >
>> > For now, I think we should put the symlinks-disabling patches
>> > (HADOOP-10020, etc) into branch-2, so that they will be part of the
>> > next releases without additional effort.
>> >
>> > I would like to see HDFS caching make it into 2.4. The APIs and
>> > implementation are beginning to stabilize, and around January it
>> > should be ok to backport to a stable branch.
>> >
>> > best,
>> > Colin
>> >
>> >
>> > On Thu, Nov 7, 2013 at 6:42 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:
>> >
>> >> Gang,
>> >>
>> >> Thinking through the next couple of releases here, appreciate f/b.
>> >>
>> >> # hadoop-2.2.1
>> >>
>> >> I was looking through commit logs and there is a *lot* of content here
>> >> (81 commits as on 11/7). Some are features/improvements and some are
>> fixes
>> >> - it's really hard to distinguish what is important and what isn't.
>> >>
>> >> I propose we start with a blank slate (i.e. blow away branch-2.2 and
>> >> start fresh from a copy of branch-2.2.0) and then be very careful and
>> >> meticulous about including only *blocker* fixes in branch-2.2. So,
>> most of
>> >> the content here comes via the next minor release (i.e. hadoop-2.3)
>> >>
>> >> In future, we continue to be *very* parsimonious about what gets into a
>> >> patch release (major.minor.patch) - in general, these should be only
>> >> *blocker* fixes or key operational issues.
>> >>
>> >> # hadoop-2.3
>> >>
>> >> I'd like to propose the following features for YARN/MR to make it into
>> >> hadoop-2.3 and punt the rest to hadoop-2.4 and beyond:
>> >> * Application History Server - This is happening in a branch and is
>> >> close; with it we can provide a reasonable experience for new
>> frameworks
>> >> being built on top of YARN.
>> >> * Bug-fixes in RM Restart
>> >> * Minimal support for long-running applications (e.g. security) via
>> >> YARN-896
>> >> * RM Fail-over via ZKFC
>> >> * Anything else?
>> >>
>> >> HDFS???
>> >>
>> >> Overall, I feel like we have a decent chance of rolling hadoop-2.3 by
>> the
>> >> end of the year.
>> >>
>> >> Thoughts?
>> >>
>> >> thanks,
>> >> Arun
>> >>
>> >>
>> >> --
>> >> Arun C. Murthy
>> >> Hortonworks Inc.
>> >> https://urldefense.proofpoint.com/v1/url?u=http://hortonworks.com/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=mPOfB21VcWnmM3DU81gzph9Quwrh%2BSSBvfQ2CNr5G0g%3D%0A&s=35573f6f0811fd2b7cc2274469110563eca6b20dce4ed5961ae05ad601d552a0
>> >>
>> >>
>> >>
>> >> --
>> >> CONFIDENTIALITY NOTICE
>> >> NOTICE: This message is intended for the use of the individual or
>> entity to
>> >> which it is addressed and may contain information that is confidential,
>> >> privileged and exempt from disclosure under applicable law. If the
>> reader
>> >> of this message is not the intended recipient, you are hereby notified
>> that
>> >> any printing, copying, dissemination, distribution, disclosure or
>> >> forwarding of this communication is strictly prohibited. If you have
>> >> received this communication in error, please contact the sender
>> immediately
>> >> and delete it from your system. Thank You.
>> >>
>> >
>> >
>> >
>> > --
>> > https://urldefense.proofpoint.com/v1/url?u=http://hortonworks.com/download/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=mPOfB21VcWnmM3DU81gzph9Quwrh%2BSSBvfQ2CNr5G0g%3D%0A&s=36bc2d49d53470c0688729dc109b1385a9fc0559599b8dca8169f00fd7e11fd4
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>> entity to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> https://urldefense.proofpoint.com/v1/url?u=http://hortonworks.com/&k=oIvRg1%2BdGAgOoM1BIlLLqw%3D%3D%0A&r=Mw3izENeqbMFnOzHo594vQ%3D%3D%0A&m=mPOfB21VcWnmM3DU81gzph9Quwrh%2BSSBvfQ2CNr5G0g%3D%0A&s=35573f6f0811fd2b7cc2274469110563eca6b20dce4ed5961ae05ad601d552a0
>>
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You
Re: Next releases
Posted by Arpit Agarwal <aa...@hortonworks.com>.
This merge to branch-2 is complete. The changes have been merged to
branch-2 and target version set to 2.4.0 (r1556076).
On Fri, Jan 3, 2014 at 4:13 PM, Arpit Agarwal <aa...@hortonworks.com>wrote:
> We plan to merge HDFS-2832 to branch-2 next week for inclusion in 2.4.
>
>
>
> On Fri, Dec 6, 2013 at 1:53 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
>> Thanks Suresh & Colin.
>>
>> Please update the Roadmap wiki with your proposals.
>>
>> As always, we will try our best to get these in - but we can collectively
>> decide to slip some of these to subsequent releases based on timelines.
>>
>> Arun
>>
>> On Dec 6, 2013, at 10:43 AM, Suresh Srinivas <su...@hortonworks.com>
>> wrote:
>>
>> > Arun,
>> >
>> > I propose the following changes for 2.3:
>> > - There have been a lot of improvements related to supporting http
>> policy.
>> > - There is a still discussion going on, but I would like to deprecate
>> > BackupNode in 2.3 as well.
>> > - We are currently working on rolling upgrades related change in HDFS.
>> We
>> > might add a couple of changes that enables rolling upgrades from 2.3
>> > onwards (hopefully we can this done by December)
>> >
>> > I propose the following for 2.4 release, if they are tested and stable:
>> > - Heterogeneous storage support - HDFS-2832
>> > - Datanode cache related change - HDFS-4949
>> > - HDFS ACLs - HDFS-4685
>> > - Rolling upgrade changes
>> >
>> > Let me know if you want me to update the wiki.
>> >
>> > Regards,
>> > Suresh
>> >
>>
>> On Dec 6, 2013, at 12:27 PM, Colin McCabe <cm...@alumni.cmu.edu> wrote:
>>
>> > If 2.4 is released in January, I think it's very unlikely to include
>> > symlinks. There is still a lot of work to be done before they're
>> > usable. You can look at the progress on HADOOP-10019. For some of
>> > the subtasks, it will require some community discussion before any
>> > code can be written.
>> >
>> > For better or worse, symlinks have not been requested by users as
>> > often as features like NFS export, HDFS caching, ACLs, etc, so effort
>> > has been focused on those instead.
>> >
>> > For now, I think we should put the symlinks-disabling patches
>> > (HADOOP-10020, etc) into branch-2, so that they will be part of the
>> > next releases without additional effort.
>> >
>> > I would like to see HDFS caching make it into 2.4. The APIs and
>> > implementation are beginning to stabilize, and around January it
>> > should be ok to backport to a stable branch.
>> >
>> > best,
>> > Colin
>> >
>> >
>> > On Thu, Nov 7, 2013 at 6:42 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:
>> >
>> >> Gang,
>> >>
>> >> Thinking through the next couple of releases here, appreciate f/b.
>> >>
>> >> # hadoop-2.2.1
>> >>
>> >> I was looking through commit logs and there is a *lot* of content here
>> >> (81 commits as on 11/7). Some are features/improvements and some are
>> fixes
>> >> - it's really hard to distinguish what is important and what isn't.
>> >>
>> >> I propose we start with a blank slate (i.e. blow away branch-2.2 and
>> >> start fresh from a copy of branch-2.2.0) and then be very careful and
>> >> meticulous about including only *blocker* fixes in branch-2.2. So,
>> most of
>> >> the content here comes via the next minor release (i.e. hadoop-2.3)
>> >>
>> >> In future, we continue to be *very* parsimonious about what gets into a
>> >> patch release (major.minor.patch) - in general, these should be only
>> >> *blocker* fixes or key operational issues.
>> >>
>> >> # hadoop-2.3
>> >>
>> >> I'd like to propose the following features for YARN/MR to make it into
>> >> hadoop-2.3 and punt the rest to hadoop-2.4 and beyond:
>> >> * Application History Server - This is happening in a branch and is
>> >> close; with it we can provide a reasonable experience for new
>> frameworks
>> >> being built on top of YARN.
>> >> * Bug-fixes in RM Restart
>> >> * Minimal support for long-running applications (e.g. security) via
>> >> YARN-896
>> >> * RM Fail-over via ZKFC
>> >> * Anything else?
>> >>
>> >> HDFS???
>> >>
>> >> Overall, I feel like we have a decent chance of rolling hadoop-2.3 by
>> the
>> >> end of the year.
>> >>
>> >> Thoughts?
>> >>
>> >> thanks,
>> >> Arun
>> >>
>> >>
>> >> --
>> >> Arun C. Murthy
>> >> Hortonworks Inc.
>> >> http://hortonworks.com/
>> >>
>> >>
>> >>
>> >> --
>> >> CONFIDENTIALITY NOTICE
>> >> NOTICE: This message is intended for the use of the individual or
>> entity to
>> >> which it is addressed and may contain information that is confidential,
>> >> privileged and exempt from disclosure under applicable law. If the
>> reader
>> >> of this message is not the intended recipient, you are hereby notified
>> that
>> >> any printing, copying, dissemination, distribution, disclosure or
>> >> forwarding of this communication is strictly prohibited. If you have
>> >> received this communication in error, please contact the sender
>> immediately
>> >> and delete it from your system. Thank You.
>> >>
>> >
>> >
>> >
>> > --
>> > http://hortonworks.com/download/
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>> entity to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Next releases
Posted by Arpit Agarwal <aa...@hortonworks.com>.
This merge to branch-2 is complete. The changes have been merged to
branch-2 and target version set to 2.4.0 (r1556076).
On Fri, Jan 3, 2014 at 4:13 PM, Arpit Agarwal <aa...@hortonworks.com>wrote:
> We plan to merge HDFS-2832 to branch-2 next week for inclusion in 2.4.
>
>
>
> On Fri, Dec 6, 2013 at 1:53 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
>> Thanks Suresh & Colin.
>>
>> Please update the Roadmap wiki with your proposals.
>>
>> As always, we will try our best to get these in - but we can collectively
>> decide to slip some of these to subsequent releases based on timelines.
>>
>> Arun
>>
>> On Dec 6, 2013, at 10:43 AM, Suresh Srinivas <su...@hortonworks.com>
>> wrote:
>>
>> > Arun,
>> >
>> > I propose the following changes for 2.3:
>> > - There have been a lot of improvements related to supporting http
>> policy.
>> > - There is a still discussion going on, but I would like to deprecate
>> > BackupNode in 2.3 as well.
>> > - We are currently working on rolling upgrades related change in HDFS.
>> We
>> > might add a couple of changes that enables rolling upgrades from 2.3
>> > onwards (hopefully we can this done by December)
>> >
>> > I propose the following for 2.4 release, if they are tested and stable:
>> > - Heterogeneous storage support - HDFS-2832
>> > - Datanode cache related change - HDFS-4949
>> > - HDFS ACLs - HDFS-4685
>> > - Rolling upgrade changes
>> >
>> > Let me know if you want me to update the wiki.
>> >
>> > Regards,
>> > Suresh
>> >
>>
>> On Dec 6, 2013, at 12:27 PM, Colin McCabe <cm...@alumni.cmu.edu> wrote:
>>
>> > If 2.4 is released in January, I think it's very unlikely to include
>> > symlinks. There is still a lot of work to be done before they're
>> > usable. You can look at the progress on HADOOP-10019. For some of
>> > the subtasks, it will require some community discussion before any
>> > code can be written.
>> >
>> > For better or worse, symlinks have not been requested by users as
>> > often as features like NFS export, HDFS caching, ACLs, etc, so effort
>> > has been focused on those instead.
>> >
>> > For now, I think we should put the symlinks-disabling patches
>> > (HADOOP-10020, etc) into branch-2, so that they will be part of the
>> > next releases without additional effort.
>> >
>> > I would like to see HDFS caching make it into 2.4. The APIs and
>> > implementation are beginning to stabilize, and around January it
>> > should be ok to backport to a stable branch.
>> >
>> > best,
>> > Colin
>> >
>> >
>> > On Thu, Nov 7, 2013 at 6:42 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:
>> >
>> >> Gang,
>> >>
>> >> Thinking through the next couple of releases here, appreciate f/b.
>> >>
>> >> # hadoop-2.2.1
>> >>
>> >> I was looking through commit logs and there is a *lot* of content here
>> >> (81 commits as on 11/7). Some are features/improvements and some are
>> fixes
>> >> - it's really hard to distinguish what is important and what isn't.
>> >>
>> >> I propose we start with a blank slate (i.e. blow away branch-2.2 and
>> >> start fresh from a copy of branch-2.2.0) and then be very careful and
>> >> meticulous about including only *blocker* fixes in branch-2.2. So,
>> most of
>> >> the content here comes via the next minor release (i.e. hadoop-2.3)
>> >>
>> >> In future, we continue to be *very* parsimonious about what gets into a
>> >> patch release (major.minor.patch) - in general, these should be only
>> >> *blocker* fixes or key operational issues.
>> >>
>> >> # hadoop-2.3
>> >>
>> >> I'd like to propose the following features for YARN/MR to make it into
>> >> hadoop-2.3 and punt the rest to hadoop-2.4 and beyond:
>> >> * Application History Server - This is happening in a branch and is
>> >> close; with it we can provide a reasonable experience for new
>> frameworks
>> >> being built on top of YARN.
>> >> * Bug-fixes in RM Restart
>> >> * Minimal support for long-running applications (e.g. security) via
>> >> YARN-896
>> >> * RM Fail-over via ZKFC
>> >> * Anything else?
>> >>
>> >> HDFS???
>> >>
>> >> Overall, I feel like we have a decent chance of rolling hadoop-2.3 by
>> the
>> >> end of the year.
>> >>
>> >> Thoughts?
>> >>
>> >> thanks,
>> >> Arun
>> >>
>> >>
>> >> --
>> >> Arun C. Murthy
>> >> Hortonworks Inc.
>> >> http://hortonworks.com/
>> >>
>> >>
>> >>
>> >> --
>> >> CONFIDENTIALITY NOTICE
>> >> NOTICE: This message is intended for the use of the individual or
>> entity to
>> >> which it is addressed and may contain information that is confidential,
>> >> privileged and exempt from disclosure under applicable law. If the
>> reader
>> >> of this message is not the intended recipient, you are hereby notified
>> that
>> >> any printing, copying, dissemination, distribution, disclosure or
>> >> forwarding of this communication is strictly prohibited. If you have
>> >> received this communication in error, please contact the sender
>> immediately
>> >> and delete it from your system. Thank You.
>> >>
>> >
>> >
>> >
>> > --
>> > http://hortonworks.com/download/
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>> entity to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
Re: Next releases
Posted by Arpit Agarwal <aa...@hortonworks.com>.
This merge to branch-2 is complete. The changes have been merged to
branch-2 and target version set to 2.4.0 (r1556076).
On Fri, Jan 3, 2014 at 4:13 PM, Arpit Agarwal <aa...@hortonworks.com>wrote:
> We plan to merge HDFS-2832 to branch-2 next week for inclusion in 2.4.
>
>
>
> On Fri, Dec 6, 2013 at 1:53 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
>> Thanks Suresh & Colin.
>>
>> Please update the Roadmap wiki with your proposals.
>>
>> As always, we will try our best to get these in - but we can collectively
>> decide to slip some of these to subsequent releases based on timelines.
>>
>> Arun
>>
>> On Dec 6, 2013, at 10:43 AM, Suresh Srinivas <su...@hortonworks.com>
>> wrote:
>>
>> > Arun,
>> >
>> > I propose the following changes for 2.3:
>> > - There have been a lot of improvements related to supporting http
>> policy.
>> > - There is a still discussion going on, but I would like to deprecate
>> > BackupNode in 2.3 as well.
>> > - We are currently working on rolling upgrades related change in HDFS.
>> We
>> > might add a couple of changes that enables rolling upgrades from 2.3
>> > onwards (hopefully we can this done by December)
>> >
>> > I propose the following for 2.4 release, if they are tested and stable:
>> > - Heterogeneous storage support - HDFS-2832
>> > - Datanode cache related change - HDFS-4949
>> > - HDFS ACLs - HDFS-4685
>> > - Rolling upgrade changes
>> >
>> > Let me know if you want me to update the wiki.
>> >
>> > Regards,
>> > Suresh
>> >
>>
>> On Dec 6, 2013, at 12:27 PM, Colin McCabe <cm...@alumni.cmu.edu> wrote:
>>
>> > If 2.4 is released in January, I think it's very unlikely to include
>> > symlinks. There is still a lot of work to be done before they're
>> > usable. You can look at the progress on HADOOP-10019. For some of
>> > the subtasks, it will require some community discussion before any
>> > code can be written.
>> >
>> > For better or worse, symlinks have not been requested by users as
>> > often as features like NFS export, HDFS caching, ACLs, etc, so effort
>> > has been focused on those instead.
>> >
>> > For now, I think we should put the symlinks-disabling patches
>> > (HADOOP-10020, etc) into branch-2, so that they will be part of the
>> > next releases without additional effort.
>> >
>> > I would like to see HDFS caching make it into 2.4. The APIs and
>> > implementation are beginning to stabilize, and around January it
>> > should be ok to backport to a stable branch.
>> >
>> > best,
>> > Colin
>> >
>> >
>> > On Thu, Nov 7, 2013 at 6:42 PM, Arun C Murthy <ac...@hortonworks.com>
>> wrote:
>> >
>> >> Gang,
>> >>
>> >> Thinking through the next couple of releases here, appreciate f/b.
>> >>
>> >> # hadoop-2.2.1
>> >>
>> >> I was looking through commit logs and there is a *lot* of content here
>> >> (81 commits as on 11/7). Some are features/improvements and some are
>> fixes
>> >> - it's really hard to distinguish what is important and what isn't.
>> >>
>> >> I propose we start with a blank slate (i.e. blow away branch-2.2 and
>> >> start fresh from a copy of branch-2.2.0) and then be very careful and
>> >> meticulous about including only *blocker* fixes in branch-2.2. So,
>> most of
>> >> the content here comes via the next minor release (i.e. hadoop-2.3)
>> >>
>> >> In future, we continue to be *very* parsimonious about what gets into a
>> >> patch release (major.minor.patch) - in general, these should be only
>> >> *blocker* fixes or key operational issues.
>> >>
>> >> # hadoop-2.3
>> >>
>> >> I'd like to propose the following features for YARN/MR to make it into
>> >> hadoop-2.3 and punt the rest to hadoop-2.4 and beyond:
>> >> * Application History Server - This is happening in a branch and is
>> >> close; with it we can provide a reasonable experience for new
>> frameworks
>> >> being built on top of YARN.
>> >> * Bug-fixes in RM Restart
>> >> * Minimal support for long-running applications (e.g. security) via
>> >> YARN-896
>> >> * RM Fail-over via ZKFC
>> >> * Anything else?
>> >>
>> >> HDFS???
>> >>
>> >> Overall, I feel like we have a decent chance of rolling hadoop-2.3 by
>> the
>> >> end of the year.
>> >>
>> >> Thoughts?
>> >>
>> >> thanks,
>> >> Arun
>> >>
>> >>
>> >> --
>> >> Arun C. Murthy
>> >> Hortonworks Inc.
>> >> http://hortonworks.com/
>> >>
>> >>
>> >>
>> >> --
>> >> CONFIDENTIALITY NOTICE
>> >> NOTICE: This message is intended for the use of the individual or
>> entity to
>> >> which it is addressed and may contain information that is confidential,
>> >> privileged and exempt from disclosure under applicable law. If the
>> reader
>> >> of this message is not the intended recipient, you are hereby notified
>> that
>> >> any printing, copying, dissemination, distribution, disclosure or
>> >> forwarding of this communication is strictly prohibited. If you have
>> >> received this communication in error, please contact the sender
>> immediately
>> >> and delete it from your system. Thank You.
>> >>
>> >
>> >
>> >
>> > --
>> > http://hortonworks.com/download/
>> >
>> > --
>> > CONFIDENTIALITY NOTICE
>> > NOTICE: This message is intended for the use of the individual or
>> entity to
>> > which it is addressed and may contain information that is confidential,
>> > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > of this message is not the intended recipient, you are hereby notified
>> that
>> > any printing, copying, dissemination, distribution, disclosure or
>> > forwarding of this communication is strictly prohibited. If you have
>> > received this communication in error, please contact the sender
>> immediately
>> > and delete it from your system. Thank You.
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity
>> to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified
>> that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
>> immediately
>> and delete it from your system. Thank You.
>>
>
>
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.