You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-dev@hadoop.apache.org by Akira AJISAKA <aj...@oss.nttdata.co.jp> on 2015/06/22 11:43:09 UTC
[DISCUSS] More Maintenance Releases
Hi everyone,
In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that
Apache Hadoop developers at Yahoo!, Twitter, and other non-distributors
work very hard to maintenance Hadoop by cherry-picking patches to their
own branches.
I want to share the work with the community. If we can cherry-pick bug
fix patches and have more maintenance releases, it'd be very happy not
only for users but also for developers who work very hard for
stabilizing their own branches.
To have more maintenance releases, I propose two changes:
* Major/Minor/Trivial bug fixes can be cherry-picked
* (Roughly) Monthly maintenance release
I would like to start the work from branch-2.6. If the change will be
accepted by the community, I'm willing to work for the maintenance, as a
release manager.
Best regards,
Akira
Re: [DISCUSS] More Maintenance Releases
Posted by Sean Busbey <bu...@cloudera.com>.
On Tue, Jun 23, 2015 at 12:30 PM, Andrew Wang <an...@cloudera.com>
wrote:
> There's no reason we have to choose 2.6 xor 2.7. If we have willing RMs and
> enough PMCs who will vote on releases, there's no reason we can't maintain
> both.
>
> However, based on the discussion at Hadoop Summit with Yahoo and Twitter,
> their interest is primarily in 2.6, and Daryn mentioned the need to get 2.6
> stable before they can move to 2.7. So, if we want to help out these big
> users, it seems like we should focus on maintaining 2.6.
>
> Allen also brought up the issue of JDK6. I see a few options (ranked best
> to worst in my eyes):
>
> * Add multi-JDK support to test-patch
> * Keep using JDK7 for precommit, and keep an eye on a nightly JDK6 run
> * Drop support for JDK6 in 2.6.x, since no one is using it anymore
>
> #3 is the most easiest and probably fine for 95% of users, but doing a big
> compat break is not how I'd want to kick off a stable release line. #2
> isn't too bad if we don't want to wait for #1.
>
>
I believe work on adding multi jdk is on-going for patch test. Should be
high on
the list once we get our dev branch up.
--
Sean
Re: [DISCUSS] More Maintenance Releases
Posted by Andrew Purtell <ap...@apache.org>.
I'm not sure we qualify as a big user, but
FWIW, we are upgrading to a Hadoop based on 2.6.0
(
with our own application of critical bugfix patches that
went in on branch-2 later
) over at Salesforce.
2.7.0 and up are scary because 2.7.0 was tagged as not ready for
production. There's a "may eat your data" sign hanging over 2.7.0 and all
subsequent releases and points along the branch-2 history.
Even using 2.6.0 comes with some trepidation because we know the HDFS
encryption feature added in that release will destroy any HBase
installation if it is turned on.
On Tue, Jun 23, 2015 at 11:55 AM, Karthik Kambatla <ka...@cloudera.com>
wrote:
> On Tue, Jun 23, 2015 at 10:30 AM, Andrew Wang <an...@cloudera.com>
> wrote:
>
> > There's no reason we have to choose 2.6 xor 2.7. If we have willing RMs
> and
> > enough PMCs who will vote on releases, there's no reason we can't
> maintain
> > both.
> >
> > However, based on the discussion at Hadoop Summit with Yahoo and Twitter,
> > their interest is primarily in 2.6, and Daryn mentioned the need to get
> 2.6
> > stable before they can move to 2.7. So, if we want to help out these big
> > users, it seems like we should focus on maintaining 2.6.
> >
>
> It would be nice to hear from the big users on the topic.
>
> They are already on 2.6 and some of them have already done a bunch of
> backports. Given it will take us a month or two (based on past experience)
> to get the first release out, it is not clear to me if we should start
> focusing on 2.6 or 2.7. But again, as you say, if there are people willing
> to RM these releases, I see value in improving both lines.
>
>
> >
> > Allen also brought up the issue of JDK6. I see a few options (ranked best
> > to worst in my eyes):
> >
> > * Add multi-JDK support to test-patch
> > * Keep using JDK7 for precommit, and keep an eye on a nightly JDK6 run
> > * Drop support for JDK6 in 2.6.x, since no one is using it anymore
> >
> > #3 is the most easiest and probably fine for 95% of users, but doing a
> big
> > compat break is not how I'd want to kick off a stable release line. #2
> > isn't too bad if we don't want to wait for #1.
> >
> > Finally, an issue that Karthik mentioned at Hadoop Summit is that
> > CHANGES.txt maintenance becomes a huge burden when doing backports after
> > the initial commit. e.g. if I was backporting something to branch-2.6,
> I'd
> > potentially need to update CHANGES.txt in trunk, branch-2, branch-2.7 to
> > move the JIRA # to the right version. If we either automated or dropped
> > CHANGES.txt, the process would be much smoother.
> >
>
> +1 for ditching CHANGES.txt.
>
> I reached out to Allen recently. He wanted to clean up his scripts more
> before dropping CHANGES.txt altogether. Should we target 2.8 for this?
>
>
> >
> > Best,
> > Andrew
> >
> > On Tue, Jun 23, 2015 at 6:05 AM, Akira AJISAKA <
> ajisakaa@oss.nttdata.co.jp
> > >
> > wrote:
> >
> > > Thanks Tsuyoshi for the comment.
> > >
> > > Targeting to 2.7 looks good to me, however, I'd like to maintenance
> > > branch-2.6 also. It's only about a half year since 2.6.0 was released.
> > > I don't want to abandon relatively new branches.
> > >
> > >
> > > On 6/23/15 16:05, Tsuyoshi Ozawa wrote:
> > >
> > >> Thank you for clarification, Karthik.
> > >>
> > >> I'd also like to work on stable release management with community.
> > >>
> > >> I'm thinking of speedy release against <next version - 1> branch for
> > >> making community feedback faster (e.g. current next version is 2.8, so
> > >> cherry-picking to 2.7 and releasing it are useful work for users).
> > >>
> > >> I know that code base of 2.7.1 is now freezing currently, I'd love to
> > >> work from 2.7.2 release if possible.
> > >>
> > >> Cheers,
> > >> - Tsuyoshi
> > >>
> > >> On Tue, Jun 23, 2015 at 5:02 AM, Colin P. McCabe <cm...@apache.org>
> > >> wrote:
> > >>
> > >>> +1 for creating a maintenance release with a more rapid release
> > >>> cadence and more effort put into stability backports. I think this
> > >>> would really be great for the project.
> > >>>
> > >>> Colin
> > >>>
> > >>> On Mon, Jun 22, 2015 at 2:43 AM, Akira AJISAKA
> > >>> <aj...@oss.nttdata.co.jp> wrote:
> > >>>
> > >>>> Hi everyone,
> > >>>>
> > >>>> In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that
> > >>>> Apache
> > >>>> Hadoop developers at Yahoo!, Twitter, and other non-distributors
> work
> > >>>> very
> > >>>> hard to maintenance Hadoop by cherry-picking patches to their own
> > >>>> branches.
> > >>>>
> > >>>> I want to share the work with the community. If we can cherry-pick
> bug
> > >>>> fix
> > >>>> patches and have more maintenance releases, it'd be very happy not
> > only
> > >>>> for
> > >>>> users but also for developers who work very hard for stabilizing
> their
> > >>>> own
> > >>>> branches.
> > >>>>
> > >>>> To have more maintenance releases, I propose two changes:
> > >>>>
> > >>>> * Major/Minor/Trivial bug fixes can be cherry-picked
> > >>>> * (Roughly) Monthly maintenance release
> > >>>>
> > >>>> I would like to start the work from branch-2.6. If the change will
> be
> > >>>> accepted by the community, I'm willing to work for the maintenance,
> > as a
> > >>>> release manager.
> > >>>>
> > >>>> Best regards,
> > >>>> Akira
> > >>>>
> > >>>
> > >
> >
>
>
>
> --
> Karthik Kambatla
> Software Engineer, Cloudera Inc.
> --------------------------------------------
> http://five.sentenc.es
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: [DISCUSS] More Maintenance Releases
Posted by Allen Wittenauer <aw...@altiscale.com>.
On Jun 26, 2015, at 3:35 PM, Andrew Wang <an...@cloudera.com> wrote:
> Allen, do you still have these concerns? You mentioned having written
> scripts to parse CHANGES.txt. I've written scripts for similar tasks, but
> what I turned to were git log and JIRA, not CHANGES.txt. I was wondering if
> this is a relic from the SVN days, since svn log was dog slow. Git log is
> quite a bit better.
git log is totally unreliable. I’ve shown that over and over and over. JIRA may not perfect, but it’s *correctable* and the restraints on what actually can be input in certain fields is a good thing.
But frankly, at this point, it doesn’t matter. We haven’t had a *single* minor release in branch-2 that didn’t break something in user- or ops- facing ways. Why stop now?
Re: [DISCUSS] More Maintenance Releases
Posted by Andrew Wang <an...@cloudera.com>.
>
> +1 for ditching CHANGES.txt.
>
> I reached out to Allen recently. He wanted to clean up his scripts more
> before dropping CHANGES.txt altogether. Should we target 2.8 for this?
I read through our previous threads on this topic, and Allen had some
compatibility concerns about dropping CHANGES.txt in a branch-2 release.
Allen, do you still have these concerns? You mentioned having written
scripts to parse CHANGES.txt. I've written scripts for similar tasks, but
what I turned to were git log and JIRA, not CHANGES.txt. I was wondering if
this is a relic from the SVN days, since svn log was dog slow. Git log is
quite a bit better.
My inclination is to just drop CHANGES.txt entirely, including in branch-2.
We already have release notes, JIRA, as well as the per-release webpage
which talks up the new features. CHANGES.txt also has never been a reliable
source of information, and I doubt it has many consumers (if any).
Best,
Andrew
Re: [DISCUSS] More Maintenance Releases
Posted by Karthik Kambatla <ka...@cloudera.com>.
On Tue, Jun 23, 2015 at 10:30 AM, Andrew Wang <an...@cloudera.com>
wrote:
> There's no reason we have to choose 2.6 xor 2.7. If we have willing RMs and
> enough PMCs who will vote on releases, there's no reason we can't maintain
> both.
>
> However, based on the discussion at Hadoop Summit with Yahoo and Twitter,
> their interest is primarily in 2.6, and Daryn mentioned the need to get 2.6
> stable before they can move to 2.7. So, if we want to help out these big
> users, it seems like we should focus on maintaining 2.6.
>
It would be nice to hear from the big users on the topic.
They are already on 2.6 and some of them have already done a bunch of
backports. Given it will take us a month or two (based on past experience)
to get the first release out, it is not clear to me if we should start
focusing on 2.6 or 2.7. But again, as you say, if there are people willing
to RM these releases, I see value in improving both lines.
>
> Allen also brought up the issue of JDK6. I see a few options (ranked best
> to worst in my eyes):
>
> * Add multi-JDK support to test-patch
> * Keep using JDK7 for precommit, and keep an eye on a nightly JDK6 run
> * Drop support for JDK6 in 2.6.x, since no one is using it anymore
>
> #3 is the most easiest and probably fine for 95% of users, but doing a big
> compat break is not how I'd want to kick off a stable release line. #2
> isn't too bad if we don't want to wait for #1.
>
> Finally, an issue that Karthik mentioned at Hadoop Summit is that
> CHANGES.txt maintenance becomes a huge burden when doing backports after
> the initial commit. e.g. if I was backporting something to branch-2.6, I'd
> potentially need to update CHANGES.txt in trunk, branch-2, branch-2.7 to
> move the JIRA # to the right version. If we either automated or dropped
> CHANGES.txt, the process would be much smoother.
>
+1 for ditching CHANGES.txt.
I reached out to Allen recently. He wanted to clean up his scripts more
before dropping CHANGES.txt altogether. Should we target 2.8 for this?
>
> Best,
> Andrew
>
> On Tue, Jun 23, 2015 at 6:05 AM, Akira AJISAKA <ajisakaa@oss.nttdata.co.jp
> >
> wrote:
>
> > Thanks Tsuyoshi for the comment.
> >
> > Targeting to 2.7 looks good to me, however, I'd like to maintenance
> > branch-2.6 also. It's only about a half year since 2.6.0 was released.
> > I don't want to abandon relatively new branches.
> >
> >
> > On 6/23/15 16:05, Tsuyoshi Ozawa wrote:
> >
> >> Thank you for clarification, Karthik.
> >>
> >> I'd also like to work on stable release management with community.
> >>
> >> I'm thinking of speedy release against <next version - 1> branch for
> >> making community feedback faster (e.g. current next version is 2.8, so
> >> cherry-picking to 2.7 and releasing it are useful work for users).
> >>
> >> I know that code base of 2.7.1 is now freezing currently, I'd love to
> >> work from 2.7.2 release if possible.
> >>
> >> Cheers,
> >> - Tsuyoshi
> >>
> >> On Tue, Jun 23, 2015 at 5:02 AM, Colin P. McCabe <cm...@apache.org>
> >> wrote:
> >>
> >>> +1 for creating a maintenance release with a more rapid release
> >>> cadence and more effort put into stability backports. I think this
> >>> would really be great for the project.
> >>>
> >>> Colin
> >>>
> >>> On Mon, Jun 22, 2015 at 2:43 AM, Akira AJISAKA
> >>> <aj...@oss.nttdata.co.jp> wrote:
> >>>
> >>>> Hi everyone,
> >>>>
> >>>> In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that
> >>>> Apache
> >>>> Hadoop developers at Yahoo!, Twitter, and other non-distributors work
> >>>> very
> >>>> hard to maintenance Hadoop by cherry-picking patches to their own
> >>>> branches.
> >>>>
> >>>> I want to share the work with the community. If we can cherry-pick bug
> >>>> fix
> >>>> patches and have more maintenance releases, it'd be very happy not
> only
> >>>> for
> >>>> users but also for developers who work very hard for stabilizing their
> >>>> own
> >>>> branches.
> >>>>
> >>>> To have more maintenance releases, I propose two changes:
> >>>>
> >>>> * Major/Minor/Trivial bug fixes can be cherry-picked
> >>>> * (Roughly) Monthly maintenance release
> >>>>
> >>>> I would like to start the work from branch-2.6. If the change will be
> >>>> accepted by the community, I'm willing to work for the maintenance,
> as a
> >>>> release manager.
> >>>>
> >>>> Best regards,
> >>>> Akira
> >>>>
> >>>
> >
>
--
Karthik Kambatla
Software Engineer, Cloudera Inc.
--------------------------------------------
http://five.sentenc.es
Re: [DISCUSS] More Maintenance Releases
Posted by Andrew Wang <an...@cloudera.com>.
There's no reason we have to choose 2.6 xor 2.7. If we have willing RMs and
enough PMCs who will vote on releases, there's no reason we can't maintain
both.
However, based on the discussion at Hadoop Summit with Yahoo and Twitter,
their interest is primarily in 2.6, and Daryn mentioned the need to get 2.6
stable before they can move to 2.7. So, if we want to help out these big
users, it seems like we should focus on maintaining 2.6.
Allen also brought up the issue of JDK6. I see a few options (ranked best
to worst in my eyes):
* Add multi-JDK support to test-patch
* Keep using JDK7 for precommit, and keep an eye on a nightly JDK6 run
* Drop support for JDK6 in 2.6.x, since no one is using it anymore
#3 is the most easiest and probably fine for 95% of users, but doing a big
compat break is not how I'd want to kick off a stable release line. #2
isn't too bad if we don't want to wait for #1.
Finally, an issue that Karthik mentioned at Hadoop Summit is that
CHANGES.txt maintenance becomes a huge burden when doing backports after
the initial commit. e.g. if I was backporting something to branch-2.6, I'd
potentially need to update CHANGES.txt in trunk, branch-2, branch-2.7 to
move the JIRA # to the right version. If we either automated or dropped
CHANGES.txt, the process would be much smoother.
Best,
Andrew
On Tue, Jun 23, 2015 at 6:05 AM, Akira AJISAKA <aj...@oss.nttdata.co.jp>
wrote:
> Thanks Tsuyoshi for the comment.
>
> Targeting to 2.7 looks good to me, however, I'd like to maintenance
> branch-2.6 also. It's only about a half year since 2.6.0 was released.
> I don't want to abandon relatively new branches.
>
>
> On 6/23/15 16:05, Tsuyoshi Ozawa wrote:
>
>> Thank you for clarification, Karthik.
>>
>> I'd also like to work on stable release management with community.
>>
>> I'm thinking of speedy release against <next version - 1> branch for
>> making community feedback faster (e.g. current next version is 2.8, so
>> cherry-picking to 2.7 and releasing it are useful work for users).
>>
>> I know that code base of 2.7.1 is now freezing currently, I'd love to
>> work from 2.7.2 release if possible.
>>
>> Cheers,
>> - Tsuyoshi
>>
>> On Tue, Jun 23, 2015 at 5:02 AM, Colin P. McCabe <cm...@apache.org>
>> wrote:
>>
>>> +1 for creating a maintenance release with a more rapid release
>>> cadence and more effort put into stability backports. I think this
>>> would really be great for the project.
>>>
>>> Colin
>>>
>>> On Mon, Jun 22, 2015 at 2:43 AM, Akira AJISAKA
>>> <aj...@oss.nttdata.co.jp> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that
>>>> Apache
>>>> Hadoop developers at Yahoo!, Twitter, and other non-distributors work
>>>> very
>>>> hard to maintenance Hadoop by cherry-picking patches to their own
>>>> branches.
>>>>
>>>> I want to share the work with the community. If we can cherry-pick bug
>>>> fix
>>>> patches and have more maintenance releases, it'd be very happy not only
>>>> for
>>>> users but also for developers who work very hard for stabilizing their
>>>> own
>>>> branches.
>>>>
>>>> To have more maintenance releases, I propose two changes:
>>>>
>>>> * Major/Minor/Trivial bug fixes can be cherry-picked
>>>> * (Roughly) Monthly maintenance release
>>>>
>>>> I would like to start the work from branch-2.6. If the change will be
>>>> accepted by the community, I'm willing to work for the maintenance, as a
>>>> release manager.
>>>>
>>>> Best regards,
>>>> Akira
>>>>
>>>
>
Re: [DISCUSS] More Maintenance Releases
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
Thanks Tsuyoshi for the comment.
Targeting to 2.7 looks good to me, however, I'd like to maintenance
branch-2.6 also. It's only about a half year since 2.6.0 was released.
I don't want to abandon relatively new branches.
On 6/23/15 16:05, Tsuyoshi Ozawa wrote:
> Thank you for clarification, Karthik.
>
> I'd also like to work on stable release management with community.
>
> I'm thinking of speedy release against <next version - 1> branch for
> making community feedback faster (e.g. current next version is 2.8, so
> cherry-picking to 2.7 and releasing it are useful work for users).
>
> I know that code base of 2.7.1 is now freezing currently, I'd love to
> work from 2.7.2 release if possible.
>
> Cheers,
> - Tsuyoshi
>
> On Tue, Jun 23, 2015 at 5:02 AM, Colin P. McCabe <cm...@apache.org> wrote:
>> +1 for creating a maintenance release with a more rapid release
>> cadence and more effort put into stability backports. I think this
>> would really be great for the project.
>>
>> Colin
>>
>> On Mon, Jun 22, 2015 at 2:43 AM, Akira AJISAKA
>> <aj...@oss.nttdata.co.jp> wrote:
>>> Hi everyone,
>>>
>>> In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that Apache
>>> Hadoop developers at Yahoo!, Twitter, and other non-distributors work very
>>> hard to maintenance Hadoop by cherry-picking patches to their own branches.
>>>
>>> I want to share the work with the community. If we can cherry-pick bug fix
>>> patches and have more maintenance releases, it'd be very happy not only for
>>> users but also for developers who work very hard for stabilizing their own
>>> branches.
>>>
>>> To have more maintenance releases, I propose two changes:
>>>
>>> * Major/Minor/Trivial bug fixes can be cherry-picked
>>> * (Roughly) Monthly maintenance release
>>>
>>> I would like to start the work from branch-2.6. If the change will be
>>> accepted by the community, I'm willing to work for the maintenance, as a
>>> release manager.
>>>
>>> Best regards,
>>> Akira
Re: [DISCUSS] More Maintenance Releases
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
Thank you for clarification, Karthik.
I'd also like to work on stable release management with community.
I'm thinking of speedy release against <next version - 1> branch for
making community feedback faster (e.g. current next version is 2.8, so
cherry-picking to 2.7 and releasing it are useful work for users).
I know that code base of 2.7.1 is now freezing currently, I'd love to
work from 2.7.2 release if possible.
Cheers,
- Tsuyoshi
On Tue, Jun 23, 2015 at 5:02 AM, Colin P. McCabe <cm...@apache.org> wrote:
> +1 for creating a maintenance release with a more rapid release
> cadence and more effort put into stability backports. I think this
> would really be great for the project.
>
> Colin
>
> On Mon, Jun 22, 2015 at 2:43 AM, Akira AJISAKA
> <aj...@oss.nttdata.co.jp> wrote:
>> Hi everyone,
>>
>> In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that Apache
>> Hadoop developers at Yahoo!, Twitter, and other non-distributors work very
>> hard to maintenance Hadoop by cherry-picking patches to their own branches.
>>
>> I want to share the work with the community. If we can cherry-pick bug fix
>> patches and have more maintenance releases, it'd be very happy not only for
>> users but also for developers who work very hard for stabilizing their own
>> branches.
>>
>> To have more maintenance releases, I propose two changes:
>>
>> * Major/Minor/Trivial bug fixes can be cherry-picked
>> * (Roughly) Monthly maintenance release
>>
>> I would like to start the work from branch-2.6. If the change will be
>> accepted by the community, I'm willing to work for the maintenance, as a
>> release manager.
>>
>> Best regards,
>> Akira
Re: [DISCUSS] More Maintenance Releases
Posted by "Colin P. McCabe" <cm...@apache.org>.
+1 for creating a maintenance release with a more rapid release
cadence and more effort put into stability backports. I think this
would really be great for the project.
Colin
On Mon, Jun 22, 2015 at 2:43 AM, Akira AJISAKA
<aj...@oss.nttdata.co.jp> wrote:
> Hi everyone,
>
> In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that Apache
> Hadoop developers at Yahoo!, Twitter, and other non-distributors work very
> hard to maintenance Hadoop by cherry-picking patches to their own branches.
>
> I want to share the work with the community. If we can cherry-pick bug fix
> patches and have more maintenance releases, it'd be very happy not only for
> users but also for developers who work very hard for stabilizing their own
> branches.
>
> To have more maintenance releases, I propose two changes:
>
> * Major/Minor/Trivial bug fixes can be cherry-picked
> * (Roughly) Monthly maintenance release
>
> I would like to start the work from branch-2.6. If the change will be
> accepted by the community, I'm willing to work for the maintenance, as a
> release manager.
>
> Best regards,
> Akira
Re: [DISCUSS] More Maintenance Releases
Posted by Sean Busbey <bu...@cloudera.com>.
More maintenance releases would be excellent.
If y'all are going to make more releases on the 2.6 line, please consider
backporting HADOOP-11710 as without it HBase is unusable on top of HDFS
encryption. It's been inconvenient that the fix is only available in a
non-production release line.
-Sean
On Mon, Jun 22, 2015 at 6:36 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi Akira,
>
> Thank you for starting interesting topic. +1 on the idea of More
> Maintenance Releases for old branches. It would be good if this
> activity is more coupled with Apache Yetus for users.
>
> BTW, I don't know one of committers, who is not PMC, can be a release
> manager. Does anyone know about this? It's described in detail as
> follows: http://hadoop.apache.org/bylaws#Decision+Making
>
> > Release Manager
> > A Release Manager (RM) is a committer who volunteers to produce a
> Release Candidate according to HowToRelease.
> >
> > Project Management Committee
> > Deciding what is distributed as products of the Apache Hadoop project.
> In particular all releases must be approved by the PMC
>
> Thanks,
> - Tsuyoshi
>
> On Mon, Jun 22, 2015 at 6:43 PM, Akira AJISAKA
> <aj...@oss.nttdata.co.jp> wrote:
> > Hi everyone,
> >
> > In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that Apache
> > Hadoop developers at Yahoo!, Twitter, and other non-distributors work
> very
> > hard to maintenance Hadoop by cherry-picking patches to their own
> branches.
> >
> > I want to share the work with the community. If we can cherry-pick bug
> fix
> > patches and have more maintenance releases, it'd be very happy not only
> for
> > users but also for developers who work very hard for stabilizing their
> own
> > branches.
> >
> > To have more maintenance releases, I propose two changes:
> >
> > * Major/Minor/Trivial bug fixes can be cherry-picked
> > * (Roughly) Monthly maintenance release
> >
> > I would like to start the work from branch-2.6. If the change will be
> > accepted by the community, I'm willing to work for the maintenance, as a
> > release manager.
> >
> > Best regards,
> > Akira
>
--
Sean
Re: [DISCUSS] More Maintenance Releases
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
Thanks Karthik for the comment.
I'm +1 for your approach to ensure stability.
On 6/22/15 23:50, Karthik Kambatla wrote:
> Thanks for starting this thread, Akira.
>
> +1 to more maintenance releases. More stable upstream releases avoids
> duplicating cherry-pick work across consumers/vendors, and shows the
> maturity of the project to users.
>
> I see value in backporting blocker/critical issues, but have mixed feelings
> about doing the same for major/minor/trivial issues. IMO, every commit has
> non-zero potential to introduce other bugs. Depending on the kind of fix
> (say, documentation), it might be okay to include these non-critical fixes.
> One approach could be to allow all bug fixes for 2.x.1, blocker/critical
> for 2.x.2, blocker for 2.x.3 (or something along those lines) to ensure
> increasing stability of maintenance releases?
>
> I am also +1 to any committer picking up RM duties for a maintenance
> release. It is healthy to have more people participate in the release
> process, so long as we have some method to maintenance release madness.
>
> A committer (who is not yet a PMC member) could be a Release Manager, but
> his vote is not binding for the release. I RM-ed the 2.5.x releases as a
> committer. RM-ing a release and voting non-binding could be a good way to
> remind the PMC to include the committer in PMC :)
>
> Cheers
> Karthik
>
> On Mon, Jun 22, 2015 at 4:36 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
>> Hi Akira,
>>
>> Thank you for starting interesting topic. +1 on the idea of More
>> Maintenance Releases for old branches. It would be good if this
>> activity is more coupled with Apache Yetus for users.
>>
>> BTW, I don't know one of committers, who is not PMC, can be a release
>> manager. Does anyone know about this? It's described in detail as
>> follows: http://hadoop.apache.org/bylaws#Decision+Making
>>
>>> Release Manager
>>> A Release Manager (RM) is a committer who volunteers to produce a
>> Release Candidate according to HowToRelease.
>>>
>>> Project Management Committee
>>> Deciding what is distributed as products of the Apache Hadoop project.
>> In particular all releases must be approved by the PMC
>>
>> Thanks,
>> - Tsuyoshi
>>
>> On Mon, Jun 22, 2015 at 6:43 PM, Akira AJISAKA
>> <aj...@oss.nttdata.co.jp> wrote:
>>> Hi everyone,
>>>
>>> In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that Apache
>>> Hadoop developers at Yahoo!, Twitter, and other non-distributors work
>> very
>>> hard to maintenance Hadoop by cherry-picking patches to their own
>> branches.
>>>
>>> I want to share the work with the community. If we can cherry-pick bug
>> fix
>>> patches and have more maintenance releases, it'd be very happy not only
>> for
>>> users but also for developers who work very hard for stabilizing their
>> own
>>> branches.
>>>
>>> To have more maintenance releases, I propose two changes:
>>>
>>> * Major/Minor/Trivial bug fixes can be cherry-picked
>>> * (Roughly) Monthly maintenance release
>>>
>>> I would like to start the work from branch-2.6. If the change will be
>>> accepted by the community, I'm willing to work for the maintenance, as a
>>> release manager.
>>>
>>> Best regards,
>>> Akira
>>
>
>
>
Re: [DISCUSS] More Maintenance Releases
Posted by Vinayakumar B <vi...@apache.org>.
+1 for the idea of maintenance releases.
Considering the amount code changes done in trunk and branch-2,
cherry-picking may not be easy and straight forward in all issues.
I would love to help in cherry-picking the fixes and reviewing them.
I would also love to help in release process.
Regards,
Vinay
On Mon, Jun 22, 2015 at 9:49 PM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
> If 2.6 is the target, someone will have to verify that any
> cherry-picked patches actually work with JDK6 since the PMC voted to
> officially kill backward compatibility in a minor release. It’s going to be
> easier and probably smarter to fix 2.7 if that’s really desired. [1]
>
> Frankly, I’d rather see effort spent on stabilizing trunk and
> ditching the now broken branch-2. We’re approaching the 4 year anniversary
> of 0.23.0’s release (which later begat 2.x, which is already past the 3
> year mark). It’s hard to claim health when its been so long since a branch
> off of trunk was cut and turned into something official.
>
> [1] Kengo and I are hard at work getting multiJDK testing working in
> Yetus, but it’s not quite ready for prime time. :( It could certain help
> here, but… it’s not very stable yet.
>
> On Jun 22, 2015, at 7:50 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
>
> > Thanks for starting this thread, Akira.
> >
> > +1 to more maintenance releases. More stable upstream releases avoids
> > duplicating cherry-pick work across consumers/vendors, and shows the
> > maturity of the project to users.
> >
> > I see value in backporting blocker/critical issues, but have mixed
> feelings
> > about doing the same for major/minor/trivial issues. IMO, every commit
> has
> > non-zero potential to introduce other bugs. Depending on the kind of fix
> > (say, documentation), it might be okay to include these non-critical
> fixes.
> > One approach could be to allow all bug fixes for 2.x.1, blocker/critical
> > for 2.x.2, blocker for 2.x.3 (or something along those lines) to ensure
> > increasing stability of maintenance releases?
> >
> > I am also +1 to any committer picking up RM duties for a maintenance
> > release. It is healthy to have more people participate in the release
> > process, so long as we have some method to maintenance release madness.
> >
> > A committer (who is not yet a PMC member) could be a Release Manager, but
> > his vote is not binding for the release. I RM-ed the 2.5.x releases as a
> > committer. RM-ing a release and voting non-binding could be a good way to
> > remind the PMC to include the committer in PMC :)
> >
> > Cheers
> > Karthik
> >
> > On Mon, Jun 22, 2015 at 4:36 AM, Tsuyoshi Ozawa <oz...@apache.org>
> wrote:
> >
> >> Hi Akira,
> >>
> >> Thank you for starting interesting topic. +1 on the idea of More
> >> Maintenance Releases for old branches. It would be good if this
> >> activity is more coupled with Apache Yetus for users.
> >>
> >> BTW, I don't know one of committers, who is not PMC, can be a release
> >> manager. Does anyone know about this? It's described in detail as
> >> follows: http://hadoop.apache.org/bylaws#Decision+Making
> >>
> >>> Release Manager
> >>> A Release Manager (RM) is a committer who volunteers to produce a
> >> Release Candidate according to HowToRelease.
> >>>
> >>> Project Management Committee
> >>> Deciding what is distributed as products of the Apache Hadoop project.
> >> In particular all releases must be approved by the PMC
> >>
> >> Thanks,
> >> - Tsuyoshi
> >>
> >> On Mon, Jun 22, 2015 at 6:43 PM, Akira AJISAKA
> >> <aj...@oss.nttdata.co.jp> wrote:
> >>> Hi everyone,
> >>>
> >>> In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that
> Apache
> >>> Hadoop developers at Yahoo!, Twitter, and other non-distributors work
> >> very
> >>> hard to maintenance Hadoop by cherry-picking patches to their own
> >> branches.
> >>>
> >>> I want to share the work with the community. If we can cherry-pick bug
> >> fix
> >>> patches and have more maintenance releases, it'd be very happy not only
> >> for
> >>> users but also for developers who work very hard for stabilizing their
> >> own
> >>> branches.
> >>>
> >>> To have more maintenance releases, I propose two changes:
> >>>
> >>> * Major/Minor/Trivial bug fixes can be cherry-picked
> >>> * (Roughly) Monthly maintenance release
> >>>
> >>> I would like to start the work from branch-2.6. If the change will be
> >>> accepted by the community, I'm willing to work for the maintenance, as
> a
> >>> release manager.
> >>>
> >>> Best regards,
> >>> Akira
> >>
> >
> >
> >
> > --
> > Karthik Kambatla
> > Software Engineer, Cloudera Inc.
> > --------------------------------------------
> > http://five.sentenc.es
>
>
Re: [DISCUSS] More Maintenance Releases
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
Thanks Allen for reminding me of supporting JDK6. If 2.6 is the target,
any commmiter needs to check a bug fix patch when cherry-picking it.
For trunk, I'm thinking we need to decide what we should do for release
3.x, in another thread. I'm okay to discuss it again.
On 6/23/15 01:19, Allen Wittenauer wrote:
>
> If 2.6 is the target, someone will have to verify that any cherry-picked patches actually work with JDK6 since the PMC voted to officially kill backward compatibility in a minor release. It’s going to be easier and probably smarter to fix 2.7 if that’s really desired. [1]
>
> Frankly, I’d rather see effort spent on stabilizing trunk and ditching the now broken branch-2. We’re approaching the 4 year anniversary of 0.23.0’s release (which later begat 2.x, which is already past the 3 year mark). It’s hard to claim health when its been so long since a branch off of trunk was cut and turned into something official.
>
> [1] Kengo and I are hard at work getting multiJDK testing working in Yetus, but it’s not quite ready for prime time. :( It could certain help here, but… it’s not very stable yet.
>
> On Jun 22, 2015, at 7:50 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
>
>> Thanks for starting this thread, Akira.
>>
>> +1 to more maintenance releases. More stable upstream releases avoids
>> duplicating cherry-pick work across consumers/vendors, and shows the
>> maturity of the project to users.
>>
>> I see value in backporting blocker/critical issues, but have mixed feelings
>> about doing the same for major/minor/trivial issues. IMO, every commit has
>> non-zero potential to introduce other bugs. Depending on the kind of fix
>> (say, documentation), it might be okay to include these non-critical fixes.
>> One approach could be to allow all bug fixes for 2.x.1, blocker/critical
>> for 2.x.2, blocker for 2.x.3 (or something along those lines) to ensure
>> increasing stability of maintenance releases?
>>
>> I am also +1 to any committer picking up RM duties for a maintenance
>> release. It is healthy to have more people participate in the release
>> process, so long as we have some method to maintenance release madness.
>>
>> A committer (who is not yet a PMC member) could be a Release Manager, but
>> his vote is not binding for the release. I RM-ed the 2.5.x releases as a
>> committer. RM-ing a release and voting non-binding could be a good way to
>> remind the PMC to include the committer in PMC :)
>>
>> Cheers
>> Karthik
>>
>> On Mon, Jun 22, 2015 at 4:36 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>>
>>> Hi Akira,
>>>
>>> Thank you for starting interesting topic. +1 on the idea of More
>>> Maintenance Releases for old branches. It would be good if this
>>> activity is more coupled with Apache Yetus for users.
>>>
>>> BTW, I don't know one of committers, who is not PMC, can be a release
>>> manager. Does anyone know about this? It's described in detail as
>>> follows: http://hadoop.apache.org/bylaws#Decision+Making
>>>
>>>> Release Manager
>>>> A Release Manager (RM) is a committer who volunteers to produce a
>>> Release Candidate according to HowToRelease.
>>>>
>>>> Project Management Committee
>>>> Deciding what is distributed as products of the Apache Hadoop project.
>>> In particular all releases must be approved by the PMC
>>>
>>> Thanks,
>>> - Tsuyoshi
>>>
>>> On Mon, Jun 22, 2015 at 6:43 PM, Akira AJISAKA
>>> <aj...@oss.nttdata.co.jp> wrote:
>>>> Hi everyone,
>>>>
>>>> In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that Apache
>>>> Hadoop developers at Yahoo!, Twitter, and other non-distributors work
>>> very
>>>> hard to maintenance Hadoop by cherry-picking patches to their own
>>> branches.
>>>>
>>>> I want to share the work with the community. If we can cherry-pick bug
>>> fix
>>>> patches and have more maintenance releases, it'd be very happy not only
>>> for
>>>> users but also for developers who work very hard for stabilizing their
>>> own
>>>> branches.
>>>>
>>>> To have more maintenance releases, I propose two changes:
>>>>
>>>> * Major/Minor/Trivial bug fixes can be cherry-picked
>>>> * (Roughly) Monthly maintenance release
>>>>
>>>> I would like to start the work from branch-2.6. If the change will be
>>>> accepted by the community, I'm willing to work for the maintenance, as a
>>>> release manager.
>>>>
>>>> Best regards,
>>>> Akira
>>>
>>
>>
>>
>> --
>> Karthik Kambatla
>> Software Engineer, Cloudera Inc.
>> --------------------------------------------
>> http://five.sentenc.es
>
Re: [DISCUSS] More Maintenance Releases
Posted by Allen Wittenauer <aw...@altiscale.com>.
If 2.6 is the target, someone will have to verify that any cherry-picked patches actually work with JDK6 since the PMC voted to officially kill backward compatibility in a minor release. It’s going to be easier and probably smarter to fix 2.7 if that’s really desired. [1]
Frankly, I’d rather see effort spent on stabilizing trunk and ditching the now broken branch-2. We’re approaching the 4 year anniversary of 0.23.0’s release (which later begat 2.x, which is already past the 3 year mark). It’s hard to claim health when its been so long since a branch off of trunk was cut and turned into something official.
[1] Kengo and I are hard at work getting multiJDK testing working in Yetus, but it’s not quite ready for prime time. :( It could certain help here, but… it’s not very stable yet.
On Jun 22, 2015, at 7:50 AM, Karthik Kambatla <ka...@cloudera.com> wrote:
> Thanks for starting this thread, Akira.
>
> +1 to more maintenance releases. More stable upstream releases avoids
> duplicating cherry-pick work across consumers/vendors, and shows the
> maturity of the project to users.
>
> I see value in backporting blocker/critical issues, but have mixed feelings
> about doing the same for major/minor/trivial issues. IMO, every commit has
> non-zero potential to introduce other bugs. Depending on the kind of fix
> (say, documentation), it might be okay to include these non-critical fixes.
> One approach could be to allow all bug fixes for 2.x.1, blocker/critical
> for 2.x.2, blocker for 2.x.3 (or something along those lines) to ensure
> increasing stability of maintenance releases?
>
> I am also +1 to any committer picking up RM duties for a maintenance
> release. It is healthy to have more people participate in the release
> process, so long as we have some method to maintenance release madness.
>
> A committer (who is not yet a PMC member) could be a Release Manager, but
> his vote is not binding for the release. I RM-ed the 2.5.x releases as a
> committer. RM-ing a release and voting non-binding could be a good way to
> remind the PMC to include the committer in PMC :)
>
> Cheers
> Karthik
>
> On Mon, Jun 22, 2015 at 4:36 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
>
>> Hi Akira,
>>
>> Thank you for starting interesting topic. +1 on the idea of More
>> Maintenance Releases for old branches. It would be good if this
>> activity is more coupled with Apache Yetus for users.
>>
>> BTW, I don't know one of committers, who is not PMC, can be a release
>> manager. Does anyone know about this? It's described in detail as
>> follows: http://hadoop.apache.org/bylaws#Decision+Making
>>
>>> Release Manager
>>> A Release Manager (RM) is a committer who volunteers to produce a
>> Release Candidate according to HowToRelease.
>>>
>>> Project Management Committee
>>> Deciding what is distributed as products of the Apache Hadoop project.
>> In particular all releases must be approved by the PMC
>>
>> Thanks,
>> - Tsuyoshi
>>
>> On Mon, Jun 22, 2015 at 6:43 PM, Akira AJISAKA
>> <aj...@oss.nttdata.co.jp> wrote:
>>> Hi everyone,
>>>
>>> In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that Apache
>>> Hadoop developers at Yahoo!, Twitter, and other non-distributors work
>> very
>>> hard to maintenance Hadoop by cherry-picking patches to their own
>> branches.
>>>
>>> I want to share the work with the community. If we can cherry-pick bug
>> fix
>>> patches and have more maintenance releases, it'd be very happy not only
>> for
>>> users but also for developers who work very hard for stabilizing their
>> own
>>> branches.
>>>
>>> To have more maintenance releases, I propose two changes:
>>>
>>> * Major/Minor/Trivial bug fixes can be cherry-picked
>>> * (Roughly) Monthly maintenance release
>>>
>>> I would like to start the work from branch-2.6. If the change will be
>>> accepted by the community, I'm willing to work for the maintenance, as a
>>> release manager.
>>>
>>> Best regards,
>>> Akira
>>
>
>
>
> --
> Karthik Kambatla
> Software Engineer, Cloudera Inc.
> --------------------------------------------
> http://five.sentenc.es
Re: [DISCUSS] More Maintenance Releases
Posted by Karthik Kambatla <ka...@cloudera.com>.
Thanks for starting this thread, Akira.
+1 to more maintenance releases. More stable upstream releases avoids
duplicating cherry-pick work across consumers/vendors, and shows the
maturity of the project to users.
I see value in backporting blocker/critical issues, but have mixed feelings
about doing the same for major/minor/trivial issues. IMO, every commit has
non-zero potential to introduce other bugs. Depending on the kind of fix
(say, documentation), it might be okay to include these non-critical fixes.
One approach could be to allow all bug fixes for 2.x.1, blocker/critical
for 2.x.2, blocker for 2.x.3 (or something along those lines) to ensure
increasing stability of maintenance releases?
I am also +1 to any committer picking up RM duties for a maintenance
release. It is healthy to have more people participate in the release
process, so long as we have some method to maintenance release madness.
A committer (who is not yet a PMC member) could be a Release Manager, but
his vote is not binding for the release. I RM-ed the 2.5.x releases as a
committer. RM-ing a release and voting non-binding could be a good way to
remind the PMC to include the committer in PMC :)
Cheers
Karthik
On Mon, Jun 22, 2015 at 4:36 AM, Tsuyoshi Ozawa <oz...@apache.org> wrote:
> Hi Akira,
>
> Thank you for starting interesting topic. +1 on the idea of More
> Maintenance Releases for old branches. It would be good if this
> activity is more coupled with Apache Yetus for users.
>
> BTW, I don't know one of committers, who is not PMC, can be a release
> manager. Does anyone know about this? It's described in detail as
> follows: http://hadoop.apache.org/bylaws#Decision+Making
>
> > Release Manager
> > A Release Manager (RM) is a committer who volunteers to produce a
> Release Candidate according to HowToRelease.
> >
> > Project Management Committee
> > Deciding what is distributed as products of the Apache Hadoop project.
> In particular all releases must be approved by the PMC
>
> Thanks,
> - Tsuyoshi
>
> On Mon, Jun 22, 2015 at 6:43 PM, Akira AJISAKA
> <aj...@oss.nttdata.co.jp> wrote:
> > Hi everyone,
> >
> > In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that Apache
> > Hadoop developers at Yahoo!, Twitter, and other non-distributors work
> very
> > hard to maintenance Hadoop by cherry-picking patches to their own
> branches.
> >
> > I want to share the work with the community. If we can cherry-pick bug
> fix
> > patches and have more maintenance releases, it'd be very happy not only
> for
> > users but also for developers who work very hard for stabilizing their
> own
> > branches.
> >
> > To have more maintenance releases, I propose two changes:
> >
> > * Major/Minor/Trivial bug fixes can be cherry-picked
> > * (Roughly) Monthly maintenance release
> >
> > I would like to start the work from branch-2.6. If the change will be
> > accepted by the community, I'm willing to work for the maintenance, as a
> > release manager.
> >
> > Best regards,
> > Akira
>
--
Karthik Kambatla
Software Engineer, Cloudera Inc.
--------------------------------------------
http://five.sentenc.es
Re: [DISCUSS] More Maintenance Releases
Posted by Tsuyoshi Ozawa <oz...@apache.org>.
Hi Akira,
Thank you for starting interesting topic. +1 on the idea of More
Maintenance Releases for old branches. It would be good if this
activity is more coupled with Apache Yetus for users.
BTW, I don't know one of committers, who is not PMC, can be a release
manager. Does anyone know about this? It's described in detail as
follows: http://hadoop.apache.org/bylaws#Decision+Making
> Release Manager
> A Release Manager (RM) is a committer who volunteers to produce a Release Candidate according to HowToRelease.
>
> Project Management Committee
> Deciding what is distributed as products of the Apache Hadoop project. In particular all releases must be approved by the PMC
Thanks,
- Tsuyoshi
On Mon, Jun 22, 2015 at 6:43 PM, Akira AJISAKA
<aj...@oss.nttdata.co.jp> wrote:
> Hi everyone,
>
> In Hadoop Summit, I joined HDFS BoF and heard from Jason Lowe that Apache
> Hadoop developers at Yahoo!, Twitter, and other non-distributors work very
> hard to maintenance Hadoop by cherry-picking patches to their own branches.
>
> I want to share the work with the community. If we can cherry-pick bug fix
> patches and have more maintenance releases, it'd be very happy not only for
> users but also for developers who work very hard for stabilizing their own
> branches.
>
> To have more maintenance releases, I propose two changes:
>
> * Major/Minor/Trivial bug fixes can be cherry-picked
> * (Roughly) Monthly maintenance release
>
> I would like to start the work from branch-2.6. If the change will be
> accepted by the community, I'm willing to work for the maintenance, as a
> release manager.
>
> Best regards,
> Akira