You are viewing a plain text version of this content. The canonical link for it is here.
Posted to yarn-dev@hadoop.apache.org by Junping Du <jd...@hortonworks.com> on 2017/11/14 01:02:24 UTC
Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current plan is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark the fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <ju...@apache.org>
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop community
has
voted to release Apache Hadoop 2.8.2, which is now available for download
from
Apache mirrors[1]. For download instructions please refer to the Apache
Hadoop
Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
our
newest stable release for entire Apache Hadoop project. For major changes
incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
This release has 315 resolved issues since previous 2.8.1 release with
following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the
downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Konstantin and Weiwei for calling attention. These two HDFS issues looks to be important to be fixed which are on my radar now.
I will hold on RC cut until we figure them out.
Thanks,
Junping
________________________________
From: Weiwei Yang <ch...@hotmail.com>
Sent: Tuesday, November 21, 2017 7:45 PM
To: Junping Du; Konstantin Shvachko
Cc: Wangda Tan; Andrew Wang; Zheng, Kai; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Agree with Konstantin. This two issues has been opened for a while but could not reach a consensus on the fix, hope this gets enough attention from the community and get them resolved.
Thanks
--
Weiwei
On 22 Nov 2017, 11:18 AM +0800, Konstantin Shvachko <sh...@gmail.com>, wrote:
I would consider these two blockers for 2.8.3 as they crash NN:
https://issues.apache.org/jira/browse/HDFS-12638
https://issues.apache.org/jira/browse/HDFS-12832
Thanks,
--Konstantin
On Tue, Nov 21, 2017 at 11:16 AM, Junping Du <jd...@hortonworks.com> wrote:
Thanks Andrew and Wangda for comments!
To me, an improvement with 17 patches is not a big problem as this is
self-contained and I didn't see a single line of delete/update on existing
code - well, arguably, patches with only adding code can also have big
impact but not likely the case here.
While the dependency discussions on HADOOP-14964 are still going on, I
will leave the decision to JIRA discussion based on which approach we will
choose(shaded?) and impact. If we cannot make consensus in short term,
probably we have to miss this in 2.8.3 release.
Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0
will get cut-off shortly.
Thanks,
Junping
________________________________
From: Wangda Tan <wheeleast@gmail.com
Sent: Tuesday, November 21, 2017 10:52 AM
To: Andrew Wang
Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org;
hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define,
improvement could be self-contained and useful, bug fix could be dangerous
in some cases. To me, If an improvement fixed some existing use case, and
the fix is self-contained. I will be open to bring such fix to maintenance
release. For example, in 2.8.2, we back ported CapacityScheduler intra
queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
big change in terms of patch size, but since it fixes broken use case
(balance user usage under Capacity Scheduler leaf queue), we backported it
to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to
committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.wang@cloudera.com<
mailto:andrew.wang@cloudera.com>> wrote:
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?
focusedCommentId=16247085&page=com.atlassian.jira.
plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jdu@hortonworks.com<mailto:jd
u@hortonworks.com>> wrote:
Thanks Kai for calling out this feature/improvement for attention and
Andrew for comments.
While I agree that maintenance release should focus on important bug fix
only, I doubt we have strict rules to disallow any features/improvements
to
land on maint release especially when those are small footprint or low
impact on existing code/features. In practice, we indeed has 77 new
features/improvements in latest 2.7.3 and 2.7.4 release.
Back to HADOOP-14964, I did a quick check and it looks like case here
belongs to self-contained improvement that has very low impact on
existing
code base, so I am OK with the improvement get landed on branch-2.8 in
case
it is well reviewed and tested.
However, as RM of branch-2.8, I have two concerns to accept it in our
2.8.3 release:
1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
for
several critical bug fixes and we should target to release it very soon -
my current plan is to cut RC out within this week inline with waiting
for 3.0.0 vote closing. Can this improvement be well tested against
branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
have strong commitment on test plan and activities in such a tight time.
2. Upgrading - I haven't heard we settle down the plan of releasing this
feature in 2.9.1 release - though I saw some discussions are going on
at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
this improvement, then users consuming this feature/improvement have no
2.9
release to upgrade or forcefully upgrade with regression. We may need a
better upgrade story here.
Pls let me know what you think. Thanks!
Thanks,
Junping
------------------------------
*From:* Andrew Wang <andrew.wang@cloudera.com<mailto:
andrew.wang@cloudera.com
*Sent:* Monday, November 20, 2017 10:22 PM
*To:* Zheng, Kai
*Cc:* Junping Du; common-dev@hadoop.apache.org<mailto:common-dev@hadoop.
apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org
;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
*Subject:* Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're
meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and
timely fashion, let's try to address that, not overload the meaning of
"maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zheng@intel.com<mailto:
kai.zheng@intel.com>> wrote:
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS
connector module? We have some Hadoop users that wish to have it by
default
for convenience, though in the past they used it by back porting
themselves. I have raised this and got thoughts from Chris and Steve.
Looks
like this is more wanted for 2.9 but I wanted to ask again here for
broad
feedback and thoughts by this chance. The back port patch is available
for
2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
we
can see some shift from 2.7.x, hence it's worth more important features
and
efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com<mailto:jdu@hortonworks.com
]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would
like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current
plan
is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark
the
fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting
closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com<ma...@gmail.com> <
dujunping@gmail.com<ma...@gmail.com>> on behalf of ??? <
junping_du@apache.org<mailto:junping_du@apache.org
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org<mailto:general@hadoop.apache.org
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop
community has voted to release Apache Hadoop 2.8.2, which is now
available
for download from Apache mirrors[1]. For download instructions please
refer
to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
and
our newest stable release for entire Apache Hadoop project. For major
changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
page[3].
This release has 315 resolved issues since previous 2.8.1 release with
following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the
downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
<mailto:common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
<mailto:common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<mailto:
hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
<mailto:hdfs-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Konstantin and Weiwei for calling attention. These two HDFS issues looks to be important to be fixed which are on my radar now.
I will hold on RC cut until we figure them out.
Thanks,
Junping
________________________________
From: Weiwei Yang <ch...@hotmail.com>
Sent: Tuesday, November 21, 2017 7:45 PM
To: Junping Du; Konstantin Shvachko
Cc: Wangda Tan; Andrew Wang; Zheng, Kai; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Agree with Konstantin. This two issues has been opened for a while but could not reach a consensus on the fix, hope this gets enough attention from the community and get them resolved.
Thanks
--
Weiwei
On 22 Nov 2017, 11:18 AM +0800, Konstantin Shvachko <sh...@gmail.com>, wrote:
I would consider these two blockers for 2.8.3 as they crash NN:
https://issues.apache.org/jira/browse/HDFS-12638
https://issues.apache.org/jira/browse/HDFS-12832
Thanks,
--Konstantin
On Tue, Nov 21, 2017 at 11:16 AM, Junping Du <jd...@hortonworks.com> wrote:
Thanks Andrew and Wangda for comments!
To me, an improvement with 17 patches is not a big problem as this is
self-contained and I didn't see a single line of delete/update on existing
code - well, arguably, patches with only adding code can also have big
impact but not likely the case here.
While the dependency discussions on HADOOP-14964 are still going on, I
will leave the decision to JIRA discussion based on which approach we will
choose(shaded?) and impact. If we cannot make consensus in short term,
probably we have to miss this in 2.8.3 release.
Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0
will get cut-off shortly.
Thanks,
Junping
________________________________
From: Wangda Tan <wheeleast@gmail.com
Sent: Tuesday, November 21, 2017 10:52 AM
To: Andrew Wang
Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org;
hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define,
improvement could be self-contained and useful, bug fix could be dangerous
in some cases. To me, If an improvement fixed some existing use case, and
the fix is self-contained. I will be open to bring such fix to maintenance
release. For example, in 2.8.2, we back ported CapacityScheduler intra
queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
big change in terms of patch size, but since it fixes broken use case
(balance user usage under Capacity Scheduler leaf queue), we backported it
to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to
committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.wang@cloudera.com<
mailto:andrew.wang@cloudera.com>> wrote:
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?
focusedCommentId=16247085&page=com.atlassian.jira.
plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jdu@hortonworks.com<mailto:jd
u@hortonworks.com>> wrote:
Thanks Kai for calling out this feature/improvement for attention and
Andrew for comments.
While I agree that maintenance release should focus on important bug fix
only, I doubt we have strict rules to disallow any features/improvements
to
land on maint release especially when those are small footprint or low
impact on existing code/features. In practice, we indeed has 77 new
features/improvements in latest 2.7.3 and 2.7.4 release.
Back to HADOOP-14964, I did a quick check and it looks like case here
belongs to self-contained improvement that has very low impact on
existing
code base, so I am OK with the improvement get landed on branch-2.8 in
case
it is well reviewed and tested.
However, as RM of branch-2.8, I have two concerns to accept it in our
2.8.3 release:
1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
for
several critical bug fixes and we should target to release it very soon -
my current plan is to cut RC out within this week inline with waiting
for 3.0.0 vote closing. Can this improvement be well tested against
branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
have strong commitment on test plan and activities in such a tight time.
2. Upgrading - I haven't heard we settle down the plan of releasing this
feature in 2.9.1 release - though I saw some discussions are going on
at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
this improvement, then users consuming this feature/improvement have no
2.9
release to upgrade or forcefully upgrade with regression. We may need a
better upgrade story here.
Pls let me know what you think. Thanks!
Thanks,
Junping
------------------------------
*From:* Andrew Wang <andrew.wang@cloudera.com<mailto:
andrew.wang@cloudera.com
*Sent:* Monday, November 20, 2017 10:22 PM
*To:* Zheng, Kai
*Cc:* Junping Du; common-dev@hadoop.apache.org<mailto:common-dev@hadoop.
apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org
;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
*Subject:* Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're
meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and
timely fashion, let's try to address that, not overload the meaning of
"maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zheng@intel.com<mailto:
kai.zheng@intel.com>> wrote:
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS
connector module? We have some Hadoop users that wish to have it by
default
for convenience, though in the past they used it by back porting
themselves. I have raised this and got thoughts from Chris and Steve.
Looks
like this is more wanted for 2.9 but I wanted to ask again here for
broad
feedback and thoughts by this chance. The back port patch is available
for
2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
we
can see some shift from 2.7.x, hence it's worth more important features
and
efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com<mailto:jdu@hortonworks.com
]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would
like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current
plan
is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark
the
fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting
closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com<ma...@gmail.com> <
dujunping@gmail.com<ma...@gmail.com>> on behalf of ??? <
junping_du@apache.org<mailto:junping_du@apache.org
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org<mailto:general@hadoop.apache.org
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop
community has voted to release Apache Hadoop 2.8.2, which is now
available
for download from Apache mirrors[1]. For download instructions please
refer
to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
and
our newest stable release for entire Apache Hadoop project. For major
changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
page[3].
This release has 315 resolved issues since previous 2.8.1 release with
following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the
downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
<mailto:common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
<mailto:common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<mailto:
hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
<mailto:hdfs-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Konstantin and Weiwei for calling attention. These two HDFS issues looks to be important to be fixed which are on my radar now.
I will hold on RC cut until we figure them out.
Thanks,
Junping
________________________________
From: Weiwei Yang <ch...@hotmail.com>
Sent: Tuesday, November 21, 2017 7:45 PM
To: Junping Du; Konstantin Shvachko
Cc: Wangda Tan; Andrew Wang; Zheng, Kai; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Agree with Konstantin. This two issues has been opened for a while but could not reach a consensus on the fix, hope this gets enough attention from the community and get them resolved.
Thanks
--
Weiwei
On 22 Nov 2017, 11:18 AM +0800, Konstantin Shvachko <sh...@gmail.com>, wrote:
I would consider these two blockers for 2.8.3 as they crash NN:
https://issues.apache.org/jira/browse/HDFS-12638
https://issues.apache.org/jira/browse/HDFS-12832
Thanks,
--Konstantin
On Tue, Nov 21, 2017 at 11:16 AM, Junping Du <jd...@hortonworks.com> wrote:
Thanks Andrew and Wangda for comments!
To me, an improvement with 17 patches is not a big problem as this is
self-contained and I didn't see a single line of delete/update on existing
code - well, arguably, patches with only adding code can also have big
impact but not likely the case here.
While the dependency discussions on HADOOP-14964 are still going on, I
will leave the decision to JIRA discussion based on which approach we will
choose(shaded?) and impact. If we cannot make consensus in short term,
probably we have to miss this in 2.8.3 release.
Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0
will get cut-off shortly.
Thanks,
Junping
________________________________
From: Wangda Tan <wheeleast@gmail.com
Sent: Tuesday, November 21, 2017 10:52 AM
To: Andrew Wang
Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org;
hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define,
improvement could be self-contained and useful, bug fix could be dangerous
in some cases. To me, If an improvement fixed some existing use case, and
the fix is self-contained. I will be open to bring such fix to maintenance
release. For example, in 2.8.2, we back ported CapacityScheduler intra
queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
big change in terms of patch size, but since it fixes broken use case
(balance user usage under Capacity Scheduler leaf queue), we backported it
to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to
committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.wang@cloudera.com<
mailto:andrew.wang@cloudera.com>> wrote:
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?
focusedCommentId=16247085&page=com.atlassian.jira.
plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jdu@hortonworks.com<mailto:jd
u@hortonworks.com>> wrote:
Thanks Kai for calling out this feature/improvement for attention and
Andrew for comments.
While I agree that maintenance release should focus on important bug fix
only, I doubt we have strict rules to disallow any features/improvements
to
land on maint release especially when those are small footprint or low
impact on existing code/features. In practice, we indeed has 77 new
features/improvements in latest 2.7.3 and 2.7.4 release.
Back to HADOOP-14964, I did a quick check and it looks like case here
belongs to self-contained improvement that has very low impact on
existing
code base, so I am OK with the improvement get landed on branch-2.8 in
case
it is well reviewed and tested.
However, as RM of branch-2.8, I have two concerns to accept it in our
2.8.3 release:
1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
for
several critical bug fixes and we should target to release it very soon -
my current plan is to cut RC out within this week inline with waiting
for 3.0.0 vote closing. Can this improvement be well tested against
branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
have strong commitment on test plan and activities in such a tight time.
2. Upgrading - I haven't heard we settle down the plan of releasing this
feature in 2.9.1 release - though I saw some discussions are going on
at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
this improvement, then users consuming this feature/improvement have no
2.9
release to upgrade or forcefully upgrade with regression. We may need a
better upgrade story here.
Pls let me know what you think. Thanks!
Thanks,
Junping
------------------------------
*From:* Andrew Wang <andrew.wang@cloudera.com<mailto:
andrew.wang@cloudera.com
*Sent:* Monday, November 20, 2017 10:22 PM
*To:* Zheng, Kai
*Cc:* Junping Du; common-dev@hadoop.apache.org<mailto:common-dev@hadoop.
apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org
;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
*Subject:* Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're
meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and
timely fashion, let's try to address that, not overload the meaning of
"maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zheng@intel.com<mailto:
kai.zheng@intel.com>> wrote:
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS
connector module? We have some Hadoop users that wish to have it by
default
for convenience, though in the past they used it by back porting
themselves. I have raised this and got thoughts from Chris and Steve.
Looks
like this is more wanted for 2.9 but I wanted to ask again here for
broad
feedback and thoughts by this chance. The back port patch is available
for
2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
we
can see some shift from 2.7.x, hence it's worth more important features
and
efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com<mailto:jdu@hortonworks.com
]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would
like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current
plan
is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark
the
fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting
closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com<ma...@gmail.com> <
dujunping@gmail.com<ma...@gmail.com>> on behalf of ??? <
junping_du@apache.org<mailto:junping_du@apache.org
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org<mailto:general@hadoop.apache.org
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop
community has voted to release Apache Hadoop 2.8.2, which is now
available
for download from Apache mirrors[1]. For download instructions please
refer
to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
and
our newest stable release for entire Apache Hadoop project. For major
changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
page[3].
This release has 315 resolved issues since previous 2.8.1 release with
following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the
downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
<mailto:common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
<mailto:common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<mailto:
hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
<mailto:hdfs-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Konstantin and Weiwei for calling attention. These two HDFS issues looks to be important to be fixed which are on my radar now.
I will hold on RC cut until we figure them out.
Thanks,
Junping
________________________________
From: Weiwei Yang <ch...@hotmail.com>
Sent: Tuesday, November 21, 2017 7:45 PM
To: Junping Du; Konstantin Shvachko
Cc: Wangda Tan; Andrew Wang; Zheng, Kai; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Agree with Konstantin. This two issues has been opened for a while but could not reach a consensus on the fix, hope this gets enough attention from the community and get them resolved.
Thanks
--
Weiwei
On 22 Nov 2017, 11:18 AM +0800, Konstantin Shvachko <sh...@gmail.com>, wrote:
I would consider these two blockers for 2.8.3 as they crash NN:
https://issues.apache.org/jira/browse/HDFS-12638
https://issues.apache.org/jira/browse/HDFS-12832
Thanks,
--Konstantin
On Tue, Nov 21, 2017 at 11:16 AM, Junping Du <jd...@hortonworks.com> wrote:
Thanks Andrew and Wangda for comments!
To me, an improvement with 17 patches is not a big problem as this is
self-contained and I didn't see a single line of delete/update on existing
code - well, arguably, patches with only adding code can also have big
impact but not likely the case here.
While the dependency discussions on HADOOP-14964 are still going on, I
will leave the decision to JIRA discussion based on which approach we will
choose(shaded?) and impact. If we cannot make consensus in short term,
probably we have to miss this in 2.8.3 release.
Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0
will get cut-off shortly.
Thanks,
Junping
________________________________
From: Wangda Tan <wheeleast@gmail.com
Sent: Tuesday, November 21, 2017 10:52 AM
To: Andrew Wang
Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org;
hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define,
improvement could be self-contained and useful, bug fix could be dangerous
in some cases. To me, If an improvement fixed some existing use case, and
the fix is self-contained. I will be open to bring such fix to maintenance
release. For example, in 2.8.2, we back ported CapacityScheduler intra
queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
big change in terms of patch size, but since it fixes broken use case
(balance user usage under Capacity Scheduler leaf queue), we backported it
to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to
committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.wang@cloudera.com<
mailto:andrew.wang@cloudera.com>> wrote:
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?
focusedCommentId=16247085&page=com.atlassian.jira.
plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jdu@hortonworks.com<mailto:jd
u@hortonworks.com>> wrote:
Thanks Kai for calling out this feature/improvement for attention and
Andrew for comments.
While I agree that maintenance release should focus on important bug fix
only, I doubt we have strict rules to disallow any features/improvements
to
land on maint release especially when those are small footprint or low
impact on existing code/features. In practice, we indeed has 77 new
features/improvements in latest 2.7.3 and 2.7.4 release.
Back to HADOOP-14964, I did a quick check and it looks like case here
belongs to self-contained improvement that has very low impact on
existing
code base, so I am OK with the improvement get landed on branch-2.8 in
case
it is well reviewed and tested.
However, as RM of branch-2.8, I have two concerns to accept it in our
2.8.3 release:
1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
for
several critical bug fixes and we should target to release it very soon -
my current plan is to cut RC out within this week inline with waiting
for 3.0.0 vote closing. Can this improvement be well tested against
branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
have strong commitment on test plan and activities in such a tight time.
2. Upgrading - I haven't heard we settle down the plan of releasing this
feature in 2.9.1 release - though I saw some discussions are going on
at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
this improvement, then users consuming this feature/improvement have no
2.9
release to upgrade or forcefully upgrade with regression. We may need a
better upgrade story here.
Pls let me know what you think. Thanks!
Thanks,
Junping
------------------------------
*From:* Andrew Wang <andrew.wang@cloudera.com<mailto:
andrew.wang@cloudera.com
*Sent:* Monday, November 20, 2017 10:22 PM
*To:* Zheng, Kai
*Cc:* Junping Du; common-dev@hadoop.apache.org<mailto:common-dev@hadoop.
apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org
;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
*Subject:* Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're
meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and
timely fashion, let's try to address that, not overload the meaning of
"maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zheng@intel.com<mailto:
kai.zheng@intel.com>> wrote:
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS
connector module? We have some Hadoop users that wish to have it by
default
for convenience, though in the past they used it by back porting
themselves. I have raised this and got thoughts from Chris and Steve.
Looks
like this is more wanted for 2.9 but I wanted to ask again here for
broad
feedback and thoughts by this chance. The back port patch is available
for
2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
we
can see some shift from 2.7.x, hence it's worth more important features
and
efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com<mailto:jdu@hortonworks.com
]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would
like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current
plan
is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark
the
fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting
closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com<ma...@gmail.com> <
dujunping@gmail.com<ma...@gmail.com>> on behalf of ??? <
junping_du@apache.org<mailto:junping_du@apache.org
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org<mailto:general@hadoop.apache.org
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop
community has voted to release Apache Hadoop 2.8.2, which is now
available
for download from Apache mirrors[1]. For download instructions please
refer
to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
and
our newest stable release for entire Apache Hadoop project. For major
changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
page[3].
This release has 315 resolved issues since previous 2.8.1 release with
following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the
downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
<mailto:common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
<mailto:common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<mailto:
hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
<mailto:hdfs-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Weiwei Yang <ch...@hotmail.com>.
Agree with Konstantin. This two issues has been opened for a while but could not reach a consensus on the fix, hope this gets enough attention from the community and get them resolved.
Thanks
--
Weiwei
On 22 Nov 2017, 11:18 AM +0800, Konstantin Shvachko <sh...@gmail.com>, wrote:
I would consider these two blockers for 2.8.3 as they crash NN:
https://issues.apache.org/jira/browse/HDFS-12638
https://issues.apache.org/jira/browse/HDFS-12832
Thanks,
--Konstantin
On Tue, Nov 21, 2017 at 11:16 AM, Junping Du <jd...@hortonworks.com> wrote:
Thanks Andrew and Wangda for comments!
To me, an improvement with 17 patches is not a big problem as this is
self-contained and I didn't see a single line of delete/update on existing
code - well, arguably, patches with only adding code can also have big
impact but not likely the case here.
While the dependency discussions on HADOOP-14964 are still going on, I
will leave the decision to JIRA discussion based on which approach we will
choose(shaded?) and impact. If we cannot make consensus in short term,
probably we have to miss this in 2.8.3 release.
Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0
will get cut-off shortly.
Thanks,
Junping
________________________________
From: Wangda Tan <wheeleast@gmail.com
Sent: Tuesday, November 21, 2017 10:52 AM
To: Andrew Wang
Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org;
hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define,
improvement could be self-contained and useful, bug fix could be dangerous
in some cases. To me, If an improvement fixed some existing use case, and
the fix is self-contained. I will be open to bring such fix to maintenance
release. For example, in 2.8.2, we back ported CapacityScheduler intra
queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
big change in terms of patch size, but since it fixes broken use case
(balance user usage under Capacity Scheduler leaf queue), we backported it
to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to
committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.wang@cloudera.com<
mailto:andrew.wang@cloudera.com>> wrote:
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?
focusedCommentId=16247085&page=com.atlassian.jira.
plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jdu@hortonworks.com<mailto:jd
u@hortonworks.com>> wrote:
Thanks Kai for calling out this feature/improvement for attention and
Andrew for comments.
While I agree that maintenance release should focus on important bug fix
only, I doubt we have strict rules to disallow any features/improvements
to
land on maint release especially when those are small footprint or low
impact on existing code/features. In practice, we indeed has 77 new
features/improvements in latest 2.7.3 and 2.7.4 release.
Back to HADOOP-14964, I did a quick check and it looks like case here
belongs to self-contained improvement that has very low impact on
existing
code base, so I am OK with the improvement get landed on branch-2.8 in
case
it is well reviewed and tested.
However, as RM of branch-2.8, I have two concerns to accept it in our
2.8.3 release:
1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
for
several critical bug fixes and we should target to release it very soon -
my current plan is to cut RC out within this week inline with waiting
for 3.0.0 vote closing. Can this improvement be well tested against
branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
have strong commitment on test plan and activities in such a tight time.
2. Upgrading - I haven't heard we settle down the plan of releasing this
feature in 2.9.1 release - though I saw some discussions are going on
at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
this improvement, then users consuming this feature/improvement have no
2.9
release to upgrade or forcefully upgrade with regression. We may need a
better upgrade story here.
Pls let me know what you think. Thanks!
Thanks,
Junping
------------------------------
*From:* Andrew Wang <andrew.wang@cloudera.com<mailto:
andrew.wang@cloudera.com
*Sent:* Monday, November 20, 2017 10:22 PM
*To:* Zheng, Kai
*Cc:* Junping Du; common-dev@hadoop.apache.org<mailto:common-dev@hadoop.
apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org
;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
*Subject:* Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're
meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and
timely fashion, let's try to address that, not overload the meaning of
"maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zheng@intel.com<mailto:
kai.zheng@intel.com>> wrote:
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS
connector module? We have some Hadoop users that wish to have it by
default
for convenience, though in the past they used it by back porting
themselves. I have raised this and got thoughts from Chris and Steve.
Looks
like this is more wanted for 2.9 but I wanted to ask again here for
broad
feedback and thoughts by this chance. The back port patch is available
for
2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
we
can see some shift from 2.7.x, hence it's worth more important features
and
efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com<mailto:jdu@hortonworks.com
]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would
like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current
plan
is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark
the
fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting
closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com<ma...@gmail.com> <
dujunping@gmail.com<ma...@gmail.com>> on behalf of 俊平堵 <
junping_du@apache.org<mailto:junping_du@apache.org
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org<mailto:general@hadoop.apache.org
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop
community has voted to release Apache Hadoop 2.8.2, which is now
available
for download from Apache mirrors[1]. For download instructions please
refer
to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
and
our newest stable release for entire Apache Hadoop project. For major
changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
page[3].
This release has 315 resolved issues since previous 2.8.1 release with
following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the
downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
<mailto:common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
<mailto:common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<mailto:
hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
<mailto:hdfs-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Weiwei Yang <ch...@hotmail.com>.
Agree with Konstantin. This two issues has been opened for a while but could not reach a consensus on the fix, hope this gets enough attention from the community and get them resolved.
Thanks
--
Weiwei
On 22 Nov 2017, 11:18 AM +0800, Konstantin Shvachko <sh...@gmail.com>, wrote:
I would consider these two blockers for 2.8.3 as they crash NN:
https://issues.apache.org/jira/browse/HDFS-12638
https://issues.apache.org/jira/browse/HDFS-12832
Thanks,
--Konstantin
On Tue, Nov 21, 2017 at 11:16 AM, Junping Du <jd...@hortonworks.com> wrote:
Thanks Andrew and Wangda for comments!
To me, an improvement with 17 patches is not a big problem as this is
self-contained and I didn't see a single line of delete/update on existing
code - well, arguably, patches with only adding code can also have big
impact but not likely the case here.
While the dependency discussions on HADOOP-14964 are still going on, I
will leave the decision to JIRA discussion based on which approach we will
choose(shaded?) and impact. If we cannot make consensus in short term,
probably we have to miss this in 2.8.3 release.
Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0
will get cut-off shortly.
Thanks,
Junping
________________________________
From: Wangda Tan <wheeleast@gmail.com
Sent: Tuesday, November 21, 2017 10:52 AM
To: Andrew Wang
Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org;
hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define,
improvement could be self-contained and useful, bug fix could be dangerous
in some cases. To me, If an improvement fixed some existing use case, and
the fix is self-contained. I will be open to bring such fix to maintenance
release. For example, in 2.8.2, we back ported CapacityScheduler intra
queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
big change in terms of patch size, but since it fixes broken use case
(balance user usage under Capacity Scheduler leaf queue), we backported it
to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to
committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.wang@cloudera.com<
mailto:andrew.wang@cloudera.com>> wrote:
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?
focusedCommentId=16247085&page=com.atlassian.jira.
plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jdu@hortonworks.com<mailto:jd
u@hortonworks.com>> wrote:
Thanks Kai for calling out this feature/improvement for attention and
Andrew for comments.
While I agree that maintenance release should focus on important bug fix
only, I doubt we have strict rules to disallow any features/improvements
to
land on maint release especially when those are small footprint or low
impact on existing code/features. In practice, we indeed has 77 new
features/improvements in latest 2.7.3 and 2.7.4 release.
Back to HADOOP-14964, I did a quick check and it looks like case here
belongs to self-contained improvement that has very low impact on
existing
code base, so I am OK with the improvement get landed on branch-2.8 in
case
it is well reviewed and tested.
However, as RM of branch-2.8, I have two concerns to accept it in our
2.8.3 release:
1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
for
several critical bug fixes and we should target to release it very soon -
my current plan is to cut RC out within this week inline with waiting
for 3.0.0 vote closing. Can this improvement be well tested against
branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
have strong commitment on test plan and activities in such a tight time.
2. Upgrading - I haven't heard we settle down the plan of releasing this
feature in 2.9.1 release - though I saw some discussions are going on
at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
this improvement, then users consuming this feature/improvement have no
2.9
release to upgrade or forcefully upgrade with regression. We may need a
better upgrade story here.
Pls let me know what you think. Thanks!
Thanks,
Junping
------------------------------
*From:* Andrew Wang <andrew.wang@cloudera.com<mailto:
andrew.wang@cloudera.com
*Sent:* Monday, November 20, 2017 10:22 PM
*To:* Zheng, Kai
*Cc:* Junping Du; common-dev@hadoop.apache.org<mailto:common-dev@hadoop.
apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org
;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
*Subject:* Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're
meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and
timely fashion, let's try to address that, not overload the meaning of
"maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zheng@intel.com<mailto:
kai.zheng@intel.com>> wrote:
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS
connector module? We have some Hadoop users that wish to have it by
default
for convenience, though in the past they used it by back porting
themselves. I have raised this and got thoughts from Chris and Steve.
Looks
like this is more wanted for 2.9 but I wanted to ask again here for
broad
feedback and thoughts by this chance. The back port patch is available
for
2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
we
can see some shift from 2.7.x, hence it's worth more important features
and
efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com<mailto:jdu@hortonworks.com
]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would
like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current
plan
is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark
the
fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting
closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com<ma...@gmail.com> <
dujunping@gmail.com<ma...@gmail.com>> on behalf of 俊平堵 <
junping_du@apache.org<mailto:junping_du@apache.org
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org<mailto:general@hadoop.apache.org
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop
community has voted to release Apache Hadoop 2.8.2, which is now
available
for download from Apache mirrors[1]. For download instructions please
refer
to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
and
our newest stable release for entire Apache Hadoop project. For major
changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
page[3].
This release has 315 resolved issues since previous 2.8.1 release with
following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the
downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
<mailto:common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
<mailto:common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<mailto:
hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
<mailto:hdfs-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Weiwei Yang <ch...@hotmail.com>.
Agree with Konstantin. This two issues has been opened for a while but could not reach a consensus on the fix, hope this gets enough attention from the community and get them resolved.
Thanks
--
Weiwei
On 22 Nov 2017, 11:18 AM +0800, Konstantin Shvachko <sh...@gmail.com>, wrote:
I would consider these two blockers for 2.8.3 as they crash NN:
https://issues.apache.org/jira/browse/HDFS-12638
https://issues.apache.org/jira/browse/HDFS-12832
Thanks,
--Konstantin
On Tue, Nov 21, 2017 at 11:16 AM, Junping Du <jd...@hortonworks.com> wrote:
Thanks Andrew and Wangda for comments!
To me, an improvement with 17 patches is not a big problem as this is
self-contained and I didn't see a single line of delete/update on existing
code - well, arguably, patches with only adding code can also have big
impact but not likely the case here.
While the dependency discussions on HADOOP-14964 are still going on, I
will leave the decision to JIRA discussion based on which approach we will
choose(shaded?) and impact. If we cannot make consensus in short term,
probably we have to miss this in 2.8.3 release.
Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0
will get cut-off shortly.
Thanks,
Junping
________________________________
From: Wangda Tan <wheeleast@gmail.com
Sent: Tuesday, November 21, 2017 10:52 AM
To: Andrew Wang
Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org;
hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define,
improvement could be self-contained and useful, bug fix could be dangerous
in some cases. To me, If an improvement fixed some existing use case, and
the fix is self-contained. I will be open to bring such fix to maintenance
release. For example, in 2.8.2, we back ported CapacityScheduler intra
queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
big change in terms of patch size, but since it fixes broken use case
(balance user usage under Capacity Scheduler leaf queue), we backported it
to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to
committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.wang@cloudera.com<
mailto:andrew.wang@cloudera.com>> wrote:
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?
focusedCommentId=16247085&page=com.atlassian.jira.
plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jdu@hortonworks.com<mailto:jd
u@hortonworks.com>> wrote:
Thanks Kai for calling out this feature/improvement for attention and
Andrew for comments.
While I agree that maintenance release should focus on important bug fix
only, I doubt we have strict rules to disallow any features/improvements
to
land on maint release especially when those are small footprint or low
impact on existing code/features. In practice, we indeed has 77 new
features/improvements in latest 2.7.3 and 2.7.4 release.
Back to HADOOP-14964, I did a quick check and it looks like case here
belongs to self-contained improvement that has very low impact on
existing
code base, so I am OK with the improvement get landed on branch-2.8 in
case
it is well reviewed and tested.
However, as RM of branch-2.8, I have two concerns to accept it in our
2.8.3 release:
1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
for
several critical bug fixes and we should target to release it very soon -
my current plan is to cut RC out within this week inline with waiting
for 3.0.0 vote closing. Can this improvement be well tested against
branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
have strong commitment on test plan and activities in such a tight time.
2. Upgrading - I haven't heard we settle down the plan of releasing this
feature in 2.9.1 release - though I saw some discussions are going on
at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
this improvement, then users consuming this feature/improvement have no
2.9
release to upgrade or forcefully upgrade with regression. We may need a
better upgrade story here.
Pls let me know what you think. Thanks!
Thanks,
Junping
------------------------------
*From:* Andrew Wang <andrew.wang@cloudera.com<mailto:
andrew.wang@cloudera.com
*Sent:* Monday, November 20, 2017 10:22 PM
*To:* Zheng, Kai
*Cc:* Junping Du; common-dev@hadoop.apache.org<mailto:common-dev@hadoop.
apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org
;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
*Subject:* Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're
meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and
timely fashion, let's try to address that, not overload the meaning of
"maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zheng@intel.com<mailto:
kai.zheng@intel.com>> wrote:
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS
connector module? We have some Hadoop users that wish to have it by
default
for convenience, though in the past they used it by back porting
themselves. I have raised this and got thoughts from Chris and Steve.
Looks
like this is more wanted for 2.9 but I wanted to ask again here for
broad
feedback and thoughts by this chance. The back port patch is available
for
2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
we
can see some shift from 2.7.x, hence it's worth more important features
and
efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com<mailto:jdu@hortonworks.com
]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would
like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current
plan
is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark
the
fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting
closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com<ma...@gmail.com> <
dujunping@gmail.com<ma...@gmail.com>> on behalf of 俊平堵 <
junping_du@apache.org<mailto:junping_du@apache.org
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org<mailto:general@hadoop.apache.org
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop
community has voted to release Apache Hadoop 2.8.2, which is now
available
for download from Apache mirrors[1]. For download instructions please
refer
to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
and
our newest stable release for entire Apache Hadoop project. For major
changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
page[3].
This release has 315 resolved issues since previous 2.8.1 release with
following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the
downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
<mailto:common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
<mailto:common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<mailto:
hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
<mailto:hdfs-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Weiwei Yang <ch...@hotmail.com>.
Agree with Konstantin. This two issues has been opened for a while but could not reach a consensus on the fix, hope this gets enough attention from the community and get them resolved.
Thanks
--
Weiwei
On 22 Nov 2017, 11:18 AM +0800, Konstantin Shvachko <sh...@gmail.com>, wrote:
I would consider these two blockers for 2.8.3 as they crash NN:
https://issues.apache.org/jira/browse/HDFS-12638
https://issues.apache.org/jira/browse/HDFS-12832
Thanks,
--Konstantin
On Tue, Nov 21, 2017 at 11:16 AM, Junping Du <jd...@hortonworks.com> wrote:
Thanks Andrew and Wangda for comments!
To me, an improvement with 17 patches is not a big problem as this is
self-contained and I didn't see a single line of delete/update on existing
code - well, arguably, patches with only adding code can also have big
impact but not likely the case here.
While the dependency discussions on HADOOP-14964 are still going on, I
will leave the decision to JIRA discussion based on which approach we will
choose(shaded?) and impact. If we cannot make consensus in short term,
probably we have to miss this in 2.8.3 release.
Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0
will get cut-off shortly.
Thanks,
Junping
________________________________
From: Wangda Tan <wheeleast@gmail.com
Sent: Tuesday, November 21, 2017 10:52 AM
To: Andrew Wang
Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org;
hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define,
improvement could be self-contained and useful, bug fix could be dangerous
in some cases. To me, If an improvement fixed some existing use case, and
the fix is self-contained. I will be open to bring such fix to maintenance
release. For example, in 2.8.2, we back ported CapacityScheduler intra
queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
big change in terms of patch size, but since it fixes broken use case
(balance user usage under Capacity Scheduler leaf queue), we backported it
to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to
committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.wang@cloudera.com<
mailto:andrew.wang@cloudera.com>> wrote:
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?
focusedCommentId=16247085&page=com.atlassian.jira.
plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jdu@hortonworks.com<mailto:jd
u@hortonworks.com>> wrote:
Thanks Kai for calling out this feature/improvement for attention and
Andrew for comments.
While I agree that maintenance release should focus on important bug fix
only, I doubt we have strict rules to disallow any features/improvements
to
land on maint release especially when those are small footprint or low
impact on existing code/features. In practice, we indeed has 77 new
features/improvements in latest 2.7.3 and 2.7.4 release.
Back to HADOOP-14964, I did a quick check and it looks like case here
belongs to self-contained improvement that has very low impact on
existing
code base, so I am OK with the improvement get landed on branch-2.8 in
case
it is well reviewed and tested.
However, as RM of branch-2.8, I have two concerns to accept it in our
2.8.3 release:
1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
for
several critical bug fixes and we should target to release it very soon -
my current plan is to cut RC out within this week inline with waiting
for 3.0.0 vote closing. Can this improvement be well tested against
branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
have strong commitment on test plan and activities in such a tight time.
2. Upgrading - I haven't heard we settle down the plan of releasing this
feature in 2.9.1 release - though I saw some discussions are going on
at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
this improvement, then users consuming this feature/improvement have no
2.9
release to upgrade or forcefully upgrade with regression. We may need a
better upgrade story here.
Pls let me know what you think. Thanks!
Thanks,
Junping
------------------------------
*From:* Andrew Wang <andrew.wang@cloudera.com<mailto:
andrew.wang@cloudera.com
*Sent:* Monday, November 20, 2017 10:22 PM
*To:* Zheng, Kai
*Cc:* Junping Du; common-dev@hadoop.apache.org<mailto:common-dev@hadoop.
apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org
;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
*Subject:* Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're
meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and
timely fashion, let's try to address that, not overload the meaning of
"maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zheng@intel.com<mailto:
kai.zheng@intel.com>> wrote:
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS
connector module? We have some Hadoop users that wish to have it by
default
for convenience, though in the past they used it by back porting
themselves. I have raised this and got thoughts from Chris and Steve.
Looks
like this is more wanted for 2.9 but I wanted to ask again here for
broad
feedback and thoughts by this chance. The back port patch is available
for
2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
we
can see some shift from 2.7.x, hence it's worth more important features
and
efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com<mailto:jdu@hortonworks.com
]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
yarn-dev@hadoop.apache.org<mailto:yarn-dev@hadoop.apache.org
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would
like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current
plan
is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark
the
fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting
closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com<ma...@gmail.com> <
dujunping@gmail.com<ma...@gmail.com>> on behalf of 俊平堵 <
junping_du@apache.org<mailto:junping_du@apache.org
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org<mailto:general@hadoop.apache.org
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop
community has voted to release Apache Hadoop 2.8.2, which is now
available
for download from Apache mirrors[1]. For download instructions please
refer
to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
and
our newest stable release for entire Apache Hadoop project. For major
changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
page[3].
This release has 315 resolved issues since previous 2.8.1 release with
following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the
downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
<mailto:common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
<mailto:common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<mailto:
hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
<mailto:hdfs-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Konstantin Shvachko <sh...@gmail.com>.
I would consider these two blockers for 2.8.3 as they crash NN:
https://issues.apache.org/jira/browse/HDFS-12638
https://issues.apache.org/jira/browse/HDFS-12832
Thanks,
--Konstantin
On Tue, Nov 21, 2017 at 11:16 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks Andrew and Wangda for comments!
>
> To me, an improvement with 17 patches is not a big problem as this is
> self-contained and I didn't see a single line of delete/update on existing
> code - well, arguably, patches with only adding code can also have big
> impact but not likely the case here.
>
> While the dependency discussions on HADOOP-14964 are still going on, I
> will leave the decision to JIRA discussion based on which approach we will
> choose(shaded?) and impact. If we cannot make consensus in short term,
> probably we have to miss this in 2.8.3 release.
>
>
> Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0
> will get cut-off shortly.
>
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Wangda Tan <wh...@gmail.com>
> Sent: Tuesday, November 21, 2017 10:52 AM
> To: Andrew Wang
> Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org
> Subject: Re: Apache Hadoop 2.8.3 Release Plan
>
> Thanks Junping for driving this.
>
> For the bug fix vs. improvement, it is actually very hard to define,
> improvement could be self-contained and useful, bug fix could be dangerous
> in some cases. To me, If an improvement fixed some existing use case, and
> the fix is self-contained. I will be open to bring such fix to maintenance
> release. For example, in 2.8.2, we back ported CapacityScheduler intra
> queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
> big change in terms of patch size, but since it fixes broken use case
> (balance user usage under Capacity Scheduler leaf queue), we backported it
> to 2.8.2 after thorough tests and validations by Yahoo.
>
> I'm not quite familiar with HADOOP-14964, I will leave the decision to
> committers who know more about the field.
>
> Just my 2 cents.
>
> Regards,
> Wangda
>
>
> On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.wang@cloudera.com<
> mailto:andrew.wang@cloudera.com>> wrote:
> The Aliyun OSS code isn't a small improvement. If you look at Sammi's
> comment
> <https://issues.apache.org/jira/browse/HADOOP-14964?
> focusedCommentId=16247085&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
> it's
> a 17 patch series that is being backported in one shot. What we're talking
> about is equivalent to merging a feature branch in a maintenance release. I
> see that Kai and Chris are having a discussion about the dependency
> changes, which indicates this is not a zero-risk change either. We really
> should not be changing dependency versions in a maintenance unless it's
> because of a bug.
>
> It's unfortunate from a timing perspective that this missed 2.9.0, but I
> still think it should wait for the next minor. Merging a feature into a
> maintenance release sets the wrong precedent.
>
> Best,
> Andrew
>
> On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
>
> > Thanks Kai for calling out this feature/improvement for attention and
> > Andrew for comments.
> >
> >
> > While I agree that maintenance release should focus on important bug fix
> > only, I doubt we have strict rules to disallow any features/improvements
> to
> > land on maint release especially when those are small footprint or low
> > impact on existing code/features. In practice, we indeed has 77 new
> > features/improvements in latest 2.7.3 and 2.7.4 release.
> >
> >
> > Back to HADOOP-14964, I did a quick check and it looks like case here
> > belongs to self-contained improvement that has very low impact on
> existing
> > code base, so I am OK with the improvement get landed on branch-2.8 in
> case
> > it is well reviewed and tested.
> >
> >
> > However, as RM of branch-2.8, I have two concerns to accept it in our
> > 2.8.3 release:
> >
> > 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
> for
> > several critical bug fixes and we should target to release it very soon -
> > my current plan is to cut RC out within this week inline with waiting
> > for 3.0.0 vote closing. Can this improvement be well tested against
> > branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> > have strong commitment on test plan and activities in such a tight time.
> >
> >
> > 2. Upgrading - I haven't heard we settle down the plan of releasing this
> > feature in 2.9.1 release - though I saw some discussions are going on
> > at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> > this improvement, then users consuming this feature/improvement have no
> 2.9
> > release to upgrade or forcefully upgrade with regression. We may need a
> > better upgrade story here.
> >
> >
> > Pls let me know what you think. Thanks!
> >
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ------------------------------
> > *From:* Andrew Wang <andrew.wang@cloudera.com<mailto:
> andrew.wang@cloudera.com>>
> > *Sent:* Monday, November 20, 2017 10:22 PM
> > *To:* Zheng, Kai
> > *Cc:* Junping Du; common-dev@hadoop.apache.org<mailto:common-dev@hadoop.
> apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org
> >;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
> >
> > I'm against including new features in maintenance releases, since they're
> > meant to be bug-fix only.
> >
> > If we're struggling with being able to deliver new features in a safe and
> > timely fashion, let's try to address that, not overload the meaning of
> > "maintenance release".
> >
> > Best,
> > Andrew
> >
> > On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zheng@intel.com<mailto:
> kai.zheng@intel.com>> wrote:
> >
> >> Hi Junping,
> >>
> >> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
> >>
> >> I have an ask, is it convenient to include the back port work for OSS
> >> connector module? We have some Hadoop users that wish to have it by
> default
> >> for convenience, though in the past they used it by back porting
> >> themselves. I have raised this and got thoughts from Chris and Steve.
> Looks
> >> like this is more wanted for 2.9 but I wanted to ask again here for
> broad
> >> feedback and thoughts by this chance. The back port patch is available
> for
> >> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
> we
> >> can see some shift from 2.7.x, hence it's worth more important features
> and
> >> efforts. How would you think? Thanks!
> >>
> >> https://issues.apache.org/jira/browse/HADOOP-14964
> >>
> >> Regards,
> >> Kai
> >>
> >> -----Original Message-----
> >> From: Junping Du [mailto:jdu@hortonworks.com<mailto:jdu@hortonworks.com
> >]
> >> Sent: Tuesday, November 14, 2017 9:02 AM
> >> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: Apache Hadoop 2.8.3 Release Plan
> >>
> >> Hi,
> >> We have several important fixes get landed on branch-2.8 and I would
> >> like to cut off branch-2.8.3 now to start 2.8.3 release work.
> >> So far, I don't see any pending blockers on 2.8.3, so my current
> plan
> >> is to cut off 1st RC of 2.8.3 in next several days:
> >> - For all coming commits to land on branch-2.8, please mark
> the
> >> fix version as 2.8.4.
> >> - If there is a really important fix for 2.8.3 and getting
> >> closed, please notify me ahead before landing it on branch-2.8.3.
> >> Please let me know if you have any thoughts or comments on the plan.
> >>
> >> Thanks,
> >>
> >> Junping
> >> ________________________________________
> >> From: dujunping@gmail.com<ma...@gmail.com> <
> dujunping@gmail.com<ma...@gmail.com>> on behalf of 俊平堵 <
> >> junping_du@apache.org<ma...@apache.org>>
> >> Sent: Friday, October 27, 2017 3:33 PM
> >> To: general@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
> >>
> >> Hi all,
> >>
> >> It gives me great pleasure to announce that the Apache Hadoop
> >> community has voted to release Apache Hadoop 2.8.2, which is now
> available
> >> for download from Apache mirrors[1]. For download instructions please
> refer
> >> to the Apache Hadoop Release page [2].
> >>
> >> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
> and
> >> our newest stable release for entire Apache Hadoop project. For major
> >> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
> page[3].
> >>
> >> This release has 315 resolved issues since previous 2.8.1 release with
> >> following
> >> breakdown:
> >> - 91 in Hadoop Common
> >> - 99 in HDFS
> >> - 105 in YARN
> >> - 20 in MapReduce
> >> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
> >>
> >> The release news is posted on the Hadoop website too, you can go to the
> >> downloads section directly [6].
> >>
> >> Thank you all for contributing to the Apache Hadoop release!
> >>
> >>
> >> Cheers,
> >>
> >> Junping
> >>
> >>
> >> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
> >>
> >> [2] http://hadoop.apache.org/releases.html
> >>
> >> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
> >>
> >> [4]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/CHANGES.2.8.2.html
> >>
> >> [5]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
> >>
> >> [6] http://hadoop.apache.org/releases.html#Download
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org>
> >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<mailto:
> hdfs-dev-unsubscribe@hadoop.apache.org>
> >> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
> >>
> >>
> >
>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Konstantin Shvachko <sh...@gmail.com>.
I would consider these two blockers for 2.8.3 as they crash NN:
https://issues.apache.org/jira/browse/HDFS-12638
https://issues.apache.org/jira/browse/HDFS-12832
Thanks,
--Konstantin
On Tue, Nov 21, 2017 at 11:16 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks Andrew and Wangda for comments!
>
> To me, an improvement with 17 patches is not a big problem as this is
> self-contained and I didn't see a single line of delete/update on existing
> code - well, arguably, patches with only adding code can also have big
> impact but not likely the case here.
>
> While the dependency discussions on HADOOP-14964 are still going on, I
> will leave the decision to JIRA discussion based on which approach we will
> choose(shaded?) and impact. If we cannot make consensus in short term,
> probably we have to miss this in 2.8.3 release.
>
>
> Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0
> will get cut-off shortly.
>
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Wangda Tan <wh...@gmail.com>
> Sent: Tuesday, November 21, 2017 10:52 AM
> To: Andrew Wang
> Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org
> Subject: Re: Apache Hadoop 2.8.3 Release Plan
>
> Thanks Junping for driving this.
>
> For the bug fix vs. improvement, it is actually very hard to define,
> improvement could be self-contained and useful, bug fix could be dangerous
> in some cases. To me, If an improvement fixed some existing use case, and
> the fix is self-contained. I will be open to bring such fix to maintenance
> release. For example, in 2.8.2, we back ported CapacityScheduler intra
> queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
> big change in terms of patch size, but since it fixes broken use case
> (balance user usage under Capacity Scheduler leaf queue), we backported it
> to 2.8.2 after thorough tests and validations by Yahoo.
>
> I'm not quite familiar with HADOOP-14964, I will leave the decision to
> committers who know more about the field.
>
> Just my 2 cents.
>
> Regards,
> Wangda
>
>
> On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.wang@cloudera.com<
> mailto:andrew.wang@cloudera.com>> wrote:
> The Aliyun OSS code isn't a small improvement. If you look at Sammi's
> comment
> <https://issues.apache.org/jira/browse/HADOOP-14964?
> focusedCommentId=16247085&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
> it's
> a 17 patch series that is being backported in one shot. What we're talking
> about is equivalent to merging a feature branch in a maintenance release. I
> see that Kai and Chris are having a discussion about the dependency
> changes, which indicates this is not a zero-risk change either. We really
> should not be changing dependency versions in a maintenance unless it's
> because of a bug.
>
> It's unfortunate from a timing perspective that this missed 2.9.0, but I
> still think it should wait for the next minor. Merging a feature into a
> maintenance release sets the wrong precedent.
>
> Best,
> Andrew
>
> On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
>
> > Thanks Kai for calling out this feature/improvement for attention and
> > Andrew for comments.
> >
> >
> > While I agree that maintenance release should focus on important bug fix
> > only, I doubt we have strict rules to disallow any features/improvements
> to
> > land on maint release especially when those are small footprint or low
> > impact on existing code/features. In practice, we indeed has 77 new
> > features/improvements in latest 2.7.3 and 2.7.4 release.
> >
> >
> > Back to HADOOP-14964, I did a quick check and it looks like case here
> > belongs to self-contained improvement that has very low impact on
> existing
> > code base, so I am OK with the improvement get landed on branch-2.8 in
> case
> > it is well reviewed and tested.
> >
> >
> > However, as RM of branch-2.8, I have two concerns to accept it in our
> > 2.8.3 release:
> >
> > 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
> for
> > several critical bug fixes and we should target to release it very soon -
> > my current plan is to cut RC out within this week inline with waiting
> > for 3.0.0 vote closing. Can this improvement be well tested against
> > branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> > have strong commitment on test plan and activities in such a tight time.
> >
> >
> > 2. Upgrading - I haven't heard we settle down the plan of releasing this
> > feature in 2.9.1 release - though I saw some discussions are going on
> > at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> > this improvement, then users consuming this feature/improvement have no
> 2.9
> > release to upgrade or forcefully upgrade with regression. We may need a
> > better upgrade story here.
> >
> >
> > Pls let me know what you think. Thanks!
> >
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ------------------------------
> > *From:* Andrew Wang <andrew.wang@cloudera.com<mailto:
> andrew.wang@cloudera.com>>
> > *Sent:* Monday, November 20, 2017 10:22 PM
> > *To:* Zheng, Kai
> > *Cc:* Junping Du; common-dev@hadoop.apache.org<mailto:common-dev@hadoop.
> apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org
> >;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
> >
> > I'm against including new features in maintenance releases, since they're
> > meant to be bug-fix only.
> >
> > If we're struggling with being able to deliver new features in a safe and
> > timely fashion, let's try to address that, not overload the meaning of
> > "maintenance release".
> >
> > Best,
> > Andrew
> >
> > On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zheng@intel.com<mailto:
> kai.zheng@intel.com>> wrote:
> >
> >> Hi Junping,
> >>
> >> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
> >>
> >> I have an ask, is it convenient to include the back port work for OSS
> >> connector module? We have some Hadoop users that wish to have it by
> default
> >> for convenience, though in the past they used it by back porting
> >> themselves. I have raised this and got thoughts from Chris and Steve.
> Looks
> >> like this is more wanted for 2.9 but I wanted to ask again here for
> broad
> >> feedback and thoughts by this chance. The back port patch is available
> for
> >> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
> we
> >> can see some shift from 2.7.x, hence it's worth more important features
> and
> >> efforts. How would you think? Thanks!
> >>
> >> https://issues.apache.org/jira/browse/HADOOP-14964
> >>
> >> Regards,
> >> Kai
> >>
> >> -----Original Message-----
> >> From: Junping Du [mailto:jdu@hortonworks.com<mailto:jdu@hortonworks.com
> >]
> >> Sent: Tuesday, November 14, 2017 9:02 AM
> >> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: Apache Hadoop 2.8.3 Release Plan
> >>
> >> Hi,
> >> We have several important fixes get landed on branch-2.8 and I would
> >> like to cut off branch-2.8.3 now to start 2.8.3 release work.
> >> So far, I don't see any pending blockers on 2.8.3, so my current
> plan
> >> is to cut off 1st RC of 2.8.3 in next several days:
> >> - For all coming commits to land on branch-2.8, please mark
> the
> >> fix version as 2.8.4.
> >> - If there is a really important fix for 2.8.3 and getting
> >> closed, please notify me ahead before landing it on branch-2.8.3.
> >> Please let me know if you have any thoughts or comments on the plan.
> >>
> >> Thanks,
> >>
> >> Junping
> >> ________________________________________
> >> From: dujunping@gmail.com<ma...@gmail.com> <
> dujunping@gmail.com<ma...@gmail.com>> on behalf of 俊平堵 <
> >> junping_du@apache.org<ma...@apache.org>>
> >> Sent: Friday, October 27, 2017 3:33 PM
> >> To: general@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
> >>
> >> Hi all,
> >>
> >> It gives me great pleasure to announce that the Apache Hadoop
> >> community has voted to release Apache Hadoop 2.8.2, which is now
> available
> >> for download from Apache mirrors[1]. For download instructions please
> refer
> >> to the Apache Hadoop Release page [2].
> >>
> >> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
> and
> >> our newest stable release for entire Apache Hadoop project. For major
> >> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
> page[3].
> >>
> >> This release has 315 resolved issues since previous 2.8.1 release with
> >> following
> >> breakdown:
> >> - 91 in Hadoop Common
> >> - 99 in HDFS
> >> - 105 in YARN
> >> - 20 in MapReduce
> >> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
> >>
> >> The release news is posted on the Hadoop website too, you can go to the
> >> downloads section directly [6].
> >>
> >> Thank you all for contributing to the Apache Hadoop release!
> >>
> >>
> >> Cheers,
> >>
> >> Junping
> >>
> >>
> >> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
> >>
> >> [2] http://hadoop.apache.org/releases.html
> >>
> >> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
> >>
> >> [4]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/CHANGES.2.8.2.html
> >>
> >> [5]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
> >>
> >> [6] http://hadoop.apache.org/releases.html#Download
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org>
> >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<mailto:
> hdfs-dev-unsubscribe@hadoop.apache.org>
> >> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
> >>
> >>
> >
>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Konstantin Shvachko <sh...@gmail.com>.
I would consider these two blockers for 2.8.3 as they crash NN:
https://issues.apache.org/jira/browse/HDFS-12638
https://issues.apache.org/jira/browse/HDFS-12832
Thanks,
--Konstantin
On Tue, Nov 21, 2017 at 11:16 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks Andrew and Wangda for comments!
>
> To me, an improvement with 17 patches is not a big problem as this is
> self-contained and I didn't see a single line of delete/update on existing
> code - well, arguably, patches with only adding code can also have big
> impact but not likely the case here.
>
> While the dependency discussions on HADOOP-14964 are still going on, I
> will leave the decision to JIRA discussion based on which approach we will
> choose(shaded?) and impact. If we cannot make consensus in short term,
> probably we have to miss this in 2.8.3 release.
>
>
> Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0
> will get cut-off shortly.
>
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Wangda Tan <wh...@gmail.com>
> Sent: Tuesday, November 21, 2017 10:52 AM
> To: Andrew Wang
> Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org
> Subject: Re: Apache Hadoop 2.8.3 Release Plan
>
> Thanks Junping for driving this.
>
> For the bug fix vs. improvement, it is actually very hard to define,
> improvement could be self-contained and useful, bug fix could be dangerous
> in some cases. To me, If an improvement fixed some existing use case, and
> the fix is self-contained. I will be open to bring such fix to maintenance
> release. For example, in 2.8.2, we back ported CapacityScheduler intra
> queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
> big change in terms of patch size, but since it fixes broken use case
> (balance user usage under Capacity Scheduler leaf queue), we backported it
> to 2.8.2 after thorough tests and validations by Yahoo.
>
> I'm not quite familiar with HADOOP-14964, I will leave the decision to
> committers who know more about the field.
>
> Just my 2 cents.
>
> Regards,
> Wangda
>
>
> On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.wang@cloudera.com<
> mailto:andrew.wang@cloudera.com>> wrote:
> The Aliyun OSS code isn't a small improvement. If you look at Sammi's
> comment
> <https://issues.apache.org/jira/browse/HADOOP-14964?
> focusedCommentId=16247085&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
> it's
> a 17 patch series that is being backported in one shot. What we're talking
> about is equivalent to merging a feature branch in a maintenance release. I
> see that Kai and Chris are having a discussion about the dependency
> changes, which indicates this is not a zero-risk change either. We really
> should not be changing dependency versions in a maintenance unless it's
> because of a bug.
>
> It's unfortunate from a timing perspective that this missed 2.9.0, but I
> still think it should wait for the next minor. Merging a feature into a
> maintenance release sets the wrong precedent.
>
> Best,
> Andrew
>
> On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
>
> > Thanks Kai for calling out this feature/improvement for attention and
> > Andrew for comments.
> >
> >
> > While I agree that maintenance release should focus on important bug fix
> > only, I doubt we have strict rules to disallow any features/improvements
> to
> > land on maint release especially when those are small footprint or low
> > impact on existing code/features. In practice, we indeed has 77 new
> > features/improvements in latest 2.7.3 and 2.7.4 release.
> >
> >
> > Back to HADOOP-14964, I did a quick check and it looks like case here
> > belongs to self-contained improvement that has very low impact on
> existing
> > code base, so I am OK with the improvement get landed on branch-2.8 in
> case
> > it is well reviewed and tested.
> >
> >
> > However, as RM of branch-2.8, I have two concerns to accept it in our
> > 2.8.3 release:
> >
> > 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
> for
> > several critical bug fixes and we should target to release it very soon -
> > my current plan is to cut RC out within this week inline with waiting
> > for 3.0.0 vote closing. Can this improvement be well tested against
> > branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> > have strong commitment on test plan and activities in such a tight time.
> >
> >
> > 2. Upgrading - I haven't heard we settle down the plan of releasing this
> > feature in 2.9.1 release - though I saw some discussions are going on
> > at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> > this improvement, then users consuming this feature/improvement have no
> 2.9
> > release to upgrade or forcefully upgrade with regression. We may need a
> > better upgrade story here.
> >
> >
> > Pls let me know what you think. Thanks!
> >
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ------------------------------
> > *From:* Andrew Wang <andrew.wang@cloudera.com<mailto:
> andrew.wang@cloudera.com>>
> > *Sent:* Monday, November 20, 2017 10:22 PM
> > *To:* Zheng, Kai
> > *Cc:* Junping Du; common-dev@hadoop.apache.org<mailto:common-dev@hadoop.
> apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org
> >;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
> >
> > I'm against including new features in maintenance releases, since they're
> > meant to be bug-fix only.
> >
> > If we're struggling with being able to deliver new features in a safe and
> > timely fashion, let's try to address that, not overload the meaning of
> > "maintenance release".
> >
> > Best,
> > Andrew
> >
> > On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zheng@intel.com<mailto:
> kai.zheng@intel.com>> wrote:
> >
> >> Hi Junping,
> >>
> >> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
> >>
> >> I have an ask, is it convenient to include the back port work for OSS
> >> connector module? We have some Hadoop users that wish to have it by
> default
> >> for convenience, though in the past they used it by back porting
> >> themselves. I have raised this and got thoughts from Chris and Steve.
> Looks
> >> like this is more wanted for 2.9 but I wanted to ask again here for
> broad
> >> feedback and thoughts by this chance. The back port patch is available
> for
> >> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
> we
> >> can see some shift from 2.7.x, hence it's worth more important features
> and
> >> efforts. How would you think? Thanks!
> >>
> >> https://issues.apache.org/jira/browse/HADOOP-14964
> >>
> >> Regards,
> >> Kai
> >>
> >> -----Original Message-----
> >> From: Junping Du [mailto:jdu@hortonworks.com<mailto:jdu@hortonworks.com
> >]
> >> Sent: Tuesday, November 14, 2017 9:02 AM
> >> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: Apache Hadoop 2.8.3 Release Plan
> >>
> >> Hi,
> >> We have several important fixes get landed on branch-2.8 and I would
> >> like to cut off branch-2.8.3 now to start 2.8.3 release work.
> >> So far, I don't see any pending blockers on 2.8.3, so my current
> plan
> >> is to cut off 1st RC of 2.8.3 in next several days:
> >> - For all coming commits to land on branch-2.8, please mark
> the
> >> fix version as 2.8.4.
> >> - If there is a really important fix for 2.8.3 and getting
> >> closed, please notify me ahead before landing it on branch-2.8.3.
> >> Please let me know if you have any thoughts or comments on the plan.
> >>
> >> Thanks,
> >>
> >> Junping
> >> ________________________________________
> >> From: dujunping@gmail.com<ma...@gmail.com> <
> dujunping@gmail.com<ma...@gmail.com>> on behalf of 俊平堵 <
> >> junping_du@apache.org<ma...@apache.org>>
> >> Sent: Friday, October 27, 2017 3:33 PM
> >> To: general@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
> >>
> >> Hi all,
> >>
> >> It gives me great pleasure to announce that the Apache Hadoop
> >> community has voted to release Apache Hadoop 2.8.2, which is now
> available
> >> for download from Apache mirrors[1]. For download instructions please
> refer
> >> to the Apache Hadoop Release page [2].
> >>
> >> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
> and
> >> our newest stable release for entire Apache Hadoop project. For major
> >> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
> page[3].
> >>
> >> This release has 315 resolved issues since previous 2.8.1 release with
> >> following
> >> breakdown:
> >> - 91 in Hadoop Common
> >> - 99 in HDFS
> >> - 105 in YARN
> >> - 20 in MapReduce
> >> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
> >>
> >> The release news is posted on the Hadoop website too, you can go to the
> >> downloads section directly [6].
> >>
> >> Thank you all for contributing to the Apache Hadoop release!
> >>
> >>
> >> Cheers,
> >>
> >> Junping
> >>
> >>
> >> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
> >>
> >> [2] http://hadoop.apache.org/releases.html
> >>
> >> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
> >>
> >> [4]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/CHANGES.2.8.2.html
> >>
> >> [5]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
> >>
> >> [6] http://hadoop.apache.org/releases.html#Download
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org>
> >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<mailto:
> hdfs-dev-unsubscribe@hadoop.apache.org>
> >> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
> >>
> >>
> >
>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Konstantin Shvachko <sh...@gmail.com>.
I would consider these two blockers for 2.8.3 as they crash NN:
https://issues.apache.org/jira/browse/HDFS-12638
https://issues.apache.org/jira/browse/HDFS-12832
Thanks,
--Konstantin
On Tue, Nov 21, 2017 at 11:16 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks Andrew and Wangda for comments!
>
> To me, an improvement with 17 patches is not a big problem as this is
> self-contained and I didn't see a single line of delete/update on existing
> code - well, arguably, patches with only adding code can also have big
> impact but not likely the case here.
>
> While the dependency discussions on HADOOP-14964 are still going on, I
> will leave the decision to JIRA discussion based on which approach we will
> choose(shaded?) and impact. If we cannot make consensus in short term,
> probably we have to miss this in 2.8.3 release.
>
>
> Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0
> will get cut-off shortly.
>
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Wangda Tan <wh...@gmail.com>
> Sent: Tuesday, November 21, 2017 10:52 AM
> To: Andrew Wang
> Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org
> Subject: Re: Apache Hadoop 2.8.3 Release Plan
>
> Thanks Junping for driving this.
>
> For the bug fix vs. improvement, it is actually very hard to define,
> improvement could be self-contained and useful, bug fix could be dangerous
> in some cases. To me, If an improvement fixed some existing use case, and
> the fix is self-contained. I will be open to bring such fix to maintenance
> release. For example, in 2.8.2, we back ported CapacityScheduler intra
> queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
> big change in terms of patch size, but since it fixes broken use case
> (balance user usage under Capacity Scheduler leaf queue), we backported it
> to 2.8.2 after thorough tests and validations by Yahoo.
>
> I'm not quite familiar with HADOOP-14964, I will leave the decision to
> committers who know more about the field.
>
> Just my 2 cents.
>
> Regards,
> Wangda
>
>
> On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <andrew.wang@cloudera.com<
> mailto:andrew.wang@cloudera.com>> wrote:
> The Aliyun OSS code isn't a small improvement. If you look at Sammi's
> comment
> <https://issues.apache.org/jira/browse/HADOOP-14964?
> focusedCommentId=16247085&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
> it's
> a 17 patch series that is being backported in one shot. What we're talking
> about is equivalent to merging a feature branch in a maintenance release. I
> see that Kai and Chris are having a discussion about the dependency
> changes, which indicates this is not a zero-risk change either. We really
> should not be changing dependency versions in a maintenance unless it's
> because of a bug.
>
> It's unfortunate from a timing perspective that this missed 2.9.0, but I
> still think it should wait for the next minor. Merging a feature into a
> maintenance release sets the wrong precedent.
>
> Best,
> Andrew
>
> On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
>
> > Thanks Kai for calling out this feature/improvement for attention and
> > Andrew for comments.
> >
> >
> > While I agree that maintenance release should focus on important bug fix
> > only, I doubt we have strict rules to disallow any features/improvements
> to
> > land on maint release especially when those are small footprint or low
> > impact on existing code/features. In practice, we indeed has 77 new
> > features/improvements in latest 2.7.3 and 2.7.4 release.
> >
> >
> > Back to HADOOP-14964, I did a quick check and it looks like case here
> > belongs to self-contained improvement that has very low impact on
> existing
> > code base, so I am OK with the improvement get landed on branch-2.8 in
> case
> > it is well reviewed and tested.
> >
> >
> > However, as RM of branch-2.8, I have two concerns to accept it in our
> > 2.8.3 release:
> >
> > 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
> for
> > several critical bug fixes and we should target to release it very soon -
> > my current plan is to cut RC out within this week inline with waiting
> > for 3.0.0 vote closing. Can this improvement be well tested against
> > branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> > have strong commitment on test plan and activities in such a tight time.
> >
> >
> > 2. Upgrading - I haven't heard we settle down the plan of releasing this
> > feature in 2.9.1 release - though I saw some discussions are going on
> > at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> > this improvement, then users consuming this feature/improvement have no
> 2.9
> > release to upgrade or forcefully upgrade with regression. We may need a
> > better upgrade story here.
> >
> >
> > Pls let me know what you think. Thanks!
> >
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ------------------------------
> > *From:* Andrew Wang <andrew.wang@cloudera.com<mailto:
> andrew.wang@cloudera.com>>
> > *Sent:* Monday, November 20, 2017 10:22 PM
> > *To:* Zheng, Kai
> > *Cc:* Junping Du; common-dev@hadoop.apache.org<mailto:common-dev@hadoop.
> apache.org>; hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org
> >;
> > mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> > *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
> >
> > I'm against including new features in maintenance releases, since they're
> > meant to be bug-fix only.
> >
> > If we're struggling with being able to deliver new features in a safe and
> > timely fashion, let's try to address that, not overload the meaning of
> > "maintenance release".
> >
> > Best,
> > Andrew
> >
> > On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <kai.zheng@intel.com<mailto:
> kai.zheng@intel.com>> wrote:
> >
> >> Hi Junping,
> >>
> >> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
> >>
> >> I have an ask, is it convenient to include the back port work for OSS
> >> connector module? We have some Hadoop users that wish to have it by
> default
> >> for convenience, though in the past they used it by back porting
> >> themselves. I have raised this and got thoughts from Chris and Steve.
> Looks
> >> like this is more wanted for 2.9 but I wanted to ask again here for
> broad
> >> feedback and thoughts by this chance. The back port patch is available
> for
> >> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
> we
> >> can see some shift from 2.7.x, hence it's worth more important features
> and
> >> efforts. How would you think? Thanks!
> >>
> >> https://issues.apache.org/jira/browse/HADOOP-14964
> >>
> >> Regards,
> >> Kai
> >>
> >> -----Original Message-----
> >> From: Junping Du [mailto:jdu@hortonworks.com<mailto:jdu@hortonworks.com
> >]
> >> Sent: Tuesday, November 14, 2017 9:02 AM
> >> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> >> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: Apache Hadoop 2.8.3 Release Plan
> >>
> >> Hi,
> >> We have several important fixes get landed on branch-2.8 and I would
> >> like to cut off branch-2.8.3 now to start 2.8.3 release work.
> >> So far, I don't see any pending blockers on 2.8.3, so my current
> plan
> >> is to cut off 1st RC of 2.8.3 in next several days:
> >> - For all coming commits to land on branch-2.8, please mark
> the
> >> fix version as 2.8.4.
> >> - If there is a really important fix for 2.8.3 and getting
> >> closed, please notify me ahead before landing it on branch-2.8.3.
> >> Please let me know if you have any thoughts or comments on the plan.
> >>
> >> Thanks,
> >>
> >> Junping
> >> ________________________________________
> >> From: dujunping@gmail.com<ma...@gmail.com> <
> dujunping@gmail.com<ma...@gmail.com>> on behalf of 俊平堵 <
> >> junping_du@apache.org<ma...@apache.org>>
> >> Sent: Friday, October 27, 2017 3:33 PM
> >> To: general@hadoop.apache.org<ma...@hadoop.apache.org>
> >> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
> >>
> >> Hi all,
> >>
> >> It gives me great pleasure to announce that the Apache Hadoop
> >> community has voted to release Apache Hadoop 2.8.2, which is now
> available
> >> for download from Apache mirrors[1]. For download instructions please
> refer
> >> to the Apache Hadoop Release page [2].
> >>
> >> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
> and
> >> our newest stable release for entire Apache Hadoop project. For major
> >> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
> page[3].
> >>
> >> This release has 315 resolved issues since previous 2.8.1 release with
> >> following
> >> breakdown:
> >> - 91 in Hadoop Common
> >> - 99 in HDFS
> >> - 105 in YARN
> >> - 20 in MapReduce
> >> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
> >>
> >> The release news is posted on the Hadoop website too, you can go to the
> >> downloads section directly [6].
> >>
> >> Thank you all for contributing to the Apache Hadoop release!
> >>
> >>
> >> Cheers,
> >>
> >> Junping
> >>
> >>
> >> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
> >>
> >> [2] http://hadoop.apache.org/releases.html
> >>
> >> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
> >>
> >> [4]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/CHANGES.2.8.2.html
> >>
> >> [5]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
> >>
> >> [6] http://hadoop.apache.org/releases.html#Download
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org>
> >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<mailto:
> hdfs-dev-unsubscribe@hadoop.apache.org>
> >> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
> >>
> >>
> >
>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Andrew and Wangda for comments!
To me, an improvement with 17 patches is not a big problem as this is self-contained and I didn't see a single line of delete/update on existing code - well, arguably, patches with only adding code can also have big impact but not likely the case here.
While the dependency discussions on HADOOP-14964 are still going on, I will leave the decision to JIRA discussion based on which approach we will choose(shaded?) and impact. If we cannot make consensus in short term, probably we have to miss this in 2.8.3 release.
Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0 will get cut-off shortly.
Thanks,
Junping
________________________________
From: Wangda Tan <wh...@gmail.com>
Sent: Tuesday, November 21, 2017 10:52 AM
To: Andrew Wang
Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define, improvement could be self-contained and useful, bug fix could be dangerous in some cases. To me, If an improvement fixed some existing use case, and the fix is self-contained. I will be open to bring such fix to maintenance release. For example, in 2.8.2, we back ported CapacityScheduler intra queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a big change in terms of patch size, but since it fixes broken use case (balance user usage under Capacity Scheduler leaf queue), we backported it to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <an...@cloudera.com>> wrote:
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?focusedCommentId=16247085&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jd...@hortonworks.com>> wrote:
> Thanks Kai for calling out this feature/improvement for attention and
> Andrew for comments.
>
>
> While I agree that maintenance release should focus on important bug fix
> only, I doubt we have strict rules to disallow any features/improvements to
> land on maint release especially when those are small footprint or low
> impact on existing code/features. In practice, we indeed has 77 new
> features/improvements in latest 2.7.3 and 2.7.4 release.
>
>
> Back to HADOOP-14964, I did a quick check and it looks like case here
> belongs to self-contained improvement that has very low impact on existing
> code base, so I am OK with the improvement get landed on branch-2.8 in case
> it is well reviewed and tested.
>
>
> However, as RM of branch-2.8, I have two concerns to accept it in our
> 2.8.3 release:
>
> 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are for
> several critical bug fixes and we should target to release it very soon -
> my current plan is to cut RC out within this week inline with waiting
> for 3.0.0 vote closing. Can this improvement be well tested against
> branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> have strong commitment on test plan and activities in such a tight time.
>
>
> 2. Upgrading - I haven't heard we settle down the plan of releasing this
> feature in 2.9.1 release - though I saw some discussions are going on
> at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> this improvement, then users consuming this feature/improvement have no 2.9
> release to upgrade or forcefully upgrade with regression. We may need a
> better upgrade story here.
>
>
> Pls let me know what you think. Thanks!
>
>
>
> Thanks,
>
>
> Junping
>
>
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>>
> *Sent:* Monday, November 20, 2017 10:22 PM
> *To:* Zheng, Kai
> *Cc:* Junping Du; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
>
> I'm against including new features in maintenance releases, since they're
> meant to be bug-fix only.
>
> If we're struggling with being able to deliver new features in a safe and
> timely fashion, let's try to address that, not overload the meaning of
> "maintenance release".
>
> Best,
> Andrew
>
> On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com>> wrote:
>
>> Hi Junping,
>>
>> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>>
>> I have an ask, is it convenient to include the back port work for OSS
>> connector module? We have some Hadoop users that wish to have it by default
>> for convenience, though in the past they used it by back porting
>> themselves. I have raised this and got thoughts from Chris and Steve. Looks
>> like this is more wanted for 2.9 but I wanted to ask again here for broad
>> feedback and thoughts by this chance. The back port patch is available for
>> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we
>> can see some shift from 2.7.x, hence it's worth more important features and
>> efforts. How would you think? Thanks!
>>
>> https://issues.apache.org/jira/browse/HADOOP-14964
>>
>> Regards,
>> Kai
>>
>> -----Original Message-----
>> From: Junping Du [mailto:jdu@hortonworks.com<ma...@hortonworks.com>]
>> Sent: Tuesday, November 14, 2017 9:02 AM
>> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Apache Hadoop 2.8.3 Release Plan
>>
>> Hi,
>> We have several important fixes get landed on branch-2.8 and I would
>> like to cut off branch-2.8.3 now to start 2.8.3 release work.
>> So far, I don't see any pending blockers on 2.8.3, so my current plan
>> is to cut off 1st RC of 2.8.3 in next several days:
>> - For all coming commits to land on branch-2.8, please mark the
>> fix version as 2.8.4.
>> - If there is a really important fix for 2.8.3 and getting
>> closed, please notify me ahead before landing it on branch-2.8.3.
>> Please let me know if you have any thoughts or comments on the plan.
>>
>> Thanks,
>>
>> Junping
>> ________________________________________
>> From: dujunping@gmail.com<ma...@gmail.com> <du...@gmail.com>> on behalf of 俊平堵 <
>> junping_du@apache.org<ma...@apache.org>>
>> Sent: Friday, October 27, 2017 3:33 PM
>> To: general@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>>
>> Hi all,
>>
>> It gives me great pleasure to announce that the Apache Hadoop
>> community has voted to release Apache Hadoop 2.8.2, which is now available
>> for download from Apache mirrors[1]. For download instructions please refer
>> to the Apache Hadoop Release page [2].
>>
>> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
>> our newest stable release for entire Apache Hadoop project. For major
>> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>>
>> This release has 315 resolved issues since previous 2.8.1 release with
>> following
>> breakdown:
>> - 91 in Hadoop Common
>> - 99 in HDFS
>> - 105 in YARN
>> - 20 in MapReduce
>> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>>
>> The release news is posted on the Hadoop website too, you can go to the
>> downloads section directly [6].
>>
>> Thank you all for contributing to the Apache Hadoop release!
>>
>>
>> Cheers,
>>
>> Junping
>>
>>
>> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>>
>> [2] http://hadoop.apache.org/releases.html
>>
>> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>>
>> [4]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/CHANGES.2.8.2.html
>>
>> [5]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>>
>> [6] http://hadoop.apache.org/releases.html#Download
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>>
>>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Andrew and Wangda for comments!
To me, an improvement with 17 patches is not a big problem as this is self-contained and I didn't see a single line of delete/update on existing code - well, arguably, patches with only adding code can also have big impact but not likely the case here.
While the dependency discussions on HADOOP-14964 are still going on, I will leave the decision to JIRA discussion based on which approach we will choose(shaded?) and impact. If we cannot make consensus in short term, probably we have to miss this in 2.8.3 release.
Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0 will get cut-off shortly.
Thanks,
Junping
________________________________
From: Wangda Tan <wh...@gmail.com>
Sent: Tuesday, November 21, 2017 10:52 AM
To: Andrew Wang
Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define, improvement could be self-contained and useful, bug fix could be dangerous in some cases. To me, If an improvement fixed some existing use case, and the fix is self-contained. I will be open to bring such fix to maintenance release. For example, in 2.8.2, we back ported CapacityScheduler intra queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a big change in terms of patch size, but since it fixes broken use case (balance user usage under Capacity Scheduler leaf queue), we backported it to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <an...@cloudera.com>> wrote:
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?focusedCommentId=16247085&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jd...@hortonworks.com>> wrote:
> Thanks Kai for calling out this feature/improvement for attention and
> Andrew for comments.
>
>
> While I agree that maintenance release should focus on important bug fix
> only, I doubt we have strict rules to disallow any features/improvements to
> land on maint release especially when those are small footprint or low
> impact on existing code/features. In practice, we indeed has 77 new
> features/improvements in latest 2.7.3 and 2.7.4 release.
>
>
> Back to HADOOP-14964, I did a quick check and it looks like case here
> belongs to self-contained improvement that has very low impact on existing
> code base, so I am OK with the improvement get landed on branch-2.8 in case
> it is well reviewed and tested.
>
>
> However, as RM of branch-2.8, I have two concerns to accept it in our
> 2.8.3 release:
>
> 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are for
> several critical bug fixes and we should target to release it very soon -
> my current plan is to cut RC out within this week inline with waiting
> for 3.0.0 vote closing. Can this improvement be well tested against
> branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> have strong commitment on test plan and activities in such a tight time.
>
>
> 2. Upgrading - I haven't heard we settle down the plan of releasing this
> feature in 2.9.1 release - though I saw some discussions are going on
> at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> this improvement, then users consuming this feature/improvement have no 2.9
> release to upgrade or forcefully upgrade with regression. We may need a
> better upgrade story here.
>
>
> Pls let me know what you think. Thanks!
>
>
>
> Thanks,
>
>
> Junping
>
>
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>>
> *Sent:* Monday, November 20, 2017 10:22 PM
> *To:* Zheng, Kai
> *Cc:* Junping Du; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
>
> I'm against including new features in maintenance releases, since they're
> meant to be bug-fix only.
>
> If we're struggling with being able to deliver new features in a safe and
> timely fashion, let's try to address that, not overload the meaning of
> "maintenance release".
>
> Best,
> Andrew
>
> On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com>> wrote:
>
>> Hi Junping,
>>
>> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>>
>> I have an ask, is it convenient to include the back port work for OSS
>> connector module? We have some Hadoop users that wish to have it by default
>> for convenience, though in the past they used it by back porting
>> themselves. I have raised this and got thoughts from Chris and Steve. Looks
>> like this is more wanted for 2.9 but I wanted to ask again here for broad
>> feedback and thoughts by this chance. The back port patch is available for
>> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we
>> can see some shift from 2.7.x, hence it's worth more important features and
>> efforts. How would you think? Thanks!
>>
>> https://issues.apache.org/jira/browse/HADOOP-14964
>>
>> Regards,
>> Kai
>>
>> -----Original Message-----
>> From: Junping Du [mailto:jdu@hortonworks.com<ma...@hortonworks.com>]
>> Sent: Tuesday, November 14, 2017 9:02 AM
>> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Apache Hadoop 2.8.3 Release Plan
>>
>> Hi,
>> We have several important fixes get landed on branch-2.8 and I would
>> like to cut off branch-2.8.3 now to start 2.8.3 release work.
>> So far, I don't see any pending blockers on 2.8.3, so my current plan
>> is to cut off 1st RC of 2.8.3 in next several days:
>> - For all coming commits to land on branch-2.8, please mark the
>> fix version as 2.8.4.
>> - If there is a really important fix for 2.8.3 and getting
>> closed, please notify me ahead before landing it on branch-2.8.3.
>> Please let me know if you have any thoughts or comments on the plan.
>>
>> Thanks,
>>
>> Junping
>> ________________________________________
>> From: dujunping@gmail.com<ma...@gmail.com> <du...@gmail.com>> on behalf of 俊平堵 <
>> junping_du@apache.org<ma...@apache.org>>
>> Sent: Friday, October 27, 2017 3:33 PM
>> To: general@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>>
>> Hi all,
>>
>> It gives me great pleasure to announce that the Apache Hadoop
>> community has voted to release Apache Hadoop 2.8.2, which is now available
>> for download from Apache mirrors[1]. For download instructions please refer
>> to the Apache Hadoop Release page [2].
>>
>> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
>> our newest stable release for entire Apache Hadoop project. For major
>> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>>
>> This release has 315 resolved issues since previous 2.8.1 release with
>> following
>> breakdown:
>> - 91 in Hadoop Common
>> - 99 in HDFS
>> - 105 in YARN
>> - 20 in MapReduce
>> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>>
>> The release news is posted on the Hadoop website too, you can go to the
>> downloads section directly [6].
>>
>> Thank you all for contributing to the Apache Hadoop release!
>>
>>
>> Cheers,
>>
>> Junping
>>
>>
>> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>>
>> [2] http://hadoop.apache.org/releases.html
>>
>> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>>
>> [4]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/CHANGES.2.8.2.html
>>
>> [5]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>>
>> [6] http://hadoop.apache.org/releases.html#Download
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>>
>>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Andrew and Wangda for comments!
To me, an improvement with 17 patches is not a big problem as this is self-contained and I didn't see a single line of delete/update on existing code - well, arguably, patches with only adding code can also have big impact but not likely the case here.
While the dependency discussions on HADOOP-14964 are still going on, I will leave the decision to JIRA discussion based on which approach we will choose(shaded?) and impact. If we cannot make consensus in short term, probably we have to miss this in 2.8.3 release.
Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0 will get cut-off shortly.
Thanks,
Junping
________________________________
From: Wangda Tan <wh...@gmail.com>
Sent: Tuesday, November 21, 2017 10:52 AM
To: Andrew Wang
Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define, improvement could be self-contained and useful, bug fix could be dangerous in some cases. To me, If an improvement fixed some existing use case, and the fix is self-contained. I will be open to bring such fix to maintenance release. For example, in 2.8.2, we back ported CapacityScheduler intra queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a big change in terms of patch size, but since it fixes broken use case (balance user usage under Capacity Scheduler leaf queue), we backported it to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <an...@cloudera.com>> wrote:
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?focusedCommentId=16247085&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jd...@hortonworks.com>> wrote:
> Thanks Kai for calling out this feature/improvement for attention and
> Andrew for comments.
>
>
> While I agree that maintenance release should focus on important bug fix
> only, I doubt we have strict rules to disallow any features/improvements to
> land on maint release especially when those are small footprint or low
> impact on existing code/features. In practice, we indeed has 77 new
> features/improvements in latest 2.7.3 and 2.7.4 release.
>
>
> Back to HADOOP-14964, I did a quick check and it looks like case here
> belongs to self-contained improvement that has very low impact on existing
> code base, so I am OK with the improvement get landed on branch-2.8 in case
> it is well reviewed and tested.
>
>
> However, as RM of branch-2.8, I have two concerns to accept it in our
> 2.8.3 release:
>
> 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are for
> several critical bug fixes and we should target to release it very soon -
> my current plan is to cut RC out within this week inline with waiting
> for 3.0.0 vote closing. Can this improvement be well tested against
> branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> have strong commitment on test plan and activities in such a tight time.
>
>
> 2. Upgrading - I haven't heard we settle down the plan of releasing this
> feature in 2.9.1 release - though I saw some discussions are going on
> at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> this improvement, then users consuming this feature/improvement have no 2.9
> release to upgrade or forcefully upgrade with regression. We may need a
> better upgrade story here.
>
>
> Pls let me know what you think. Thanks!
>
>
>
> Thanks,
>
>
> Junping
>
>
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>>
> *Sent:* Monday, November 20, 2017 10:22 PM
> *To:* Zheng, Kai
> *Cc:* Junping Du; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
>
> I'm against including new features in maintenance releases, since they're
> meant to be bug-fix only.
>
> If we're struggling with being able to deliver new features in a safe and
> timely fashion, let's try to address that, not overload the meaning of
> "maintenance release".
>
> Best,
> Andrew
>
> On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com>> wrote:
>
>> Hi Junping,
>>
>> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>>
>> I have an ask, is it convenient to include the back port work for OSS
>> connector module? We have some Hadoop users that wish to have it by default
>> for convenience, though in the past they used it by back porting
>> themselves. I have raised this and got thoughts from Chris and Steve. Looks
>> like this is more wanted for 2.9 but I wanted to ask again here for broad
>> feedback and thoughts by this chance. The back port patch is available for
>> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we
>> can see some shift from 2.7.x, hence it's worth more important features and
>> efforts. How would you think? Thanks!
>>
>> https://issues.apache.org/jira/browse/HADOOP-14964
>>
>> Regards,
>> Kai
>>
>> -----Original Message-----
>> From: Junping Du [mailto:jdu@hortonworks.com<ma...@hortonworks.com>]
>> Sent: Tuesday, November 14, 2017 9:02 AM
>> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Apache Hadoop 2.8.3 Release Plan
>>
>> Hi,
>> We have several important fixes get landed on branch-2.8 and I would
>> like to cut off branch-2.8.3 now to start 2.8.3 release work.
>> So far, I don't see any pending blockers on 2.8.3, so my current plan
>> is to cut off 1st RC of 2.8.3 in next several days:
>> - For all coming commits to land on branch-2.8, please mark the
>> fix version as 2.8.4.
>> - If there is a really important fix for 2.8.3 and getting
>> closed, please notify me ahead before landing it on branch-2.8.3.
>> Please let me know if you have any thoughts or comments on the plan.
>>
>> Thanks,
>>
>> Junping
>> ________________________________________
>> From: dujunping@gmail.com<ma...@gmail.com> <du...@gmail.com>> on behalf of 俊平堵 <
>> junping_du@apache.org<ma...@apache.org>>
>> Sent: Friday, October 27, 2017 3:33 PM
>> To: general@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>>
>> Hi all,
>>
>> It gives me great pleasure to announce that the Apache Hadoop
>> community has voted to release Apache Hadoop 2.8.2, which is now available
>> for download from Apache mirrors[1]. For download instructions please refer
>> to the Apache Hadoop Release page [2].
>>
>> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
>> our newest stable release for entire Apache Hadoop project. For major
>> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>>
>> This release has 315 resolved issues since previous 2.8.1 release with
>> following
>> breakdown:
>> - 91 in Hadoop Common
>> - 99 in HDFS
>> - 105 in YARN
>> - 20 in MapReduce
>> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>>
>> The release news is posted on the Hadoop website too, you can go to the
>> downloads section directly [6].
>>
>> Thank you all for contributing to the Apache Hadoop release!
>>
>>
>> Cheers,
>>
>> Junping
>>
>>
>> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>>
>> [2] http://hadoop.apache.org/releases.html
>>
>> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>>
>> [4]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/CHANGES.2.8.2.html
>>
>> [5]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>>
>> [6] http://hadoop.apache.org/releases.html#Download
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>>
>>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Andrew and Wangda for comments!
To me, an improvement with 17 patches is not a big problem as this is self-contained and I didn't see a single line of delete/update on existing code - well, arguably, patches with only adding code can also have big impact but not likely the case here.
While the dependency discussions on HADOOP-14964 are still going on, I will leave the decision to JIRA discussion based on which approach we will choose(shaded?) and impact. If we cannot make consensus in short term, probably we have to miss this in 2.8.3 release.
Okay. Last call for blocker/critical fixes landing on branch-2.8.3. RC0 will get cut-off shortly.
Thanks,
Junping
________________________________
From: Wangda Tan <wh...@gmail.com>
Sent: Tuesday, November 21, 2017 10:52 AM
To: Andrew Wang
Cc: Junping Du; Zheng, Kai; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define, improvement could be self-contained and useful, bug fix could be dangerous in some cases. To me, If an improvement fixed some existing use case, and the fix is self-contained. I will be open to bring such fix to maintenance release. For example, in 2.8.2, we back ported CapacityScheduler intra queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a big change in terms of patch size, but since it fixes broken use case (balance user usage under Capacity Scheduler leaf queue), we backported it to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <an...@cloudera.com>> wrote:
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?focusedCommentId=16247085&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jd...@hortonworks.com>> wrote:
> Thanks Kai for calling out this feature/improvement for attention and
> Andrew for comments.
>
>
> While I agree that maintenance release should focus on important bug fix
> only, I doubt we have strict rules to disallow any features/improvements to
> land on maint release especially when those are small footprint or low
> impact on existing code/features. In practice, we indeed has 77 new
> features/improvements in latest 2.7.3 and 2.7.4 release.
>
>
> Back to HADOOP-14964, I did a quick check and it looks like case here
> belongs to self-contained improvement that has very low impact on existing
> code base, so I am OK with the improvement get landed on branch-2.8 in case
> it is well reviewed and tested.
>
>
> However, as RM of branch-2.8, I have two concerns to accept it in our
> 2.8.3 release:
>
> 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are for
> several critical bug fixes and we should target to release it very soon -
> my current plan is to cut RC out within this week inline with waiting
> for 3.0.0 vote closing. Can this improvement be well tested against
> branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> have strong commitment on test plan and activities in such a tight time.
>
>
> 2. Upgrading - I haven't heard we settle down the plan of releasing this
> feature in 2.9.1 release - though I saw some discussions are going on
> at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> this improvement, then users consuming this feature/improvement have no 2.9
> release to upgrade or forcefully upgrade with regression. We may need a
> better upgrade story here.
>
>
> Pls let me know what you think. Thanks!
>
>
>
> Thanks,
>
>
> Junping
>
>
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>>
> *Sent:* Monday, November 20, 2017 10:22 PM
> *To:* Zheng, Kai
> *Cc:* Junping Du; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
>
> I'm against including new features in maintenance releases, since they're
> meant to be bug-fix only.
>
> If we're struggling with being able to deliver new features in a safe and
> timely fashion, let's try to address that, not overload the meaning of
> "maintenance release".
>
> Best,
> Andrew
>
> On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com>> wrote:
>
>> Hi Junping,
>>
>> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>>
>> I have an ask, is it convenient to include the back port work for OSS
>> connector module? We have some Hadoop users that wish to have it by default
>> for convenience, though in the past they used it by back porting
>> themselves. I have raised this and got thoughts from Chris and Steve. Looks
>> like this is more wanted for 2.9 but I wanted to ask again here for broad
>> feedback and thoughts by this chance. The back port patch is available for
>> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we
>> can see some shift from 2.7.x, hence it's worth more important features and
>> efforts. How would you think? Thanks!
>>
>> https://issues.apache.org/jira/browse/HADOOP-14964
>>
>> Regards,
>> Kai
>>
>> -----Original Message-----
>> From: Junping Du [mailto:jdu@hortonworks.com<ma...@hortonworks.com>]
>> Sent: Tuesday, November 14, 2017 9:02 AM
>> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Apache Hadoop 2.8.3 Release Plan
>>
>> Hi,
>> We have several important fixes get landed on branch-2.8 and I would
>> like to cut off branch-2.8.3 now to start 2.8.3 release work.
>> So far, I don't see any pending blockers on 2.8.3, so my current plan
>> is to cut off 1st RC of 2.8.3 in next several days:
>> - For all coming commits to land on branch-2.8, please mark the
>> fix version as 2.8.4.
>> - If there is a really important fix for 2.8.3 and getting
>> closed, please notify me ahead before landing it on branch-2.8.3.
>> Please let me know if you have any thoughts or comments on the plan.
>>
>> Thanks,
>>
>> Junping
>> ________________________________________
>> From: dujunping@gmail.com<ma...@gmail.com> <du...@gmail.com>> on behalf of 俊平堵 <
>> junping_du@apache.org<ma...@apache.org>>
>> Sent: Friday, October 27, 2017 3:33 PM
>> To: general@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>>
>> Hi all,
>>
>> It gives me great pleasure to announce that the Apache Hadoop
>> community has voted to release Apache Hadoop 2.8.2, which is now available
>> for download from Apache mirrors[1]. For download instructions please refer
>> to the Apache Hadoop Release page [2].
>>
>> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
>> our newest stable release for entire Apache Hadoop project. For major
>> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>>
>> This release has 315 resolved issues since previous 2.8.1 release with
>> following
>> breakdown:
>> - 91 in Hadoop Common
>> - 99 in HDFS
>> - 105 in YARN
>> - 20 in MapReduce
>> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>>
>> The release news is posted on the Hadoop website too, you can go to the
>> downloads section directly [6].
>>
>> Thank you all for contributing to the Apache Hadoop release!
>>
>>
>> Cheers,
>>
>> Junping
>>
>>
>> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>>
>> [2] http://hadoop.apache.org/releases.html
>>
>> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>>
>> [4]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/CHANGES.2.8.2.html
>>
>> [5]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>>
>> [6] http://hadoop.apache.org/releases.html#Download
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>>
>>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Wangda Tan <wh...@gmail.com>.
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define,
improvement could be self-contained and useful, bug fix could be dangerous
in some cases. To me, If an improvement fixed some existing use case, and
the fix is self-contained. I will be open to bring such fix to maintenance
release. For example, in 2.8.2, we back ported CapacityScheduler intra
queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
big change in terms of patch size, but since it fixes broken use case
(balance user usage under Capacity Scheduler leaf queue), we backported it
to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to
committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <an...@cloudera.com>
wrote:
> The Aliyun OSS code isn't a small improvement. If you look at Sammi's
> comment
> <https://issues.apache.org/jira/browse/HADOOP-14964?
> focusedCommentId=16247085&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
> it's
> a 17 patch series that is being backported in one shot. What we're talking
> about is equivalent to merging a feature branch in a maintenance release. I
> see that Kai and Chris are having a discussion about the dependency
> changes, which indicates this is not a zero-risk change either. We really
> should not be changing dependency versions in a maintenance unless it's
> because of a bug.
>
> It's unfortunate from a timing perspective that this missed 2.9.0, but I
> still think it should wait for the next minor. Merging a feature into a
> maintenance release sets the wrong precedent.
>
> Best,
> Andrew
>
> On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks Kai for calling out this feature/improvement for attention and
> > Andrew for comments.
> >
> >
> > While I agree that maintenance release should focus on important bug fix
> > only, I doubt we have strict rules to disallow any features/improvements
> to
> > land on maint release especially when those are small footprint or low
> > impact on existing code/features. In practice, we indeed has 77 new
> > features/improvements in latest 2.7.3 and 2.7.4 release.
> >
> >
> > Back to HADOOP-14964, I did a quick check and it looks like case here
> > belongs to self-contained improvement that has very low impact on
> existing
> > code base, so I am OK with the improvement get landed on branch-2.8 in
> case
> > it is well reviewed and tested.
> >
> >
> > However, as RM of branch-2.8, I have two concerns to accept it in our
> > 2.8.3 release:
> >
> > 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
> for
> > several critical bug fixes and we should target to release it very soon -
> > my current plan is to cut RC out within this week inline with waiting
> > for 3.0.0 vote closing. Can this improvement be well tested against
> > branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> > have strong commitment on test plan and activities in such a tight time.
> >
> >
> > 2. Upgrading - I haven't heard we settle down the plan of releasing this
> > feature in 2.9.1 release - though I saw some discussions are going on
> > at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> > this improvement, then users consuming this feature/improvement have no
> 2.9
> > release to upgrade or forcefully upgrade with regression. We may need a
> > better upgrade story here.
> >
> >
> > Pls let me know what you think. Thanks!
> >
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ------------------------------
> > *From:* Andrew Wang <an...@cloudera.com>
> > *Sent:* Monday, November 20, 2017 10:22 PM
> > *To:* Zheng, Kai
> > *Cc:* Junping Du; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org;
> > mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> > *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
> >
> > I'm against including new features in maintenance releases, since they're
> > meant to be bug-fix only.
> >
> > If we're struggling with being able to deliver new features in a safe and
> > timely fashion, let's try to address that, not overload the meaning of
> > "maintenance release".
> >
> > Best,
> > Andrew
> >
> > On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
> >
> >> Hi Junping,
> >>
> >> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
> >>
> >> I have an ask, is it convenient to include the back port work for OSS
> >> connector module? We have some Hadoop users that wish to have it by
> default
> >> for convenience, though in the past they used it by back porting
> >> themselves. I have raised this and got thoughts from Chris and Steve.
> Looks
> >> like this is more wanted for 2.9 but I wanted to ask again here for
> broad
> >> feedback and thoughts by this chance. The back port patch is available
> for
> >> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
> we
> >> can see some shift from 2.7.x, hence it's worth more important features
> and
> >> efforts. How would you think? Thanks!
> >>
> >> https://issues.apache.org/jira/browse/HADOOP-14964
> >>
> >> Regards,
> >> Kai
> >>
> >> -----Original Message-----
> >> From: Junping Du [mailto:jdu@hortonworks.com]
> >> Sent: Tuesday, November 14, 2017 9:02 AM
> >> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> >> Subject: Apache Hadoop 2.8.3 Release Plan
> >>
> >> Hi,
> >> We have several important fixes get landed on branch-2.8 and I would
> >> like to cut off branch-2.8.3 now to start 2.8.3 release work.
> >> So far, I don't see any pending blockers on 2.8.3, so my current
> plan
> >> is to cut off 1st RC of 2.8.3 in next several days:
> >> - For all coming commits to land on branch-2.8, please mark
> the
> >> fix version as 2.8.4.
> >> - If there is a really important fix for 2.8.3 and getting
> >> closed, please notify me ahead before landing it on branch-2.8.3.
> >> Please let me know if you have any thoughts or comments on the plan.
> >>
> >> Thanks,
> >>
> >> Junping
> >> ________________________________________
> >> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
> >> junping_du@apache.org>
> >> Sent: Friday, October 27, 2017 3:33 PM
> >> To: general@hadoop.apache.org
> >> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
> >>
> >> Hi all,
> >>
> >> It gives me great pleasure to announce that the Apache Hadoop
> >> community has voted to release Apache Hadoop 2.8.2, which is now
> available
> >> for download from Apache mirrors[1]. For download instructions please
> refer
> >> to the Apache Hadoop Release page [2].
> >>
> >> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
> and
> >> our newest stable release for entire Apache Hadoop project. For major
> >> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
> page[3].
> >>
> >> This release has 315 resolved issues since previous 2.8.1 release with
> >> following
> >> breakdown:
> >> - 91 in Hadoop Common
> >> - 99 in HDFS
> >> - 105 in YARN
> >> - 20 in MapReduce
> >> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
> >>
> >> The release news is posted on the Hadoop website too, you can go to the
> >> downloads section directly [6].
> >>
> >> Thank you all for contributing to the Apache Hadoop release!
> >>
> >>
> >> Cheers,
> >>
> >> Junping
> >>
> >>
> >> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
> >>
> >> [2] http://hadoop.apache.org/releases.html
> >>
> >> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
> >>
> >> [4]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/CHANGES.2.8.2.html
> >>
> >> [5]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
> >>
> >> [6] http://hadoop.apache.org/releases.html#Download
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >>
> >>
> >
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Wangda Tan <wh...@gmail.com>.
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define,
improvement could be self-contained and useful, bug fix could be dangerous
in some cases. To me, If an improvement fixed some existing use case, and
the fix is self-contained. I will be open to bring such fix to maintenance
release. For example, in 2.8.2, we back ported CapacityScheduler intra
queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
big change in terms of patch size, but since it fixes broken use case
(balance user usage under Capacity Scheduler leaf queue), we backported it
to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to
committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <an...@cloudera.com>
wrote:
> The Aliyun OSS code isn't a small improvement. If you look at Sammi's
> comment
> <https://issues.apache.org/jira/browse/HADOOP-14964?
> focusedCommentId=16247085&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
> it's
> a 17 patch series that is being backported in one shot. What we're talking
> about is equivalent to merging a feature branch in a maintenance release. I
> see that Kai and Chris are having a discussion about the dependency
> changes, which indicates this is not a zero-risk change either. We really
> should not be changing dependency versions in a maintenance unless it's
> because of a bug.
>
> It's unfortunate from a timing perspective that this missed 2.9.0, but I
> still think it should wait for the next minor. Merging a feature into a
> maintenance release sets the wrong precedent.
>
> Best,
> Andrew
>
> On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks Kai for calling out this feature/improvement for attention and
> > Andrew for comments.
> >
> >
> > While I agree that maintenance release should focus on important bug fix
> > only, I doubt we have strict rules to disallow any features/improvements
> to
> > land on maint release especially when those are small footprint or low
> > impact on existing code/features. In practice, we indeed has 77 new
> > features/improvements in latest 2.7.3 and 2.7.4 release.
> >
> >
> > Back to HADOOP-14964, I did a quick check and it looks like case here
> > belongs to self-contained improvement that has very low impact on
> existing
> > code base, so I am OK with the improvement get landed on branch-2.8 in
> case
> > it is well reviewed and tested.
> >
> >
> > However, as RM of branch-2.8, I have two concerns to accept it in our
> > 2.8.3 release:
> >
> > 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
> for
> > several critical bug fixes and we should target to release it very soon -
> > my current plan is to cut RC out within this week inline with waiting
> > for 3.0.0 vote closing. Can this improvement be well tested against
> > branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> > have strong commitment on test plan and activities in such a tight time.
> >
> >
> > 2. Upgrading - I haven't heard we settle down the plan of releasing this
> > feature in 2.9.1 release - though I saw some discussions are going on
> > at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> > this improvement, then users consuming this feature/improvement have no
> 2.9
> > release to upgrade or forcefully upgrade with regression. We may need a
> > better upgrade story here.
> >
> >
> > Pls let me know what you think. Thanks!
> >
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ------------------------------
> > *From:* Andrew Wang <an...@cloudera.com>
> > *Sent:* Monday, November 20, 2017 10:22 PM
> > *To:* Zheng, Kai
> > *Cc:* Junping Du; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org;
> > mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> > *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
> >
> > I'm against including new features in maintenance releases, since they're
> > meant to be bug-fix only.
> >
> > If we're struggling with being able to deliver new features in a safe and
> > timely fashion, let's try to address that, not overload the meaning of
> > "maintenance release".
> >
> > Best,
> > Andrew
> >
> > On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
> >
> >> Hi Junping,
> >>
> >> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
> >>
> >> I have an ask, is it convenient to include the back port work for OSS
> >> connector module? We have some Hadoop users that wish to have it by
> default
> >> for convenience, though in the past they used it by back porting
> >> themselves. I have raised this and got thoughts from Chris and Steve.
> Looks
> >> like this is more wanted for 2.9 but I wanted to ask again here for
> broad
> >> feedback and thoughts by this chance. The back port patch is available
> for
> >> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
> we
> >> can see some shift from 2.7.x, hence it's worth more important features
> and
> >> efforts. How would you think? Thanks!
> >>
> >> https://issues.apache.org/jira/browse/HADOOP-14964
> >>
> >> Regards,
> >> Kai
> >>
> >> -----Original Message-----
> >> From: Junping Du [mailto:jdu@hortonworks.com]
> >> Sent: Tuesday, November 14, 2017 9:02 AM
> >> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> >> Subject: Apache Hadoop 2.8.3 Release Plan
> >>
> >> Hi,
> >> We have several important fixes get landed on branch-2.8 and I would
> >> like to cut off branch-2.8.3 now to start 2.8.3 release work.
> >> So far, I don't see any pending blockers on 2.8.3, so my current
> plan
> >> is to cut off 1st RC of 2.8.3 in next several days:
> >> - For all coming commits to land on branch-2.8, please mark
> the
> >> fix version as 2.8.4.
> >> - If there is a really important fix for 2.8.3 and getting
> >> closed, please notify me ahead before landing it on branch-2.8.3.
> >> Please let me know if you have any thoughts or comments on the plan.
> >>
> >> Thanks,
> >>
> >> Junping
> >> ________________________________________
> >> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
> >> junping_du@apache.org>
> >> Sent: Friday, October 27, 2017 3:33 PM
> >> To: general@hadoop.apache.org
> >> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
> >>
> >> Hi all,
> >>
> >> It gives me great pleasure to announce that the Apache Hadoop
> >> community has voted to release Apache Hadoop 2.8.2, which is now
> available
> >> for download from Apache mirrors[1]. For download instructions please
> refer
> >> to the Apache Hadoop Release page [2].
> >>
> >> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
> and
> >> our newest stable release for entire Apache Hadoop project. For major
> >> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
> page[3].
> >>
> >> This release has 315 resolved issues since previous 2.8.1 release with
> >> following
> >> breakdown:
> >> - 91 in Hadoop Common
> >> - 99 in HDFS
> >> - 105 in YARN
> >> - 20 in MapReduce
> >> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
> >>
> >> The release news is posted on the Hadoop website too, you can go to the
> >> downloads section directly [6].
> >>
> >> Thank you all for contributing to the Apache Hadoop release!
> >>
> >>
> >> Cheers,
> >>
> >> Junping
> >>
> >>
> >> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
> >>
> >> [2] http://hadoop.apache.org/releases.html
> >>
> >> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
> >>
> >> [4]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/CHANGES.2.8.2.html
> >>
> >> [5]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
> >>
> >> [6] http://hadoop.apache.org/releases.html#Download
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >>
> >>
> >
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Wangda Tan <wh...@gmail.com>.
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define,
improvement could be self-contained and useful, bug fix could be dangerous
in some cases. To me, If an improvement fixed some existing use case, and
the fix is self-contained. I will be open to bring such fix to maintenance
release. For example, in 2.8.2, we back ported CapacityScheduler intra
queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
big change in terms of patch size, but since it fixes broken use case
(balance user usage under Capacity Scheduler leaf queue), we backported it
to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to
committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <an...@cloudera.com>
wrote:
> The Aliyun OSS code isn't a small improvement. If you look at Sammi's
> comment
> <https://issues.apache.org/jira/browse/HADOOP-14964?
> focusedCommentId=16247085&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
> it's
> a 17 patch series that is being backported in one shot. What we're talking
> about is equivalent to merging a feature branch in a maintenance release. I
> see that Kai and Chris are having a discussion about the dependency
> changes, which indicates this is not a zero-risk change either. We really
> should not be changing dependency versions in a maintenance unless it's
> because of a bug.
>
> It's unfortunate from a timing perspective that this missed 2.9.0, but I
> still think it should wait for the next minor. Merging a feature into a
> maintenance release sets the wrong precedent.
>
> Best,
> Andrew
>
> On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks Kai for calling out this feature/improvement for attention and
> > Andrew for comments.
> >
> >
> > While I agree that maintenance release should focus on important bug fix
> > only, I doubt we have strict rules to disallow any features/improvements
> to
> > land on maint release especially when those are small footprint or low
> > impact on existing code/features. In practice, we indeed has 77 new
> > features/improvements in latest 2.7.3 and 2.7.4 release.
> >
> >
> > Back to HADOOP-14964, I did a quick check and it looks like case here
> > belongs to self-contained improvement that has very low impact on
> existing
> > code base, so I am OK with the improvement get landed on branch-2.8 in
> case
> > it is well reviewed and tested.
> >
> >
> > However, as RM of branch-2.8, I have two concerns to accept it in our
> > 2.8.3 release:
> >
> > 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
> for
> > several critical bug fixes and we should target to release it very soon -
> > my current plan is to cut RC out within this week inline with waiting
> > for 3.0.0 vote closing. Can this improvement be well tested against
> > branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> > have strong commitment on test plan and activities in such a tight time.
> >
> >
> > 2. Upgrading - I haven't heard we settle down the plan of releasing this
> > feature in 2.9.1 release - though I saw some discussions are going on
> > at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> > this improvement, then users consuming this feature/improvement have no
> 2.9
> > release to upgrade or forcefully upgrade with regression. We may need a
> > better upgrade story here.
> >
> >
> > Pls let me know what you think. Thanks!
> >
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ------------------------------
> > *From:* Andrew Wang <an...@cloudera.com>
> > *Sent:* Monday, November 20, 2017 10:22 PM
> > *To:* Zheng, Kai
> > *Cc:* Junping Du; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org;
> > mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> > *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
> >
> > I'm against including new features in maintenance releases, since they're
> > meant to be bug-fix only.
> >
> > If we're struggling with being able to deliver new features in a safe and
> > timely fashion, let's try to address that, not overload the meaning of
> > "maintenance release".
> >
> > Best,
> > Andrew
> >
> > On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
> >
> >> Hi Junping,
> >>
> >> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
> >>
> >> I have an ask, is it convenient to include the back port work for OSS
> >> connector module? We have some Hadoop users that wish to have it by
> default
> >> for convenience, though in the past they used it by back porting
> >> themselves. I have raised this and got thoughts from Chris and Steve.
> Looks
> >> like this is more wanted for 2.9 but I wanted to ask again here for
> broad
> >> feedback and thoughts by this chance. The back port patch is available
> for
> >> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
> we
> >> can see some shift from 2.7.x, hence it's worth more important features
> and
> >> efforts. How would you think? Thanks!
> >>
> >> https://issues.apache.org/jira/browse/HADOOP-14964
> >>
> >> Regards,
> >> Kai
> >>
> >> -----Original Message-----
> >> From: Junping Du [mailto:jdu@hortonworks.com]
> >> Sent: Tuesday, November 14, 2017 9:02 AM
> >> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> >> Subject: Apache Hadoop 2.8.3 Release Plan
> >>
> >> Hi,
> >> We have several important fixes get landed on branch-2.8 and I would
> >> like to cut off branch-2.8.3 now to start 2.8.3 release work.
> >> So far, I don't see any pending blockers on 2.8.3, so my current
> plan
> >> is to cut off 1st RC of 2.8.3 in next several days:
> >> - For all coming commits to land on branch-2.8, please mark
> the
> >> fix version as 2.8.4.
> >> - If there is a really important fix for 2.8.3 and getting
> >> closed, please notify me ahead before landing it on branch-2.8.3.
> >> Please let me know if you have any thoughts or comments on the plan.
> >>
> >> Thanks,
> >>
> >> Junping
> >> ________________________________________
> >> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
> >> junping_du@apache.org>
> >> Sent: Friday, October 27, 2017 3:33 PM
> >> To: general@hadoop.apache.org
> >> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
> >>
> >> Hi all,
> >>
> >> It gives me great pleasure to announce that the Apache Hadoop
> >> community has voted to release Apache Hadoop 2.8.2, which is now
> available
> >> for download from Apache mirrors[1]. For download instructions please
> refer
> >> to the Apache Hadoop Release page [2].
> >>
> >> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
> and
> >> our newest stable release for entire Apache Hadoop project. For major
> >> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
> page[3].
> >>
> >> This release has 315 resolved issues since previous 2.8.1 release with
> >> following
> >> breakdown:
> >> - 91 in Hadoop Common
> >> - 99 in HDFS
> >> - 105 in YARN
> >> - 20 in MapReduce
> >> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
> >>
> >> The release news is posted on the Hadoop website too, you can go to the
> >> downloads section directly [6].
> >>
> >> Thank you all for contributing to the Apache Hadoop release!
> >>
> >>
> >> Cheers,
> >>
> >> Junping
> >>
> >>
> >> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
> >>
> >> [2] http://hadoop.apache.org/releases.html
> >>
> >> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
> >>
> >> [4]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/CHANGES.2.8.2.html
> >>
> >> [5]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
> >>
> >> [6] http://hadoop.apache.org/releases.html#Download
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >>
> >>
> >
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Wangda Tan <wh...@gmail.com>.
Thanks Junping for driving this.
For the bug fix vs. improvement, it is actually very hard to define,
improvement could be self-contained and useful, bug fix could be dangerous
in some cases. To me, If an improvement fixed some existing use case, and
the fix is self-contained. I will be open to bring such fix to maintenance
release. For example, in 2.8.2, we back ported CapacityScheduler intra
queue preemption https://issues.apache.org/jira/browse/YARN-2113. It is a
big change in terms of patch size, but since it fixes broken use case
(balance user usage under Capacity Scheduler leaf queue), we backported it
to 2.8.2 after thorough tests and validations by Yahoo.
I'm not quite familiar with HADOOP-14964, I will leave the decision to
committers who know more about the field.
Just my 2 cents.
Regards,
Wangda
On Tue, Nov 21, 2017 at 10:21 AM, Andrew Wang <an...@cloudera.com>
wrote:
> The Aliyun OSS code isn't a small improvement. If you look at Sammi's
> comment
> <https://issues.apache.org/jira/browse/HADOOP-14964?
> focusedCommentId=16247085&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
> it's
> a 17 patch series that is being backported in one shot. What we're talking
> about is equivalent to merging a feature branch in a maintenance release. I
> see that Kai and Chris are having a discussion about the dependency
> changes, which indicates this is not a zero-risk change either. We really
> should not be changing dependency versions in a maintenance unless it's
> because of a bug.
>
> It's unfortunate from a timing perspective that this missed 2.9.0, but I
> still think it should wait for the next minor. Merging a feature into a
> maintenance release sets the wrong precedent.
>
> Best,
> Andrew
>
> On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> > Thanks Kai for calling out this feature/improvement for attention and
> > Andrew for comments.
> >
> >
> > While I agree that maintenance release should focus on important bug fix
> > only, I doubt we have strict rules to disallow any features/improvements
> to
> > land on maint release especially when those are small footprint or low
> > impact on existing code/features. In practice, we indeed has 77 new
> > features/improvements in latest 2.7.3 and 2.7.4 release.
> >
> >
> > Back to HADOOP-14964, I did a quick check and it looks like case here
> > belongs to self-contained improvement that has very low impact on
> existing
> > code base, so I am OK with the improvement get landed on branch-2.8 in
> case
> > it is well reviewed and tested.
> >
> >
> > However, as RM of branch-2.8, I have two concerns to accept it in our
> > 2.8.3 release:
> >
> > 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are
> for
> > several critical bug fixes and we should target to release it very soon -
> > my current plan is to cut RC out within this week inline with waiting
> > for 3.0.0 vote closing. Can this improvement be well tested against
> > branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> > have strong commitment on test plan and activities in such a tight time.
> >
> >
> > 2. Upgrading - I haven't heard we settle down the plan of releasing this
> > feature in 2.9.1 release - though I saw some discussions are going on
> > at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> > this improvement, then users consuming this feature/improvement have no
> 2.9
> > release to upgrade or forcefully upgrade with regression. We may need a
> > better upgrade story here.
> >
> >
> > Pls let me know what you think. Thanks!
> >
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ------------------------------
> > *From:* Andrew Wang <an...@cloudera.com>
> > *Sent:* Monday, November 20, 2017 10:22 PM
> > *To:* Zheng, Kai
> > *Cc:* Junping Du; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org;
> > mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> > *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
> >
> > I'm against including new features in maintenance releases, since they're
> > meant to be bug-fix only.
> >
> > If we're struggling with being able to deliver new features in a safe and
> > timely fashion, let's try to address that, not overload the meaning of
> > "maintenance release".
> >
> > Best,
> > Andrew
> >
> > On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
> >
> >> Hi Junping,
> >>
> >> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
> >>
> >> I have an ask, is it convenient to include the back port work for OSS
> >> connector module? We have some Hadoop users that wish to have it by
> default
> >> for convenience, though in the past they used it by back porting
> >> themselves. I have raised this and got thoughts from Chris and Steve.
> Looks
> >> like this is more wanted for 2.9 but I wanted to ask again here for
> broad
> >> feedback and thoughts by this chance. The back port patch is available
> for
> >> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as
> we
> >> can see some shift from 2.7.x, hence it's worth more important features
> and
> >> efforts. How would you think? Thanks!
> >>
> >> https://issues.apache.org/jira/browse/HADOOP-14964
> >>
> >> Regards,
> >> Kai
> >>
> >> -----Original Message-----
> >> From: Junping Du [mailto:jdu@hortonworks.com]
> >> Sent: Tuesday, November 14, 2017 9:02 AM
> >> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> >> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> >> Subject: Apache Hadoop 2.8.3 Release Plan
> >>
> >> Hi,
> >> We have several important fixes get landed on branch-2.8 and I would
> >> like to cut off branch-2.8.3 now to start 2.8.3 release work.
> >> So far, I don't see any pending blockers on 2.8.3, so my current
> plan
> >> is to cut off 1st RC of 2.8.3 in next several days:
> >> - For all coming commits to land on branch-2.8, please mark
> the
> >> fix version as 2.8.4.
> >> - If there is a really important fix for 2.8.3 and getting
> >> closed, please notify me ahead before landing it on branch-2.8.3.
> >> Please let me know if you have any thoughts or comments on the plan.
> >>
> >> Thanks,
> >>
> >> Junping
> >> ________________________________________
> >> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
> >> junping_du@apache.org>
> >> Sent: Friday, October 27, 2017 3:33 PM
> >> To: general@hadoop.apache.org
> >> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
> >>
> >> Hi all,
> >>
> >> It gives me great pleasure to announce that the Apache Hadoop
> >> community has voted to release Apache Hadoop 2.8.2, which is now
> available
> >> for download from Apache mirrors[1]. For download instructions please
> refer
> >> to the Apache Hadoop Release page [2].
> >>
> >> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
> and
> >> our newest stable release for entire Apache Hadoop project. For major
> >> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
> page[3].
> >>
> >> This release has 315 resolved issues since previous 2.8.1 release with
> >> following
> >> breakdown:
> >> - 91 in Hadoop Common
> >> - 99 in HDFS
> >> - 105 in YARN
> >> - 20 in MapReduce
> >> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
> >>
> >> The release news is posted on the Hadoop website too, you can go to the
> >> downloads section directly [6].
> >>
> >> Thank you all for contributing to the Apache Hadoop release!
> >>
> >>
> >> Cheers,
> >>
> >> Junping
> >>
> >>
> >> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
> >>
> >> [2] http://hadoop.apache.org/releases.html
> >>
> >> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
> >>
> >> [4]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/CHANGES.2.8.2.html
> >>
> >> [5]
> >> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
> >> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
> >>
> >> [6] http://hadoop.apache.org/releases.html#Download
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
> >>
> >>
> >
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Andrew Wang <an...@cloudera.com>.
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?focusedCommentId=16247085&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks Kai for calling out this feature/improvement for attention and
> Andrew for comments.
>
>
> While I agree that maintenance release should focus on important bug fix
> only, I doubt we have strict rules to disallow any features/improvements to
> land on maint release especially when those are small footprint or low
> impact on existing code/features. In practice, we indeed has 77 new
> features/improvements in latest 2.7.3 and 2.7.4 release.
>
>
> Back to HADOOP-14964, I did a quick check and it looks like case here
> belongs to self-contained improvement that has very low impact on existing
> code base, so I am OK with the improvement get landed on branch-2.8 in case
> it is well reviewed and tested.
>
>
> However, as RM of branch-2.8, I have two concerns to accept it in our
> 2.8.3 release:
>
> 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are for
> several critical bug fixes and we should target to release it very soon -
> my current plan is to cut RC out within this week inline with waiting
> for 3.0.0 vote closing. Can this improvement be well tested against
> branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> have strong commitment on test plan and activities in such a tight time.
>
>
> 2. Upgrading - I haven't heard we settle down the plan of releasing this
> feature in 2.9.1 release - though I saw some discussions are going on
> at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> this improvement, then users consuming this feature/improvement have no 2.9
> release to upgrade or forcefully upgrade with regression. We may need a
> better upgrade story here.
>
>
> Pls let me know what you think. Thanks!
>
>
>
> Thanks,
>
>
> Junping
>
>
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>
> *Sent:* Monday, November 20, 2017 10:22 PM
> *To:* Zheng, Kai
> *Cc:* Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
>
> I'm against including new features in maintenance releases, since they're
> meant to be bug-fix only.
>
> If we're struggling with being able to deliver new features in a safe and
> timely fashion, let's try to address that, not overload the meaning of
> "maintenance release".
>
> Best,
> Andrew
>
> On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
>
>> Hi Junping,
>>
>> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>>
>> I have an ask, is it convenient to include the back port work for OSS
>> connector module? We have some Hadoop users that wish to have it by default
>> for convenience, though in the past they used it by back porting
>> themselves. I have raised this and got thoughts from Chris and Steve. Looks
>> like this is more wanted for 2.9 but I wanted to ask again here for broad
>> feedback and thoughts by this chance. The back port patch is available for
>> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we
>> can see some shift from 2.7.x, hence it's worth more important features and
>> efforts. How would you think? Thanks!
>>
>> https://issues.apache.org/jira/browse/HADOOP-14964
>>
>> Regards,
>> Kai
>>
>> -----Original Message-----
>> From: Junping Du [mailto:jdu@hortonworks.com]
>> Sent: Tuesday, November 14, 2017 9:02 AM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Subject: Apache Hadoop 2.8.3 Release Plan
>>
>> Hi,
>> We have several important fixes get landed on branch-2.8 and I would
>> like to cut off branch-2.8.3 now to start 2.8.3 release work.
>> So far, I don't see any pending blockers on 2.8.3, so my current plan
>> is to cut off 1st RC of 2.8.3 in next several days:
>> - For all coming commits to land on branch-2.8, please mark the
>> fix version as 2.8.4.
>> - If there is a really important fix for 2.8.3 and getting
>> closed, please notify me ahead before landing it on branch-2.8.3.
>> Please let me know if you have any thoughts or comments on the plan.
>>
>> Thanks,
>>
>> Junping
>> ________________________________________
>> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
>> junping_du@apache.org>
>> Sent: Friday, October 27, 2017 3:33 PM
>> To: general@hadoop.apache.org
>> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>>
>> Hi all,
>>
>> It gives me great pleasure to announce that the Apache Hadoop
>> community has voted to release Apache Hadoop 2.8.2, which is now available
>> for download from Apache mirrors[1]. For download instructions please refer
>> to the Apache Hadoop Release page [2].
>>
>> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
>> our newest stable release for entire Apache Hadoop project. For major
>> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>>
>> This release has 315 resolved issues since previous 2.8.1 release with
>> following
>> breakdown:
>> - 91 in Hadoop Common
>> - 99 in HDFS
>> - 105 in YARN
>> - 20 in MapReduce
>> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>>
>> The release news is posted on the Hadoop website too, you can go to the
>> downloads section directly [6].
>>
>> Thank you all for contributing to the Apache Hadoop release!
>>
>>
>> Cheers,
>>
>> Junping
>>
>>
>> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>>
>> [2] http://hadoop.apache.org/releases.html
>>
>> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>>
>> [4]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/CHANGES.2.8.2.html
>>
>> [5]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>>
>> [6] http://hadoop.apache.org/releases.html#Download
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>
>>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Andrew Wang <an...@cloudera.com>.
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?focusedCommentId=16247085&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks Kai for calling out this feature/improvement for attention and
> Andrew for comments.
>
>
> While I agree that maintenance release should focus on important bug fix
> only, I doubt we have strict rules to disallow any features/improvements to
> land on maint release especially when those are small footprint or low
> impact on existing code/features. In practice, we indeed has 77 new
> features/improvements in latest 2.7.3 and 2.7.4 release.
>
>
> Back to HADOOP-14964, I did a quick check and it looks like case here
> belongs to self-contained improvement that has very low impact on existing
> code base, so I am OK with the improvement get landed on branch-2.8 in case
> it is well reviewed and tested.
>
>
> However, as RM of branch-2.8, I have two concerns to accept it in our
> 2.8.3 release:
>
> 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are for
> several critical bug fixes and we should target to release it very soon -
> my current plan is to cut RC out within this week inline with waiting
> for 3.0.0 vote closing. Can this improvement be well tested against
> branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> have strong commitment on test plan and activities in such a tight time.
>
>
> 2. Upgrading - I haven't heard we settle down the plan of releasing this
> feature in 2.9.1 release - though I saw some discussions are going on
> at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> this improvement, then users consuming this feature/improvement have no 2.9
> release to upgrade or forcefully upgrade with regression. We may need a
> better upgrade story here.
>
>
> Pls let me know what you think. Thanks!
>
>
>
> Thanks,
>
>
> Junping
>
>
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>
> *Sent:* Monday, November 20, 2017 10:22 PM
> *To:* Zheng, Kai
> *Cc:* Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
>
> I'm against including new features in maintenance releases, since they're
> meant to be bug-fix only.
>
> If we're struggling with being able to deliver new features in a safe and
> timely fashion, let's try to address that, not overload the meaning of
> "maintenance release".
>
> Best,
> Andrew
>
> On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
>
>> Hi Junping,
>>
>> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>>
>> I have an ask, is it convenient to include the back port work for OSS
>> connector module? We have some Hadoop users that wish to have it by default
>> for convenience, though in the past they used it by back porting
>> themselves. I have raised this and got thoughts from Chris and Steve. Looks
>> like this is more wanted for 2.9 but I wanted to ask again here for broad
>> feedback and thoughts by this chance. The back port patch is available for
>> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we
>> can see some shift from 2.7.x, hence it's worth more important features and
>> efforts. How would you think? Thanks!
>>
>> https://issues.apache.org/jira/browse/HADOOP-14964
>>
>> Regards,
>> Kai
>>
>> -----Original Message-----
>> From: Junping Du [mailto:jdu@hortonworks.com]
>> Sent: Tuesday, November 14, 2017 9:02 AM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Subject: Apache Hadoop 2.8.3 Release Plan
>>
>> Hi,
>> We have several important fixes get landed on branch-2.8 and I would
>> like to cut off branch-2.8.3 now to start 2.8.3 release work.
>> So far, I don't see any pending blockers on 2.8.3, so my current plan
>> is to cut off 1st RC of 2.8.3 in next several days:
>> - For all coming commits to land on branch-2.8, please mark the
>> fix version as 2.8.4.
>> - If there is a really important fix for 2.8.3 and getting
>> closed, please notify me ahead before landing it on branch-2.8.3.
>> Please let me know if you have any thoughts or comments on the plan.
>>
>> Thanks,
>>
>> Junping
>> ________________________________________
>> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
>> junping_du@apache.org>
>> Sent: Friday, October 27, 2017 3:33 PM
>> To: general@hadoop.apache.org
>> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>>
>> Hi all,
>>
>> It gives me great pleasure to announce that the Apache Hadoop
>> community has voted to release Apache Hadoop 2.8.2, which is now available
>> for download from Apache mirrors[1]. For download instructions please refer
>> to the Apache Hadoop Release page [2].
>>
>> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
>> our newest stable release for entire Apache Hadoop project. For major
>> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>>
>> This release has 315 resolved issues since previous 2.8.1 release with
>> following
>> breakdown:
>> - 91 in Hadoop Common
>> - 99 in HDFS
>> - 105 in YARN
>> - 20 in MapReduce
>> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>>
>> The release news is posted on the Hadoop website too, you can go to the
>> downloads section directly [6].
>>
>> Thank you all for contributing to the Apache Hadoop release!
>>
>>
>> Cheers,
>>
>> Junping
>>
>>
>> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>>
>> [2] http://hadoop.apache.org/releases.html
>>
>> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>>
>> [4]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/CHANGES.2.8.2.html
>>
>> [5]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>>
>> [6] http://hadoop.apache.org/releases.html#Download
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>
>>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Andrew Wang <an...@cloudera.com>.
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?focusedCommentId=16247085&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks Kai for calling out this feature/improvement for attention and
> Andrew for comments.
>
>
> While I agree that maintenance release should focus on important bug fix
> only, I doubt we have strict rules to disallow any features/improvements to
> land on maint release especially when those are small footprint or low
> impact on existing code/features. In practice, we indeed has 77 new
> features/improvements in latest 2.7.3 and 2.7.4 release.
>
>
> Back to HADOOP-14964, I did a quick check and it looks like case here
> belongs to self-contained improvement that has very low impact on existing
> code base, so I am OK with the improvement get landed on branch-2.8 in case
> it is well reviewed and tested.
>
>
> However, as RM of branch-2.8, I have two concerns to accept it in our
> 2.8.3 release:
>
> 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are for
> several critical bug fixes and we should target to release it very soon -
> my current plan is to cut RC out within this week inline with waiting
> for 3.0.0 vote closing. Can this improvement be well tested against
> branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> have strong commitment on test plan and activities in such a tight time.
>
>
> 2. Upgrading - I haven't heard we settle down the plan of releasing this
> feature in 2.9.1 release - though I saw some discussions are going on
> at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> this improvement, then users consuming this feature/improvement have no 2.9
> release to upgrade or forcefully upgrade with regression. We may need a
> better upgrade story here.
>
>
> Pls let me know what you think. Thanks!
>
>
>
> Thanks,
>
>
> Junping
>
>
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>
> *Sent:* Monday, November 20, 2017 10:22 PM
> *To:* Zheng, Kai
> *Cc:* Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
>
> I'm against including new features in maintenance releases, since they're
> meant to be bug-fix only.
>
> If we're struggling with being able to deliver new features in a safe and
> timely fashion, let's try to address that, not overload the meaning of
> "maintenance release".
>
> Best,
> Andrew
>
> On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
>
>> Hi Junping,
>>
>> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>>
>> I have an ask, is it convenient to include the back port work for OSS
>> connector module? We have some Hadoop users that wish to have it by default
>> for convenience, though in the past they used it by back porting
>> themselves. I have raised this and got thoughts from Chris and Steve. Looks
>> like this is more wanted for 2.9 but I wanted to ask again here for broad
>> feedback and thoughts by this chance. The back port patch is available for
>> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we
>> can see some shift from 2.7.x, hence it's worth more important features and
>> efforts. How would you think? Thanks!
>>
>> https://issues.apache.org/jira/browse/HADOOP-14964
>>
>> Regards,
>> Kai
>>
>> -----Original Message-----
>> From: Junping Du [mailto:jdu@hortonworks.com]
>> Sent: Tuesday, November 14, 2017 9:02 AM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Subject: Apache Hadoop 2.8.3 Release Plan
>>
>> Hi,
>> We have several important fixes get landed on branch-2.8 and I would
>> like to cut off branch-2.8.3 now to start 2.8.3 release work.
>> So far, I don't see any pending blockers on 2.8.3, so my current plan
>> is to cut off 1st RC of 2.8.3 in next several days:
>> - For all coming commits to land on branch-2.8, please mark the
>> fix version as 2.8.4.
>> - If there is a really important fix for 2.8.3 and getting
>> closed, please notify me ahead before landing it on branch-2.8.3.
>> Please let me know if you have any thoughts or comments on the plan.
>>
>> Thanks,
>>
>> Junping
>> ________________________________________
>> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
>> junping_du@apache.org>
>> Sent: Friday, October 27, 2017 3:33 PM
>> To: general@hadoop.apache.org
>> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>>
>> Hi all,
>>
>> It gives me great pleasure to announce that the Apache Hadoop
>> community has voted to release Apache Hadoop 2.8.2, which is now available
>> for download from Apache mirrors[1]. For download instructions please refer
>> to the Apache Hadoop Release page [2].
>>
>> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
>> our newest stable release for entire Apache Hadoop project. For major
>> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>>
>> This release has 315 resolved issues since previous 2.8.1 release with
>> following
>> breakdown:
>> - 91 in Hadoop Common
>> - 99 in HDFS
>> - 105 in YARN
>> - 20 in MapReduce
>> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>>
>> The release news is posted on the Hadoop website too, you can go to the
>> downloads section directly [6].
>>
>> Thank you all for contributing to the Apache Hadoop release!
>>
>>
>> Cheers,
>>
>> Junping
>>
>>
>> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>>
>> [2] http://hadoop.apache.org/releases.html
>>
>> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>>
>> [4]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/CHANGES.2.8.2.html
>>
>> [5]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>>
>> [6] http://hadoop.apache.org/releases.html#Download
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>
>>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Andrew Wang <an...@cloudera.com>.
The Aliyun OSS code isn't a small improvement. If you look at Sammi's
comment
<https://issues.apache.org/jira/browse/HADOOP-14964?focusedCommentId=16247085&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16247085>,
it's
a 17 patch series that is being backported in one shot. What we're talking
about is equivalent to merging a feature branch in a maintenance release. I
see that Kai and Chris are having a discussion about the dependency
changes, which indicates this is not a zero-risk change either. We really
should not be changing dependency versions in a maintenance unless it's
because of a bug.
It's unfortunate from a timing perspective that this missed 2.9.0, but I
still think it should wait for the next minor. Merging a feature into a
maintenance release sets the wrong precedent.
Best,
Andrew
On Tue, Nov 21, 2017 at 1:08 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks Kai for calling out this feature/improvement for attention and
> Andrew for comments.
>
>
> While I agree that maintenance release should focus on important bug fix
> only, I doubt we have strict rules to disallow any features/improvements to
> land on maint release especially when those are small footprint or low
> impact on existing code/features. In practice, we indeed has 77 new
> features/improvements in latest 2.7.3 and 2.7.4 release.
>
>
> Back to HADOOP-14964, I did a quick check and it looks like case here
> belongs to self-contained improvement that has very low impact on existing
> code base, so I am OK with the improvement get landed on branch-2.8 in case
> it is well reviewed and tested.
>
>
> However, as RM of branch-2.8, I have two concerns to accept it in our
> 2.8.3 release:
>
> 1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are for
> several critical bug fixes and we should target to release it very soon -
> my current plan is to cut RC out within this week inline with waiting
> for 3.0.0 vote closing. Can this improvement be well tested against
> branch-2.8.3 within this strictly timeline? It seems a bit rush unless we
> have strong commitment on test plan and activities in such a tight time.
>
>
> 2. Upgrading - I haven't heard we settle down the plan of releasing this
> feature in 2.9.1 release - though I saw some discussions are going on
> at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes
> this improvement, then users consuming this feature/improvement have no 2.9
> release to upgrade or forcefully upgrade with regression. We may need a
> better upgrade story here.
>
>
> Pls let me know what you think. Thanks!
>
>
>
> Thanks,
>
>
> Junping
>
>
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>
> *Sent:* Monday, November 20, 2017 10:22 PM
> *To:* Zheng, Kai
> *Cc:* Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> *Subject:* Re: Apache Hadoop 2.8.3 Release Plan
>
> I'm against including new features in maintenance releases, since they're
> meant to be bug-fix only.
>
> If we're struggling with being able to deliver new features in a safe and
> timely fashion, let's try to address that, not overload the meaning of
> "maintenance release".
>
> Best,
> Andrew
>
> On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
>
>> Hi Junping,
>>
>> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>>
>> I have an ask, is it convenient to include the back port work for OSS
>> connector module? We have some Hadoop users that wish to have it by default
>> for convenience, though in the past they used it by back porting
>> themselves. I have raised this and got thoughts from Chris and Steve. Looks
>> like this is more wanted for 2.9 but I wanted to ask again here for broad
>> feedback and thoughts by this chance. The back port patch is available for
>> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we
>> can see some shift from 2.7.x, hence it's worth more important features and
>> efforts. How would you think? Thanks!
>>
>> https://issues.apache.org/jira/browse/HADOOP-14964
>>
>> Regards,
>> Kai
>>
>> -----Original Message-----
>> From: Junping Du [mailto:jdu@hortonworks.com]
>> Sent: Tuesday, November 14, 2017 9:02 AM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Subject: Apache Hadoop 2.8.3 Release Plan
>>
>> Hi,
>> We have several important fixes get landed on branch-2.8 and I would
>> like to cut off branch-2.8.3 now to start 2.8.3 release work.
>> So far, I don't see any pending blockers on 2.8.3, so my current plan
>> is to cut off 1st RC of 2.8.3 in next several days:
>> - For all coming commits to land on branch-2.8, please mark the
>> fix version as 2.8.4.
>> - If there is a really important fix for 2.8.3 and getting
>> closed, please notify me ahead before landing it on branch-2.8.3.
>> Please let me know if you have any thoughts or comments on the plan.
>>
>> Thanks,
>>
>> Junping
>> ________________________________________
>> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
>> junping_du@apache.org>
>> Sent: Friday, October 27, 2017 3:33 PM
>> To: general@hadoop.apache.org
>> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>>
>> Hi all,
>>
>> It gives me great pleasure to announce that the Apache Hadoop
>> community has voted to release Apache Hadoop 2.8.2, which is now available
>> for download from Apache mirrors[1]. For download instructions please refer
>> to the Apache Hadoop Release page [2].
>>
>> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
>> our newest stable release for entire Apache Hadoop project. For major
>> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>>
>> This release has 315 resolved issues since previous 2.8.1 release with
>> following
>> breakdown:
>> - 91 in Hadoop Common
>> - 99 in HDFS
>> - 105 in YARN
>> - 20 in MapReduce
>> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>>
>> The release news is posted on the Hadoop website too, you can go to the
>> downloads section directly [6].
>>
>> Thank you all for contributing to the Apache Hadoop release!
>>
>>
>> Cheers,
>>
>> Junping
>>
>>
>> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>>
>> [2] http://hadoop.apache.org/releases.html
>>
>> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>>
>> [4]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/CHANGES.2.8.2.html
>>
>> [5]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/had
>> oop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>>
>> [6] http://hadoop.apache.org/releases.html#Download
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>
>>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Kai for calling out this feature/improvement for attention and Andrew for comments.
While I agree that maintenance release should focus on important bug fix only, I doubt we have strict rules to disallow any features/improvements to land on maint release especially when those are small footprint or low impact on existing code/features. In practice, we indeed has 77 new features/improvements in latest 2.7.3 and 2.7.4 release.
Back to HADOOP-14964, I did a quick check and it looks like case here belongs to self-contained improvement that has very low impact on existing code base, so I am OK with the improvement get landed on branch-2.8 in case it is well reviewed and tested.
However, as RM of branch-2.8, I have two concerns to accept it in our 2.8.3 release:
1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are for several critical bug fixes and we should target to release it very soon - my current plan is to cut RC out within this week inline with waiting for 3.0.0 vote closing. Can this improvement be well tested against branch-2.8.3 within this strictly timeline? It seems a bit rush unless we have strong commitment on test plan and activities in such a tight time.
2. Upgrading - I haven't heard we settle down the plan of releasing this feature in 2.9.1 release - though I saw some discussions are going on at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes this improvement, then users consuming this feature/improvement have no 2.9 release to upgrade or forcefully upgrade with regression. We may need a better upgrade story here.
Pls let me know what you think. Thanks!
Thanks,
Junping
________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Monday, November 20, 2017 10:22 PM
To: Zheng, Kai
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that, not overload the meaning of "maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com>> wrote:
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS connector module? We have some Hadoop users that wish to have it by default for convenience, though in the past they used it by back porting themselves. I have raised this and got thoughts from Chris and Steve. Looks like this is more wanted for 2.9 but I wanted to ask again here for broad feedback and thoughts by this chance. The back port patch is available for 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we can see some shift from 2.7.x, hence it's worth more important features and efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com<ma...@hortonworks.com>]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current plan is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark the fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com<ma...@gmail.com> <du...@gmail.com>> on behalf of ??? <ju...@apache.org>>
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 2.8.2, which is now available for download from Apache mirrors[1]. For download instructions please refer to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and our newest stable release for entire Apache Hadoop project. For major changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
This release has 315 resolved issues since previous 2.8.1 release with following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: common-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Andrew Wang <an...@cloudera.com>.
>
>
> >> If we're struggling with being able to deliver new features in a safe
> and timely fashion, let's try to address that...
>
> This is interesting. Do you aware any means to do that? Thanks!
>
> I've mentioned this a few times on the lists before, but our biggest gap
in keeping branches releasable is automated integration testing.
I think we try to put too much into our minor releases, and features arrive
before they're baked. Having automated integration testing helps with this.
When we were finally able to turn on CI for the 3.0.0 release branch, we
started finding bugs much sooner after they were introduced, which made it
easier to revert before too much other code was built on top. The early
alphas felt Sisyphean at times, with bugs being introduced faster than we
could uncover and fix them.
A smaller example would be release validation. I've long wanted a nightly
Jenkins job that makes an RC and runs some basic checks on it. We end up
rolling extra RCs for small stuff that could have been caught earlier.
Best,
Andrew
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Andrew Wang <an...@cloudera.com>.
>
>
> >> If we're struggling with being able to deliver new features in a safe
> and timely fashion, let's try to address that...
>
> This is interesting. Do you aware any means to do that? Thanks!
>
> I've mentioned this a few times on the lists before, but our biggest gap
in keeping branches releasable is automated integration testing.
I think we try to put too much into our minor releases, and features arrive
before they're baked. Having automated integration testing helps with this.
When we were finally able to turn on CI for the 3.0.0 release branch, we
started finding bugs much sooner after they were introduced, which made it
easier to revert before too much other code was built on top. The early
alphas felt Sisyphean at times, with bugs being introduced faster than we
could uncover and fix them.
A smaller example would be release validation. I've long wanted a nightly
Jenkins job that makes an RC and runs some basic checks on it. We end up
rolling extra RCs for small stuff that could have been caught earlier.
Best,
Andrew
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Andrew Wang <an...@cloudera.com>.
>
>
> >> If we're struggling with being able to deliver new features in a safe
> and timely fashion, let's try to address that...
>
> This is interesting. Do you aware any means to do that? Thanks!
>
> I've mentioned this a few times on the lists before, but our biggest gap
in keeping branches releasable is automated integration testing.
I think we try to put too much into our minor releases, and features arrive
before they're baked. Having automated integration testing helps with this.
When we were finally able to turn on CI for the 3.0.0 release branch, we
started finding bugs much sooner after they were introduced, which made it
easier to revert before too much other code was built on top. The early
alphas felt Sisyphean at times, with bugs being introduced faster than we
could uncover and fix them.
A smaller example would be release validation. I've long wanted a nightly
Jenkins job that makes an RC and runs some basic checks on it. We end up
rolling extra RCs for small stuff that could have been caught earlier.
Best,
Andrew
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Andrew Wang <an...@cloudera.com>.
>
>
> >> If we're struggling with being able to deliver new features in a safe
> and timely fashion, let's try to address that...
>
> This is interesting. Do you aware any means to do that? Thanks!
>
> I've mentioned this a few times on the lists before, but our biggest gap
in keeping branches releasable is automated integration testing.
I think we try to put too much into our minor releases, and features arrive
before they're baked. Having automated integration testing helps with this.
When we were finally able to turn on CI for the 3.0.0 release branch, we
started finding bugs much sooner after they were introduced, which made it
easier to revert before too much other code was built on top. The early
alphas felt Sisyphean at times, with bugs being introduced faster than we
could uncover and fix them.
A smaller example would be release validation. I've long wanted a nightly
Jenkins job that makes an RC and runs some basic checks on it. We end up
rolling extra RCs for small stuff that could have been caught earlier.
Best,
Andrew
RE: Apache Hadoop 2.8.3 Release Plan
Posted by "Zheng, Kai" <ka...@intel.com>.
Thanks Andrew for the comments.
Yes, if we're "strictly" following the "maintenance release" practice, that'd be great and it's never my intent to overload it and cause mess.
>> If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that...
This is interesting. Do you aware any means to do that? Thanks!
Regards,
Kai
-----Original Message-----
From: Andrew Wang [mailto:andrew.wang@cloudera.com]
Sent: Tuesday, November 21, 2017 2:22 PM
To: Zheng, Kai <ka...@intel.com>
Cc: Junping Du <jd...@hortonworks.com>; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that, not overload the meaning of "maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
> Hi Junping,
>
> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>
> I have an ask, is it convenient to include the back port work for OSS
> connector module? We have some Hadoop users that wish to have it by
> default for convenience, though in the past they used it by back
> porting themselves. I have raised this and got thoughts from Chris and
> Steve. Looks like this is more wanted for 2.9 but I wanted to ask
> again here for broad feedback and thoughts by this chance. The back
> port patch is available for
> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising
> as we can see some shift from 2.7.x, hence it's worth more important
> features and efforts. How would you think? Thanks!
>
> https://issues.apache.org/jira/browse/HADOOP-14964
>
> Regards,
> Kai
>
> -----Original Message-----
> From: Junping Du [mailto:jdu@hortonworks.com]
> Sent: Tuesday, November 14, 2017 9:02 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Apache Hadoop 2.8.3 Release Plan
>
> Hi,
> We have several important fixes get landed on branch-2.8 and I
> would like to cut off branch-2.8.3 now to start 2.8.3 release work.
> So far, I don't see any pending blockers on 2.8.3, so my current
> plan is to cut off 1st RC of 2.8.3 in next several days:
> - For all coming commits to land on branch-2.8, please mark
> the fix version as 2.8.4.
> - If there is a really important fix for 2.8.3 and getting
> closed, please notify me ahead before landing it on branch-2.8.3.
> Please let me know if you have any thoughts or comments on the plan.
>
> Thanks,
>
> Junping
> ________________________________________
> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
> junping_du@apache.org>
> Sent: Friday, October 27, 2017 3:33 PM
> To: general@hadoop.apache.org
> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>
> Hi all,
>
> It gives me great pleasure to announce that the Apache Hadoop
> community has voted to release Apache Hadoop 2.8.2, which is now
> available for download from Apache mirrors[1]. For download
> instructions please refer to the Apache Hadoop Release page [2].
>
> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
> and our newest stable release for entire Apache Hadoop project. For
> major changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>
> This release has 315 resolved issues since previous 2.8.1 release with
> following
> breakdown:
> - 91 in Hadoop Common
> - 99 in HDFS
> - 105 in YARN
> - 20 in MapReduce
> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>
> The release news is posted on the Hadoop website too, you can go to
> the downloads section directly [6].
>
> Thank you all for contributing to the Apache Hadoop release!
>
>
> Cheers,
>
> Junping
>
>
> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>
> [2] http://hadoop.apache.org/releases.html
>
> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>
> [4]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/CHANGES.2.8.2.html
>
> [5]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>
> [6] http://hadoop.apache.org/releases.html#Download
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Kai for calling out this feature/improvement for attention and Andrew for comments.
While I agree that maintenance release should focus on important bug fix only, I doubt we have strict rules to disallow any features/improvements to land on maint release especially when those are small footprint or low impact on existing code/features. In practice, we indeed has 77 new features/improvements in latest 2.7.3 and 2.7.4 release.
Back to HADOOP-14964, I did a quick check and it looks like case here belongs to self-contained improvement that has very low impact on existing code base, so I am OK with the improvement get landed on branch-2.8 in case it is well reviewed and tested.
However, as RM of branch-2.8, I have two concerns to accept it in our 2.8.3 release:
1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are for several critical bug fixes and we should target to release it very soon - my current plan is to cut RC out within this week inline with waiting for 3.0.0 vote closing. Can this improvement be well tested against branch-2.8.3 within this strictly timeline? It seems a bit rush unless we have strong commitment on test plan and activities in such a tight time.
2. Upgrading - I haven't heard we settle down the plan of releasing this feature in 2.9.1 release - though I saw some discussions are going on at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes this improvement, then users consuming this feature/improvement have no 2.9 release to upgrade or forcefully upgrade with regression. We may need a better upgrade story here.
Pls let me know what you think. Thanks!
Thanks,
Junping
________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Monday, November 20, 2017 10:22 PM
To: Zheng, Kai
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that, not overload the meaning of "maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com>> wrote:
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS connector module? We have some Hadoop users that wish to have it by default for convenience, though in the past they used it by back porting themselves. I have raised this and got thoughts from Chris and Steve. Looks like this is more wanted for 2.9 but I wanted to ask again here for broad feedback and thoughts by this chance. The back port patch is available for 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we can see some shift from 2.7.x, hence it's worth more important features and efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com<ma...@hortonworks.com>]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current plan is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark the fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com<ma...@gmail.com> <du...@gmail.com>> on behalf of ??? <ju...@apache.org>>
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 2.8.2, which is now available for download from Apache mirrors[1]. For download instructions please refer to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and our newest stable release for entire Apache Hadoop project. For major changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
This release has 315 resolved issues since previous 2.8.1 release with following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: common-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
RE: Apache Hadoop 2.8.3 Release Plan
Posted by "Zheng, Kai" <ka...@intel.com>.
Thanks Andrew for the comments.
Yes, if we're "strictly" following the "maintenance release" practice, that'd be great and it's never my intent to overload it and cause mess.
>> If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that...
This is interesting. Do you aware any means to do that? Thanks!
Regards,
Kai
-----Original Message-----
From: Andrew Wang [mailto:andrew.wang@cloudera.com]
Sent: Tuesday, November 21, 2017 2:22 PM
To: Zheng, Kai <ka...@intel.com>
Cc: Junping Du <jd...@hortonworks.com>; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that, not overload the meaning of "maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
> Hi Junping,
>
> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>
> I have an ask, is it convenient to include the back port work for OSS
> connector module? We have some Hadoop users that wish to have it by
> default for convenience, though in the past they used it by back
> porting themselves. I have raised this and got thoughts from Chris and
> Steve. Looks like this is more wanted for 2.9 but I wanted to ask
> again here for broad feedback and thoughts by this chance. The back
> port patch is available for
> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising
> as we can see some shift from 2.7.x, hence it's worth more important
> features and efforts. How would you think? Thanks!
>
> https://issues.apache.org/jira/browse/HADOOP-14964
>
> Regards,
> Kai
>
> -----Original Message-----
> From: Junping Du [mailto:jdu@hortonworks.com]
> Sent: Tuesday, November 14, 2017 9:02 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Apache Hadoop 2.8.3 Release Plan
>
> Hi,
> We have several important fixes get landed on branch-2.8 and I
> would like to cut off branch-2.8.3 now to start 2.8.3 release work.
> So far, I don't see any pending blockers on 2.8.3, so my current
> plan is to cut off 1st RC of 2.8.3 in next several days:
> - For all coming commits to land on branch-2.8, please mark
> the fix version as 2.8.4.
> - If there is a really important fix for 2.8.3 and getting
> closed, please notify me ahead before landing it on branch-2.8.3.
> Please let me know if you have any thoughts or comments on the plan.
>
> Thanks,
>
> Junping
> ________________________________________
> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
> junping_du@apache.org>
> Sent: Friday, October 27, 2017 3:33 PM
> To: general@hadoop.apache.org
> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>
> Hi all,
>
> It gives me great pleasure to announce that the Apache Hadoop
> community has voted to release Apache Hadoop 2.8.2, which is now
> available for download from Apache mirrors[1]. For download
> instructions please refer to the Apache Hadoop Release page [2].
>
> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
> and our newest stable release for entire Apache Hadoop project. For
> major changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>
> This release has 315 resolved issues since previous 2.8.1 release with
> following
> breakdown:
> - 91 in Hadoop Common
> - 99 in HDFS
> - 105 in YARN
> - 20 in MapReduce
> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>
> The release news is posted on the Hadoop website too, you can go to
> the downloads section directly [6].
>
> Thank you all for contributing to the Apache Hadoop release!
>
>
> Cheers,
>
> Junping
>
>
> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>
> [2] http://hadoop.apache.org/releases.html
>
> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>
> [4]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/CHANGES.2.8.2.html
>
> [5]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>
> [6] http://hadoop.apache.org/releases.html#Download
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
RE: Apache Hadoop 2.8.3 Release Plan
Posted by "Zheng, Kai" <ka...@intel.com>.
Thanks Andrew for the comments.
Yes, if we're "strictly" following the "maintenance release" practice, that'd be great and it's never my intent to overload it and cause mess.
>> If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that...
This is interesting. Do you aware any means to do that? Thanks!
Regards,
Kai
-----Original Message-----
From: Andrew Wang [mailto:andrew.wang@cloudera.com]
Sent: Tuesday, November 21, 2017 2:22 PM
To: Zheng, Kai <ka...@intel.com>
Cc: Junping Du <jd...@hortonworks.com>; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that, not overload the meaning of "maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
> Hi Junping,
>
> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>
> I have an ask, is it convenient to include the back port work for OSS
> connector module? We have some Hadoop users that wish to have it by
> default for convenience, though in the past they used it by back
> porting themselves. I have raised this and got thoughts from Chris and
> Steve. Looks like this is more wanted for 2.9 but I wanted to ask
> again here for broad feedback and thoughts by this chance. The back
> port patch is available for
> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising
> as we can see some shift from 2.7.x, hence it's worth more important
> features and efforts. How would you think? Thanks!
>
> https://issues.apache.org/jira/browse/HADOOP-14964
>
> Regards,
> Kai
>
> -----Original Message-----
> From: Junping Du [mailto:jdu@hortonworks.com]
> Sent: Tuesday, November 14, 2017 9:02 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Apache Hadoop 2.8.3 Release Plan
>
> Hi,
> We have several important fixes get landed on branch-2.8 and I
> would like to cut off branch-2.8.3 now to start 2.8.3 release work.
> So far, I don't see any pending blockers on 2.8.3, so my current
> plan is to cut off 1st RC of 2.8.3 in next several days:
> - For all coming commits to land on branch-2.8, please mark
> the fix version as 2.8.4.
> - If there is a really important fix for 2.8.3 and getting
> closed, please notify me ahead before landing it on branch-2.8.3.
> Please let me know if you have any thoughts or comments on the plan.
>
> Thanks,
>
> Junping
> ________________________________________
> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
> junping_du@apache.org>
> Sent: Friday, October 27, 2017 3:33 PM
> To: general@hadoop.apache.org
> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>
> Hi all,
>
> It gives me great pleasure to announce that the Apache Hadoop
> community has voted to release Apache Hadoop 2.8.2, which is now
> available for download from Apache mirrors[1]. For download
> instructions please refer to the Apache Hadoop Release page [2].
>
> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
> and our newest stable release for entire Apache Hadoop project. For
> major changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>
> This release has 315 resolved issues since previous 2.8.1 release with
> following
> breakdown:
> - 91 in Hadoop Common
> - 99 in HDFS
> - 105 in YARN
> - 20 in MapReduce
> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>
> The release news is posted on the Hadoop website too, you can go to
> the downloads section directly [6].
>
> Thank you all for contributing to the Apache Hadoop release!
>
>
> Cheers,
>
> Junping
>
>
> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>
> [2] http://hadoop.apache.org/releases.html
>
> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>
> [4]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/CHANGES.2.8.2.html
>
> [5]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>
> [6] http://hadoop.apache.org/releases.html#Download
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Kai for calling out this feature/improvement for attention and Andrew for comments.
While I agree that maintenance release should focus on important bug fix only, I doubt we have strict rules to disallow any features/improvements to land on maint release especially when those are small footprint or low impact on existing code/features. In practice, we indeed has 77 new features/improvements in latest 2.7.3 and 2.7.4 release.
Back to HADOOP-14964, I did a quick check and it looks like case here belongs to self-contained improvement that has very low impact on existing code base, so I am OK with the improvement get landed on branch-2.8 in case it is well reviewed and tested.
However, as RM of branch-2.8, I have two concerns to accept it in our 2.8.3 release:
1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are for several critical bug fixes and we should target to release it very soon - my current plan is to cut RC out within this week inline with waiting for 3.0.0 vote closing. Can this improvement be well tested against branch-2.8.3 within this strictly timeline? It seems a bit rush unless we have strong commitment on test plan and activities in such a tight time.
2. Upgrading - I haven't heard we settle down the plan of releasing this feature in 2.9.1 release - though I saw some discussions are going on at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes this improvement, then users consuming this feature/improvement have no 2.9 release to upgrade or forcefully upgrade with regression. We may need a better upgrade story here.
Pls let me know what you think. Thanks!
Thanks,
Junping
________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Monday, November 20, 2017 10:22 PM
To: Zheng, Kai
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that, not overload the meaning of "maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com>> wrote:
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS connector module? We have some Hadoop users that wish to have it by default for convenience, though in the past they used it by back porting themselves. I have raised this and got thoughts from Chris and Steve. Looks like this is more wanted for 2.9 but I wanted to ask again here for broad feedback and thoughts by this chance. The back port patch is available for 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we can see some shift from 2.7.x, hence it's worth more important features and efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com<ma...@hortonworks.com>]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current plan is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark the fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com<ma...@gmail.com> <du...@gmail.com>> on behalf of ??? <ju...@apache.org>>
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 2.8.2, which is now available for download from Apache mirrors[1]. For download instructions please refer to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and our newest stable release for entire Apache Hadoop project. For major changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
This release has 315 resolved issues since previous 2.8.1 release with following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: common-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
RE: Apache Hadoop 2.8.3 Release Plan
Posted by "Zheng, Kai" <ka...@intel.com>.
Thanks Andrew for the comments.
Yes, if we're "strictly" following the "maintenance release" practice, that'd be great and it's never my intent to overload it and cause mess.
>> If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that...
This is interesting. Do you aware any means to do that? Thanks!
Regards,
Kai
-----Original Message-----
From: Andrew Wang [mailto:andrew.wang@cloudera.com]
Sent: Tuesday, November 21, 2017 2:22 PM
To: Zheng, Kai <ka...@intel.com>
Cc: Junping Du <jd...@hortonworks.com>; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that, not overload the meaning of "maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
> Hi Junping,
>
> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>
> I have an ask, is it convenient to include the back port work for OSS
> connector module? We have some Hadoop users that wish to have it by
> default for convenience, though in the past they used it by back
> porting themselves. I have raised this and got thoughts from Chris and
> Steve. Looks like this is more wanted for 2.9 but I wanted to ask
> again here for broad feedback and thoughts by this chance. The back
> port patch is available for
> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising
> as we can see some shift from 2.7.x, hence it's worth more important
> features and efforts. How would you think? Thanks!
>
> https://issues.apache.org/jira/browse/HADOOP-14964
>
> Regards,
> Kai
>
> -----Original Message-----
> From: Junping Du [mailto:jdu@hortonworks.com]
> Sent: Tuesday, November 14, 2017 9:02 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Apache Hadoop 2.8.3 Release Plan
>
> Hi,
> We have several important fixes get landed on branch-2.8 and I
> would like to cut off branch-2.8.3 now to start 2.8.3 release work.
> So far, I don't see any pending blockers on 2.8.3, so my current
> plan is to cut off 1st RC of 2.8.3 in next several days:
> - For all coming commits to land on branch-2.8, please mark
> the fix version as 2.8.4.
> - If there is a really important fix for 2.8.3 and getting
> closed, please notify me ahead before landing it on branch-2.8.3.
> Please let me know if you have any thoughts or comments on the plan.
>
> Thanks,
>
> Junping
> ________________________________________
> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
> junping_du@apache.org>
> Sent: Friday, October 27, 2017 3:33 PM
> To: general@hadoop.apache.org
> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>
> Hi all,
>
> It gives me great pleasure to announce that the Apache Hadoop
> community has voted to release Apache Hadoop 2.8.2, which is now
> available for download from Apache mirrors[1]. For download
> instructions please refer to the Apache Hadoop Release page [2].
>
> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line
> and our newest stable release for entire Apache Hadoop project. For
> major changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>
> This release has 315 resolved issues since previous 2.8.1 release with
> following
> breakdown:
> - 91 in Hadoop Common
> - 99 in HDFS
> - 105 in YARN
> - 20 in MapReduce
> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>
> The release news is posted on the Hadoop website too, you can go to
> the downloads section directly [6].
>
> Thank you all for contributing to the Apache Hadoop release!
>
>
> Cheers,
>
> Junping
>
>
> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>
> [2] http://hadoop.apache.org/releases.html
>
> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>
> [4]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/CHANGES.2.8.2.html
>
> [5]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>
> [6] http://hadoop.apache.org/releases.html#Download
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Daniel Templeton <da...@cloudera.com>.
Doh. Mailer dropped some of the lists. Replying again to avoid
fragmenting the discussion...
Still +1 to Andrew's comments.
Daniel
On 11/21/17 7:53 AM, Daniel Templeton wrote:
> +1
>
> Daniel
>
> On 11/20/17 10:22 PM, Andrew Wang wrote:
>> I'm against including new features in maintenance releases, since
>> they're
>> meant to be bug-fix only.
>>
>> If we're struggling with being able to deliver new features in a safe
>> and
>> timely fashion, let's try to address that, not overload the meaning of
>> "maintenance release".
>>
>> Best,
>> Andrew
>>
>> On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
>>
>>> Hi Junping,
>>>
>>> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>>>
>>> I have an ask, is it convenient to include the back port work for OSS
>>> connector module? We have some Hadoop users that wish to have it by
>>> default
>>> for convenience, though in the past they used it by back porting
>>> themselves. I have raised this and got thoughts from Chris and
>>> Steve. Looks
>>> like this is more wanted for 2.9 but I wanted to ask again here for
>>> broad
>>> feedback and thoughts by this chance. The back port patch is
>>> available for
>>> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising
>>> as we
>>> can see some shift from 2.7.x, hence it's worth more important
>>> features and
>>> efforts. How would you think? Thanks!
>>>
>>> https://issues.apache.org/jira/browse/HADOOP-14964
>>>
>>> Regards,
>>> Kai
>>>
>>> -----Original Message-----
>>> From: Junping Du [mailto:jdu@hortonworks.com]
>>> Sent: Tuesday, November 14, 2017 9:02 AM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Subject: Apache Hadoop 2.8.3 Release Plan
>>>
>>> Hi,
>>> We have several important fixes get landed on branch-2.8 and I
>>> would
>>> like to cut off branch-2.8.3 now to start 2.8.3 release work.
>>> So far, I don't see any pending blockers on 2.8.3, so my
>>> current plan
>>> is to cut off 1st RC of 2.8.3 in next several days:
>>> - For all coming commits to land on branch-2.8, please
>>> mark the
>>> fix version as 2.8.4.
>>> - If there is a really important fix for 2.8.3 and getting
>>> closed, please notify me ahead before landing it on branch-2.8.3.
>>> Please let me know if you have any thoughts or comments on the
>>> plan.
>>>
>>> Thanks,
>>>
>>> Junping
>>> ________________________________________
>>> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
>>> junping_du@apache.org>
>>> Sent: Friday, October 27, 2017 3:33 PM
>>> To: general@hadoop.apache.org
>>> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>>>
>>> Hi all,
>>>
>>> It gives me great pleasure to announce that the Apache Hadoop
>>> community has voted to release Apache Hadoop 2.8.2, which is now
>>> available
>>> for download from Apache mirrors[1]. For download instructions
>>> please refer
>>> to the Apache Hadoop Release page [2].
>>>
>>> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8
>>> line and
>>> our newest stable release for entire Apache Hadoop project. For major
>>> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
>>> page[3].
>>>
>>> This release has 315 resolved issues since previous 2.8.1 release with
>>> following
>>> breakdown:
>>> - 91 in Hadoop Common
>>> - 99 in HDFS
>>> - 105 in YARN
>>> - 20 in MapReduce
>>> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>>>
>>> The release news is posted on the Hadoop website too, you can go to the
>>> downloads section directly [6].
>>>
>>> Thank you all for contributing to the Apache Hadoop release!
>>>
>>>
>>> Cheers,
>>>
>>> Junping
>>>
>>>
>>> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>>>
>>> [2] http://hadoop.apache.org/releases.html
>>>
>>> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>>>
>>> [4]
>>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
>>> hadoop-common/release/2.8.2/CHANGES.2.8.2.html
>>>
>>> [5]
>>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
>>> hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>>>
>>> [6] http://hadoop.apache.org/releases.html#Download
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>>
>>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Daniel Templeton <da...@cloudera.com>.
Doh. Mailer dropped some of the lists. Replying again to avoid
fragmenting the discussion...
Still +1 to Andrew's comments.
Daniel
On 11/21/17 7:53 AM, Daniel Templeton wrote:
> +1
>
> Daniel
>
> On 11/20/17 10:22 PM, Andrew Wang wrote:
>> I'm against including new features in maintenance releases, since
>> they're
>> meant to be bug-fix only.
>>
>> If we're struggling with being able to deliver new features in a safe
>> and
>> timely fashion, let's try to address that, not overload the meaning of
>> "maintenance release".
>>
>> Best,
>> Andrew
>>
>> On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
>>
>>> Hi Junping,
>>>
>>> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>>>
>>> I have an ask, is it convenient to include the back port work for OSS
>>> connector module? We have some Hadoop users that wish to have it by
>>> default
>>> for convenience, though in the past they used it by back porting
>>> themselves. I have raised this and got thoughts from Chris and
>>> Steve. Looks
>>> like this is more wanted for 2.9 but I wanted to ask again here for
>>> broad
>>> feedback and thoughts by this chance. The back port patch is
>>> available for
>>> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising
>>> as we
>>> can see some shift from 2.7.x, hence it's worth more important
>>> features and
>>> efforts. How would you think? Thanks!
>>>
>>> https://issues.apache.org/jira/browse/HADOOP-14964
>>>
>>> Regards,
>>> Kai
>>>
>>> -----Original Message-----
>>> From: Junping Du [mailto:jdu@hortonworks.com]
>>> Sent: Tuesday, November 14, 2017 9:02 AM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Subject: Apache Hadoop 2.8.3 Release Plan
>>>
>>> Hi,
>>> We have several important fixes get landed on branch-2.8 and I
>>> would
>>> like to cut off branch-2.8.3 now to start 2.8.3 release work.
>>> So far, I don't see any pending blockers on 2.8.3, so my
>>> current plan
>>> is to cut off 1st RC of 2.8.3 in next several days:
>>> - For all coming commits to land on branch-2.8, please
>>> mark the
>>> fix version as 2.8.4.
>>> - If there is a really important fix for 2.8.3 and getting
>>> closed, please notify me ahead before landing it on branch-2.8.3.
>>> Please let me know if you have any thoughts or comments on the
>>> plan.
>>>
>>> Thanks,
>>>
>>> Junping
>>> ________________________________________
>>> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
>>> junping_du@apache.org>
>>> Sent: Friday, October 27, 2017 3:33 PM
>>> To: general@hadoop.apache.org
>>> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>>>
>>> Hi all,
>>>
>>> It gives me great pleasure to announce that the Apache Hadoop
>>> community has voted to release Apache Hadoop 2.8.2, which is now
>>> available
>>> for download from Apache mirrors[1]. For download instructions
>>> please refer
>>> to the Apache Hadoop Release page [2].
>>>
>>> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8
>>> line and
>>> our newest stable release for entire Apache Hadoop project. For major
>>> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
>>> page[3].
>>>
>>> This release has 315 resolved issues since previous 2.8.1 release with
>>> following
>>> breakdown:
>>> - 91 in Hadoop Common
>>> - 99 in HDFS
>>> - 105 in YARN
>>> - 20 in MapReduce
>>> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>>>
>>> The release news is posted on the Hadoop website too, you can go to the
>>> downloads section directly [6].
>>>
>>> Thank you all for contributing to the Apache Hadoop release!
>>>
>>>
>>> Cheers,
>>>
>>> Junping
>>>
>>>
>>> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>>>
>>> [2] http://hadoop.apache.org/releases.html
>>>
>>> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>>>
>>> [4]
>>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
>>> hadoop-common/release/2.8.2/CHANGES.2.8.2.html
>>>
>>> [5]
>>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
>>> hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>>>
>>> [6] http://hadoop.apache.org/releases.html#Download
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>>
>>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Daniel Templeton <da...@cloudera.com>.
Doh. Mailer dropped some of the lists. Replying again to avoid
fragmenting the discussion...
Still +1 to Andrew's comments.
Daniel
On 11/21/17 7:53 AM, Daniel Templeton wrote:
> +1
>
> Daniel
>
> On 11/20/17 10:22 PM, Andrew Wang wrote:
>> I'm against including new features in maintenance releases, since
>> they're
>> meant to be bug-fix only.
>>
>> If we're struggling with being able to deliver new features in a safe
>> and
>> timely fashion, let's try to address that, not overload the meaning of
>> "maintenance release".
>>
>> Best,
>> Andrew
>>
>> On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
>>
>>> Hi Junping,
>>>
>>> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>>>
>>> I have an ask, is it convenient to include the back port work for OSS
>>> connector module? We have some Hadoop users that wish to have it by
>>> default
>>> for convenience, though in the past they used it by back porting
>>> themselves. I have raised this and got thoughts from Chris and
>>> Steve. Looks
>>> like this is more wanted for 2.9 but I wanted to ask again here for
>>> broad
>>> feedback and thoughts by this chance. The back port patch is
>>> available for
>>> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising
>>> as we
>>> can see some shift from 2.7.x, hence it's worth more important
>>> features and
>>> efforts. How would you think? Thanks!
>>>
>>> https://issues.apache.org/jira/browse/HADOOP-14964
>>>
>>> Regards,
>>> Kai
>>>
>>> -----Original Message-----
>>> From: Junping Du [mailto:jdu@hortonworks.com]
>>> Sent: Tuesday, November 14, 2017 9:02 AM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Subject: Apache Hadoop 2.8.3 Release Plan
>>>
>>> Hi,
>>> We have several important fixes get landed on branch-2.8 and I
>>> would
>>> like to cut off branch-2.8.3 now to start 2.8.3 release work.
>>> So far, I don't see any pending blockers on 2.8.3, so my
>>> current plan
>>> is to cut off 1st RC of 2.8.3 in next several days:
>>> - For all coming commits to land on branch-2.8, please
>>> mark the
>>> fix version as 2.8.4.
>>> - If there is a really important fix for 2.8.3 and getting
>>> closed, please notify me ahead before landing it on branch-2.8.3.
>>> Please let me know if you have any thoughts or comments on the
>>> plan.
>>>
>>> Thanks,
>>>
>>> Junping
>>> ________________________________________
>>> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
>>> junping_du@apache.org>
>>> Sent: Friday, October 27, 2017 3:33 PM
>>> To: general@hadoop.apache.org
>>> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>>>
>>> Hi all,
>>>
>>> It gives me great pleasure to announce that the Apache Hadoop
>>> community has voted to release Apache Hadoop 2.8.2, which is now
>>> available
>>> for download from Apache mirrors[1]. For download instructions
>>> please refer
>>> to the Apache Hadoop Release page [2].
>>>
>>> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8
>>> line and
>>> our newest stable release for entire Apache Hadoop project. For major
>>> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
>>> page[3].
>>>
>>> This release has 315 resolved issues since previous 2.8.1 release with
>>> following
>>> breakdown:
>>> - 91 in Hadoop Common
>>> - 99 in HDFS
>>> - 105 in YARN
>>> - 20 in MapReduce
>>> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>>>
>>> The release news is posted on the Hadoop website too, you can go to the
>>> downloads section directly [6].
>>>
>>> Thank you all for contributing to the Apache Hadoop release!
>>>
>>>
>>> Cheers,
>>>
>>> Junping
>>>
>>>
>>> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>>>
>>> [2] http://hadoop.apache.org/releases.html
>>>
>>> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>>>
>>> [4]
>>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
>>> hadoop-common/release/2.8.2/CHANGES.2.8.2.html
>>>
>>> [5]
>>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
>>> hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>>>
>>> [6] http://hadoop.apache.org/releases.html#Download
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>>
>>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Daniel Templeton <da...@cloudera.com>.
Doh. Mailer dropped some of the lists. Replying again to avoid
fragmenting the discussion...
Still +1 to Andrew's comments.
Daniel
On 11/21/17 7:53 AM, Daniel Templeton wrote:
> +1
>
> Daniel
>
> On 11/20/17 10:22 PM, Andrew Wang wrote:
>> I'm against including new features in maintenance releases, since
>> they're
>> meant to be bug-fix only.
>>
>> If we're struggling with being able to deliver new features in a safe
>> and
>> timely fashion, let's try to address that, not overload the meaning of
>> "maintenance release".
>>
>> Best,
>> Andrew
>>
>> On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
>>
>>> Hi Junping,
>>>
>>> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>>>
>>> I have an ask, is it convenient to include the back port work for OSS
>>> connector module? We have some Hadoop users that wish to have it by
>>> default
>>> for convenience, though in the past they used it by back porting
>>> themselves. I have raised this and got thoughts from Chris and
>>> Steve. Looks
>>> like this is more wanted for 2.9 but I wanted to ask again here for
>>> broad
>>> feedback and thoughts by this chance. The back port patch is
>>> available for
>>> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising
>>> as we
>>> can see some shift from 2.7.x, hence it's worth more important
>>> features and
>>> efforts. How would you think? Thanks!
>>>
>>> https://issues.apache.org/jira/browse/HADOOP-14964
>>>
>>> Regards,
>>> Kai
>>>
>>> -----Original Message-----
>>> From: Junping Du [mailto:jdu@hortonworks.com]
>>> Sent: Tuesday, November 14, 2017 9:02 AM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Subject: Apache Hadoop 2.8.3 Release Plan
>>>
>>> Hi,
>>> We have several important fixes get landed on branch-2.8 and I
>>> would
>>> like to cut off branch-2.8.3 now to start 2.8.3 release work.
>>> So far, I don't see any pending blockers on 2.8.3, so my
>>> current plan
>>> is to cut off 1st RC of 2.8.3 in next several days:
>>> - For all coming commits to land on branch-2.8, please
>>> mark the
>>> fix version as 2.8.4.
>>> - If there is a really important fix for 2.8.3 and getting
>>> closed, please notify me ahead before landing it on branch-2.8.3.
>>> Please let me know if you have any thoughts or comments on the
>>> plan.
>>>
>>> Thanks,
>>>
>>> Junping
>>> ________________________________________
>>> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
>>> junping_du@apache.org>
>>> Sent: Friday, October 27, 2017 3:33 PM
>>> To: general@hadoop.apache.org
>>> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>>>
>>> Hi all,
>>>
>>> It gives me great pleasure to announce that the Apache Hadoop
>>> community has voted to release Apache Hadoop 2.8.2, which is now
>>> available
>>> for download from Apache mirrors[1]. For download instructions
>>> please refer
>>> to the Apache Hadoop Release page [2].
>>>
>>> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8
>>> line and
>>> our newest stable release for entire Apache Hadoop project. For major
>>> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main
>>> page[3].
>>>
>>> This release has 315 resolved issues since previous 2.8.1 release with
>>> following
>>> breakdown:
>>> - 91 in Hadoop Common
>>> - 99 in HDFS
>>> - 105 in YARN
>>> - 20 in MapReduce
>>> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>>>
>>> The release news is posted on the Hadoop website too, you can go to the
>>> downloads section directly [6].
>>>
>>> Thank you all for contributing to the Apache Hadoop release!
>>>
>>>
>>> Cheers,
>>>
>>> Junping
>>>
>>>
>>> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>>>
>>> [2] http://hadoop.apache.org/releases.html
>>>
>>> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>>>
>>> [4]
>>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
>>> hadoop-common/release/2.8.2/CHANGES.2.8.2.html
>>>
>>> [5]
>>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
>>> hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>>>
>>> [6] http://hadoop.apache.org/releases.html#Download
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>>
>>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Daniel Templeton <da...@cloudera.com>.
+1
Daniel
On 11/20/17 10:22 PM, Andrew Wang wrote:
> I'm against including new features in maintenance releases, since they're
> meant to be bug-fix only.
>
> If we're struggling with being able to deliver new features in a safe and
> timely fashion, let's try to address that, not overload the meaning of
> "maintenance release".
>
> Best,
> Andrew
>
> On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
>
>> Hi Junping,
>>
>> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>>
>> I have an ask, is it convenient to include the back port work for OSS
>> connector module? We have some Hadoop users that wish to have it by default
>> for convenience, though in the past they used it by back porting
>> themselves. I have raised this and got thoughts from Chris and Steve. Looks
>> like this is more wanted for 2.9 but I wanted to ask again here for broad
>> feedback and thoughts by this chance. The back port patch is available for
>> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we
>> can see some shift from 2.7.x, hence it's worth more important features and
>> efforts. How would you think? Thanks!
>>
>> https://issues.apache.org/jira/browse/HADOOP-14964
>>
>> Regards,
>> Kai
>>
>> -----Original Message-----
>> From: Junping Du [mailto:jdu@hortonworks.com]
>> Sent: Tuesday, November 14, 2017 9:02 AM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Subject: Apache Hadoop 2.8.3 Release Plan
>>
>> Hi,
>> We have several important fixes get landed on branch-2.8 and I would
>> like to cut off branch-2.8.3 now to start 2.8.3 release work.
>> So far, I don't see any pending blockers on 2.8.3, so my current plan
>> is to cut off 1st RC of 2.8.3 in next several days:
>> - For all coming commits to land on branch-2.8, please mark the
>> fix version as 2.8.4.
>> - If there is a really important fix for 2.8.3 and getting
>> closed, please notify me ahead before landing it on branch-2.8.3.
>> Please let me know if you have any thoughts or comments on the plan.
>>
>> Thanks,
>>
>> Junping
>> ________________________________________
>> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
>> junping_du@apache.org>
>> Sent: Friday, October 27, 2017 3:33 PM
>> To: general@hadoop.apache.org
>> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>>
>> Hi all,
>>
>> It gives me great pleasure to announce that the Apache Hadoop
>> community has voted to release Apache Hadoop 2.8.2, which is now available
>> for download from Apache mirrors[1]. For download instructions please refer
>> to the Apache Hadoop Release page [2].
>>
>> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
>> our newest stable release for entire Apache Hadoop project. For major
>> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>>
>> This release has 315 resolved issues since previous 2.8.1 release with
>> following
>> breakdown:
>> - 91 in Hadoop Common
>> - 99 in HDFS
>> - 105 in YARN
>> - 20 in MapReduce
>> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>>
>> The release news is posted on the Hadoop website too, you can go to the
>> downloads section directly [6].
>>
>> Thank you all for contributing to the Apache Hadoop release!
>>
>>
>> Cheers,
>>
>> Junping
>>
>>
>> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>>
>> [2] http://hadoop.apache.org/releases.html
>>
>> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>>
>> [4]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
>> hadoop-common/release/2.8.2/CHANGES.2.8.2.html
>>
>> [5]
>> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
>> hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>>
>> [6] http://hadoop.apache.org/releases.html#Download
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Kai for calling out this feature/improvement for attention and Andrew for comments.
While I agree that maintenance release should focus on important bug fix only, I doubt we have strict rules to disallow any features/improvements to land on maint release especially when those are small footprint or low impact on existing code/features. In practice, we indeed has 77 new features/improvements in latest 2.7.3 and 2.7.4 release.
Back to HADOOP-14964, I did a quick check and it looks like case here belongs to self-contained improvement that has very low impact on existing code base, so I am OK with the improvement get landed on branch-2.8 in case it is well reviewed and tested.
However, as RM of branch-2.8, I have two concerns to accept it in our 2.8.3 release:
1. Timing - as I mentioned in beginning, the main purpose of 2.8.3 are for several critical bug fixes and we should target to release it very soon - my current plan is to cut RC out within this week inline with waiting for 3.0.0 vote closing. Can this improvement be well tested against branch-2.8.3 within this strictly timeline? It seems a bit rush unless we have strong commitment on test plan and activities in such a tight time.
2. Upgrading - I haven't heard we settle down the plan of releasing this feature in 2.9.1 release - though I saw some discussions are going on at HADOOP-14964. Assume 2.8.3 is released ahead of 2.9.1 and it includes this improvement, then users consuming this feature/improvement have no 2.9 release to upgrade or forcefully upgrade with regression. We may need a better upgrade story here.
Pls let me know what you think. Thanks!
Thanks,
Junping
________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Monday, November 20, 2017 10:22 PM
To: Zheng, Kai
Cc: Junping Du; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: Apache Hadoop 2.8.3 Release Plan
I'm against including new features in maintenance releases, since they're meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and timely fashion, let's try to address that, not overload the meaning of "maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com>> wrote:
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS connector module? We have some Hadoop users that wish to have it by default for convenience, though in the past they used it by back porting themselves. I have raised this and got thoughts from Chris and Steve. Looks like this is more wanted for 2.9 but I wanted to ask again here for broad feedback and thoughts by this chance. The back port patch is available for 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we can see some shift from 2.7.x, hence it's worth more important features and efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com<ma...@hortonworks.com>]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current plan is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark the fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com<ma...@gmail.com> <du...@gmail.com>> on behalf of ??? <ju...@apache.org>>
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 2.8.2, which is now available for download from Apache mirrors[1]. For download instructions please refer to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and our newest stable release for entire Apache Hadoop project. For major changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
This release has 315 resolved issues since previous 2.8.1 release with following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: common-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Andrew Wang <an...@cloudera.com>.
I'm against including new features in maintenance releases, since they're
meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and
timely fashion, let's try to address that, not overload the meaning of
"maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
> Hi Junping,
>
> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>
> I have an ask, is it convenient to include the back port work for OSS
> connector module? We have some Hadoop users that wish to have it by default
> for convenience, though in the past they used it by back porting
> themselves. I have raised this and got thoughts from Chris and Steve. Looks
> like this is more wanted for 2.9 but I wanted to ask again here for broad
> feedback and thoughts by this chance. The back port patch is available for
> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we
> can see some shift from 2.7.x, hence it's worth more important features and
> efforts. How would you think? Thanks!
>
> https://issues.apache.org/jira/browse/HADOOP-14964
>
> Regards,
> Kai
>
> -----Original Message-----
> From: Junping Du [mailto:jdu@hortonworks.com]
> Sent: Tuesday, November 14, 2017 9:02 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Apache Hadoop 2.8.3 Release Plan
>
> Hi,
> We have several important fixes get landed on branch-2.8 and I would
> like to cut off branch-2.8.3 now to start 2.8.3 release work.
> So far, I don't see any pending blockers on 2.8.3, so my current plan
> is to cut off 1st RC of 2.8.3 in next several days:
> - For all coming commits to land on branch-2.8, please mark the
> fix version as 2.8.4.
> - If there is a really important fix for 2.8.3 and getting
> closed, please notify me ahead before landing it on branch-2.8.3.
> Please let me know if you have any thoughts or comments on the plan.
>
> Thanks,
>
> Junping
> ________________________________________
> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
> junping_du@apache.org>
> Sent: Friday, October 27, 2017 3:33 PM
> To: general@hadoop.apache.org
> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>
> Hi all,
>
> It gives me great pleasure to announce that the Apache Hadoop
> community has voted to release Apache Hadoop 2.8.2, which is now available
> for download from Apache mirrors[1]. For download instructions please refer
> to the Apache Hadoop Release page [2].
>
> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
> our newest stable release for entire Apache Hadoop project. For major
> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>
> This release has 315 resolved issues since previous 2.8.1 release with
> following
> breakdown:
> - 91 in Hadoop Common
> - 99 in HDFS
> - 105 in YARN
> - 20 in MapReduce
> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>
> The release news is posted on the Hadoop website too, you can go to the
> downloads section directly [6].
>
> Thank you all for contributing to the Apache Hadoop release!
>
>
> Cheers,
>
> Junping
>
>
> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>
> [2] http://hadoop.apache.org/releases.html
>
> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>
> [4]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/CHANGES.2.8.2.html
>
> [5]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>
> [6] http://hadoop.apache.org/releases.html#Download
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Andrew Wang <an...@cloudera.com>.
I'm against including new features in maintenance releases, since they're
meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and
timely fashion, let's try to address that, not overload the meaning of
"maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
> Hi Junping,
>
> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>
> I have an ask, is it convenient to include the back port work for OSS
> connector module? We have some Hadoop users that wish to have it by default
> for convenience, though in the past they used it by back porting
> themselves. I have raised this and got thoughts from Chris and Steve. Looks
> like this is more wanted for 2.9 but I wanted to ask again here for broad
> feedback and thoughts by this chance. The back port patch is available for
> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we
> can see some shift from 2.7.x, hence it's worth more important features and
> efforts. How would you think? Thanks!
>
> https://issues.apache.org/jira/browse/HADOOP-14964
>
> Regards,
> Kai
>
> -----Original Message-----
> From: Junping Du [mailto:jdu@hortonworks.com]
> Sent: Tuesday, November 14, 2017 9:02 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Apache Hadoop 2.8.3 Release Plan
>
> Hi,
> We have several important fixes get landed on branch-2.8 and I would
> like to cut off branch-2.8.3 now to start 2.8.3 release work.
> So far, I don't see any pending blockers on 2.8.3, so my current plan
> is to cut off 1st RC of 2.8.3 in next several days:
> - For all coming commits to land on branch-2.8, please mark the
> fix version as 2.8.4.
> - If there is a really important fix for 2.8.3 and getting
> closed, please notify me ahead before landing it on branch-2.8.3.
> Please let me know if you have any thoughts or comments on the plan.
>
> Thanks,
>
> Junping
> ________________________________________
> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
> junping_du@apache.org>
> Sent: Friday, October 27, 2017 3:33 PM
> To: general@hadoop.apache.org
> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>
> Hi all,
>
> It gives me great pleasure to announce that the Apache Hadoop
> community has voted to release Apache Hadoop 2.8.2, which is now available
> for download from Apache mirrors[1]. For download instructions please refer
> to the Apache Hadoop Release page [2].
>
> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
> our newest stable release for entire Apache Hadoop project. For major
> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>
> This release has 315 resolved issues since previous 2.8.1 release with
> following
> breakdown:
> - 91 in Hadoop Common
> - 99 in HDFS
> - 105 in YARN
> - 20 in MapReduce
> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>
> The release news is posted on the Hadoop website too, you can go to the
> downloads section directly [6].
>
> Thank you all for contributing to the Apache Hadoop release!
>
>
> Cheers,
>
> Junping
>
>
> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>
> [2] http://hadoop.apache.org/releases.html
>
> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>
> [4]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/CHANGES.2.8.2.html
>
> [5]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>
> [6] http://hadoop.apache.org/releases.html#Download
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Andrew Wang <an...@cloudera.com>.
I'm against including new features in maintenance releases, since they're
meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and
timely fashion, let's try to address that, not overload the meaning of
"maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
> Hi Junping,
>
> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>
> I have an ask, is it convenient to include the back port work for OSS
> connector module? We have some Hadoop users that wish to have it by default
> for convenience, though in the past they used it by back porting
> themselves. I have raised this and got thoughts from Chris and Steve. Looks
> like this is more wanted for 2.9 but I wanted to ask again here for broad
> feedback and thoughts by this chance. The back port patch is available for
> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we
> can see some shift from 2.7.x, hence it's worth more important features and
> efforts. How would you think? Thanks!
>
> https://issues.apache.org/jira/browse/HADOOP-14964
>
> Regards,
> Kai
>
> -----Original Message-----
> From: Junping Du [mailto:jdu@hortonworks.com]
> Sent: Tuesday, November 14, 2017 9:02 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Apache Hadoop 2.8.3 Release Plan
>
> Hi,
> We have several important fixes get landed on branch-2.8 and I would
> like to cut off branch-2.8.3 now to start 2.8.3 release work.
> So far, I don't see any pending blockers on 2.8.3, so my current plan
> is to cut off 1st RC of 2.8.3 in next several days:
> - For all coming commits to land on branch-2.8, please mark the
> fix version as 2.8.4.
> - If there is a really important fix for 2.8.3 and getting
> closed, please notify me ahead before landing it on branch-2.8.3.
> Please let me know if you have any thoughts or comments on the plan.
>
> Thanks,
>
> Junping
> ________________________________________
> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
> junping_du@apache.org>
> Sent: Friday, October 27, 2017 3:33 PM
> To: general@hadoop.apache.org
> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>
> Hi all,
>
> It gives me great pleasure to announce that the Apache Hadoop
> community has voted to release Apache Hadoop 2.8.2, which is now available
> for download from Apache mirrors[1]. For download instructions please refer
> to the Apache Hadoop Release page [2].
>
> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
> our newest stable release for entire Apache Hadoop project. For major
> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>
> This release has 315 resolved issues since previous 2.8.1 release with
> following
> breakdown:
> - 91 in Hadoop Common
> - 99 in HDFS
> - 105 in YARN
> - 20 in MapReduce
> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>
> The release news is posted on the Hadoop website too, you can go to the
> downloads section directly [6].
>
> Thank you all for contributing to the Apache Hadoop release!
>
>
> Cheers,
>
> Junping
>
>
> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>
> [2] http://hadoop.apache.org/releases.html
>
> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>
> [4]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/CHANGES.2.8.2.html
>
> [5]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>
> [6] http://hadoop.apache.org/releases.html#Download
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: Apache Hadoop 2.8.3 Release Plan
Posted by Andrew Wang <an...@cloudera.com>.
I'm against including new features in maintenance releases, since they're
meant to be bug-fix only.
If we're struggling with being able to deliver new features in a safe and
timely fashion, let's try to address that, not overload the meaning of
"maintenance release".
Best,
Andrew
On Mon, Nov 20, 2017 at 5:20 PM, Zheng, Kai <ka...@intel.com> wrote:
> Hi Junping,
>
> Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
>
> I have an ask, is it convenient to include the back port work for OSS
> connector module? We have some Hadoop users that wish to have it by default
> for convenience, though in the past they used it by back porting
> themselves. I have raised this and got thoughts from Chris and Steve. Looks
> like this is more wanted for 2.9 but I wanted to ask again here for broad
> feedback and thoughts by this chance. The back port patch is available for
> 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we
> can see some shift from 2.7.x, hence it's worth more important features and
> efforts. How would you think? Thanks!
>
> https://issues.apache.org/jira/browse/HADOOP-14964
>
> Regards,
> Kai
>
> -----Original Message-----
> From: Junping Du [mailto:jdu@hortonworks.com]
> Sent: Tuesday, November 14, 2017 9:02 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Apache Hadoop 2.8.3 Release Plan
>
> Hi,
> We have several important fixes get landed on branch-2.8 and I would
> like to cut off branch-2.8.3 now to start 2.8.3 release work.
> So far, I don't see any pending blockers on 2.8.3, so my current plan
> is to cut off 1st RC of 2.8.3 in next several days:
> - For all coming commits to land on branch-2.8, please mark the
> fix version as 2.8.4.
> - If there is a really important fix for 2.8.3 and getting
> closed, please notify me ahead before landing it on branch-2.8.3.
> Please let me know if you have any thoughts or comments on the plan.
>
> Thanks,
>
> Junping
> ________________________________________
> From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <
> junping_du@apache.org>
> Sent: Friday, October 27, 2017 3:33 PM
> To: general@hadoop.apache.org
> Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
>
> Hi all,
>
> It gives me great pleasure to announce that the Apache Hadoop
> community has voted to release Apache Hadoop 2.8.2, which is now available
> for download from Apache mirrors[1]. For download instructions please refer
> to the Apache Hadoop Release page [2].
>
> Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and
> our newest stable release for entire Apache Hadoop project. For major
> changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
>
> This release has 315 resolved issues since previous 2.8.1 release with
> following
> breakdown:
> - 91 in Hadoop Common
> - 99 in HDFS
> - 105 in YARN
> - 20 in MapReduce
> Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
>
> The release news is posted on the Hadoop website too, you can go to the
> downloads section directly [6].
>
> Thank you all for contributing to the Apache Hadoop release!
>
>
> Cheers,
>
> Junping
>
>
> [1] http://www.apache.org/dyn/closer.cgi/hadoop/common
>
> [2] http://hadoop.apache.org/releases.html
>
> [3] http://hadoop.apache.org/docs/r2.8.2/index.html
>
> [4]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/CHANGES.2.8.2.html
>
> [5]
> http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/
> hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
>
> [6] http://hadoop.apache.org/releases.html#Download
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
RE: Apache Hadoop 2.8.3 Release Plan
Posted by "Zheng, Kai" <ka...@intel.com>.
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS connector module? We have some Hadoop users that wish to have it by default for convenience, though in the past they used it by back porting themselves. I have raised this and got thoughts from Chris and Steve. Looks like this is more wanted for 2.9 but I wanted to ask again here for broad feedback and thoughts by this chance. The back port patch is available for 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we can see some shift from 2.7.x, hence it's worth more important features and efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current plan is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark the fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <ju...@apache.org>
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 2.8.2, which is now available for download from Apache mirrors[1]. For download instructions please refer to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and our newest stable release for entire Apache Hadoop project. For major changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
This release has 315 resolved issues since previous 2.8.1 release with following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
RE: Apache Hadoop 2.8.3 Release Plan
Posted by "Zheng, Kai" <ka...@intel.com>.
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS connector module? We have some Hadoop users that wish to have it by default for convenience, though in the past they used it by back porting themselves. I have raised this and got thoughts from Chris and Steve. Looks like this is more wanted for 2.9 but I wanted to ask again here for broad feedback and thoughts by this chance. The back port patch is available for 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we can see some shift from 2.7.x, hence it's worth more important features and efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current plan is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark the fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <ju...@apache.org>
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 2.8.2, which is now available for download from Apache mirrors[1]. For download instructions please refer to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and our newest stable release for entire Apache Hadoop project. For major changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
This release has 315 resolved issues since previous 2.8.1 release with following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
RE: Apache Hadoop 2.8.3 Release Plan
Posted by "Zheng, Kai" <ka...@intel.com>.
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS connector module? We have some Hadoop users that wish to have it by default for convenience, though in the past they used it by back porting themselves. I have raised this and got thoughts from Chris and Steve. Looks like this is more wanted for 2.9 but I wanted to ask again here for broad feedback and thoughts by this chance. The back port patch is available for 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we can see some shift from 2.7.x, hence it's worth more important features and efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current plan is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark the fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <ju...@apache.org>
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 2.8.2, which is now available for download from Apache mirrors[1]. For download instructions please refer to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and our newest stable release for entire Apache Hadoop project. For major changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
This release has 315 resolved issues since previous 2.8.1 release with following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
RE: Apache Hadoop 2.8.3 Release Plan
Posted by "Zheng, Kai" <ka...@intel.com>.
Hi Junping,
Thank you for making 2.8.2 happen and now planning the 2.8.3 release.
I have an ask, is it convenient to include the back port work for OSS connector module? We have some Hadoop users that wish to have it by default for convenience, though in the past they used it by back porting themselves. I have raised this and got thoughts from Chris and Steve. Looks like this is more wanted for 2.9 but I wanted to ask again here for broad feedback and thoughts by this chance. The back port patch is available for 2.8 and the one for branch-2 was already in. IMO, 2.8.x is promising as we can see some shift from 2.7.x, hence it's worth more important features and efforts. How would you think? Thanks!
https://issues.apache.org/jira/browse/HADOOP-14964
Regards,
Kai
-----Original Message-----
From: Junping Du [mailto:jdu@hortonworks.com]
Sent: Tuesday, November 14, 2017 9:02 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Apache Hadoop 2.8.3 Release Plan
Hi,
We have several important fixes get landed on branch-2.8 and I would like to cut off branch-2.8.3 now to start 2.8.3 release work.
So far, I don't see any pending blockers on 2.8.3, so my current plan is to cut off 1st RC of 2.8.3 in next several days:
- For all coming commits to land on branch-2.8, please mark the fix version as 2.8.4.
- If there is a really important fix for 2.8.3 and getting closed, please notify me ahead before landing it on branch-2.8.3.
Please let me know if you have any thoughts or comments on the plan.
Thanks,
Junping
________________________________________
From: dujunping@gmail.com <du...@gmail.com> on behalf of 俊平堵 <ju...@apache.org>
Sent: Friday, October 27, 2017 3:33 PM
To: general@hadoop.apache.org
Subject: [ANNOUNCE] Apache Hadoop 2.8.2 Release.
Hi all,
It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 2.8.2, which is now available for download from Apache mirrors[1]. For download instructions please refer to the Apache Hadoop Release page [2].
Apache Hadoop 2.8.2 is the first GA release of Apache Hadoop 2.8 line and our newest stable release for entire Apache Hadoop project. For major changes incuded in Hadoop 2.8 line, please refer Hadoop 2.8.2 main page[3].
This release has 315 resolved issues since previous 2.8.1 release with following
breakdown:
- 91 in Hadoop Common
- 99 in HDFS
- 105 in YARN
- 20 in MapReduce
Please read the log of CHANGES[4] and RELEASENOTES[5] for more details.
The release news is posted on the Hadoop website too, you can go to the downloads section directly [6].
Thank you all for contributing to the Apache Hadoop release!
Cheers,
Junping
[1] http://www.apache.org/dyn/closer.cgi/hadoop/common
[2] http://hadoop.apache.org/releases.html
[3] http://hadoop.apache.org/docs/r2.8.2/index.html
[4]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/CHANGES.2.8.2.html
[5]
http://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/release/2.8.2/RELEASENOTES.2.8.2.html
[6] http://hadoop.apache.org/releases.html#Download
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org