You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Vinod Vavilapalli <vi...@hortonworks.com> on 2015/11/11 21:13:55 UTC
Re: [DISCUSS] Looking to a 2.8.0 release
Agreed on not mixing this with major release discussions.
Okay, I just finished my review of 2.8 content.
A quick summary follows.
Current state of originally planned items
- Nearly Done / Done and so need to close down quickly
— Support *both* JDK7 and JDK8 runtimes HADOOP-11090
— Supporting non-exclusive node-labels: YARN-3214: Done, can push as an alpha / beta feature
— Support priorities across applications within the same queue YARN-1963: Can push as an alpha / beta feature
- Definitely have to move out into 2.9 and beyond
— Early work for disk and network isolation in YARN: YARN-2139, YARN-2140: Early noise, some critical pieces designed, done but not a lot of movement of late
— Classpath isolation for downstream clients HADOOP-11656: Lots of chatter a while ago, not much movement of late
— Support for Erasure Codes in HDFS HDFS-7285<https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the interest of stability / bake-in
Non-planned features that went into 2.8.0
— The overall list of new features: https://issues.apache.org/jira/issues/?filter=12333994
— HDFS-6200 Create a separate jar for hdfs-client: Compatible improvement - no dimension of alpha/betaness here.
— HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
— Stability improvements ready to use:
— HDFS-8008 Support client-side back off when the datanodes are congested
— HDFS-8009 Signal congestion on the DataNode
— YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well most of it anyways. Can push as an alpha feature.
— YARN-1197 Support changing resources of an allocated container: Can push as an alpha/beta feature
Items in progress to think about in 2 weeks
— YARN Timeline Service v1.5 - YARN-4233: A short term bridge before YARN-2928 comes around. I think this should go in given the tremendous activity recently.
— YARN Timeline Service Next generation: YARN-2928: Lots of momentum, but clearly a work in progress. Two options here
— If it is safe to ship it into 2.8 in a disable manner, we can get the early code into trunk and all the way int o2.8.
— If it is not safe, it organically rolls over into 2.9
— Compatibility tools to catch backwards, forwards compatibility issues at patch submission, release times. Some of it is captured at YARN-3292. This also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
This is my plan of action for now in terms of the release itself
- Cut a branch about two weeks from now
- Do an RC mid next month (leaving ~4weeks since branch-cut)
- As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
— gaining downstream adoption
— wider testing,
— yet reserving our right to fix any inadvertent incompatibilities introduced.
If we can get answers on “Items to think about now” during this and next week, we will overall be in good shape.
Thoughts?
Thanks
+Vinod
PS:As you may have noted above, this time around, I want to do something that we’ve always wanted to do, but never explicitly did. I’m calling out readiness of each feature as they stand today so we can inform our users better of what they can start relying on in production clusters.
On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cm...@apache.org>> wrote:
I think it makes sense to have a 2.8 release since there are a
tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
release seems like something we should consider separately since it
would not have the same compatibility guarantees as a 2.8 release.
There's a pretty big delta between trunk and 2.8 as well.
cheers,
Colin
On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cd...@apache.org>> wrote:
With two active sustaining branches (2.6, 2.7), what would you think
of releasing trunk as 3.x instead of pushing 2.8? There are many new
features (EC, Y1197, etc.), and trunk could be the source of several
alpha/beta releases before we fork the 3.x line. -C
On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
<vi...@hortonworks.com>> wrote:
As you may have noted, 2.8.0 got completely derailed what with 2.7.x and the unusually long 2.6.1 release.
With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2 and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.
I’ll do a quick survey of where the individual features are and the amount of content already present in 2.8 and kick-start 2.8.0 process again.
+Vinod
On Apr 21, 2015, at 2:39 PM, vinodkv@apache.org<ma...@apache.org> wrote:
With 2.7.0 out of the way, and with more maintenance releases to stabilize it, I propose we start thinking about 2.8.0.
Here's my first cut of the proposal, will update the Roadmap wiki.
- Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
- Compatibility tools to catch backwards, forwards compatibility issues at patch submission, release times. Some of it is captured at YARN-3292. This also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
- HADOOP-11656 Classpath isolation for downstream clients
- Support for Erasure Codes in HDFS HDFS-7285
- Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
- YARN Timeline Service Next generation: YARN-2928. At least branch-merge + early peek.
- Supporting non-exclusive node-labels: YARN-3214
I'm experimenting with more agile 2.7.x releases and would like to continue the same by volunteering as the RM for 2.8.x too.
Given the long time we took with 2.7.0, the timeline I am looking at is 8-12 weeks. We can pick as many features as they finish along and make a more predictable releases instead of holding up releases for ever.
Thoughts?
Thanks
+Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Vavilapalli <vi...@hortonworks.com>.
Please ignore this reply, was supposed to be a nudge on internal lists.
+Vinod
On Nov 11, 2015, at 1:15 PM, Vinod Vavilapalli <vi...@hortonworks.com>> wrote:
Started a thread on 2.8.0. Please give feedback etc. I’ll need help from both the teams to get this done right.
+Vinod
Begin forwarded message:
From: Vinod Kumar Vavilapalli <vi...@hortonworks.com>>
Subject: Re: [DISCUSS] Looking to a 2.8.0 release
Date: November 11, 2015 at 12:13:54 PM PST
To: Hadoop Common <co...@hadoop.apache.org>>, "yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>" <ya...@hadoop.apache.org>>, "hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>" <hd...@hadoop.apache.org>>, mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Cc: "vinodkv@apache.org<ma...@apache.org>" <vi...@apache.org>>
Agreed on not mixing this with major release discussions.
Okay, I just finished my review of 2.8 content.
A quick summary follows.
Current state of originally planned items
- Nearly Done / Done and so need to close down quickly
— Support *both* JDK7 and JDK8 runtimes HADOOP-11090
— Supporting non-exclusive node-labels: YARN-3214: Done, can push as an alpha / beta feature
— Support priorities across applications within the same queue YARN-1963: Can push as an alpha / beta feature
- Definitely have to move out into 2.9 and beyond
— Early work for disk and network isolation in YARN: YARN-2139, YARN-2140: Early noise, some critical pieces designed, done but not a lot of movement of late
— Classpath isolation for downstream clients HADOOP-11656: Lots of chatter a while ago, not much movement of late
— Support for Erasure Codes in HDFS HDFS-7285<https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the interest of stability / bake-in
Non-planned features that went into 2.8.0
— The overall list of new features: https://issues.apache.org/jira/issues/?filter=12333994
— HDFS-6200 Create a separate jar for hdfs-client: Compatible improvement - no dimension of alpha/betaness here.
— HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
— Stability improvements ready to use:
— HDFS-8008 Support client-side back off when the datanodes are congested
— HDFS-8009 Signal congestion on the DataNode
— YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well most of it anyways. Can push as an alpha feature.
— YARN-1197 Support changing resources of an allocated container: Can push as an alpha/beta feature
Items in progress to think about in 2 weeks
— YARN Timeline Service v1.5 - YARN-4233: A short term bridge before YARN-2928 comes around. I think this should go in given the tremendous activity recently.
— YARN Timeline Service Next generation: YARN-2928: Lots of momentum, but clearly a work in progress. Two options here
— If it is safe to ship it into 2.8 in a disable manner, we can get the early code into trunk and all the way int o2.8.
— If it is not safe, it organically rolls over into 2.9
— Compatibility tools to catch backwards, forwards compatibility issues at patch submission, release times. Some of it is captured at YARN-3292. This also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
This is my plan of action for now in terms of the release itself
- Cut a branch about two weeks from now
- Do an RC mid next month (leaving ~4weeks since branch-cut)
- As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
— gaining downstream adoption
— wider testing,
— yet reserving our right to fix any inadvertent incompatibilities introduced.
If we can get answers on “Items to think about now” during this and next week, we will overall be in good shape.
Thoughts?
Thanks
+Vinod
PS:As you may have noted above, this time around, I want to do something that we’ve always wanted to do, but never explicitly did. I’m calling out readiness of each feature as they stand today so we can inform our users better of what they can start relying on in production clusters.
On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cm...@apache.org>> wrote:
I think it makes sense to have a 2.8 release since there are a
tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
release seems like something we should consider separately since it
would not have the same compatibility guarantees as a 2.8 release.
There's a pretty big delta between trunk and 2.8 as well.
cheers,
Colin
On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cd...@apache.org>> wrote:
With two active sustaining branches (2.6, 2.7), what would you think
of releasing trunk as 3.x instead of pushing 2.8? There are many new
features (EC, Y1197, etc.), and trunk could be the source of several
alpha/beta releases before we fork the 3.x line. -C
On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
<vi...@hortonworks.com>> wrote:
As you may have noted, 2.8.0 got completely derailed what with 2.7.x and the unusually long 2.6.1 release.
With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2 and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.
I’ll do a quick survey of where the individual features are and the amount of content already present in 2.8 and kick-start 2.8.0 process again.
+Vinod
On Apr 21, 2015, at 2:39 PM, vinodkv@apache.org<ma...@apache.org> wrote:
With 2.7.0 out of the way, and with more maintenance releases to stabilize it, I propose we start thinking about 2.8.0.
Here's my first cut of the proposal, will update the Roadmap wiki.
- Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
- Compatibility tools to catch backwards, forwards compatibility issues at patch submission, release times. Some of it is captured at YARN-3292. This also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
- HADOOP-11656 Classpath isolation for downstream clients
- Support for Erasure Codes in HDFS HDFS-7285
- Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
- YARN Timeline Service Next generation: YARN-2928. At least branch-merge + early peek.
- Supporting non-exclusive node-labels: YARN-3214
I'm experimenting with more agile 2.7.x releases and would like to continue the same by volunteering as the RM for 2.8.x too.
Given the long time we took with 2.7.0, the timeline I am looking at is 8-12 weeks. We can pick as many features as they finish along and make a more predictable releases instead of holding up releases for ever.
Thoughts?
Thanks
+Vinod
Fwd: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Vavilapalli <vi...@hortonworks.com>.
Started a thread on 2.8.0. Please give feedback etc. I’ll need help from both the teams to get this done right.
+Vinod
Begin forwarded message:
From: Vinod Kumar Vavilapalli <vi...@hortonworks.com>>
Subject: Re: [DISCUSS] Looking to a 2.8.0 release
Date: November 11, 2015 at 12:13:54 PM PST
To: Hadoop Common <co...@hadoop.apache.org>>, "yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>" <ya...@hadoop.apache.org>>, "hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>" <hd...@hadoop.apache.org>>, mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Cc: "vinodkv@apache.org<ma...@apache.org>" <vi...@apache.org>>
Agreed on not mixing this with major release discussions.
Okay, I just finished my review of 2.8 content.
A quick summary follows.
Current state of originally planned items
- Nearly Done / Done and so need to close down quickly
— Support *both* JDK7 and JDK8 runtimes HADOOP-11090
— Supporting non-exclusive node-labels: YARN-3214: Done, can push as an alpha / beta feature
— Support priorities across applications within the same queue YARN-1963: Can push as an alpha / beta feature
- Definitely have to move out into 2.9 and beyond
— Early work for disk and network isolation in YARN: YARN-2139, YARN-2140: Early noise, some critical pieces designed, done but not a lot of movement of late
— Classpath isolation for downstream clients HADOOP-11656: Lots of chatter a while ago, not much movement of late
— Support for Erasure Codes in HDFS HDFS-7285<https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the interest of stability / bake-in
Non-planned features that went into 2.8.0
— The overall list of new features: https://issues.apache.org/jira/issues/?filter=12333994
— HDFS-6200 Create a separate jar for hdfs-client: Compatible improvement - no dimension of alpha/betaness here.
— HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
— Stability improvements ready to use:
— HDFS-8008 Support client-side back off when the datanodes are congested
— HDFS-8009 Signal congestion on the DataNode
— YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well most of it anyways. Can push as an alpha feature.
— YARN-1197 Support changing resources of an allocated container: Can push as an alpha/beta feature
Items in progress to think about in 2 weeks
— YARN Timeline Service v1.5 - YARN-4233: A short term bridge before YARN-2928 comes around. I think this should go in given the tremendous activity recently.
— YARN Timeline Service Next generation: YARN-2928: Lots of momentum, but clearly a work in progress. Two options here
— If it is safe to ship it into 2.8 in a disable manner, we can get the early code into trunk and all the way int o2.8.
— If it is not safe, it organically rolls over into 2.9
— Compatibility tools to catch backwards, forwards compatibility issues at patch submission, release times. Some of it is captured at YARN-3292. This also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
This is my plan of action for now in terms of the release itself
- Cut a branch about two weeks from now
- Do an RC mid next month (leaving ~4weeks since branch-cut)
- As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
— gaining downstream adoption
— wider testing,
— yet reserving our right to fix any inadvertent incompatibilities introduced.
If we can get answers on “Items to think about now” during this and next week, we will overall be in good shape.
Thoughts?
Thanks
+Vinod
PS:As you may have noted above, this time around, I want to do something that we’ve always wanted to do, but never explicitly did. I’m calling out readiness of each feature as they stand today so we can inform our users better of what they can start relying on in production clusters.
On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cm...@apache.org>> wrote:
I think it makes sense to have a 2.8 release since there are a
tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
release seems like something we should consider separately since it
would not have the same compatibility guarantees as a 2.8 release.
There's a pretty big delta between trunk and 2.8 as well.
cheers,
Colin
On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cd...@apache.org>> wrote:
With two active sustaining branches (2.6, 2.7), what would you think
of releasing trunk as 3.x instead of pushing 2.8? There are many new
features (EC, Y1197, etc.), and trunk could be the source of several
alpha/beta releases before we fork the 3.x line. -C
On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
<vi...@hortonworks.com>> wrote:
As you may have noted, 2.8.0 got completely derailed what with 2.7.x and the unusually long 2.6.1 release.
With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2 and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.
I’ll do a quick survey of where the individual features are and the amount of content already present in 2.8 and kick-start 2.8.0 process again.
+Vinod
On Apr 21, 2015, at 2:39 PM, vinodkv@apache.org<ma...@apache.org> wrote:
With 2.7.0 out of the way, and with more maintenance releases to stabilize it, I propose we start thinking about 2.8.0.
Here's my first cut of the proposal, will update the Roadmap wiki.
- Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
- Compatibility tools to catch backwards, forwards compatibility issues at patch submission, release times. Some of it is captured at YARN-3292. This also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
- HADOOP-11656 Classpath isolation for downstream clients
- Support for Erasure Codes in HDFS HDFS-7285
- Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
- YARN Timeline Service Next generation: YARN-2928. At least branch-merge + early peek.
- Supporting non-exclusive node-labels: YARN-3214
I'm experimenting with more agile 2.7.x releases and would like to continue the same by volunteering as the RM for 2.8.x too.
Given the long time we took with 2.7.0, the timeline I am looking at is 8-12 weeks. We can pick as many features as they finish along and make a more predictable releases instead of holding up releases for ever.
Thoughts?
Thanks
+Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Karthik Kambatla <ka...@cloudera.com>.
I am really against the notion of calling x.y.0 releases alpha/beta; it is
very confusing. If we think a release is alpha/beta quality, why not
release it as x.y.0-alpha or x.y.0-beta, and follow it up eventually with
x.y.0 GA.
I am in favor of labeling any of the newer features to be of alpha/beta
quality.
SharedCache is another close to done feature.
On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vinodkv@hortonworks.com
> wrote:
> Agreed on not mixing this with major release discussions.
>
> Okay, I just finished my review of 2.8 content.
>
> A quick summary follows.
>
> Current state of originally planned items
>
> - Nearly Done / Done and so need to close down quickly
> — Support *both* JDK7 and JDK8 runtimes HADOOP-11090
> — Supporting non-exclusive node-labels: YARN-3214: Done, can push as
> an alpha / beta feature
> — Support priorities across applications within the same queue
> YARN-1963: Can push as an alpha / beta feature
>
> - Definitely have to move out into 2.9 and beyond
> — Early work for disk and network isolation in YARN: YARN-2139,
> YARN-2140: Early noise, some critical pieces designed, done but not a lot
> of movement of late
> — Classpath isolation for downstream clients HADOOP-11656: Lots of
> chatter a while ago, not much movement of late
> — Support for Erasure Codes in HDFS HDFS-7285<
> https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the
> interest of stability / bake-in
>
> Non-planned features that went into 2.8.0
>
> — The overall list of new features:
> https://issues.apache.org/jira/issues/?filter=12333994
> — HDFS-6200 Create a separate jar for hdfs-client: Compatible
> improvement - no dimension of alpha/betaness here.
> — HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
> — Stability improvements ready to use:
> — HDFS-8008 Support client-side back off when the datanodes are
> congested
> — HDFS-8009 Signal congestion on the DataNode
> — YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> most of it anyways. Can push as an alpha feature.
> — YARN-1197 Support changing resources of an allocated container: Can
> push as an alpha/beta feature
>
> Items in progress to think about in 2 weeks
>
> — YARN Timeline Service v1.5 - YARN-4233: A short term bridge before
> YARN-2928 comes around. I think this should go in given the tremendous
> activity recently.
> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
> but clearly a work in progress. Two options here
> — If it is safe to ship it into 2.8 in a disable manner, we can
> get the early code into trunk and all the way int o2.8.
> — If it is not safe, it organically rolls over into 2.9
> — Compatibility tools to catch backwards, forwards compatibility
> issues at patch submission, release times. Some of it is captured at
> YARN-3292. This also involves resurrecting jdiff
> (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
>
> This is my plan of action for now in terms of the release itself
>
> - Cut a branch about two weeks from now
> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> - As with 2.7.x series, the first release will still be called as early /
> alpha release in the interest of
> — gaining downstream adoption
> — wider testing,
> — yet reserving our right to fix any inadvertent incompatibilities
> introduced.
>
> If we can get answers on “Items to think about now” during this and next
> week, we will overall be in good shape.
>
> Thoughts?
>
> Thanks
> +Vinod
> PS:As you may have noted above, this time around, I want to do something
> that we’ve always wanted to do, but never explicitly did. I’m calling out
> readiness of each feature as they stand today so we can inform our users
> better of what they can start relying on in production clusters.
>
>
> On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cmccabe@apache.org<mailto:
> cmccabe@apache.org>> wrote:
>
> I think it makes sense to have a 2.8 release since there are a
> tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
> release seems like something we should consider separately since it
> would not have the same compatibility guarantees as a 2.8 release.
> There's a pretty big delta between trunk and 2.8 as well.
>
> cheers,
> Colin
>
> On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cdouglas@apache.org
> <ma...@apache.org>> wrote:
> With two active sustaining branches (2.6, 2.7), what would you think
> of releasing trunk as 3.x instead of pushing 2.8? There are many new
> features (EC, Y1197, etc.), and trunk could be the source of several
> alpha/beta releases before we fork the 3.x line. -C
>
> On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
> <vi...@hortonworks.com>> wrote:
> As you may have noted, 2.8.0 got completely derailed what with 2.7.x and
> the unusually long 2.6.1 release.
>
> With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2
> and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.
>
> I’ll do a quick survey of where the individual features are and the amount
> of content already present in 2.8 and kick-start 2.8.0 process again.
>
> +Vinod
>
>
> On Apr 21, 2015, at 2:39 PM, vinodkv@apache.org<ma...@apache.org>
> wrote:
>
> With 2.7.0 out of the way, and with more maintenance releases to stabilize
> it, I propose we start thinking about 2.8.0.
>
> Here's my first cut of the proposal, will update the Roadmap wiki.
> - Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
> - Compatibility tools to catch backwards, forwards compatibility issues at
> patch submission, release times. Some of it is captured at YARN-3292. This
> also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310)
> and/or investing in new tools.
> - HADOOP-11656 Classpath isolation for downstream clients
> - Support for Erasure Codes in HDFS HDFS-7285
> - Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
> - YARN Timeline Service Next generation: YARN-2928. At least branch-merge
> + early peek.
> - Supporting non-exclusive node-labels: YARN-3214
>
> I'm experimenting with more agile 2.7.x releases and would like to
> continue the same by volunteering as the RM for 2.8.x too.
>
> Given the long time we took with 2.7.0, the timeline I am looking at is
> 8-12 weeks. We can pick as many features as they finish along and make a
> more predictable releases instead of holding up releases for ever.
>
> Thoughts?
>
> Thanks
> +Vinod
>
>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Andrew Wang <an...@cloudera.com>.
>
>
> maybe discuss having a list @ release time. As an example, s3 and
> encryption at rest shipped in beta stage... what's in 2.8 that "we don't
> yet trust ourselves?". Me, I'd put erasure coding in there just because
> I've no familiarity with it
>
> Quick clarification, EC isn't scheduled for 2.8. IMO it's still an open
question whether we want to include in any branch-2 release. Elliot
(wearing his Facebook hat) said he'd be hesitant to deploy it because of
the significant NN changes. This might apply to our other big users like
Yahoo or Twitter.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Andrew Wang <an...@cloudera.com>.
>
>
> maybe discuss having a list @ release time. As an example, s3 and
> encryption at rest shipped in beta stage... what's in 2.8 that "we don't
> yet trust ourselves?". Me, I'd put erasure coding in there just because
> I've no familiarity with it
>
> Quick clarification, EC isn't scheduled for 2.8. IMO it's still an open
question whether we want to include in any branch-2 release. Elliot
(wearing his Facebook hat) said he'd be hesitant to deploy it because of
the significant NN changes. This might apply to our other big users like
Yahoo or Twitter.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Andrew Wang <an...@cloudera.com>.
>
>
> maybe discuss having a list @ release time. As an example, s3 and
> encryption at rest shipped in beta stage... what's in 2.8 that "we don't
> yet trust ourselves?". Me, I'd put erasure coding in there just because
> I've no familiarity with it
>
> Quick clarification, EC isn't scheduled for 2.8. IMO it's still an open
question whether we want to include in any branch-2 release. Elliot
(wearing his Facebook hat) said he'd be hesitant to deploy it because of
the significant NN changes. This might apply to our other big users like
Yahoo or Twitter.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Andrew Wang <an...@cloudera.com>.
>
>
> maybe discuss having a list @ release time. As an example, s3 and
> encryption at rest shipped in beta stage... what's in 2.8 that "we don't
> yet trust ourselves?". Me, I'd put erasure coding in there just because
> I've no familiarity with it
>
> Quick clarification, EC isn't scheduled for 2.8. IMO it's still an open
question whether we want to include in any branch-2 release. Elliot
(wearing his Facebook hat) said he'd be hesitant to deploy it because of
the significant NN changes. This might apply to our other big users like
Yahoo or Twitter.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Steve Loughran <st...@hortonworks.com>.
> On 25 Nov 2015, at 22:01, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Tx for your comments, Andrew!
>
> I did talk about it in a few discussions in the past related to this but yes, we never codified the feature-level alpha/beta tags. Part of the reason why I never pushed for such a codification is that (a) it is a subjective decision that the feature contributors usually have the best say on and (2) voting on the alpha-ness / beta-ness may not be a productive exercise in non-trivial number of cases (as I have seen with the release-level tags, some users think an alpha release is of production quality enough for _their_ use-cases).
>
> That said, I agree about noting down our general recommendations on what an alpha feature means, what a beta feature means etc. Let me file a JIRA for this.
maybe discuss having a list @ release time. As an example, s3 and encryption at rest shipped in beta stage... what's in 2.8 that "we don't yet trust ourselves?". Me, I'd put erasure coding in there just because I've no familiarity with it
>
> The second point you made is absolutely true. Atleast on YARN / MR side, I usually end up traversing (some if not all of) alpha features and making sure the corresponding APIs are explicitly marked private or public unstable / evolving. I do think that there is a lot of value in us getting more systematic with this - how about we do this for the feature list of 2.8 and evolve the process?
>
> In general, may be we could have a list of ‘check-list’ JIRAs that we always address before every release. Few things already come to my mind:
> - Mark which features are alpha / beta and make sure the corresponding APIs, public interfaces reflect the state
+ have people add JIRAs for the next version to actually mark things as stable/out of beta
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Steve Loughran <st...@hortonworks.com>.
> On 25 Nov 2015, at 22:01, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Tx for your comments, Andrew!
>
> I did talk about it in a few discussions in the past related to this but yes, we never codified the feature-level alpha/beta tags. Part of the reason why I never pushed for such a codification is that (a) it is a subjective decision that the feature contributors usually have the best say on and (2) voting on the alpha-ness / beta-ness may not be a productive exercise in non-trivial number of cases (as I have seen with the release-level tags, some users think an alpha release is of production quality enough for _their_ use-cases).
>
> That said, I agree about noting down our general recommendations on what an alpha feature means, what a beta feature means etc. Let me file a JIRA for this.
maybe discuss having a list @ release time. As an example, s3 and encryption at rest shipped in beta stage... what's in 2.8 that "we don't yet trust ourselves?". Me, I'd put erasure coding in there just because I've no familiarity with it
>
> The second point you made is absolutely true. Atleast on YARN / MR side, I usually end up traversing (some if not all of) alpha features and making sure the corresponding APIs are explicitly marked private or public unstable / evolving. I do think that there is a lot of value in us getting more systematic with this - how about we do this for the feature list of 2.8 and evolve the process?
>
> In general, may be we could have a list of ‘check-list’ JIRAs that we always address before every release. Few things already come to my mind:
> - Mark which features are alpha / beta and make sure the corresponding APIs, public interfaces reflect the state
+ have people add JIRAs for the next version to actually mark things as stable/out of beta
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Andrew Wang <an...@cloudera.com>.
SGTM, thanks Vinod! LMK if you need reviews on any of that.
Regarding the release checklist, another item I'd add is updating the
release notes in the project documentation, we've forgotten in the past.
On Wed, Nov 25, 2015 at 2:01 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:
> Tx for your comments, Andrew!
>
> I did talk about it in a few discussions in the past related to this but
> yes, we never codified the feature-level alpha/beta tags. Part of the
> reason why I never pushed for such a codification is that (a) it is a
> subjective decision that the feature contributors usually have the best say
> on and (2) voting on the alpha-ness / beta-ness may not be a productive
> exercise in non-trivial number of cases (as I have seen with the
> release-level tags, some users think an alpha release is of production
> quality enough for _their_ use-cases).
>
> That said, I agree about noting down our general recommendations on what
> an alpha feature means, what a beta feature means etc. Let me file a JIRA
> for this.
>
> The second point you made is absolutely true. Atleast on YARN / MR side, I
> usually end up traversing (some if not all of) alpha features and making
> sure the corresponding APIs are explicitly marked private or public
> unstable / evolving. I do think that there is a lot of value in us getting
> more systematic with this - how about we do this for the feature list of
> 2.8 and evolve the process?
>
> In general, may be we could have a list of ‘check-list’ JIRAs that we
> always address before every release. Few things already come to my mind:
> - Mark which features are alpha / beta and make sure the corresponding
> APIs, public interfaces reflect the state
> - Revise all newly added configuration properties to make sure they
> follow our general naming patterns. New contributors sometimes create
> non-standard properties that we come to regret supporting.
> - Generate a list of newly added public entry-points and validate that
> they are all indeed meant to be public
> - [...]
>
> Thoughts?
>
> +Vinod
>
>
> > On Nov 25, 2015, at 11:47 AM, Andrew Wang <an...@cloudera.com>
> wrote:
> >
> > Hey Vinod,
> >
> > I'm fine with the idea of alpha/beta marking in the abstract, but had a
> > question: do we define these terms in our compatibility policy or
> > elsewhere? I think it's commonly understood among us developers (alpha
> > means not fully tested and API unstable, beta means it's not fully tested
> > but is API stable), but it'd be good to have it written down.
> >
> > Also I think we've only done alpha/beta tagging at the release-level
> > previously which is a simpler story to tell users. So it's important for
> > this release that alpha features set their interface stability
> annotations
> > to "evolving". There isn't a corresponding annotation for "interface
> > quality", but IMO that's overkill.
> >
> > Thanks,
> > Andrew
> >
> > On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli <
> > vinodkv@apache.org> wrote:
> >
> >> This is the current state from the feedback I gathered.
> >> - Support priorities across applications within the same queue YARN-1963
> >> — Can push as an alpha / beta feature per Sunil
> >> - YARN-1197 Support changing resources of an allocated container:
> >> — Can push as an alpha/beta feature per Wangda
> >> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> >> most of it anyways.
> >> — Can push as an alpha feature.
> >> - YARN Timeline Service v1.5 - YARN-4233
> >> — Should include per Li Lu
> >> - YARN Timeline Service Next generation: YARN-2928
> >> — Per analysis from Sangjin, drop this from 2.8.
> >>
> >> One open feature status
> >> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
> >>
> >> Updated the Roadmap wiki with the same.
> >>
> >> Thanks
> >> +Vinod
> >>
> >>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
> >>>
> >>> I reviewed the current state of the YARN-2928 changes regarding its
> >> impact
> >>> if the timeline service v.2 is disabled. It does appear that there are
> a
> >>> lot of things that still do get created and enabled unconditionally
> >>> regardless of configuration. While this is understandable when we were
> >>> working to implement the feature, this clearly needs to be cleaned up
> so
> >>> that when disabled the timeline service v.2 doesn't impact other
> things.
> >>>
> >>> I filed a JIRA for that work:
> >>> https://issues.apache.org/jira/browse/YARN-4356
> >>>
> >>> We need to complete it before we can merge.
> >>>
> >>> Somewhat related is the status of the configuration and what it means
> in
> >>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.).
> I
> >>> know there is an ongoing discussion regarding YARN-4183. We'll need to
> >>> reflect the outcome of that discussion.
> >>>
> >>> My overall impression of whether this can be done for 2.8 is that it
> >> looks
> >>> rather challenging given the suggested timeframe. We also need to
> >> complete
> >>> several major tasks before it is ready.
> >>>
> >>> Sangjin
> >>>
> >>>
> >>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
> >>>
> >>>>
> >>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
> >>>> vinodkv@hortonworks.com> wrote:
> >>>>
> >>>>> — YARN Timeline Service Next generation: YARN-2928: Lots of
> >> momentum,
> >>>>> but clearly a work in progress. Two options here
> >>>>> — If it is safe to ship it into 2.8 in a disable manner, we can
> >>>>> get the early code into trunk and all the way int o2.8.
> >>>>> — If it is not safe, it organically rolls over into 2.9
> >>>>>
> >>>>
> >>>> I'll review the changes on YARN-2928 to see what impact it has (if
> any)
> >> if
> >>>> the timeline service v.2 is disabled.
> >>>>
> >>>> Another condition for it to make 2.8 is whether the branch will be in
> a
> >>>> shape in a couple of weeks such that it adds value for folks that want
> >> to
> >>>> test it. Hopefully it will become clearer soon.
> >>>>
> >>>> Sangjin
> >>>>
> >>
> >>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Andrew Wang <an...@cloudera.com>.
SGTM, thanks Vinod! LMK if you need reviews on any of that.
Regarding the release checklist, another item I'd add is updating the
release notes in the project documentation, we've forgotten in the past.
On Wed, Nov 25, 2015 at 2:01 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:
> Tx for your comments, Andrew!
>
> I did talk about it in a few discussions in the past related to this but
> yes, we never codified the feature-level alpha/beta tags. Part of the
> reason why I never pushed for such a codification is that (a) it is a
> subjective decision that the feature contributors usually have the best say
> on and (2) voting on the alpha-ness / beta-ness may not be a productive
> exercise in non-trivial number of cases (as I have seen with the
> release-level tags, some users think an alpha release is of production
> quality enough for _their_ use-cases).
>
> That said, I agree about noting down our general recommendations on what
> an alpha feature means, what a beta feature means etc. Let me file a JIRA
> for this.
>
> The second point you made is absolutely true. Atleast on YARN / MR side, I
> usually end up traversing (some if not all of) alpha features and making
> sure the corresponding APIs are explicitly marked private or public
> unstable / evolving. I do think that there is a lot of value in us getting
> more systematic with this - how about we do this for the feature list of
> 2.8 and evolve the process?
>
> In general, may be we could have a list of ‘check-list’ JIRAs that we
> always address before every release. Few things already come to my mind:
> - Mark which features are alpha / beta and make sure the corresponding
> APIs, public interfaces reflect the state
> - Revise all newly added configuration properties to make sure they
> follow our general naming patterns. New contributors sometimes create
> non-standard properties that we come to regret supporting.
> - Generate a list of newly added public entry-points and validate that
> they are all indeed meant to be public
> - [...]
>
> Thoughts?
>
> +Vinod
>
>
> > On Nov 25, 2015, at 11:47 AM, Andrew Wang <an...@cloudera.com>
> wrote:
> >
> > Hey Vinod,
> >
> > I'm fine with the idea of alpha/beta marking in the abstract, but had a
> > question: do we define these terms in our compatibility policy or
> > elsewhere? I think it's commonly understood among us developers (alpha
> > means not fully tested and API unstable, beta means it's not fully tested
> > but is API stable), but it'd be good to have it written down.
> >
> > Also I think we've only done alpha/beta tagging at the release-level
> > previously which is a simpler story to tell users. So it's important for
> > this release that alpha features set their interface stability
> annotations
> > to "evolving". There isn't a corresponding annotation for "interface
> > quality", but IMO that's overkill.
> >
> > Thanks,
> > Andrew
> >
> > On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli <
> > vinodkv@apache.org> wrote:
> >
> >> This is the current state from the feedback I gathered.
> >> - Support priorities across applications within the same queue YARN-1963
> >> — Can push as an alpha / beta feature per Sunil
> >> - YARN-1197 Support changing resources of an allocated container:
> >> — Can push as an alpha/beta feature per Wangda
> >> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> >> most of it anyways.
> >> — Can push as an alpha feature.
> >> - YARN Timeline Service v1.5 - YARN-4233
> >> — Should include per Li Lu
> >> - YARN Timeline Service Next generation: YARN-2928
> >> — Per analysis from Sangjin, drop this from 2.8.
> >>
> >> One open feature status
> >> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
> >>
> >> Updated the Roadmap wiki with the same.
> >>
> >> Thanks
> >> +Vinod
> >>
> >>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
> >>>
> >>> I reviewed the current state of the YARN-2928 changes regarding its
> >> impact
> >>> if the timeline service v.2 is disabled. It does appear that there are
> a
> >>> lot of things that still do get created and enabled unconditionally
> >>> regardless of configuration. While this is understandable when we were
> >>> working to implement the feature, this clearly needs to be cleaned up
> so
> >>> that when disabled the timeline service v.2 doesn't impact other
> things.
> >>>
> >>> I filed a JIRA for that work:
> >>> https://issues.apache.org/jira/browse/YARN-4356
> >>>
> >>> We need to complete it before we can merge.
> >>>
> >>> Somewhat related is the status of the configuration and what it means
> in
> >>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.).
> I
> >>> know there is an ongoing discussion regarding YARN-4183. We'll need to
> >>> reflect the outcome of that discussion.
> >>>
> >>> My overall impression of whether this can be done for 2.8 is that it
> >> looks
> >>> rather challenging given the suggested timeframe. We also need to
> >> complete
> >>> several major tasks before it is ready.
> >>>
> >>> Sangjin
> >>>
> >>>
> >>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
> >>>
> >>>>
> >>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
> >>>> vinodkv@hortonworks.com> wrote:
> >>>>
> >>>>> — YARN Timeline Service Next generation: YARN-2928: Lots of
> >> momentum,
> >>>>> but clearly a work in progress. Two options here
> >>>>> — If it is safe to ship it into 2.8 in a disable manner, we can
> >>>>> get the early code into trunk and all the way int o2.8.
> >>>>> — If it is not safe, it organically rolls over into 2.9
> >>>>>
> >>>>
> >>>> I'll review the changes on YARN-2928 to see what impact it has (if
> any)
> >> if
> >>>> the timeline service v.2 is disabled.
> >>>>
> >>>> Another condition for it to make 2.8 is whether the branch will be in
> a
> >>>> shape in a couple of weeks such that it adds value for folks that want
> >> to
> >>>> test it. Hopefully it will become clearer soon.
> >>>>
> >>>> Sangjin
> >>>>
> >>
> >>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Andrew Wang <an...@cloudera.com>.
SGTM, thanks Vinod! LMK if you need reviews on any of that.
Regarding the release checklist, another item I'd add is updating the
release notes in the project documentation, we've forgotten in the past.
On Wed, Nov 25, 2015 at 2:01 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:
> Tx for your comments, Andrew!
>
> I did talk about it in a few discussions in the past related to this but
> yes, we never codified the feature-level alpha/beta tags. Part of the
> reason why I never pushed for such a codification is that (a) it is a
> subjective decision that the feature contributors usually have the best say
> on and (2) voting on the alpha-ness / beta-ness may not be a productive
> exercise in non-trivial number of cases (as I have seen with the
> release-level tags, some users think an alpha release is of production
> quality enough for _their_ use-cases).
>
> That said, I agree about noting down our general recommendations on what
> an alpha feature means, what a beta feature means etc. Let me file a JIRA
> for this.
>
> The second point you made is absolutely true. Atleast on YARN / MR side, I
> usually end up traversing (some if not all of) alpha features and making
> sure the corresponding APIs are explicitly marked private or public
> unstable / evolving. I do think that there is a lot of value in us getting
> more systematic with this - how about we do this for the feature list of
> 2.8 and evolve the process?
>
> In general, may be we could have a list of ‘check-list’ JIRAs that we
> always address before every release. Few things already come to my mind:
> - Mark which features are alpha / beta and make sure the corresponding
> APIs, public interfaces reflect the state
> - Revise all newly added configuration properties to make sure they
> follow our general naming patterns. New contributors sometimes create
> non-standard properties that we come to regret supporting.
> - Generate a list of newly added public entry-points and validate that
> they are all indeed meant to be public
> - [...]
>
> Thoughts?
>
> +Vinod
>
>
> > On Nov 25, 2015, at 11:47 AM, Andrew Wang <an...@cloudera.com>
> wrote:
> >
> > Hey Vinod,
> >
> > I'm fine with the idea of alpha/beta marking in the abstract, but had a
> > question: do we define these terms in our compatibility policy or
> > elsewhere? I think it's commonly understood among us developers (alpha
> > means not fully tested and API unstable, beta means it's not fully tested
> > but is API stable), but it'd be good to have it written down.
> >
> > Also I think we've only done alpha/beta tagging at the release-level
> > previously which is a simpler story to tell users. So it's important for
> > this release that alpha features set their interface stability
> annotations
> > to "evolving". There isn't a corresponding annotation for "interface
> > quality", but IMO that's overkill.
> >
> > Thanks,
> > Andrew
> >
> > On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli <
> > vinodkv@apache.org> wrote:
> >
> >> This is the current state from the feedback I gathered.
> >> - Support priorities across applications within the same queue YARN-1963
> >> — Can push as an alpha / beta feature per Sunil
> >> - YARN-1197 Support changing resources of an allocated container:
> >> — Can push as an alpha/beta feature per Wangda
> >> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> >> most of it anyways.
> >> — Can push as an alpha feature.
> >> - YARN Timeline Service v1.5 - YARN-4233
> >> — Should include per Li Lu
> >> - YARN Timeline Service Next generation: YARN-2928
> >> — Per analysis from Sangjin, drop this from 2.8.
> >>
> >> One open feature status
> >> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
> >>
> >> Updated the Roadmap wiki with the same.
> >>
> >> Thanks
> >> +Vinod
> >>
> >>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
> >>>
> >>> I reviewed the current state of the YARN-2928 changes regarding its
> >> impact
> >>> if the timeline service v.2 is disabled. It does appear that there are
> a
> >>> lot of things that still do get created and enabled unconditionally
> >>> regardless of configuration. While this is understandable when we were
> >>> working to implement the feature, this clearly needs to be cleaned up
> so
> >>> that when disabled the timeline service v.2 doesn't impact other
> things.
> >>>
> >>> I filed a JIRA for that work:
> >>> https://issues.apache.org/jira/browse/YARN-4356
> >>>
> >>> We need to complete it before we can merge.
> >>>
> >>> Somewhat related is the status of the configuration and what it means
> in
> >>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.).
> I
> >>> know there is an ongoing discussion regarding YARN-4183. We'll need to
> >>> reflect the outcome of that discussion.
> >>>
> >>> My overall impression of whether this can be done for 2.8 is that it
> >> looks
> >>> rather challenging given the suggested timeframe. We also need to
> >> complete
> >>> several major tasks before it is ready.
> >>>
> >>> Sangjin
> >>>
> >>>
> >>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
> >>>
> >>>>
> >>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
> >>>> vinodkv@hortonworks.com> wrote:
> >>>>
> >>>>> — YARN Timeline Service Next generation: YARN-2928: Lots of
> >> momentum,
> >>>>> but clearly a work in progress. Two options here
> >>>>> — If it is safe to ship it into 2.8 in a disable manner, we can
> >>>>> get the early code into trunk and all the way int o2.8.
> >>>>> — If it is not safe, it organically rolls over into 2.9
> >>>>>
> >>>>
> >>>> I'll review the changes on YARN-2928 to see what impact it has (if
> any)
> >> if
> >>>> the timeline service v.2 is disabled.
> >>>>
> >>>> Another condition for it to make 2.8 is whether the branch will be in
> a
> >>>> shape in a couple of weeks such that it adds value for folks that want
> >> to
> >>>> test it. Hopefully it will become clearer soon.
> >>>>
> >>>> Sangjin
> >>>>
> >>
> >>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Steve Loughran <st...@hortonworks.com>.
> On 25 Nov 2015, at 22:01, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Tx for your comments, Andrew!
>
> I did talk about it in a few discussions in the past related to this but yes, we never codified the feature-level alpha/beta tags. Part of the reason why I never pushed for such a codification is that (a) it is a subjective decision that the feature contributors usually have the best say on and (2) voting on the alpha-ness / beta-ness may not be a productive exercise in non-trivial number of cases (as I have seen with the release-level tags, some users think an alpha release is of production quality enough for _their_ use-cases).
>
> That said, I agree about noting down our general recommendations on what an alpha feature means, what a beta feature means etc. Let me file a JIRA for this.
maybe discuss having a list @ release time. As an example, s3 and encryption at rest shipped in beta stage... what's in 2.8 that "we don't yet trust ourselves?". Me, I'd put erasure coding in there just because I've no familiarity with it
>
> The second point you made is absolutely true. Atleast on YARN / MR side, I usually end up traversing (some if not all of) alpha features and making sure the corresponding APIs are explicitly marked private or public unstable / evolving. I do think that there is a lot of value in us getting more systematic with this - how about we do this for the feature list of 2.8 and evolve the process?
>
> In general, may be we could have a list of ‘check-list’ JIRAs that we always address before every release. Few things already come to my mind:
> - Mark which features are alpha / beta and make sure the corresponding APIs, public interfaces reflect the state
+ have people add JIRAs for the next version to actually mark things as stable/out of beta
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Steve Loughran <st...@hortonworks.com>.
> On 25 Nov 2015, at 22:01, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Tx for your comments, Andrew!
>
> I did talk about it in a few discussions in the past related to this but yes, we never codified the feature-level alpha/beta tags. Part of the reason why I never pushed for such a codification is that (a) it is a subjective decision that the feature contributors usually have the best say on and (2) voting on the alpha-ness / beta-ness may not be a productive exercise in non-trivial number of cases (as I have seen with the release-level tags, some users think an alpha release is of production quality enough for _their_ use-cases).
>
> That said, I agree about noting down our general recommendations on what an alpha feature means, what a beta feature means etc. Let me file a JIRA for this.
maybe discuss having a list @ release time. As an example, s3 and encryption at rest shipped in beta stage... what's in 2.8 that "we don't yet trust ourselves?". Me, I'd put erasure coding in there just because I've no familiarity with it
>
> The second point you made is absolutely true. Atleast on YARN / MR side, I usually end up traversing (some if not all of) alpha features and making sure the corresponding APIs are explicitly marked private or public unstable / evolving. I do think that there is a lot of value in us getting more systematic with this - how about we do this for the feature list of 2.8 and evolve the process?
>
> In general, may be we could have a list of ‘check-list’ JIRAs that we always address before every release. Few things already come to my mind:
> - Mark which features are alpha / beta and make sure the corresponding APIs, public interfaces reflect the state
+ have people add JIRAs for the next version to actually mark things as stable/out of beta
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Andrew Wang <an...@cloudera.com>.
SGTM, thanks Vinod! LMK if you need reviews on any of that.
Regarding the release checklist, another item I'd add is updating the
release notes in the project documentation, we've forgotten in the past.
On Wed, Nov 25, 2015 at 2:01 PM, Vinod Kumar Vavilapalli <vinodkv@apache.org
> wrote:
> Tx for your comments, Andrew!
>
> I did talk about it in a few discussions in the past related to this but
> yes, we never codified the feature-level alpha/beta tags. Part of the
> reason why I never pushed for such a codification is that (a) it is a
> subjective decision that the feature contributors usually have the best say
> on and (2) voting on the alpha-ness / beta-ness may not be a productive
> exercise in non-trivial number of cases (as I have seen with the
> release-level tags, some users think an alpha release is of production
> quality enough for _their_ use-cases).
>
> That said, I agree about noting down our general recommendations on what
> an alpha feature means, what a beta feature means etc. Let me file a JIRA
> for this.
>
> The second point you made is absolutely true. Atleast on YARN / MR side, I
> usually end up traversing (some if not all of) alpha features and making
> sure the corresponding APIs are explicitly marked private or public
> unstable / evolving. I do think that there is a lot of value in us getting
> more systematic with this - how about we do this for the feature list of
> 2.8 and evolve the process?
>
> In general, may be we could have a list of ‘check-list’ JIRAs that we
> always address before every release. Few things already come to my mind:
> - Mark which features are alpha / beta and make sure the corresponding
> APIs, public interfaces reflect the state
> - Revise all newly added configuration properties to make sure they
> follow our general naming patterns. New contributors sometimes create
> non-standard properties that we come to regret supporting.
> - Generate a list of newly added public entry-points and validate that
> they are all indeed meant to be public
> - [...]
>
> Thoughts?
>
> +Vinod
>
>
> > On Nov 25, 2015, at 11:47 AM, Andrew Wang <an...@cloudera.com>
> wrote:
> >
> > Hey Vinod,
> >
> > I'm fine with the idea of alpha/beta marking in the abstract, but had a
> > question: do we define these terms in our compatibility policy or
> > elsewhere? I think it's commonly understood among us developers (alpha
> > means not fully tested and API unstable, beta means it's not fully tested
> > but is API stable), but it'd be good to have it written down.
> >
> > Also I think we've only done alpha/beta tagging at the release-level
> > previously which is a simpler story to tell users. So it's important for
> > this release that alpha features set their interface stability
> annotations
> > to "evolving". There isn't a corresponding annotation for "interface
> > quality", but IMO that's overkill.
> >
> > Thanks,
> > Andrew
> >
> > On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli <
> > vinodkv@apache.org> wrote:
> >
> >> This is the current state from the feedback I gathered.
> >> - Support priorities across applications within the same queue YARN-1963
> >> — Can push as an alpha / beta feature per Sunil
> >> - YARN-1197 Support changing resources of an allocated container:
> >> — Can push as an alpha/beta feature per Wangda
> >> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> >> most of it anyways.
> >> — Can push as an alpha feature.
> >> - YARN Timeline Service v1.5 - YARN-4233
> >> — Should include per Li Lu
> >> - YARN Timeline Service Next generation: YARN-2928
> >> — Per analysis from Sangjin, drop this from 2.8.
> >>
> >> One open feature status
> >> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
> >>
> >> Updated the Roadmap wiki with the same.
> >>
> >> Thanks
> >> +Vinod
> >>
> >>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
> >>>
> >>> I reviewed the current state of the YARN-2928 changes regarding its
> >> impact
> >>> if the timeline service v.2 is disabled. It does appear that there are
> a
> >>> lot of things that still do get created and enabled unconditionally
> >>> regardless of configuration. While this is understandable when we were
> >>> working to implement the feature, this clearly needs to be cleaned up
> so
> >>> that when disabled the timeline service v.2 doesn't impact other
> things.
> >>>
> >>> I filed a JIRA for that work:
> >>> https://issues.apache.org/jira/browse/YARN-4356
> >>>
> >>> We need to complete it before we can merge.
> >>>
> >>> Somewhat related is the status of the configuration and what it means
> in
> >>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.).
> I
> >>> know there is an ongoing discussion regarding YARN-4183. We'll need to
> >>> reflect the outcome of that discussion.
> >>>
> >>> My overall impression of whether this can be done for 2.8 is that it
> >> looks
> >>> rather challenging given the suggested timeframe. We also need to
> >> complete
> >>> several major tasks before it is ready.
> >>>
> >>> Sangjin
> >>>
> >>>
> >>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
> >>>
> >>>>
> >>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
> >>>> vinodkv@hortonworks.com> wrote:
> >>>>
> >>>>> — YARN Timeline Service Next generation: YARN-2928: Lots of
> >> momentum,
> >>>>> but clearly a work in progress. Two options here
> >>>>> — If it is safe to ship it into 2.8 in a disable manner, we can
> >>>>> get the early code into trunk and all the way int o2.8.
> >>>>> — If it is not safe, it organically rolls over into 2.9
> >>>>>
> >>>>
> >>>> I'll review the changes on YARN-2928 to see what impact it has (if
> any)
> >> if
> >>>> the timeline service v.2 is disabled.
> >>>>
> >>>> Another condition for it to make 2.8 is whether the branch will be in
> a
> >>>> shape in a couple of weeks such that it adds value for folks that want
> >> to
> >>>> test it. Hopefully it will become clearer soon.
> >>>>
> >>>> Sangjin
> >>>>
> >>
> >>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Tx for your comments, Andrew!
I did talk about it in a few discussions in the past related to this but yes, we never codified the feature-level alpha/beta tags. Part of the reason why I never pushed for such a codification is that (a) it is a subjective decision that the feature contributors usually have the best say on and (2) voting on the alpha-ness / beta-ness may not be a productive exercise in non-trivial number of cases (as I have seen with the release-level tags, some users think an alpha release is of production quality enough for _their_ use-cases).
That said, I agree about noting down our general recommendations on what an alpha feature means, what a beta feature means etc. Let me file a JIRA for this.
The second point you made is absolutely true. Atleast on YARN / MR side, I usually end up traversing (some if not all of) alpha features and making sure the corresponding APIs are explicitly marked private or public unstable / evolving. I do think that there is a lot of value in us getting more systematic with this - how about we do this for the feature list of 2.8 and evolve the process?
In general, may be we could have a list of ‘check-list’ JIRAs that we always address before every release. Few things already come to my mind:
- Mark which features are alpha / beta and make sure the corresponding APIs, public interfaces reflect the state
- Revise all newly added configuration properties to make sure they follow our general naming patterns. New contributors sometimes create non-standard properties that we come to regret supporting.
- Generate a list of newly added public entry-points and validate that they are all indeed meant to be public
- [...]
Thoughts?
+Vinod
> On Nov 25, 2015, at 11:47 AM, Andrew Wang <an...@cloudera.com> wrote:
>
> Hey Vinod,
>
> I'm fine with the idea of alpha/beta marking in the abstract, but had a
> question: do we define these terms in our compatibility policy or
> elsewhere? I think it's commonly understood among us developers (alpha
> means not fully tested and API unstable, beta means it's not fully tested
> but is API stable), but it'd be good to have it written down.
>
> Also I think we've only done alpha/beta tagging at the release-level
> previously which is a simpler story to tell users. So it's important for
> this release that alpha features set their interface stability annotations
> to "evolving". There isn't a corresponding annotation for "interface
> quality", but IMO that's overkill.
>
> Thanks,
> Andrew
>
> On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>> This is the current state from the feedback I gathered.
>> - Support priorities across applications within the same queue YARN-1963
>> — Can push as an alpha / beta feature per Sunil
>> - YARN-1197 Support changing resources of an allocated container:
>> — Can push as an alpha/beta feature per Wangda
>> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>> most of it anyways.
>> — Can push as an alpha feature.
>> - YARN Timeline Service v1.5 - YARN-4233
>> — Should include per Li Lu
>> - YARN Timeline Service Next generation: YARN-2928
>> — Per analysis from Sangjin, drop this from 2.8.
>>
>> One open feature status
>> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>>
>> Updated the Roadmap wiki with the same.
>>
>> Thanks
>> +Vinod
>>
>>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>
>>> I reviewed the current state of the YARN-2928 changes regarding its
>> impact
>>> if the timeline service v.2 is disabled. It does appear that there are a
>>> lot of things that still do get created and enabled unconditionally
>>> regardless of configuration. While this is understandable when we were
>>> working to implement the feature, this clearly needs to be cleaned up so
>>> that when disabled the timeline service v.2 doesn't impact other things.
>>>
>>> I filed a JIRA for that work:
>>> https://issues.apache.org/jira/browse/YARN-4356
>>>
>>> We need to complete it before we can merge.
>>>
>>> Somewhat related is the status of the configuration and what it means in
>>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
>>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>>> reflect the outcome of that discussion.
>>>
>>> My overall impression of whether this can be done for 2.8 is that it
>> looks
>>> rather challenging given the suggested timeframe. We also need to
>> complete
>>> several major tasks before it is ready.
>>>
>>> Sangjin
>>>
>>>
>>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>>
>>>>
>>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>> — YARN Timeline Service Next generation: YARN-2928: Lots of
>> momentum,
>>>>> but clearly a work in progress. Two options here
>>>>> — If it is safe to ship it into 2.8 in a disable manner, we can
>>>>> get the early code into trunk and all the way int o2.8.
>>>>> — If it is not safe, it organically rolls over into 2.9
>>>>>
>>>>
>>>> I'll review the changes on YARN-2928 to see what impact it has (if any)
>> if
>>>> the timeline service v.2 is disabled.
>>>>
>>>> Another condition for it to make 2.8 is whether the branch will be in a
>>>> shape in a couple of weeks such that it adds value for folks that want
>> to
>>>> test it. Hopefully it will become clearer soon.
>>>>
>>>> Sangjin
>>>>
>>
>>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Tx for your comments, Andrew!
I did talk about it in a few discussions in the past related to this but yes, we never codified the feature-level alpha/beta tags. Part of the reason why I never pushed for such a codification is that (a) it is a subjective decision that the feature contributors usually have the best say on and (2) voting on the alpha-ness / beta-ness may not be a productive exercise in non-trivial number of cases (as I have seen with the release-level tags, some users think an alpha release is of production quality enough for _their_ use-cases).
That said, I agree about noting down our general recommendations on what an alpha feature means, what a beta feature means etc. Let me file a JIRA for this.
The second point you made is absolutely true. Atleast on YARN / MR side, I usually end up traversing (some if not all of) alpha features and making sure the corresponding APIs are explicitly marked private or public unstable / evolving. I do think that there is a lot of value in us getting more systematic with this - how about we do this for the feature list of 2.8 and evolve the process?
In general, may be we could have a list of ‘check-list’ JIRAs that we always address before every release. Few things already come to my mind:
- Mark which features are alpha / beta and make sure the corresponding APIs, public interfaces reflect the state
- Revise all newly added configuration properties to make sure they follow our general naming patterns. New contributors sometimes create non-standard properties that we come to regret supporting.
- Generate a list of newly added public entry-points and validate that they are all indeed meant to be public
- [...]
Thoughts?
+Vinod
> On Nov 25, 2015, at 11:47 AM, Andrew Wang <an...@cloudera.com> wrote:
>
> Hey Vinod,
>
> I'm fine with the idea of alpha/beta marking in the abstract, but had a
> question: do we define these terms in our compatibility policy or
> elsewhere? I think it's commonly understood among us developers (alpha
> means not fully tested and API unstable, beta means it's not fully tested
> but is API stable), but it'd be good to have it written down.
>
> Also I think we've only done alpha/beta tagging at the release-level
> previously which is a simpler story to tell users. So it's important for
> this release that alpha features set their interface stability annotations
> to "evolving". There isn't a corresponding annotation for "interface
> quality", but IMO that's overkill.
>
> Thanks,
> Andrew
>
> On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>> This is the current state from the feedback I gathered.
>> - Support priorities across applications within the same queue YARN-1963
>> — Can push as an alpha / beta feature per Sunil
>> - YARN-1197 Support changing resources of an allocated container:
>> — Can push as an alpha/beta feature per Wangda
>> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>> most of it anyways.
>> — Can push as an alpha feature.
>> - YARN Timeline Service v1.5 - YARN-4233
>> — Should include per Li Lu
>> - YARN Timeline Service Next generation: YARN-2928
>> — Per analysis from Sangjin, drop this from 2.8.
>>
>> One open feature status
>> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>>
>> Updated the Roadmap wiki with the same.
>>
>> Thanks
>> +Vinod
>>
>>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>
>>> I reviewed the current state of the YARN-2928 changes regarding its
>> impact
>>> if the timeline service v.2 is disabled. It does appear that there are a
>>> lot of things that still do get created and enabled unconditionally
>>> regardless of configuration. While this is understandable when we were
>>> working to implement the feature, this clearly needs to be cleaned up so
>>> that when disabled the timeline service v.2 doesn't impact other things.
>>>
>>> I filed a JIRA for that work:
>>> https://issues.apache.org/jira/browse/YARN-4356
>>>
>>> We need to complete it before we can merge.
>>>
>>> Somewhat related is the status of the configuration and what it means in
>>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
>>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>>> reflect the outcome of that discussion.
>>>
>>> My overall impression of whether this can be done for 2.8 is that it
>> looks
>>> rather challenging given the suggested timeframe. We also need to
>> complete
>>> several major tasks before it is ready.
>>>
>>> Sangjin
>>>
>>>
>>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>>
>>>>
>>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>> — YARN Timeline Service Next generation: YARN-2928: Lots of
>> momentum,
>>>>> but clearly a work in progress. Two options here
>>>>> — If it is safe to ship it into 2.8 in a disable manner, we can
>>>>> get the early code into trunk and all the way int o2.8.
>>>>> — If it is not safe, it organically rolls over into 2.9
>>>>>
>>>>
>>>> I'll review the changes on YARN-2928 to see what impact it has (if any)
>> if
>>>> the timeline service v.2 is disabled.
>>>>
>>>> Another condition for it to make 2.8 is whether the branch will be in a
>>>> shape in a couple of weeks such that it adds value for folks that want
>> to
>>>> test it. Hopefully it will become clearer soon.
>>>>
>>>> Sangjin
>>>>
>>
>>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Tx for your comments, Andrew!
I did talk about it in a few discussions in the past related to this but yes, we never codified the feature-level alpha/beta tags. Part of the reason why I never pushed for such a codification is that (a) it is a subjective decision that the feature contributors usually have the best say on and (2) voting on the alpha-ness / beta-ness may not be a productive exercise in non-trivial number of cases (as I have seen with the release-level tags, some users think an alpha release is of production quality enough for _their_ use-cases).
That said, I agree about noting down our general recommendations on what an alpha feature means, what a beta feature means etc. Let me file a JIRA for this.
The second point you made is absolutely true. Atleast on YARN / MR side, I usually end up traversing (some if not all of) alpha features and making sure the corresponding APIs are explicitly marked private or public unstable / evolving. I do think that there is a lot of value in us getting more systematic with this - how about we do this for the feature list of 2.8 and evolve the process?
In general, may be we could have a list of ‘check-list’ JIRAs that we always address before every release. Few things already come to my mind:
- Mark which features are alpha / beta and make sure the corresponding APIs, public interfaces reflect the state
- Revise all newly added configuration properties to make sure they follow our general naming patterns. New contributors sometimes create non-standard properties that we come to regret supporting.
- Generate a list of newly added public entry-points and validate that they are all indeed meant to be public
- [...]
Thoughts?
+Vinod
> On Nov 25, 2015, at 11:47 AM, Andrew Wang <an...@cloudera.com> wrote:
>
> Hey Vinod,
>
> I'm fine with the idea of alpha/beta marking in the abstract, but had a
> question: do we define these terms in our compatibility policy or
> elsewhere? I think it's commonly understood among us developers (alpha
> means not fully tested and API unstable, beta means it's not fully tested
> but is API stable), but it'd be good to have it written down.
>
> Also I think we've only done alpha/beta tagging at the release-level
> previously which is a simpler story to tell users. So it's important for
> this release that alpha features set their interface stability annotations
> to "evolving". There isn't a corresponding annotation for "interface
> quality", but IMO that's overkill.
>
> Thanks,
> Andrew
>
> On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>> This is the current state from the feedback I gathered.
>> - Support priorities across applications within the same queue YARN-1963
>> — Can push as an alpha / beta feature per Sunil
>> - YARN-1197 Support changing resources of an allocated container:
>> — Can push as an alpha/beta feature per Wangda
>> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>> most of it anyways.
>> — Can push as an alpha feature.
>> - YARN Timeline Service v1.5 - YARN-4233
>> — Should include per Li Lu
>> - YARN Timeline Service Next generation: YARN-2928
>> — Per analysis from Sangjin, drop this from 2.8.
>>
>> One open feature status
>> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>>
>> Updated the Roadmap wiki with the same.
>>
>> Thanks
>> +Vinod
>>
>>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>
>>> I reviewed the current state of the YARN-2928 changes regarding its
>> impact
>>> if the timeline service v.2 is disabled. It does appear that there are a
>>> lot of things that still do get created and enabled unconditionally
>>> regardless of configuration. While this is understandable when we were
>>> working to implement the feature, this clearly needs to be cleaned up so
>>> that when disabled the timeline service v.2 doesn't impact other things.
>>>
>>> I filed a JIRA for that work:
>>> https://issues.apache.org/jira/browse/YARN-4356
>>>
>>> We need to complete it before we can merge.
>>>
>>> Somewhat related is the status of the configuration and what it means in
>>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
>>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>>> reflect the outcome of that discussion.
>>>
>>> My overall impression of whether this can be done for 2.8 is that it
>> looks
>>> rather challenging given the suggested timeframe. We also need to
>> complete
>>> several major tasks before it is ready.
>>>
>>> Sangjin
>>>
>>>
>>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>>
>>>>
>>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>> — YARN Timeline Service Next generation: YARN-2928: Lots of
>> momentum,
>>>>> but clearly a work in progress. Two options here
>>>>> — If it is safe to ship it into 2.8 in a disable manner, we can
>>>>> get the early code into trunk and all the way int o2.8.
>>>>> — If it is not safe, it organically rolls over into 2.9
>>>>>
>>>>
>>>> I'll review the changes on YARN-2928 to see what impact it has (if any)
>> if
>>>> the timeline service v.2 is disabled.
>>>>
>>>> Another condition for it to make 2.8 is whether the branch will be in a
>>>> shape in a couple of weeks such that it adds value for folks that want
>> to
>>>> test it. Hopefully it will become clearer soon.
>>>>
>>>> Sangjin
>>>>
>>
>>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Tx for your comments, Andrew!
I did talk about it in a few discussions in the past related to this but yes, we never codified the feature-level alpha/beta tags. Part of the reason why I never pushed for such a codification is that (a) it is a subjective decision that the feature contributors usually have the best say on and (2) voting on the alpha-ness / beta-ness may not be a productive exercise in non-trivial number of cases (as I have seen with the release-level tags, some users think an alpha release is of production quality enough for _their_ use-cases).
That said, I agree about noting down our general recommendations on what an alpha feature means, what a beta feature means etc. Let me file a JIRA for this.
The second point you made is absolutely true. Atleast on YARN / MR side, I usually end up traversing (some if not all of) alpha features and making sure the corresponding APIs are explicitly marked private or public unstable / evolving. I do think that there is a lot of value in us getting more systematic with this - how about we do this for the feature list of 2.8 and evolve the process?
In general, may be we could have a list of ‘check-list’ JIRAs that we always address before every release. Few things already come to my mind:
- Mark which features are alpha / beta and make sure the corresponding APIs, public interfaces reflect the state
- Revise all newly added configuration properties to make sure they follow our general naming patterns. New contributors sometimes create non-standard properties that we come to regret supporting.
- Generate a list of newly added public entry-points and validate that they are all indeed meant to be public
- [...]
Thoughts?
+Vinod
> On Nov 25, 2015, at 11:47 AM, Andrew Wang <an...@cloudera.com> wrote:
>
> Hey Vinod,
>
> I'm fine with the idea of alpha/beta marking in the abstract, but had a
> question: do we define these terms in our compatibility policy or
> elsewhere? I think it's commonly understood among us developers (alpha
> means not fully tested and API unstable, beta means it's not fully tested
> but is API stable), but it'd be good to have it written down.
>
> Also I think we've only done alpha/beta tagging at the release-level
> previously which is a simpler story to tell users. So it's important for
> this release that alpha features set their interface stability annotations
> to "evolving". There isn't a corresponding annotation for "interface
> quality", but IMO that's overkill.
>
> Thanks,
> Andrew
>
> On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli <
> vinodkv@apache.org> wrote:
>
>> This is the current state from the feedback I gathered.
>> - Support priorities across applications within the same queue YARN-1963
>> — Can push as an alpha / beta feature per Sunil
>> - YARN-1197 Support changing resources of an allocated container:
>> — Can push as an alpha/beta feature per Wangda
>> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>> most of it anyways.
>> — Can push as an alpha feature.
>> - YARN Timeline Service v1.5 - YARN-4233
>> — Should include per Li Lu
>> - YARN Timeline Service Next generation: YARN-2928
>> — Per analysis from Sangjin, drop this from 2.8.
>>
>> One open feature status
>> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>>
>> Updated the Roadmap wiki with the same.
>>
>> Thanks
>> +Vinod
>>
>>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>
>>> I reviewed the current state of the YARN-2928 changes regarding its
>> impact
>>> if the timeline service v.2 is disabled. It does appear that there are a
>>> lot of things that still do get created and enabled unconditionally
>>> regardless of configuration. While this is understandable when we were
>>> working to implement the feature, this clearly needs to be cleaned up so
>>> that when disabled the timeline service v.2 doesn't impact other things.
>>>
>>> I filed a JIRA for that work:
>>> https://issues.apache.org/jira/browse/YARN-4356
>>>
>>> We need to complete it before we can merge.
>>>
>>> Somewhat related is the status of the configuration and what it means in
>>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
>>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>>> reflect the outcome of that discussion.
>>>
>>> My overall impression of whether this can be done for 2.8 is that it
>> looks
>>> rather challenging given the suggested timeframe. We also need to
>> complete
>>> several major tasks before it is ready.
>>>
>>> Sangjin
>>>
>>>
>>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>>
>>>>
>>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>> — YARN Timeline Service Next generation: YARN-2928: Lots of
>> momentum,
>>>>> but clearly a work in progress. Two options here
>>>>> — If it is safe to ship it into 2.8 in a disable manner, we can
>>>>> get the early code into trunk and all the way int o2.8.
>>>>> — If it is not safe, it organically rolls over into 2.9
>>>>>
>>>>
>>>> I'll review the changes on YARN-2928 to see what impact it has (if any)
>> if
>>>> the timeline service v.2 is disabled.
>>>>
>>>> Another condition for it to make 2.8 is whether the branch will be in a
>>>> shape in a couple of weeks such that it adds value for folks that want
>> to
>>>> test it. Hopefully it will become clearer soon.
>>>>
>>>> Sangjin
>>>>
>>
>>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Andrew Wang <an...@cloudera.com>.
Hey Vinod,
I'm fine with the idea of alpha/beta marking in the abstract, but had a
question: do we define these terms in our compatibility policy or
elsewhere? I think it's commonly understood among us developers (alpha
means not fully tested and API unstable, beta means it's not fully tested
but is API stable), but it'd be good to have it written down.
Also I think we've only done alpha/beta tagging at the release-level
previously which is a simpler story to tell users. So it's important for
this release that alpha features set their interface stability annotations
to "evolving". There isn't a corresponding annotation for "interface
quality", but IMO that's overkill.
Thanks,
Andrew
On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:
> This is the current state from the feedback I gathered.
> - Support priorities across applications within the same queue YARN-1963
> — Can push as an alpha / beta feature per Sunil
> - YARN-1197 Support changing resources of an allocated container:
> — Can push as an alpha/beta feature per Wangda
> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> most of it anyways.
> — Can push as an alpha feature.
> - YARN Timeline Service v1.5 - YARN-4233
> — Should include per Li Lu
> - YARN Timeline Service Next generation: YARN-2928
> — Per analysis from Sangjin, drop this from 2.8.
>
> One open feature status
> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>
> Updated the Roadmap wiki with the same.
>
> Thanks
> +Vinod
>
> > On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
> >
> > I reviewed the current state of the YARN-2928 changes regarding its
> impact
> > if the timeline service v.2 is disabled. It does appear that there are a
> > lot of things that still do get created and enabled unconditionally
> > regardless of configuration. While this is understandable when we were
> > working to implement the feature, this clearly needs to be cleaned up so
> > that when disabled the timeline service v.2 doesn't impact other things.
> >
> > I filed a JIRA for that work:
> > https://issues.apache.org/jira/browse/YARN-4356
> >
> > We need to complete it before we can merge.
> >
> > Somewhat related is the status of the configuration and what it means in
> > various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
> > know there is an ongoing discussion regarding YARN-4183. We'll need to
> > reflect the outcome of that discussion.
> >
> > My overall impression of whether this can be done for 2.8 is that it
> looks
> > rather challenging given the suggested timeframe. We also need to
> complete
> > several major tasks before it is ready.
> >
> > Sangjin
> >
> >
> > On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
> >
> >>
> >> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
> >> vinodkv@hortonworks.com> wrote:
> >>
> >>> — YARN Timeline Service Next generation: YARN-2928: Lots of
> momentum,
> >>> but clearly a work in progress. Two options here
> >>> — If it is safe to ship it into 2.8 in a disable manner, we can
> >>> get the early code into trunk and all the way int o2.8.
> >>> — If it is not safe, it organically rolls over into 2.9
> >>>
> >>
> >> I'll review the changes on YARN-2928 to see what impact it has (if any)
> if
> >> the timeline service v.2 is disabled.
> >>
> >> Another condition for it to make 2.8 is whether the branch will be in a
> >> shape in a couple of weeks such that it adds value for folks that want
> to
> >> test it. Hopefully it will become clearer soon.
> >>
> >> Sangjin
> >>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Andrew Wang <an...@cloudera.com>.
Hey Vinod,
I'm fine with the idea of alpha/beta marking in the abstract, but had a
question: do we define these terms in our compatibility policy or
elsewhere? I think it's commonly understood among us developers (alpha
means not fully tested and API unstable, beta means it's not fully tested
but is API stable), but it'd be good to have it written down.
Also I think we've only done alpha/beta tagging at the release-level
previously which is a simpler story to tell users. So it's important for
this release that alpha features set their interface stability annotations
to "evolving". There isn't a corresponding annotation for "interface
quality", but IMO that's overkill.
Thanks,
Andrew
On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:
> This is the current state from the feedback I gathered.
> - Support priorities across applications within the same queue YARN-1963
> — Can push as an alpha / beta feature per Sunil
> - YARN-1197 Support changing resources of an allocated container:
> — Can push as an alpha/beta feature per Wangda
> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> most of it anyways.
> — Can push as an alpha feature.
> - YARN Timeline Service v1.5 - YARN-4233
> — Should include per Li Lu
> - YARN Timeline Service Next generation: YARN-2928
> — Per analysis from Sangjin, drop this from 2.8.
>
> One open feature status
> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>
> Updated the Roadmap wiki with the same.
>
> Thanks
> +Vinod
>
> > On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
> >
> > I reviewed the current state of the YARN-2928 changes regarding its
> impact
> > if the timeline service v.2 is disabled. It does appear that there are a
> > lot of things that still do get created and enabled unconditionally
> > regardless of configuration. While this is understandable when we were
> > working to implement the feature, this clearly needs to be cleaned up so
> > that when disabled the timeline service v.2 doesn't impact other things.
> >
> > I filed a JIRA for that work:
> > https://issues.apache.org/jira/browse/YARN-4356
> >
> > We need to complete it before we can merge.
> >
> > Somewhat related is the status of the configuration and what it means in
> > various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
> > know there is an ongoing discussion regarding YARN-4183. We'll need to
> > reflect the outcome of that discussion.
> >
> > My overall impression of whether this can be done for 2.8 is that it
> looks
> > rather challenging given the suggested timeframe. We also need to
> complete
> > several major tasks before it is ready.
> >
> > Sangjin
> >
> >
> > On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
> >
> >>
> >> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
> >> vinodkv@hortonworks.com> wrote:
> >>
> >>> — YARN Timeline Service Next generation: YARN-2928: Lots of
> momentum,
> >>> but clearly a work in progress. Two options here
> >>> — If it is safe to ship it into 2.8 in a disable manner, we can
> >>> get the early code into trunk and all the way int o2.8.
> >>> — If it is not safe, it organically rolls over into 2.9
> >>>
> >>
> >> I'll review the changes on YARN-2928 to see what impact it has (if any)
> if
> >> the timeline service v.2 is disabled.
> >>
> >> Another condition for it to make 2.8 is whether the branch will be in a
> >> shape in a couple of weeks such that it adds value for folks that want
> to
> >> test it. Hopefully it will become clearer soon.
> >>
> >> Sangjin
> >>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Andrew Wang <an...@cloudera.com>.
Hey Vinod,
I'm fine with the idea of alpha/beta marking in the abstract, but had a
question: do we define these terms in our compatibility policy or
elsewhere? I think it's commonly understood among us developers (alpha
means not fully tested and API unstable, beta means it's not fully tested
but is API stable), but it'd be good to have it written down.
Also I think we've only done alpha/beta tagging at the release-level
previously which is a simpler story to tell users. So it's important for
this release that alpha features set their interface stability annotations
to "evolving". There isn't a corresponding annotation for "interface
quality", but IMO that's overkill.
Thanks,
Andrew
On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:
> This is the current state from the feedback I gathered.
> - Support priorities across applications within the same queue YARN-1963
> — Can push as an alpha / beta feature per Sunil
> - YARN-1197 Support changing resources of an allocated container:
> — Can push as an alpha/beta feature per Wangda
> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> most of it anyways.
> — Can push as an alpha feature.
> - YARN Timeline Service v1.5 - YARN-4233
> — Should include per Li Lu
> - YARN Timeline Service Next generation: YARN-2928
> — Per analysis from Sangjin, drop this from 2.8.
>
> One open feature status
> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>
> Updated the Roadmap wiki with the same.
>
> Thanks
> +Vinod
>
> > On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
> >
> > I reviewed the current state of the YARN-2928 changes regarding its
> impact
> > if the timeline service v.2 is disabled. It does appear that there are a
> > lot of things that still do get created and enabled unconditionally
> > regardless of configuration. While this is understandable when we were
> > working to implement the feature, this clearly needs to be cleaned up so
> > that when disabled the timeline service v.2 doesn't impact other things.
> >
> > I filed a JIRA for that work:
> > https://issues.apache.org/jira/browse/YARN-4356
> >
> > We need to complete it before we can merge.
> >
> > Somewhat related is the status of the configuration and what it means in
> > various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
> > know there is an ongoing discussion regarding YARN-4183. We'll need to
> > reflect the outcome of that discussion.
> >
> > My overall impression of whether this can be done for 2.8 is that it
> looks
> > rather challenging given the suggested timeframe. We also need to
> complete
> > several major tasks before it is ready.
> >
> > Sangjin
> >
> >
> > On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
> >
> >>
> >> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
> >> vinodkv@hortonworks.com> wrote:
> >>
> >>> — YARN Timeline Service Next generation: YARN-2928: Lots of
> momentum,
> >>> but clearly a work in progress. Two options here
> >>> — If it is safe to ship it into 2.8 in a disable manner, we can
> >>> get the early code into trunk and all the way int o2.8.
> >>> — If it is not safe, it organically rolls over into 2.9
> >>>
> >>
> >> I'll review the changes on YARN-2928 to see what impact it has (if any)
> if
> >> the timeline service v.2 is disabled.
> >>
> >> Another condition for it to make 2.8 is whether the branch will be in a
> >> shape in a couple of weeks such that it adds value for folks that want
> to
> >> test it. Hopefully it will become clearer soon.
> >>
> >> Sangjin
> >>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Chris Nauroth <cn...@hortonworks.com>.
+1. Thanks, Vinod.
--Chris Nauroth
On 11/25/15, 1:45 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org> wrote:
>Okay, tx for this clarification Chris! I dug more into this and now
>realized the actual scope of this. Given the the limited nature of this
>feature (non-Namenode etc) and the WIP nature of the larger umbrella
>HADOOP-11744, we will ship the feature but I’ll stop calling this out as
>a notable feature.
>
>Thanks
>+Vinod
>
>
>> On Nov 25, 2015, at 12:04 PM, Chris Nauroth <cn...@hortonworks.com>
>>wrote:
>>
>> Hi Vinod,
>>
>> The HDFS-8155 work is complete in branch-2 already, so feel free to
>> include it in the roadmap.
>>
>> For those watching the thread that aren't familiar with HDFS-8155, I
>>want
>> to call out that it was a client-side change only. The WebHDFS client
>>is
>> capable of obtaining OAuth2 tokens and passing them along in its HTTP
>> requests. The NameNode and DataNode server side currently do not have
>>any
>> support for OAuth2, so overall, this feature is only useful in some very
>> unique deployment architectures right now. This is all discussed
>> explicitly in documentation committed with HDFS-8155, but I wanted to
>> prevent any mistaken assumptions for people only reading this thread.
>>
>> --Chris Nauroth
>>
>>
>>
>>
>> On 11/25/15, 11:08 AM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>> wrote:
>>
>>> This is the current state from the feedback I gathered.
>>> - Support priorities across applications within the same queue
>>>YARN-1963
>>> ‹ Can push as an alpha / beta feature per Sunil
>>> - YARN-1197 Support changing resources of an allocated container:
>>> ‹ Can push as an alpha/beta feature per Wangda
>>> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>>> most of it anyways.
>>> ‹ Can push as an alpha feature.
>>> - YARN Timeline Service v1.5 - YARN-4233
>>> ‹ Should include per Li Lu
>>> - YARN Timeline Service Next generation: YARN-2928
>>> ‹ Per analysis from Sangjin, drop this from 2.8.
>>>
>>> One open feature status
>>> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>>>
>>> Updated the Roadmap wiki with the same.
>>>
>>> Thanks
>>> +Vinod
>>>
>>>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>>
>>>> I reviewed the current state of the YARN-2928 changes regarding its
>>>> impact
>>>> if the timeline service v.2 is disabled. It does appear that there
>>>>are a
>>>> lot of things that still do get created and enabled unconditionally
>>>> regardless of configuration. While this is understandable when we were
>>>> working to implement the feature, this clearly needs to be cleaned up
>>>>so
>>>> that when disabled the timeline service v.2 doesn't impact other
>>>>things.
>>>>
>>>> I filed a JIRA for that work:
>>>> https://issues.apache.org/jira/browse/YARN-4356
>>>>
>>>> We need to complete it before we can merge.
>>>>
>>>> Somewhat related is the status of the configuration and what it means
>>>>in
>>>> various contexts (client/app-side vs. server-side, v.1 vs. v.2,
>>>>etc.). I
>>>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>>>> reflect the outcome of that discussion.
>>>>
>>>> My overall impression of whether this can be done for 2.8 is that it
>>>> looks
>>>> rather challenging given the suggested timeframe. We also need to
>>>> complete
>>>> several major tasks before it is ready.
>>>>
>>>> Sangjin
>>>>
>>>>
>>>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>>>
>>>>>
>>>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>>>> vinodkv@hortonworks.com> wrote:
>>>>>
>>>>>> ‹ YARN Timeline Service Next generation: YARN-2928: Lots of
>>>>>> momentum,
>>>>>> but clearly a work in progress. Two options here
>>>>>> ‹ If it is safe to ship it into 2.8 in a disable manner, we
>>>>>>can
>>>>>> get the early code into trunk and all the way int o2.8.
>>>>>> ‹ If it is not safe, it organically rolls over into 2.9
>>>>>>
>>>>>
>>>>> I'll review the changes on YARN-2928 to see what impact it has (if
>>>>> any) if
>>>>> the timeline service v.2 is disabled.
>>>>>
>>>>> Another condition for it to make 2.8 is whether the branch will be
>>>>>in a
>>>>> shape in a couple of weeks such that it adds value for folks that
>>>>>want
>>>>> to
>>>>> test it. Hopefully it will become clearer soon.
>>>>>
>>>>> Sangjin
>>>>>
>>>
>>
>>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Chris Nauroth <cn...@hortonworks.com>.
+1. Thanks, Vinod.
--Chris Nauroth
On 11/25/15, 1:45 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org> wrote:
>Okay, tx for this clarification Chris! I dug more into this and now
>realized the actual scope of this. Given the the limited nature of this
>feature (non-Namenode etc) and the WIP nature of the larger umbrella
>HADOOP-11744, we will ship the feature but I’ll stop calling this out as
>a notable feature.
>
>Thanks
>+Vinod
>
>
>> On Nov 25, 2015, at 12:04 PM, Chris Nauroth <cn...@hortonworks.com>
>>wrote:
>>
>> Hi Vinod,
>>
>> The HDFS-8155 work is complete in branch-2 already, so feel free to
>> include it in the roadmap.
>>
>> For those watching the thread that aren't familiar with HDFS-8155, I
>>want
>> to call out that it was a client-side change only. The WebHDFS client
>>is
>> capable of obtaining OAuth2 tokens and passing them along in its HTTP
>> requests. The NameNode and DataNode server side currently do not have
>>any
>> support for OAuth2, so overall, this feature is only useful in some very
>> unique deployment architectures right now. This is all discussed
>> explicitly in documentation committed with HDFS-8155, but I wanted to
>> prevent any mistaken assumptions for people only reading this thread.
>>
>> --Chris Nauroth
>>
>>
>>
>>
>> On 11/25/15, 11:08 AM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>> wrote:
>>
>>> This is the current state from the feedback I gathered.
>>> - Support priorities across applications within the same queue
>>>YARN-1963
>>> ‹ Can push as an alpha / beta feature per Sunil
>>> - YARN-1197 Support changing resources of an allocated container:
>>> ‹ Can push as an alpha/beta feature per Wangda
>>> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>>> most of it anyways.
>>> ‹ Can push as an alpha feature.
>>> - YARN Timeline Service v1.5 - YARN-4233
>>> ‹ Should include per Li Lu
>>> - YARN Timeline Service Next generation: YARN-2928
>>> ‹ Per analysis from Sangjin, drop this from 2.8.
>>>
>>> One open feature status
>>> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>>>
>>> Updated the Roadmap wiki with the same.
>>>
>>> Thanks
>>> +Vinod
>>>
>>>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>>
>>>> I reviewed the current state of the YARN-2928 changes regarding its
>>>> impact
>>>> if the timeline service v.2 is disabled. It does appear that there
>>>>are a
>>>> lot of things that still do get created and enabled unconditionally
>>>> regardless of configuration. While this is understandable when we were
>>>> working to implement the feature, this clearly needs to be cleaned up
>>>>so
>>>> that when disabled the timeline service v.2 doesn't impact other
>>>>things.
>>>>
>>>> I filed a JIRA for that work:
>>>> https://issues.apache.org/jira/browse/YARN-4356
>>>>
>>>> We need to complete it before we can merge.
>>>>
>>>> Somewhat related is the status of the configuration and what it means
>>>>in
>>>> various contexts (client/app-side vs. server-side, v.1 vs. v.2,
>>>>etc.). I
>>>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>>>> reflect the outcome of that discussion.
>>>>
>>>> My overall impression of whether this can be done for 2.8 is that it
>>>> looks
>>>> rather challenging given the suggested timeframe. We also need to
>>>> complete
>>>> several major tasks before it is ready.
>>>>
>>>> Sangjin
>>>>
>>>>
>>>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>>>
>>>>>
>>>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>>>> vinodkv@hortonworks.com> wrote:
>>>>>
>>>>>> ‹ YARN Timeline Service Next generation: YARN-2928: Lots of
>>>>>> momentum,
>>>>>> but clearly a work in progress. Two options here
>>>>>> ‹ If it is safe to ship it into 2.8 in a disable manner, we
>>>>>>can
>>>>>> get the early code into trunk and all the way int o2.8.
>>>>>> ‹ If it is not safe, it organically rolls over into 2.9
>>>>>>
>>>>>
>>>>> I'll review the changes on YARN-2928 to see what impact it has (if
>>>>> any) if
>>>>> the timeline service v.2 is disabled.
>>>>>
>>>>> Another condition for it to make 2.8 is whether the branch will be
>>>>>in a
>>>>> shape in a couple of weeks such that it adds value for folks that
>>>>>want
>>>>> to
>>>>> test it. Hopefully it will become clearer soon.
>>>>>
>>>>> Sangjin
>>>>>
>>>
>>
>>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Chris Nauroth <cn...@hortonworks.com>.
+1. Thanks, Vinod.
--Chris Nauroth
On 11/25/15, 1:45 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org> wrote:
>Okay, tx for this clarification Chris! I dug more into this and now
>realized the actual scope of this. Given the the limited nature of this
>feature (non-Namenode etc) and the WIP nature of the larger umbrella
>HADOOP-11744, we will ship the feature but I’ll stop calling this out as
>a notable feature.
>
>Thanks
>+Vinod
>
>
>> On Nov 25, 2015, at 12:04 PM, Chris Nauroth <cn...@hortonworks.com>
>>wrote:
>>
>> Hi Vinod,
>>
>> The HDFS-8155 work is complete in branch-2 already, so feel free to
>> include it in the roadmap.
>>
>> For those watching the thread that aren't familiar with HDFS-8155, I
>>want
>> to call out that it was a client-side change only. The WebHDFS client
>>is
>> capable of obtaining OAuth2 tokens and passing them along in its HTTP
>> requests. The NameNode and DataNode server side currently do not have
>>any
>> support for OAuth2, so overall, this feature is only useful in some very
>> unique deployment architectures right now. This is all discussed
>> explicitly in documentation committed with HDFS-8155, but I wanted to
>> prevent any mistaken assumptions for people only reading this thread.
>>
>> --Chris Nauroth
>>
>>
>>
>>
>> On 11/25/15, 11:08 AM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>> wrote:
>>
>>> This is the current state from the feedback I gathered.
>>> - Support priorities across applications within the same queue
>>>YARN-1963
>>> ‹ Can push as an alpha / beta feature per Sunil
>>> - YARN-1197 Support changing resources of an allocated container:
>>> ‹ Can push as an alpha/beta feature per Wangda
>>> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>>> most of it anyways.
>>> ‹ Can push as an alpha feature.
>>> - YARN Timeline Service v1.5 - YARN-4233
>>> ‹ Should include per Li Lu
>>> - YARN Timeline Service Next generation: YARN-2928
>>> ‹ Per analysis from Sangjin, drop this from 2.8.
>>>
>>> One open feature status
>>> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>>>
>>> Updated the Roadmap wiki with the same.
>>>
>>> Thanks
>>> +Vinod
>>>
>>>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>>
>>>> I reviewed the current state of the YARN-2928 changes regarding its
>>>> impact
>>>> if the timeline service v.2 is disabled. It does appear that there
>>>>are a
>>>> lot of things that still do get created and enabled unconditionally
>>>> regardless of configuration. While this is understandable when we were
>>>> working to implement the feature, this clearly needs to be cleaned up
>>>>so
>>>> that when disabled the timeline service v.2 doesn't impact other
>>>>things.
>>>>
>>>> I filed a JIRA for that work:
>>>> https://issues.apache.org/jira/browse/YARN-4356
>>>>
>>>> We need to complete it before we can merge.
>>>>
>>>> Somewhat related is the status of the configuration and what it means
>>>>in
>>>> various contexts (client/app-side vs. server-side, v.1 vs. v.2,
>>>>etc.). I
>>>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>>>> reflect the outcome of that discussion.
>>>>
>>>> My overall impression of whether this can be done for 2.8 is that it
>>>> looks
>>>> rather challenging given the suggested timeframe. We also need to
>>>> complete
>>>> several major tasks before it is ready.
>>>>
>>>> Sangjin
>>>>
>>>>
>>>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>>>
>>>>>
>>>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>>>> vinodkv@hortonworks.com> wrote:
>>>>>
>>>>>> ‹ YARN Timeline Service Next generation: YARN-2928: Lots of
>>>>>> momentum,
>>>>>> but clearly a work in progress. Two options here
>>>>>> ‹ If it is safe to ship it into 2.8 in a disable manner, we
>>>>>>can
>>>>>> get the early code into trunk and all the way int o2.8.
>>>>>> ‹ If it is not safe, it organically rolls over into 2.9
>>>>>>
>>>>>
>>>>> I'll review the changes on YARN-2928 to see what impact it has (if
>>>>> any) if
>>>>> the timeline service v.2 is disabled.
>>>>>
>>>>> Another condition for it to make 2.8 is whether the branch will be
>>>>>in a
>>>>> shape in a couple of weeks such that it adds value for folks that
>>>>>want
>>>>> to
>>>>> test it. Hopefully it will become clearer soon.
>>>>>
>>>>> Sangjin
>>>>>
>>>
>>
>>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Chris Nauroth <cn...@hortonworks.com>.
+1. Thanks, Vinod.
--Chris Nauroth
On 11/25/15, 1:45 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org> wrote:
>Okay, tx for this clarification Chris! I dug more into this and now
>realized the actual scope of this. Given the the limited nature of this
>feature (non-Namenode etc) and the WIP nature of the larger umbrella
>HADOOP-11744, we will ship the feature but I’ll stop calling this out as
>a notable feature.
>
>Thanks
>+Vinod
>
>
>> On Nov 25, 2015, at 12:04 PM, Chris Nauroth <cn...@hortonworks.com>
>>wrote:
>>
>> Hi Vinod,
>>
>> The HDFS-8155 work is complete in branch-2 already, so feel free to
>> include it in the roadmap.
>>
>> For those watching the thread that aren't familiar with HDFS-8155, I
>>want
>> to call out that it was a client-side change only. The WebHDFS client
>>is
>> capable of obtaining OAuth2 tokens and passing them along in its HTTP
>> requests. The NameNode and DataNode server side currently do not have
>>any
>> support for OAuth2, so overall, this feature is only useful in some very
>> unique deployment architectures right now. This is all discussed
>> explicitly in documentation committed with HDFS-8155, but I wanted to
>> prevent any mistaken assumptions for people only reading this thread.
>>
>> --Chris Nauroth
>>
>>
>>
>>
>> On 11/25/15, 11:08 AM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>> wrote:
>>
>>> This is the current state from the feedback I gathered.
>>> - Support priorities across applications within the same queue
>>>YARN-1963
>>> ‹ Can push as an alpha / beta feature per Sunil
>>> - YARN-1197 Support changing resources of an allocated container:
>>> ‹ Can push as an alpha/beta feature per Wangda
>>> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>>> most of it anyways.
>>> ‹ Can push as an alpha feature.
>>> - YARN Timeline Service v1.5 - YARN-4233
>>> ‹ Should include per Li Lu
>>> - YARN Timeline Service Next generation: YARN-2928
>>> ‹ Per analysis from Sangjin, drop this from 2.8.
>>>
>>> One open feature status
>>> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>>>
>>> Updated the Roadmap wiki with the same.
>>>
>>> Thanks
>>> +Vinod
>>>
>>>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>>
>>>> I reviewed the current state of the YARN-2928 changes regarding its
>>>> impact
>>>> if the timeline service v.2 is disabled. It does appear that there
>>>>are a
>>>> lot of things that still do get created and enabled unconditionally
>>>> regardless of configuration. While this is understandable when we were
>>>> working to implement the feature, this clearly needs to be cleaned up
>>>>so
>>>> that when disabled the timeline service v.2 doesn't impact other
>>>>things.
>>>>
>>>> I filed a JIRA for that work:
>>>> https://issues.apache.org/jira/browse/YARN-4356
>>>>
>>>> We need to complete it before we can merge.
>>>>
>>>> Somewhat related is the status of the configuration and what it means
>>>>in
>>>> various contexts (client/app-side vs. server-side, v.1 vs. v.2,
>>>>etc.). I
>>>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>>>> reflect the outcome of that discussion.
>>>>
>>>> My overall impression of whether this can be done for 2.8 is that it
>>>> looks
>>>> rather challenging given the suggested timeframe. We also need to
>>>> complete
>>>> several major tasks before it is ready.
>>>>
>>>> Sangjin
>>>>
>>>>
>>>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>>>
>>>>>
>>>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>>>> vinodkv@hortonworks.com> wrote:
>>>>>
>>>>>> ‹ YARN Timeline Service Next generation: YARN-2928: Lots of
>>>>>> momentum,
>>>>>> but clearly a work in progress. Two options here
>>>>>> ‹ If it is safe to ship it into 2.8 in a disable manner, we
>>>>>>can
>>>>>> get the early code into trunk and all the way int o2.8.
>>>>>> ‹ If it is not safe, it organically rolls over into 2.9
>>>>>>
>>>>>
>>>>> I'll review the changes on YARN-2928 to see what impact it has (if
>>>>> any) if
>>>>> the timeline service v.2 is disabled.
>>>>>
>>>>> Another condition for it to make 2.8 is whether the branch will be
>>>>>in a
>>>>> shape in a couple of weeks such that it adds value for folks that
>>>>>want
>>>>> to
>>>>> test it. Hopefully it will become clearer soon.
>>>>>
>>>>> Sangjin
>>>>>
>>>
>>
>>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Okay, tx for this clarification Chris! I dug more into this and now realized the actual scope of this. Given the the limited nature of this feature (non-Namenode etc) and the WIP nature of the larger umbrella HADOOP-11744, we will ship the feature but I’ll stop calling this out as a notable feature.
Thanks
+Vinod
> On Nov 25, 2015, at 12:04 PM, Chris Nauroth <cn...@hortonworks.com> wrote:
>
> Hi Vinod,
>
> The HDFS-8155 work is complete in branch-2 already, so feel free to
> include it in the roadmap.
>
> For those watching the thread that aren't familiar with HDFS-8155, I want
> to call out that it was a client-side change only. The WebHDFS client is
> capable of obtaining OAuth2 tokens and passing them along in its HTTP
> requests. The NameNode and DataNode server side currently do not have any
> support for OAuth2, so overall, this feature is only useful in some very
> unique deployment architectures right now. This is all discussed
> explicitly in documentation committed with HDFS-8155, but I wanted to
> prevent any mistaken assumptions for people only reading this thread.
>
> --Chris Nauroth
>
>
>
>
> On 11/25/15, 11:08 AM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
> wrote:
>
>> This is the current state from the feedback I gathered.
>> - Support priorities across applications within the same queue YARN-1963
>> ‹ Can push as an alpha / beta feature per Sunil
>> - YARN-1197 Support changing resources of an allocated container:
>> ‹ Can push as an alpha/beta feature per Wangda
>> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>> most of it anyways.
>> ‹ Can push as an alpha feature.
>> - YARN Timeline Service v1.5 - YARN-4233
>> ‹ Should include per Li Lu
>> - YARN Timeline Service Next generation: YARN-2928
>> ‹ Per analysis from Sangjin, drop this from 2.8.
>>
>> One open feature status
>> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>>
>> Updated the Roadmap wiki with the same.
>>
>> Thanks
>> +Vinod
>>
>>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>
>>> I reviewed the current state of the YARN-2928 changes regarding its
>>> impact
>>> if the timeline service v.2 is disabled. It does appear that there are a
>>> lot of things that still do get created and enabled unconditionally
>>> regardless of configuration. While this is understandable when we were
>>> working to implement the feature, this clearly needs to be cleaned up so
>>> that when disabled the timeline service v.2 doesn't impact other things.
>>>
>>> I filed a JIRA for that work:
>>> https://issues.apache.org/jira/browse/YARN-4356
>>>
>>> We need to complete it before we can merge.
>>>
>>> Somewhat related is the status of the configuration and what it means in
>>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
>>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>>> reflect the outcome of that discussion.
>>>
>>> My overall impression of whether this can be done for 2.8 is that it
>>> looks
>>> rather challenging given the suggested timeframe. We also need to
>>> complete
>>> several major tasks before it is ready.
>>>
>>> Sangjin
>>>
>>>
>>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>>
>>>>
>>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>> ‹ YARN Timeline Service Next generation: YARN-2928: Lots of
>>>>> momentum,
>>>>> but clearly a work in progress. Two options here
>>>>> ‹ If it is safe to ship it into 2.8 in a disable manner, we can
>>>>> get the early code into trunk and all the way int o2.8.
>>>>> ‹ If it is not safe, it organically rolls over into 2.9
>>>>>
>>>>
>>>> I'll review the changes on YARN-2928 to see what impact it has (if
>>>> any) if
>>>> the timeline service v.2 is disabled.
>>>>
>>>> Another condition for it to make 2.8 is whether the branch will be in a
>>>> shape in a couple of weeks such that it adds value for folks that want
>>>> to
>>>> test it. Hopefully it will become clearer soon.
>>>>
>>>> Sangjin
>>>>
>>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Okay, tx for this clarification Chris! I dug more into this and now realized the actual scope of this. Given the the limited nature of this feature (non-Namenode etc) and the WIP nature of the larger umbrella HADOOP-11744, we will ship the feature but I’ll stop calling this out as a notable feature.
Thanks
+Vinod
> On Nov 25, 2015, at 12:04 PM, Chris Nauroth <cn...@hortonworks.com> wrote:
>
> Hi Vinod,
>
> The HDFS-8155 work is complete in branch-2 already, so feel free to
> include it in the roadmap.
>
> For those watching the thread that aren't familiar with HDFS-8155, I want
> to call out that it was a client-side change only. The WebHDFS client is
> capable of obtaining OAuth2 tokens and passing them along in its HTTP
> requests. The NameNode and DataNode server side currently do not have any
> support for OAuth2, so overall, this feature is only useful in some very
> unique deployment architectures right now. This is all discussed
> explicitly in documentation committed with HDFS-8155, but I wanted to
> prevent any mistaken assumptions for people only reading this thread.
>
> --Chris Nauroth
>
>
>
>
> On 11/25/15, 11:08 AM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
> wrote:
>
>> This is the current state from the feedback I gathered.
>> - Support priorities across applications within the same queue YARN-1963
>> ‹ Can push as an alpha / beta feature per Sunil
>> - YARN-1197 Support changing resources of an allocated container:
>> ‹ Can push as an alpha/beta feature per Wangda
>> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>> most of it anyways.
>> ‹ Can push as an alpha feature.
>> - YARN Timeline Service v1.5 - YARN-4233
>> ‹ Should include per Li Lu
>> - YARN Timeline Service Next generation: YARN-2928
>> ‹ Per analysis from Sangjin, drop this from 2.8.
>>
>> One open feature status
>> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>>
>> Updated the Roadmap wiki with the same.
>>
>> Thanks
>> +Vinod
>>
>>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>
>>> I reviewed the current state of the YARN-2928 changes regarding its
>>> impact
>>> if the timeline service v.2 is disabled. It does appear that there are a
>>> lot of things that still do get created and enabled unconditionally
>>> regardless of configuration. While this is understandable when we were
>>> working to implement the feature, this clearly needs to be cleaned up so
>>> that when disabled the timeline service v.2 doesn't impact other things.
>>>
>>> I filed a JIRA for that work:
>>> https://issues.apache.org/jira/browse/YARN-4356
>>>
>>> We need to complete it before we can merge.
>>>
>>> Somewhat related is the status of the configuration and what it means in
>>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
>>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>>> reflect the outcome of that discussion.
>>>
>>> My overall impression of whether this can be done for 2.8 is that it
>>> looks
>>> rather challenging given the suggested timeframe. We also need to
>>> complete
>>> several major tasks before it is ready.
>>>
>>> Sangjin
>>>
>>>
>>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>>
>>>>
>>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>> ‹ YARN Timeline Service Next generation: YARN-2928: Lots of
>>>>> momentum,
>>>>> but clearly a work in progress. Two options here
>>>>> ‹ If it is safe to ship it into 2.8 in a disable manner, we can
>>>>> get the early code into trunk and all the way int o2.8.
>>>>> ‹ If it is not safe, it organically rolls over into 2.9
>>>>>
>>>>
>>>> I'll review the changes on YARN-2928 to see what impact it has (if
>>>> any) if
>>>> the timeline service v.2 is disabled.
>>>>
>>>> Another condition for it to make 2.8 is whether the branch will be in a
>>>> shape in a couple of weeks such that it adds value for folks that want
>>>> to
>>>> test it. Hopefully it will become clearer soon.
>>>>
>>>> Sangjin
>>>>
>>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Okay, tx for this clarification Chris! I dug more into this and now realized the actual scope of this. Given the the limited nature of this feature (non-Namenode etc) and the WIP nature of the larger umbrella HADOOP-11744, we will ship the feature but I’ll stop calling this out as a notable feature.
Thanks
+Vinod
> On Nov 25, 2015, at 12:04 PM, Chris Nauroth <cn...@hortonworks.com> wrote:
>
> Hi Vinod,
>
> The HDFS-8155 work is complete in branch-2 already, so feel free to
> include it in the roadmap.
>
> For those watching the thread that aren't familiar with HDFS-8155, I want
> to call out that it was a client-side change only. The WebHDFS client is
> capable of obtaining OAuth2 tokens and passing them along in its HTTP
> requests. The NameNode and DataNode server side currently do not have any
> support for OAuth2, so overall, this feature is only useful in some very
> unique deployment architectures right now. This is all discussed
> explicitly in documentation committed with HDFS-8155, but I wanted to
> prevent any mistaken assumptions for people only reading this thread.
>
> --Chris Nauroth
>
>
>
>
> On 11/25/15, 11:08 AM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
> wrote:
>
>> This is the current state from the feedback I gathered.
>> - Support priorities across applications within the same queue YARN-1963
>> ‹ Can push as an alpha / beta feature per Sunil
>> - YARN-1197 Support changing resources of an allocated container:
>> ‹ Can push as an alpha/beta feature per Wangda
>> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>> most of it anyways.
>> ‹ Can push as an alpha feature.
>> - YARN Timeline Service v1.5 - YARN-4233
>> ‹ Should include per Li Lu
>> - YARN Timeline Service Next generation: YARN-2928
>> ‹ Per analysis from Sangjin, drop this from 2.8.
>>
>> One open feature status
>> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>>
>> Updated the Roadmap wiki with the same.
>>
>> Thanks
>> +Vinod
>>
>>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>
>>> I reviewed the current state of the YARN-2928 changes regarding its
>>> impact
>>> if the timeline service v.2 is disabled. It does appear that there are a
>>> lot of things that still do get created and enabled unconditionally
>>> regardless of configuration. While this is understandable when we were
>>> working to implement the feature, this clearly needs to be cleaned up so
>>> that when disabled the timeline service v.2 doesn't impact other things.
>>>
>>> I filed a JIRA for that work:
>>> https://issues.apache.org/jira/browse/YARN-4356
>>>
>>> We need to complete it before we can merge.
>>>
>>> Somewhat related is the status of the configuration and what it means in
>>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
>>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>>> reflect the outcome of that discussion.
>>>
>>> My overall impression of whether this can be done for 2.8 is that it
>>> looks
>>> rather challenging given the suggested timeframe. We also need to
>>> complete
>>> several major tasks before it is ready.
>>>
>>> Sangjin
>>>
>>>
>>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>>
>>>>
>>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>> ‹ YARN Timeline Service Next generation: YARN-2928: Lots of
>>>>> momentum,
>>>>> but clearly a work in progress. Two options here
>>>>> ‹ If it is safe to ship it into 2.8 in a disable manner, we can
>>>>> get the early code into trunk and all the way int o2.8.
>>>>> ‹ If it is not safe, it organically rolls over into 2.9
>>>>>
>>>>
>>>> I'll review the changes on YARN-2928 to see what impact it has (if
>>>> any) if
>>>> the timeline service v.2 is disabled.
>>>>
>>>> Another condition for it to make 2.8 is whether the branch will be in a
>>>> shape in a couple of weeks such that it adds value for folks that want
>>>> to
>>>> test it. Hopefully it will become clearer soon.
>>>>
>>>> Sangjin
>>>>
>>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Okay, tx for this clarification Chris! I dug more into this and now realized the actual scope of this. Given the the limited nature of this feature (non-Namenode etc) and the WIP nature of the larger umbrella HADOOP-11744, we will ship the feature but I’ll stop calling this out as a notable feature.
Thanks
+Vinod
> On Nov 25, 2015, at 12:04 PM, Chris Nauroth <cn...@hortonworks.com> wrote:
>
> Hi Vinod,
>
> The HDFS-8155 work is complete in branch-2 already, so feel free to
> include it in the roadmap.
>
> For those watching the thread that aren't familiar with HDFS-8155, I want
> to call out that it was a client-side change only. The WebHDFS client is
> capable of obtaining OAuth2 tokens and passing them along in its HTTP
> requests. The NameNode and DataNode server side currently do not have any
> support for OAuth2, so overall, this feature is only useful in some very
> unique deployment architectures right now. This is all discussed
> explicitly in documentation committed with HDFS-8155, but I wanted to
> prevent any mistaken assumptions for people only reading this thread.
>
> --Chris Nauroth
>
>
>
>
> On 11/25/15, 11:08 AM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
> wrote:
>
>> This is the current state from the feedback I gathered.
>> - Support priorities across applications within the same queue YARN-1963
>> ‹ Can push as an alpha / beta feature per Sunil
>> - YARN-1197 Support changing resources of an allocated container:
>> ‹ Can push as an alpha/beta feature per Wangda
>> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>> most of it anyways.
>> ‹ Can push as an alpha feature.
>> - YARN Timeline Service v1.5 - YARN-4233
>> ‹ Should include per Li Lu
>> - YARN Timeline Service Next generation: YARN-2928
>> ‹ Per analysis from Sangjin, drop this from 2.8.
>>
>> One open feature status
>> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>>
>> Updated the Roadmap wiki with the same.
>>
>> Thanks
>> +Vinod
>>
>>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>
>>> I reviewed the current state of the YARN-2928 changes regarding its
>>> impact
>>> if the timeline service v.2 is disabled. It does appear that there are a
>>> lot of things that still do get created and enabled unconditionally
>>> regardless of configuration. While this is understandable when we were
>>> working to implement the feature, this clearly needs to be cleaned up so
>>> that when disabled the timeline service v.2 doesn't impact other things.
>>>
>>> I filed a JIRA for that work:
>>> https://issues.apache.org/jira/browse/YARN-4356
>>>
>>> We need to complete it before we can merge.
>>>
>>> Somewhat related is the status of the configuration and what it means in
>>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
>>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>>> reflect the outcome of that discussion.
>>>
>>> My overall impression of whether this can be done for 2.8 is that it
>>> looks
>>> rather challenging given the suggested timeframe. We also need to
>>> complete
>>> several major tasks before it is ready.
>>>
>>> Sangjin
>>>
>>>
>>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>>
>>>>
>>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>> ‹ YARN Timeline Service Next generation: YARN-2928: Lots of
>>>>> momentum,
>>>>> but clearly a work in progress. Two options here
>>>>> ‹ If it is safe to ship it into 2.8 in a disable manner, we can
>>>>> get the early code into trunk and all the way int o2.8.
>>>>> ‹ If it is not safe, it organically rolls over into 2.9
>>>>>
>>>>
>>>> I'll review the changes on YARN-2928 to see what impact it has (if
>>>> any) if
>>>> the timeline service v.2 is disabled.
>>>>
>>>> Another condition for it to make 2.8 is whether the branch will be in a
>>>> shape in a couple of weeks such that it adds value for folks that want
>>>> to
>>>> test it. Hopefully it will become clearer soon.
>>>>
>>>> Sangjin
>>>>
>>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hi Vinod,
The HDFS-8155 work is complete in branch-2 already, so feel free to
include it in the roadmap.
For those watching the thread that aren't familiar with HDFS-8155, I want
to call out that it was a client-side change only. The WebHDFS client is
capable of obtaining OAuth2 tokens and passing them along in its HTTP
requests. The NameNode and DataNode server side currently do not have any
support for OAuth2, so overall, this feature is only useful in some very
unique deployment architectures right now. This is all discussed
explicitly in documentation committed with HDFS-8155, but I wanted to
prevent any mistaken assumptions for people only reading this thread.
--Chris Nauroth
On 11/25/15, 11:08 AM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
wrote:
>This is the current state from the feedback I gathered.
> - Support priorities across applications within the same queue YARN-1963
> ‹ Can push as an alpha / beta feature per Sunil
> - YARN-1197 Support changing resources of an allocated container:
> ‹ Can push as an alpha/beta feature per Wangda
> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>most of it anyways.
> ‹ Can push as an alpha feature.
> - YARN Timeline Service v1.5 - YARN-4233
> ‹ Should include per Li Lu
> - YARN Timeline Service Next generation: YARN-2928
> ‹ Per analysis from Sangjin, drop this from 2.8.
>
>One open feature status
> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>
>Updated the Roadmap wiki with the same.
>
>Thanks
>+Vinod
>
>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>
>> I reviewed the current state of the YARN-2928 changes regarding its
>>impact
>> if the timeline service v.2 is disabled. It does appear that there are a
>> lot of things that still do get created and enabled unconditionally
>> regardless of configuration. While this is understandable when we were
>> working to implement the feature, this clearly needs to be cleaned up so
>> that when disabled the timeline service v.2 doesn't impact other things.
>>
>> I filed a JIRA for that work:
>> https://issues.apache.org/jira/browse/YARN-4356
>>
>> We need to complete it before we can merge.
>>
>> Somewhat related is the status of the configuration and what it means in
>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>> reflect the outcome of that discussion.
>>
>> My overall impression of whether this can be done for 2.8 is that it
>>looks
>> rather challenging given the suggested timeframe. We also need to
>>complete
>> several major tasks before it is ready.
>>
>> Sangjin
>>
>>
>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>
>>>
>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>> vinodkv@hortonworks.com> wrote:
>>>
>>>> ‹ YARN Timeline Service Next generation: YARN-2928: Lots of
>>>>momentum,
>>>> but clearly a work in progress. Two options here
>>>> ‹ If it is safe to ship it into 2.8 in a disable manner, we can
>>>> get the early code into trunk and all the way int o2.8.
>>>> ‹ If it is not safe, it organically rolls over into 2.9
>>>>
>>>
>>> I'll review the changes on YARN-2928 to see what impact it has (if
>>>any) if
>>> the timeline service v.2 is disabled.
>>>
>>> Another condition for it to make 2.8 is whether the branch will be in a
>>> shape in a couple of weeks such that it adds value for folks that want
>>>to
>>> test it. Hopefully it will become clearer soon.
>>>
>>> Sangjin
>>>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hi Vinod,
The HDFS-8155 work is complete in branch-2 already, so feel free to
include it in the roadmap.
For those watching the thread that aren't familiar with HDFS-8155, I want
to call out that it was a client-side change only. The WebHDFS client is
capable of obtaining OAuth2 tokens and passing them along in its HTTP
requests. The NameNode and DataNode server side currently do not have any
support for OAuth2, so overall, this feature is only useful in some very
unique deployment architectures right now. This is all discussed
explicitly in documentation committed with HDFS-8155, but I wanted to
prevent any mistaken assumptions for people only reading this thread.
--Chris Nauroth
On 11/25/15, 11:08 AM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
wrote:
>This is the current state from the feedback I gathered.
> - Support priorities across applications within the same queue YARN-1963
> ‹ Can push as an alpha / beta feature per Sunil
> - YARN-1197 Support changing resources of an allocated container:
> ‹ Can push as an alpha/beta feature per Wangda
> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>most of it anyways.
> ‹ Can push as an alpha feature.
> - YARN Timeline Service v1.5 - YARN-4233
> ‹ Should include per Li Lu
> - YARN Timeline Service Next generation: YARN-2928
> ‹ Per analysis from Sangjin, drop this from 2.8.
>
>One open feature status
> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>
>Updated the Roadmap wiki with the same.
>
>Thanks
>+Vinod
>
>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>
>> I reviewed the current state of the YARN-2928 changes regarding its
>>impact
>> if the timeline service v.2 is disabled. It does appear that there are a
>> lot of things that still do get created and enabled unconditionally
>> regardless of configuration. While this is understandable when we were
>> working to implement the feature, this clearly needs to be cleaned up so
>> that when disabled the timeline service v.2 doesn't impact other things.
>>
>> I filed a JIRA for that work:
>> https://issues.apache.org/jira/browse/YARN-4356
>>
>> We need to complete it before we can merge.
>>
>> Somewhat related is the status of the configuration and what it means in
>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>> reflect the outcome of that discussion.
>>
>> My overall impression of whether this can be done for 2.8 is that it
>>looks
>> rather challenging given the suggested timeframe. We also need to
>>complete
>> several major tasks before it is ready.
>>
>> Sangjin
>>
>>
>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>
>>>
>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>> vinodkv@hortonworks.com> wrote:
>>>
>>>> ‹ YARN Timeline Service Next generation: YARN-2928: Lots of
>>>>momentum,
>>>> but clearly a work in progress. Two options here
>>>> ‹ If it is safe to ship it into 2.8 in a disable manner, we can
>>>> get the early code into trunk and all the way int o2.8.
>>>> ‹ If it is not safe, it organically rolls over into 2.9
>>>>
>>>
>>> I'll review the changes on YARN-2928 to see what impact it has (if
>>>any) if
>>> the timeline service v.2 is disabled.
>>>
>>> Another condition for it to make 2.8 is whether the branch will be in a
>>> shape in a couple of weeks such that it adds value for folks that want
>>>to
>>> test it. Hopefully it will become clearer soon.
>>>
>>> Sangjin
>>>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hi Vinod,
The HDFS-8155 work is complete in branch-2 already, so feel free to
include it in the roadmap.
For those watching the thread that aren't familiar with HDFS-8155, I want
to call out that it was a client-side change only. The WebHDFS client is
capable of obtaining OAuth2 tokens and passing them along in its HTTP
requests. The NameNode and DataNode server side currently do not have any
support for OAuth2, so overall, this feature is only useful in some very
unique deployment architectures right now. This is all discussed
explicitly in documentation committed with HDFS-8155, but I wanted to
prevent any mistaken assumptions for people only reading this thread.
--Chris Nauroth
On 11/25/15, 11:08 AM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
wrote:
>This is the current state from the feedback I gathered.
> - Support priorities across applications within the same queue YARN-1963
> ‹ Can push as an alpha / beta feature per Sunil
> - YARN-1197 Support changing resources of an allocated container:
> ‹ Can push as an alpha/beta feature per Wangda
> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>most of it anyways.
> ‹ Can push as an alpha feature.
> - YARN Timeline Service v1.5 - YARN-4233
> ‹ Should include per Li Lu
> - YARN Timeline Service Next generation: YARN-2928
> ‹ Per analysis from Sangjin, drop this from 2.8.
>
>One open feature status
> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>
>Updated the Roadmap wiki with the same.
>
>Thanks
>+Vinod
>
>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>
>> I reviewed the current state of the YARN-2928 changes regarding its
>>impact
>> if the timeline service v.2 is disabled. It does appear that there are a
>> lot of things that still do get created and enabled unconditionally
>> regardless of configuration. While this is understandable when we were
>> working to implement the feature, this clearly needs to be cleaned up so
>> that when disabled the timeline service v.2 doesn't impact other things.
>>
>> I filed a JIRA for that work:
>> https://issues.apache.org/jira/browse/YARN-4356
>>
>> We need to complete it before we can merge.
>>
>> Somewhat related is the status of the configuration and what it means in
>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>> reflect the outcome of that discussion.
>>
>> My overall impression of whether this can be done for 2.8 is that it
>>looks
>> rather challenging given the suggested timeframe. We also need to
>>complete
>> several major tasks before it is ready.
>>
>> Sangjin
>>
>>
>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>
>>>
>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>> vinodkv@hortonworks.com> wrote:
>>>
>>>> ‹ YARN Timeline Service Next generation: YARN-2928: Lots of
>>>>momentum,
>>>> but clearly a work in progress. Two options here
>>>> ‹ If it is safe to ship it into 2.8 in a disable manner, we can
>>>> get the early code into trunk and all the way int o2.8.
>>>> ‹ If it is not safe, it organically rolls over into 2.9
>>>>
>>>
>>> I'll review the changes on YARN-2928 to see what impact it has (if
>>>any) if
>>> the timeline service v.2 is disabled.
>>>
>>> Another condition for it to make 2.8 is whether the branch will be in a
>>> shape in a couple of weeks such that it adds value for folks that want
>>>to
>>> test it. Hopefully it will become clearer soon.
>>>
>>> Sangjin
>>>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Chris Nauroth <cn...@hortonworks.com>.
Hi Vinod,
The HDFS-8155 work is complete in branch-2 already, so feel free to
include it in the roadmap.
For those watching the thread that aren't familiar with HDFS-8155, I want
to call out that it was a client-side change only. The WebHDFS client is
capable of obtaining OAuth2 tokens and passing them along in its HTTP
requests. The NameNode and DataNode server side currently do not have any
support for OAuth2, so overall, this feature is only useful in some very
unique deployment architectures right now. This is all discussed
explicitly in documentation committed with HDFS-8155, but I wanted to
prevent any mistaken assumptions for people only reading this thread.
--Chris Nauroth
On 11/25/15, 11:08 AM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
wrote:
>This is the current state from the feedback I gathered.
> - Support priorities across applications within the same queue YARN-1963
> ‹ Can push as an alpha / beta feature per Sunil
> - YARN-1197 Support changing resources of an allocated container:
> ‹ Can push as an alpha/beta feature per Wangda
> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
>most of it anyways.
> ‹ Can push as an alpha feature.
> - YARN Timeline Service v1.5 - YARN-4233
> ‹ Should include per Li Lu
> - YARN Timeline Service Next generation: YARN-2928
> ‹ Per analysis from Sangjin, drop this from 2.8.
>
>One open feature status
> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>
>Updated the Roadmap wiki with the same.
>
>Thanks
>+Vinod
>
>> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>>
>> I reviewed the current state of the YARN-2928 changes regarding its
>>impact
>> if the timeline service v.2 is disabled. It does appear that there are a
>> lot of things that still do get created and enabled unconditionally
>> regardless of configuration. While this is understandable when we were
>> working to implement the feature, this clearly needs to be cleaned up so
>> that when disabled the timeline service v.2 doesn't impact other things.
>>
>> I filed a JIRA for that work:
>> https://issues.apache.org/jira/browse/YARN-4356
>>
>> We need to complete it before we can merge.
>>
>> Somewhat related is the status of the configuration and what it means in
>> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
>> know there is an ongoing discussion regarding YARN-4183. We'll need to
>> reflect the outcome of that discussion.
>>
>> My overall impression of whether this can be done for 2.8 is that it
>>looks
>> rather challenging given the suggested timeframe. We also need to
>>complete
>> several major tasks before it is ready.
>>
>> Sangjin
>>
>>
>> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>>
>>>
>>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>>> vinodkv@hortonworks.com> wrote:
>>>
>>>> ‹ YARN Timeline Service Next generation: YARN-2928: Lots of
>>>>momentum,
>>>> but clearly a work in progress. Two options here
>>>> ‹ If it is safe to ship it into 2.8 in a disable manner, we can
>>>> get the early code into trunk and all the way int o2.8.
>>>> ‹ If it is not safe, it organically rolls over into 2.9
>>>>
>>>
>>> I'll review the changes on YARN-2928 to see what impact it has (if
>>>any) if
>>> the timeline service v.2 is disabled.
>>>
>>> Another condition for it to make 2.8 is whether the branch will be in a
>>> shape in a couple of weeks such that it adds value for folks that want
>>>to
>>> test it. Hopefully it will become clearer soon.
>>>
>>> Sangjin
>>>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Andrew Wang <an...@cloudera.com>.
Hey Vinod,
I'm fine with the idea of alpha/beta marking in the abstract, but had a
question: do we define these terms in our compatibility policy or
elsewhere? I think it's commonly understood among us developers (alpha
means not fully tested and API unstable, beta means it's not fully tested
but is API stable), but it'd be good to have it written down.
Also I think we've only done alpha/beta tagging at the release-level
previously which is a simpler story to tell users. So it's important for
this release that alpha features set their interface stability annotations
to "evolving". There isn't a corresponding annotation for "interface
quality", but IMO that's overkill.
Thanks,
Andrew
On Wed, Nov 25, 2015 at 11:08 AM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:
> This is the current state from the feedback I gathered.
> - Support priorities across applications within the same queue YARN-1963
> — Can push as an alpha / beta feature per Sunil
> - YARN-1197 Support changing resources of an allocated container:
> — Can push as an alpha/beta feature per Wangda
> - YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> most of it anyways.
> — Can push as an alpha feature.
> - YARN Timeline Service v1.5 - YARN-4233
> — Should include per Li Lu
> - YARN Timeline Service Next generation: YARN-2928
> — Per analysis from Sangjin, drop this from 2.8.
>
> One open feature status
> - HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
>
> Updated the Roadmap wiki with the same.
>
> Thanks
> +Vinod
>
> > On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
> >
> > I reviewed the current state of the YARN-2928 changes regarding its
> impact
> > if the timeline service v.2 is disabled. It does appear that there are a
> > lot of things that still do get created and enabled unconditionally
> > regardless of configuration. While this is understandable when we were
> > working to implement the feature, this clearly needs to be cleaned up so
> > that when disabled the timeline service v.2 doesn't impact other things.
> >
> > I filed a JIRA for that work:
> > https://issues.apache.org/jira/browse/YARN-4356
> >
> > We need to complete it before we can merge.
> >
> > Somewhat related is the status of the configuration and what it means in
> > various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
> > know there is an ongoing discussion regarding YARN-4183. We'll need to
> > reflect the outcome of that discussion.
> >
> > My overall impression of whether this can be done for 2.8 is that it
> looks
> > rather challenging given the suggested timeframe. We also need to
> complete
> > several major tasks before it is ready.
> >
> > Sangjin
> >
> >
> > On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
> >
> >>
> >> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
> >> vinodkv@hortonworks.com> wrote:
> >>
> >>> — YARN Timeline Service Next generation: YARN-2928: Lots of
> momentum,
> >>> but clearly a work in progress. Two options here
> >>> — If it is safe to ship it into 2.8 in a disable manner, we can
> >>> get the early code into trunk and all the way int o2.8.
> >>> — If it is not safe, it organically rolls over into 2.9
> >>>
> >>
> >> I'll review the changes on YARN-2928 to see what impact it has (if any)
> if
> >> the timeline service v.2 is disabled.
> >>
> >> Another condition for it to make 2.8 is whether the branch will be in a
> >> shape in a couple of weeks such that it adds value for folks that want
> to
> >> test it. Hopefully it will become clearer soon.
> >>
> >> Sangjin
> >>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
This is the current state from the feedback I gathered.
- Support priorities across applications within the same queue YARN-1963
— Can push as an alpha / beta feature per Sunil
- YARN-1197 Support changing resources of an allocated container:
— Can push as an alpha/beta feature per Wangda
- YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well most of it anyways.
— Can push as an alpha feature.
- YARN Timeline Service v1.5 - YARN-4233
— Should include per Li Lu
- YARN Timeline Service Next generation: YARN-2928
— Per analysis from Sangjin, drop this from 2.8.
One open feature status
- HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
Updated the Roadmap wiki with the same.
Thanks
+Vinod
> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>
> I reviewed the current state of the YARN-2928 changes regarding its impact
> if the timeline service v.2 is disabled. It does appear that there are a
> lot of things that still do get created and enabled unconditionally
> regardless of configuration. While this is understandable when we were
> working to implement the feature, this clearly needs to be cleaned up so
> that when disabled the timeline service v.2 doesn't impact other things.
>
> I filed a JIRA for that work:
> https://issues.apache.org/jira/browse/YARN-4356
>
> We need to complete it before we can merge.
>
> Somewhat related is the status of the configuration and what it means in
> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
> know there is an ongoing discussion regarding YARN-4183. We'll need to
> reflect the outcome of that discussion.
>
> My overall impression of whether this can be done for 2.8 is that it looks
> rather challenging given the suggested timeframe. We also need to complete
> several major tasks before it is ready.
>
> Sangjin
>
>
> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>
>>
>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>> vinodkv@hortonworks.com> wrote:
>>
>>> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
>>> but clearly a work in progress. Two options here
>>> — If it is safe to ship it into 2.8 in a disable manner, we can
>>> get the early code into trunk and all the way int o2.8.
>>> — If it is not safe, it organically rolls over into 2.9
>>>
>>
>> I'll review the changes on YARN-2928 to see what impact it has (if any) if
>> the timeline service v.2 is disabled.
>>
>> Another condition for it to make 2.8 is whether the branch will be in a
>> shape in a couple of weeks such that it adds value for folks that want to
>> test it. Hopefully it will become clearer soon.
>>
>> Sangjin
>>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
This is the current state from the feedback I gathered.
- Support priorities across applications within the same queue YARN-1963
— Can push as an alpha / beta feature per Sunil
- YARN-1197 Support changing resources of an allocated container:
— Can push as an alpha/beta feature per Wangda
- YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well most of it anyways.
— Can push as an alpha feature.
- YARN Timeline Service v1.5 - YARN-4233
— Should include per Li Lu
- YARN Timeline Service Next generation: YARN-2928
— Per analysis from Sangjin, drop this from 2.8.
One open feature status
- HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
Updated the Roadmap wiki with the same.
Thanks
+Vinod
> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>
> I reviewed the current state of the YARN-2928 changes regarding its impact
> if the timeline service v.2 is disabled. It does appear that there are a
> lot of things that still do get created and enabled unconditionally
> regardless of configuration. While this is understandable when we were
> working to implement the feature, this clearly needs to be cleaned up so
> that when disabled the timeline service v.2 doesn't impact other things.
>
> I filed a JIRA for that work:
> https://issues.apache.org/jira/browse/YARN-4356
>
> We need to complete it before we can merge.
>
> Somewhat related is the status of the configuration and what it means in
> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
> know there is an ongoing discussion regarding YARN-4183. We'll need to
> reflect the outcome of that discussion.
>
> My overall impression of whether this can be done for 2.8 is that it looks
> rather challenging given the suggested timeframe. We also need to complete
> several major tasks before it is ready.
>
> Sangjin
>
>
> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>
>>
>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>> vinodkv@hortonworks.com> wrote:
>>
>>> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
>>> but clearly a work in progress. Two options here
>>> — If it is safe to ship it into 2.8 in a disable manner, we can
>>> get the early code into trunk and all the way int o2.8.
>>> — If it is not safe, it organically rolls over into 2.9
>>>
>>
>> I'll review the changes on YARN-2928 to see what impact it has (if any) if
>> the timeline service v.2 is disabled.
>>
>> Another condition for it to make 2.8 is whether the branch will be in a
>> shape in a couple of weeks such that it adds value for folks that want to
>> test it. Hopefully it will become clearer soon.
>>
>> Sangjin
>>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
This is the current state from the feedback I gathered.
- Support priorities across applications within the same queue YARN-1963
— Can push as an alpha / beta feature per Sunil
- YARN-1197 Support changing resources of an allocated container:
— Can push as an alpha/beta feature per Wangda
- YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well most of it anyways.
— Can push as an alpha feature.
- YARN Timeline Service v1.5 - YARN-4233
— Should include per Li Lu
- YARN Timeline Service Next generation: YARN-2928
— Per analysis from Sangjin, drop this from 2.8.
One open feature status
- HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
Updated the Roadmap wiki with the same.
Thanks
+Vinod
> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>
> I reviewed the current state of the YARN-2928 changes regarding its impact
> if the timeline service v.2 is disabled. It does appear that there are a
> lot of things that still do get created and enabled unconditionally
> regardless of configuration. While this is understandable when we were
> working to implement the feature, this clearly needs to be cleaned up so
> that when disabled the timeline service v.2 doesn't impact other things.
>
> I filed a JIRA for that work:
> https://issues.apache.org/jira/browse/YARN-4356
>
> We need to complete it before we can merge.
>
> Somewhat related is the status of the configuration and what it means in
> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
> know there is an ongoing discussion regarding YARN-4183. We'll need to
> reflect the outcome of that discussion.
>
> My overall impression of whether this can be done for 2.8 is that it looks
> rather challenging given the suggested timeframe. We also need to complete
> several major tasks before it is ready.
>
> Sangjin
>
>
> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>
>>
>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>> vinodkv@hortonworks.com> wrote:
>>
>>> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
>>> but clearly a work in progress. Two options here
>>> — If it is safe to ship it into 2.8 in a disable manner, we can
>>> get the early code into trunk and all the way int o2.8.
>>> — If it is not safe, it organically rolls over into 2.9
>>>
>>
>> I'll review the changes on YARN-2928 to see what impact it has (if any) if
>> the timeline service v.2 is disabled.
>>
>> Another condition for it to make 2.8 is whether the branch will be in a
>> shape in a couple of weeks such that it adds value for folks that want to
>> test it. Hopefully it will become clearer soon.
>>
>> Sangjin
>>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
This is the current state from the feedback I gathered.
- Support priorities across applications within the same queue YARN-1963
— Can push as an alpha / beta feature per Sunil
- YARN-1197 Support changing resources of an allocated container:
— Can push as an alpha/beta feature per Wangda
- YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well most of it anyways.
— Can push as an alpha feature.
- YARN Timeline Service v1.5 - YARN-4233
— Should include per Li Lu
- YARN Timeline Service Next generation: YARN-2928
— Per analysis from Sangjin, drop this from 2.8.
One open feature status
- HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
Updated the Roadmap wiki with the same.
Thanks
+Vinod
> On Nov 13, 2015, at 12:12 PM, Sangjin Lee <sj...@apache.org> wrote:
>
> I reviewed the current state of the YARN-2928 changes regarding its impact
> if the timeline service v.2 is disabled. It does appear that there are a
> lot of things that still do get created and enabled unconditionally
> regardless of configuration. While this is understandable when we were
> working to implement the feature, this clearly needs to be cleaned up so
> that when disabled the timeline service v.2 doesn't impact other things.
>
> I filed a JIRA for that work:
> https://issues.apache.org/jira/browse/YARN-4356
>
> We need to complete it before we can merge.
>
> Somewhat related is the status of the configuration and what it means in
> various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
> know there is an ongoing discussion regarding YARN-4183. We'll need to
> reflect the outcome of that discussion.
>
> My overall impression of whether this can be done for 2.8 is that it looks
> rather challenging given the suggested timeframe. We also need to complete
> several major tasks before it is ready.
>
> Sangjin
>
>
> On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>
>>
>> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
>> vinodkv@hortonworks.com> wrote:
>>
>>> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
>>> but clearly a work in progress. Two options here
>>> — If it is safe to ship it into 2.8 in a disable manner, we can
>>> get the early code into trunk and all the way int o2.8.
>>> — If it is not safe, it organically rolls over into 2.9
>>>
>>
>> I'll review the changes on YARN-2928 to see what impact it has (if any) if
>> the timeline service v.2 is disabled.
>>
>> Another condition for it to make 2.8 is whether the branch will be in a
>> shape in a couple of weeks such that it adds value for folks that want to
>> test it. Hopefully it will become clearer soon.
>>
>> Sangjin
>>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Sangjin Lee <sj...@apache.org>.
I reviewed the current state of the YARN-2928 changes regarding its impact
if the timeline service v.2 is disabled. It does appear that there are a
lot of things that still do get created and enabled unconditionally
regardless of configuration. While this is understandable when we were
working to implement the feature, this clearly needs to be cleaned up so
that when disabled the timeline service v.2 doesn't impact other things.
I filed a JIRA for that work:
https://issues.apache.org/jira/browse/YARN-4356
We need to complete it before we can merge.
Somewhat related is the status of the configuration and what it means in
various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
know there is an ongoing discussion regarding YARN-4183. We'll need to
reflect the outcome of that discussion.
My overall impression of whether this can be done for 2.8 is that it looks
rather challenging given the suggested timeframe. We also need to complete
several major tasks before it is ready.
Sangjin
On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>
> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
> vinodkv@hortonworks.com> wrote:
>
>> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
>> but clearly a work in progress. Two options here
>> — If it is safe to ship it into 2.8 in a disable manner, we can
>> get the early code into trunk and all the way int o2.8.
>> — If it is not safe, it organically rolls over into 2.9
>>
>
> I'll review the changes on YARN-2928 to see what impact it has (if any) if
> the timeline service v.2 is disabled.
>
> Another condition for it to make 2.8 is whether the branch will be in a
> shape in a couple of weeks such that it adds value for folks that want to
> test it. Hopefully it will become clearer soon.
>
> Sangjin
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Sangjin Lee <sj...@apache.org>.
I reviewed the current state of the YARN-2928 changes regarding its impact
if the timeline service v.2 is disabled. It does appear that there are a
lot of things that still do get created and enabled unconditionally
regardless of configuration. While this is understandable when we were
working to implement the feature, this clearly needs to be cleaned up so
that when disabled the timeline service v.2 doesn't impact other things.
I filed a JIRA for that work:
https://issues.apache.org/jira/browse/YARN-4356
We need to complete it before we can merge.
Somewhat related is the status of the configuration and what it means in
various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
know there is an ongoing discussion regarding YARN-4183. We'll need to
reflect the outcome of that discussion.
My overall impression of whether this can be done for 2.8 is that it looks
rather challenging given the suggested timeframe. We also need to complete
several major tasks before it is ready.
Sangjin
On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>
> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
> vinodkv@hortonworks.com> wrote:
>
>> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
>> but clearly a work in progress. Two options here
>> — If it is safe to ship it into 2.8 in a disable manner, we can
>> get the early code into trunk and all the way int o2.8.
>> — If it is not safe, it organically rolls over into 2.9
>>
>
> I'll review the changes on YARN-2928 to see what impact it has (if any) if
> the timeline service v.2 is disabled.
>
> Another condition for it to make 2.8 is whether the branch will be in a
> shape in a couple of weeks such that it adds value for folks that want to
> test it. Hopefully it will become clearer soon.
>
> Sangjin
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Sangjin Lee <sj...@apache.org>.
I reviewed the current state of the YARN-2928 changes regarding its impact
if the timeline service v.2 is disabled. It does appear that there are a
lot of things that still do get created and enabled unconditionally
regardless of configuration. While this is understandable when we were
working to implement the feature, this clearly needs to be cleaned up so
that when disabled the timeline service v.2 doesn't impact other things.
I filed a JIRA for that work:
https://issues.apache.org/jira/browse/YARN-4356
We need to complete it before we can merge.
Somewhat related is the status of the configuration and what it means in
various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
know there is an ongoing discussion regarding YARN-4183. We'll need to
reflect the outcome of that discussion.
My overall impression of whether this can be done for 2.8 is that it looks
rather challenging given the suggested timeframe. We also need to complete
several major tasks before it is ready.
Sangjin
On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>
> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
> vinodkv@hortonworks.com> wrote:
>
>> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
>> but clearly a work in progress. Two options here
>> — If it is safe to ship it into 2.8 in a disable manner, we can
>> get the early code into trunk and all the way int o2.8.
>> — If it is not safe, it organically rolls over into 2.9
>>
>
> I'll review the changes on YARN-2928 to see what impact it has (if any) if
> the timeline service v.2 is disabled.
>
> Another condition for it to make 2.8 is whether the branch will be in a
> shape in a couple of weeks such that it adds value for folks that want to
> test it. Hopefully it will become clearer soon.
>
> Sangjin
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Sangjin Lee <sj...@apache.org>.
I reviewed the current state of the YARN-2928 changes regarding its impact
if the timeline service v.2 is disabled. It does appear that there are a
lot of things that still do get created and enabled unconditionally
regardless of configuration. While this is understandable when we were
working to implement the feature, this clearly needs to be cleaned up so
that when disabled the timeline service v.2 doesn't impact other things.
I filed a JIRA for that work:
https://issues.apache.org/jira/browse/YARN-4356
We need to complete it before we can merge.
Somewhat related is the status of the configuration and what it means in
various contexts (client/app-side vs. server-side, v.1 vs. v.2, etc.). I
know there is an ongoing discussion regarding YARN-4183. We'll need to
reflect the outcome of that discussion.
My overall impression of whether this can be done for 2.8 is that it looks
rather challenging given the suggested timeframe. We also need to complete
several major tasks before it is ready.
Sangjin
On Wed, Nov 11, 2015 at 5:49 PM, Sangjin Lee <sj...@gmail.com> wrote:
>
> On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <
> vinodkv@hortonworks.com> wrote:
>
>> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
>> but clearly a work in progress. Two options here
>> — If it is safe to ship it into 2.8 in a disable manner, we can
>> get the early code into trunk and all the way int o2.8.
>> — If it is not safe, it organically rolls over into 2.9
>>
>
> I'll review the changes on YARN-2928 to see what impact it has (if any) if
> the timeline service v.2 is disabled.
>
> Another condition for it to make 2.8 is whether the branch will be in a
> shape in a couple of weeks such that it adds value for folks that want to
> test it. Hopefully it will become clearer soon.
>
> Sangjin
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Sangjin Lee <sj...@gmail.com>.
On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vinodkv@hortonworks.com
> wrote:
> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
> but clearly a work in progress. Two options here
> — If it is safe to ship it into 2.8 in a disable manner, we can
> get the early code into trunk and all the way int o2.8.
> — If it is not safe, it organically rolls over into 2.9
>
I'll review the changes on YARN-2928 to see what impact it has (if any) if
the timeline service v.2 is disabled.
Another condition for it to make 2.8 is whether the branch will be in a
shape in a couple of weeks such that it adds value for folks that want to
test it. Hopefully it will become clearer soon.
Sangjin
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Haohui,
It’ll help to document this whole line of discussion about hdfs jar change and its impact/non-impact for existing users so there is less confusion.
Thanks
+Vinod
> On Nov 11, 2015, at 3:26 PM, Haohui Mai <ri...@gmail.com> wrote:
>
> bq. If and only if they take the Hadoop class path at face value.
> Many applications don’t because of conflicting dependencies and
> instead import specific jars.
>
> We do make the assumptions that applications need to pick up all the
> dependency (either automatically or manually). The situation is
> similar with adding a new dependency into hdfs in a minor release.
>
> Maven / gradle obviously help, but I'd love to hear more about it how
> you get it to work. In trunk hadoop-env.sh adds 118 jars into the
> class path. Are you manually importing 118 jars for every single
> applications?
>
>
>
> On Wed, Nov 11, 2015 at 3:09 PM, Haohui Mai <ri...@gmail.com> wrote:
>> bq. currently pulling in hadoop-client gives downstream apps
>> hadoop-hdfs-client, but not hadoop-hdfs server side, right?
>>
>> Right now hadoop-client pulls in hadoop-hdfs directly to ensure a
>> smooth transition. Maybe we can revisit the decision in the 2.9 / 3.x?
>>
>> On Wed, Nov 11, 2015 at 3:00 PM, Steve Loughran <st...@hortonworks.com> wrote:
>>>
>>>> On 11 Nov 2015, at 22:15, Haohui Mai <ri...@gmail.com> wrote:
>>>>
>>>> bq. it basically makes the assumption that everyone recompiles for
>>>> every minor release.
>>>>
>>>> I don't think that the statement holds. HDFS-6200 keeps classes in the
>>>> same package. hdfs-client becomes a transitive dependency of the
>>>> original hdfs jar.
>>>>
>>>> Applications continue to work without recompilation as the classes
>>>> will be in the same name and will be available in the classpath. They
>>>> have the option of switching to depending only on hdfs-client to
>>>> minimize the dependency when they are comfortable.
>>>>
>>>> I'm not claiming that there are no bugs in HDFS-6200, but just like
>>>> other features we discover bugs and fix them continuously.
>>>>
>>>> ~Haohui
>>>>
>>>
>>> currently pulling in hadoop-client gives downstream apps hadoop-hdfs-client, but not hadoop-hdfs server side, right?
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Chris Nauroth <cn...@hortonworks.com>.
Regarding interface visibility/stability, I'm aware of 2 relevant JIRAs
right now.
HADOOP-10776 proposes to mark Public some of the security plumbing like
the FileSystem delegation token methods, UserGroupInformation, Token and
Credentials. At this point, I think we are only fooling ourselves trying
to treat these as Private or LimitedPrivate. I believe they are de facto
Public, because downstream applications just don't have any other reliable
way to do what these interfaces do.
HADOOP-12600 proposes to mark FileContext Stable. I expect this one is
simply an oversight.
I've taken the bold step of marking both issues as 2.8.0 blockers. We can
of course reconsider if this is controversial.
--Chris Nauroth
On 11/25/15, 11:30 AM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
wrote:
>Steve,
>
>
>> There's a lot of stuff in 2.8; I note that I'd like to see the s3a perf
>>improvements & openstack fixes in there: for which I need reviewers. I
>>don't have the spare time to do this myself.
>
>If you think they are useful, it helps to file tickets (or point out
>existing tickets), start discussion etc w.r.t these areas in order to
>attract contributors.
>
>
>> -likewise, DFSConfigKeys stayed in hdfs-server. I know it's tagged as
>>@Private, but it's long been where all the string constants for HDFS
>>options live. Forcing users to retype them in their own source is not
>>only dangerous (it only encourages typos), it actually stops you using
>>your IDE finding out where those constants get used.
>
>> We do now have a set of keys in the client, HdfsClientConfigKeys, but
>>these are still declared as @Private. Which is a mistake for the reasons
>>above, and because it encourages hadoop developers to assume that they
>>are free to make whatever changes they want to this code, and if it
>>breaks something, say "it was tagged as private²
>
>
>If these are worth going after, please file tickets under HDFS-6200 if
>they don¹t exist already.
>
>
>>
>> 1. We have to recognise that a lot of things marked @Private are in
>>fact essential for clients to use. Not just constants, but actual
>>classes.
>>
>> 2. We have to look hard at @LimitedPrivate and question the legitimacy
>>of tagging things as so, especially anything
>>"@InterfaceAudience.LimitedPrivate({""MapReduce"}) ‹because any YARN app
>>you write ends up needing those classes. For evidence, look at
>>DistributedShell's imports, and pick a few at random: NMClientAsyncImpl,
>>ConverterUtils being easy targets.
>
>There are existing tickets for some of these under YARN-1953 that need
>some developer love.
>
>
>> Returning to the pending 2.8.0 release, there's a way to find out
>>what's going to break: build and test things against the snapshots,
>>without waiting for the beta releases and expecting the downstream
>>projects to do it for you. If they don't build, that's a success: you've
>>found a compatibility problem to fix. If they don't test, well that's
>>trouble ‹you are in finger pointing time.
>
>
>I¹ve tried doing this in the past without much success. Some downstream
>components did pick up RCs but a majority of them needed a release -
>hence my current approach.
>
>Thanks
>+Vinod
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Steve,
> There's a lot of stuff in 2.8; I note that I'd like to see the s3a perf improvements & openstack fixes in there: for which I need reviewers. I don't have the spare time to do this myself.
If you think they are useful, it helps to file tickets (or point out existing tickets), start discussion etc w.r.t these areas in order to attract contributors.
> -likewise, DFSConfigKeys stayed in hdfs-server. I know it's tagged as @Private, but it's long been where all the string constants for HDFS options live. Forcing users to retype them in their own source is not only dangerous (it only encourages typos), it actually stops you using your IDE finding out where those constants get used.
> We do now have a set of keys in the client, HdfsClientConfigKeys, but these are still declared as @Private. Which is a mistake for the reasons above, and because it encourages hadoop developers to assume that they are free to make whatever changes they want to this code, and if it breaks something, say "it was tagged as private”
If these are worth going after, please file tickets under HDFS-6200 if they don’t exist already.
>
> 1. We have to recognise that a lot of things marked @Private are in fact essential for clients to use. Not just constants, but actual classes.
>
> 2. We have to look hard at @LimitedPrivate and question the legitimacy of tagging things as so, especially anything "@InterfaceAudience.LimitedPrivate({""MapReduce"}) —because any YARN app you write ends up needing those classes. For evidence, look at DistributedShell's imports, and pick a few at random: NMClientAsyncImpl, ConverterUtils being easy targets.
There are existing tickets for some of these under YARN-1953 that need some developer love.
> Returning to the pending 2.8.0 release, there's a way to find out what's going to break: build and test things against the snapshots, without waiting for the beta releases and expecting the downstream projects to do it for you. If they don't build, that's a success: you've found a compatibility problem to fix. If they don't test, well that's trouble —you are in finger pointing time.
I’ve tried doing this in the past without much success. Some downstream components did pick up RCs but a majority of them needed a release - hence my current approach.
Thanks
+Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Steve Loughran <st...@hortonworks.com>.
There's a lot of stuff in 2.8; I note that I'd like to see the s3a perf improvements & openstack fixes in there: for which I need reviewers. I don't have the spare time to do this myself.
I've already been building & testing both Apache Slider (incubating) and Apache Spark against both 2.8.0-SNAPSHOT & 3.0.0-SNAPSHOT.
What's been troublesome for builds which use maven as the way of managing dependencies (I'm ignoring the fact that spark *also* has an SBT build with ivy doing dep management)? HDFS client
-hadoop-hdfs-client pulled HdfsConfiguration. I'd been explicitly creating this to force in hdfs-default.xml & hdfs-site.xml loading, so that I could do sanity checks on things like security settings prior to attempting AM launch.
-likewise, DFSConfigKeys stayed in hdfs-server. I know it's tagged as @Private, but it's long been where all the string constants for HDFS options live. Forcing users to retype them in their own source is not only dangerous (it only encourages typos), it actually stops you using your IDE finding out where those constants get used.
We do now have a set of keys in the client, HdfsClientConfigKeys, but these are still declared as @Private. Which is a mistake for the reasons above, and because it encourages hadoop developers to assume that they are free to make whatever changes they want to this code, and if it breaks something, say "it was tagged as private"
1. We have to recognise that a lot of things marked @Private are in fact essential for clients to use. Not just constants, but actual classes.
2. We have to look hard at @LimitedPrivate and question the legitimacy of tagging things as so, especially anything "@InterfaceAudience.LimitedPrivate({""MapReduce"}) —because any YARN app you write ends up needing those classes. For evidence, look at DistributedShell's imports, and pick a few at random: NMClientAsyncImpl, ConverterUtils being easy targets.
3. Or for real fun, UGI: @InterfaceAudience.LimitedPrivate({"HDFS", "MapReduce", "HBase", "Hive", "Oozie"})
I'd advocate marking all "MapReduce" as "YarnApp" and have people working on those classes accept that they will be used downstream and treat changes with caution. Yes, they may be messy, but how things are used. At least with a modern IDE you can add in the downstream projects and identify those uses with ease.
In the end SLIDER-948 addressed the problems for me. I switched to pulling in hadoop-hdfs *and* copied and pasted all the DFSConfigurationKeys I used into my own file of constants.
HDFS-9301 should make these changes things I could revert —and other projects not notice them ever existing —but I've left them them in to isolate me from any more situations like this. To be completely ruthless: I don't trust that code to not break my builds any more.
Behaviour-wise, I've not seen much in the way of changes; all tests work the same. Oh and Spark wouldn't compile against 3.0 as an exception tagged as @Deprecated since Hadoop 0.18 got pulled. Trivially fixed.
Returning to the pending 2.8.0 release, there's a way to find out what's going to break: build and test things against the snapshots, without waiting for the beta releases and expecting the downstream projects to do it for you. If they don't build, that's a success: you've found a compatibility problem to fix. If they don't test, well that's trouble —you are in finger pointing time.
-Steve
> On 11 Nov 2015, at 23:26, Haohui Mai <ri...@gmail.com> wrote:
>
> bq. If and only if they take the Hadoop class path at face value.
> Many applications don’t because of conflicting dependencies and
> instead import specific jars.
>
> We do make the assumptions that applications need to pick up all the
> dependency (either automatically or manually). The situation is
> similar with adding a new dependency into hdfs in a minor release.
>
> Maven / gradle obviously help, but I'd love to hear more about it how
> you get it to work. In trunk hadoop-env.sh adds 118 jars into the
> class path. Are you manually importing 118 jars for every single
> applications?
>
>
>
> On Wed, Nov 11, 2015 at 3:09 PM, Haohui Mai <ri...@gmail.com> wrote:
>> bq. currently pulling in hadoop-client gives downstream apps
>> hadoop-hdfs-client, but not hadoop-hdfs server side, right?
>>
>> Right now hadoop-client pulls in hadoop-hdfs directly to ensure a
>> smooth transition. Maybe we can revisit the decision in the 2.9 / 3.x?
>>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Haohui Mai <ri...@gmail.com>.
bq. If and only if they take the Hadoop class path at face value.
Many applications don’t because of conflicting dependencies and
instead import specific jars.
We do make the assumptions that applications need to pick up all the
dependency (either automatically or manually). The situation is
similar with adding a new dependency into hdfs in a minor release.
Maven / gradle obviously help, but I'd love to hear more about it how
you get it to work. In trunk hadoop-env.sh adds 118 jars into the
class path. Are you manually importing 118 jars for every single
applications?
On Wed, Nov 11, 2015 at 3:09 PM, Haohui Mai <ri...@gmail.com> wrote:
> bq. currently pulling in hadoop-client gives downstream apps
> hadoop-hdfs-client, but not hadoop-hdfs server side, right?
>
> Right now hadoop-client pulls in hadoop-hdfs directly to ensure a
> smooth transition. Maybe we can revisit the decision in the 2.9 / 3.x?
>
> On Wed, Nov 11, 2015 at 3:00 PM, Steve Loughran <st...@hortonworks.com> wrote:
>>
>>> On 11 Nov 2015, at 22:15, Haohui Mai <ri...@gmail.com> wrote:
>>>
>>> bq. it basically makes the assumption that everyone recompiles for
>>> every minor release.
>>>
>>> I don't think that the statement holds. HDFS-6200 keeps classes in the
>>> same package. hdfs-client becomes a transitive dependency of the
>>> original hdfs jar.
>>>
>>> Applications continue to work without recompilation as the classes
>>> will be in the same name and will be available in the classpath. They
>>> have the option of switching to depending only on hdfs-client to
>>> minimize the dependency when they are comfortable.
>>>
>>> I'm not claiming that there are no bugs in HDFS-6200, but just like
>>> other features we discover bugs and fix them continuously.
>>>
>>> ~Haohui
>>>
>>
>> currently pulling in hadoop-client gives downstream apps hadoop-hdfs-client, but not hadoop-hdfs server side, right?
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Haohui Mai <ri...@gmail.com>.
bq. currently pulling in hadoop-client gives downstream apps
hadoop-hdfs-client, but not hadoop-hdfs server side, right?
Right now hadoop-client pulls in hadoop-hdfs directly to ensure a
smooth transition. Maybe we can revisit the decision in the 2.9 / 3.x?
On Wed, Nov 11, 2015 at 3:00 PM, Steve Loughran <st...@hortonworks.com> wrote:
>
>> On 11 Nov 2015, at 22:15, Haohui Mai <ri...@gmail.com> wrote:
>>
>> bq. it basically makes the assumption that everyone recompiles for
>> every minor release.
>>
>> I don't think that the statement holds. HDFS-6200 keeps classes in the
>> same package. hdfs-client becomes a transitive dependency of the
>> original hdfs jar.
>>
>> Applications continue to work without recompilation as the classes
>> will be in the same name and will be available in the classpath. They
>> have the option of switching to depending only on hdfs-client to
>> minimize the dependency when they are comfortable.
>>
>> I'm not claiming that there are no bugs in HDFS-6200, but just like
>> other features we discover bugs and fix them continuously.
>>
>> ~Haohui
>>
>
> currently pulling in hadoop-client gives downstream apps hadoop-hdfs-client, but not hadoop-hdfs server side, right?
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Steve Loughran <st...@hortonworks.com>.
> On 11 Nov 2015, at 22:15, Haohui Mai <ri...@gmail.com> wrote:
>
> bq. it basically makes the assumption that everyone recompiles for
> every minor release.
>
> I don't think that the statement holds. HDFS-6200 keeps classes in the
> same package. hdfs-client becomes a transitive dependency of the
> original hdfs jar.
>
> Applications continue to work without recompilation as the classes
> will be in the same name and will be available in the classpath. They
> have the option of switching to depending only on hdfs-client to
> minimize the dependency when they are comfortable.
>
> I'm not claiming that there are no bugs in HDFS-6200, but just like
> other features we discover bugs and fix them continuously.
>
> ~Haohui
>
currently pulling in hadoop-client gives downstream apps hadoop-hdfs-client, but not hadoop-hdfs server side, right?
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 11, 2015, at 2:15 PM, Haohui Mai <ri...@gmail.com> wrote:
>
> bq. it basically makes the assumption that everyone recompiles for
> every minor release.
>
> I don't think that the statement holds. HDFS-6200 keeps classes in the
> same package. hdfs-client becomes a transitive dependency of the
> original hdfs jar.
>
> Applications continue to work without recompilation as the classes
> will be in the same name and will be available in the class path.
If and only if they take the Hadoop class path at face value. Many applications don’t because of conflicting dependencies and instead import specific jars.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Haohui Mai <ri...@gmail.com>.
bq. it basically makes the assumption that everyone recompiles for
every minor release.
I don't think that the statement holds. HDFS-6200 keeps classes in the
same package. hdfs-client becomes a transitive dependency of the
original hdfs jar.
Applications continue to work without recompilation as the classes
will be in the same name and will be available in the classpath. They
have the option of switching to depending only on hdfs-client to
minimize the dependency when they are comfortable.
I'm not claiming that there are no bugs in HDFS-6200, but just like
other features we discover bugs and fix them continuously.
~Haohui
On Wed, Nov 11, 2015 at 12:43 PM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>> On Nov 11, 2015, at 12:13 PM, Vinod Vavilapalli <vi...@hortonworks.com> wrote:
>>
>> — HDFS-6200 Create a separate jar for hdfs-client: Compatible improvement - no dimension of alpha/betaness here.
>
> IMO: this feels like a massive break in backwards compatibility. Anyone who is looking for specific methods in specific jars are going to have a bad time. Also, it seems as though every week a new issue crops up that is related to this change. Is Slider still having problems with it? The reasoning “well, the pom sets the dependencies so it’s ok” feels like an *extremely weak* reason this wasn’t marked incompatible— it basically makes the assumption that everyone recompiles for every minor release.
>
>> — Compatibility tools to catch backwards, forwards compatibility issues at patch submission, release times. Some of it is captured at YARN-3292. This also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
>
> There has been talk in the past about adding Java ACC support to Yetus.
>
>> Thoughts?
>
> I’d rather see efforts on 3.x than another disastrous 2.x release. The track record is not good. At least a new major will signify that danger looms ahead. We’re already treating 2.x minor releases as effectively major (see the list of incompatible JIRAs) so what different does it make if we do 2.x vs. 3.x anyway?
>
>>
>> Thanks
>> +Vinod
>> PS:As you may have noted above, this time around, I want to do something that we’ve always wanted to do, but never explicitly did. I’m calling out readiness of each feature as they stand today so we can inform our users better of what they can start relying on in production clusters.
>
> … except some of these changes are so deep reaching that even if you don’t use the feature, you’re still impacted by it ...
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Karthik Kambatla <ka...@cloudera.com>.
Did we consider cutting a branch-3 that borrows relatively compatible
patches from trunk to run the 3.x line? That said, I would like for us to
really tighten our compatibility policies and actually stick to them
starting the next major release.
On Wed, Nov 11, 2015 at 1:11 PM, Vinod Vavilapalli <vi...@hortonworks.com>
wrote:
> I’ll let others comment on specific features.
>
> Regarding the 3.x vs 2.x point, as I noted before on other threads, given
> all the incompatibilities in trunk it will be ways off before users can run
> their production workloads on a 3.x release. Therefore, as I was proposing
> before, we should continue the 2.x lines, but soon get started on rolling
> out a release candidate based off trunk.
>
> Like with 2.8, I’d like to go back and prepare some notes on trunk’s
> content so we can objectively discuss about it.
>
> +Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Karthik Kambatla <ka...@cloudera.com>.
Did we consider cutting a branch-3 that borrows relatively compatible
patches from trunk to run the 3.x line? That said, I would like for us to
really tighten our compatibility policies and actually stick to them
starting the next major release.
On Wed, Nov 11, 2015 at 1:11 PM, Vinod Vavilapalli <vi...@hortonworks.com>
wrote:
> I’ll let others comment on specific features.
>
> Regarding the 3.x vs 2.x point, as I noted before on other threads, given
> all the incompatibilities in trunk it will be ways off before users can run
> their production workloads on a 3.x release. Therefore, as I was proposing
> before, we should continue the 2.x lines, but soon get started on rolling
> out a release candidate based off trunk.
>
> Like with 2.8, I’d like to go back and prepare some notes on trunk’s
> content so we can objectively discuss about it.
>
> +Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 25, 2015, at 11:23 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> There are 40 odd incompatible changes in 3.x: https://issues.apache.org/jira/issues/?jql=project%20in%20%28HADOOP%2C%20YARN%2C%20HDFS%2C%20MAPREDUCE%29%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%203.0.0%20AND%20fixVersion%20not%20in%20%282.6.2%2C%202.6.3%2C%202.7.1%2C%202.7.2%2C%202.7.3%2C%202.8.0%29%20and%20%22Hadoop%20Flags%22%20in%20%28%22Incompatible%20change%22%29%20ORDER%20BY%20key%20ASC%2C%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>
> Need to dig deeper on their impact. Clearly all my local shell scripts completely stopped working, it will be good to have some bridging there to help users migrate.
I think you should file a JIRA on what actually broke. I’m genuinely curious.
> Like I said before, I will spend more time on trunk only changes in order to kick-start a 3.x discussion.
>
> What are the incompatible changes in the 2.x line that you are talking about?
Thanks for confirming what I’ve always suspected.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 25, 2015, at 11:23 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> There are 40 odd incompatible changes in 3.x: https://issues.apache.org/jira/issues/?jql=project%20in%20%28HADOOP%2C%20YARN%2C%20HDFS%2C%20MAPREDUCE%29%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%203.0.0%20AND%20fixVersion%20not%20in%20%282.6.2%2C%202.6.3%2C%202.7.1%2C%202.7.2%2C%202.7.3%2C%202.8.0%29%20and%20%22Hadoop%20Flags%22%20in%20%28%22Incompatible%20change%22%29%20ORDER%20BY%20key%20ASC%2C%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>
> Need to dig deeper on their impact. Clearly all my local shell scripts completely stopped working, it will be good to have some bridging there to help users migrate.
I think you should file a JIRA on what actually broke. I’m genuinely curious.
> Like I said before, I will spend more time on trunk only changes in order to kick-start a 3.x discussion.
>
> What are the incompatible changes in the 2.x line that you are talking about?
Thanks for confirming what I’ve always suspected.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 25, 2015, at 11:23 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> There are 40 odd incompatible changes in 3.x: https://issues.apache.org/jira/issues/?jql=project%20in%20%28HADOOP%2C%20YARN%2C%20HDFS%2C%20MAPREDUCE%29%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%203.0.0%20AND%20fixVersion%20not%20in%20%282.6.2%2C%202.6.3%2C%202.7.1%2C%202.7.2%2C%202.7.3%2C%202.8.0%29%20and%20%22Hadoop%20Flags%22%20in%20%28%22Incompatible%20change%22%29%20ORDER%20BY%20key%20ASC%2C%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>
> Need to dig deeper on their impact. Clearly all my local shell scripts completely stopped working, it will be good to have some bridging there to help users migrate.
I think you should file a JIRA on what actually broke. I’m genuinely curious.
> Like I said before, I will spend more time on trunk only changes in order to kick-start a 3.x discussion.
>
> What are the incompatible changes in the 2.x line that you are talking about?
Thanks for confirming what I’ve always suspected.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 25, 2015, at 11:23 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> There are 40 odd incompatible changes in 3.x: https://issues.apache.org/jira/issues/?jql=project%20in%20%28HADOOP%2C%20YARN%2C%20HDFS%2C%20MAPREDUCE%29%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%203.0.0%20AND%20fixVersion%20not%20in%20%282.6.2%2C%202.6.3%2C%202.7.1%2C%202.7.2%2C%202.7.3%2C%202.8.0%29%20and%20%22Hadoop%20Flags%22%20in%20%28%22Incompatible%20change%22%29%20ORDER%20BY%20key%20ASC%2C%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
>
> Need to dig deeper on their impact. Clearly all my local shell scripts completely stopped working, it will be good to have some bridging there to help users migrate.
I think you should file a JIRA on what actually broke. I’m genuinely curious.
> Like I said before, I will spend more time on trunk only changes in order to kick-start a 3.x discussion.
>
> What are the incompatible changes in the 2.x line that you are talking about?
Thanks for confirming what I’ve always suspected.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
There are 40 odd incompatible changes in 3.x: https://issues.apache.org/jira/issues/?jql=project%20in%20%28HADOOP%2C%20YARN%2C%20HDFS%2C%20MAPREDUCE%29%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%203.0.0%20AND%20fixVersion%20not%20in%20%282.6.2%2C%202.6.3%2C%202.7.1%2C%202.7.2%2C%202.7.3%2C%202.8.0%29%20and%20%22Hadoop%20Flags%22%20in%20%28%22Incompatible%20change%22%29%20ORDER%20BY%20key%20ASC%2C%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
Need to dig deeper on their impact. Clearly all my local shell scripts completely stopped working, it will be good to have some bridging there to help users migrate.
Like I said before, I will spend more time on trunk only changes in order to kick-start a 3.x discussion.
What are the incompatible changes in the 2.x line that you are talking about?
+Vinod
> On Nov 11, 2015, at 2:15 PM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>
>> On Nov 11, 2015, at 1:11 PM, Vinod Vavilapalli <vi...@hortonworks.com> wrote:
>>
>> I’ll let others comment on specific features.
>>
>> Regarding the 3.x vs 2.x point, as I noted before on other threads, given all the incompatibilities in trunk it will be ways off before users can run their production workloads on a 3.x release.
>
> [citation needed]
>
> Seriously. Back that statement up especially in light of there having been more incompatibilities in all the 2.x releases combined than in 3.x.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
There are 40 odd incompatible changes in 3.x: https://issues.apache.org/jira/issues/?jql=project%20in%20%28HADOOP%2C%20YARN%2C%20HDFS%2C%20MAPREDUCE%29%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%203.0.0%20AND%20fixVersion%20not%20in%20%282.6.2%2C%202.6.3%2C%202.7.1%2C%202.7.2%2C%202.7.3%2C%202.8.0%29%20and%20%22Hadoop%20Flags%22%20in%20%28%22Incompatible%20change%22%29%20ORDER%20BY%20key%20ASC%2C%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
Need to dig deeper on their impact. Clearly all my local shell scripts completely stopped working, it will be good to have some bridging there to help users migrate.
Like I said before, I will spend more time on trunk only changes in order to kick-start a 3.x discussion.
What are the incompatible changes in the 2.x line that you are talking about?
+Vinod
> On Nov 11, 2015, at 2:15 PM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>
>> On Nov 11, 2015, at 1:11 PM, Vinod Vavilapalli <vi...@hortonworks.com> wrote:
>>
>> I’ll let others comment on specific features.
>>
>> Regarding the 3.x vs 2.x point, as I noted before on other threads, given all the incompatibilities in trunk it will be ways off before users can run their production workloads on a 3.x release.
>
> [citation needed]
>
> Seriously. Back that statement up especially in light of there having been more incompatibilities in all the 2.x releases combined than in 3.x.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
There are 40 odd incompatible changes in 3.x: https://issues.apache.org/jira/issues/?jql=project%20in%20%28HADOOP%2C%20YARN%2C%20HDFS%2C%20MAPREDUCE%29%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%203.0.0%20AND%20fixVersion%20not%20in%20%282.6.2%2C%202.6.3%2C%202.7.1%2C%202.7.2%2C%202.7.3%2C%202.8.0%29%20and%20%22Hadoop%20Flags%22%20in%20%28%22Incompatible%20change%22%29%20ORDER%20BY%20key%20ASC%2C%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
Need to dig deeper on their impact. Clearly all my local shell scripts completely stopped working, it will be good to have some bridging there to help users migrate.
Like I said before, I will spend more time on trunk only changes in order to kick-start a 3.x discussion.
What are the incompatible changes in the 2.x line that you are talking about?
+Vinod
> On Nov 11, 2015, at 2:15 PM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>
>> On Nov 11, 2015, at 1:11 PM, Vinod Vavilapalli <vi...@hortonworks.com> wrote:
>>
>> I’ll let others comment on specific features.
>>
>> Regarding the 3.x vs 2.x point, as I noted before on other threads, given all the incompatibilities in trunk it will be ways off before users can run their production workloads on a 3.x release.
>
> [citation needed]
>
> Seriously. Back that statement up especially in light of there having been more incompatibilities in all the 2.x releases combined than in 3.x.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
There are 40 odd incompatible changes in 3.x: https://issues.apache.org/jira/issues/?jql=project%20in%20%28HADOOP%2C%20YARN%2C%20HDFS%2C%20MAPREDUCE%29%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%203.0.0%20AND%20fixVersion%20not%20in%20%282.6.2%2C%202.6.3%2C%202.7.1%2C%202.7.2%2C%202.7.3%2C%202.8.0%29%20and%20%22Hadoop%20Flags%22%20in%20%28%22Incompatible%20change%22%29%20ORDER%20BY%20key%20ASC%2C%20due%20ASC%2C%20priority%20DESC%2C%20created%20ASC
Need to dig deeper on their impact. Clearly all my local shell scripts completely stopped working, it will be good to have some bridging there to help users migrate.
Like I said before, I will spend more time on trunk only changes in order to kick-start a 3.x discussion.
What are the incompatible changes in the 2.x line that you are talking about?
+Vinod
> On Nov 11, 2015, at 2:15 PM, Allen Wittenauer <aw...@altiscale.com> wrote:
>
>
>> On Nov 11, 2015, at 1:11 PM, Vinod Vavilapalli <vi...@hortonworks.com> wrote:
>>
>> I’ll let others comment on specific features.
>>
>> Regarding the 3.x vs 2.x point, as I noted before on other threads, given all the incompatibilities in trunk it will be ways off before users can run their production workloads on a 3.x release.
>
> [citation needed]
>
> Seriously. Back that statement up especially in light of there having been more incompatibilities in all the 2.x releases combined than in 3.x.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 11, 2015, at 1:11 PM, Vinod Vavilapalli <vi...@hortonworks.com> wrote:
>
> I’ll let others comment on specific features.
>
> Regarding the 3.x vs 2.x point, as I noted before on other threads, given all the incompatibilities in trunk it will be ways off before users can run their production workloads on a 3.x release.
[citation needed]
Seriously. Back that statement up especially in light of there having been more incompatibilities in all the 2.x releases combined than in 3.x.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Sunil Govind <su...@gmail.com>.
Thank you Vinod for starting this discussion.
+1 for getting a beta/alpha version for Application Priority (YARN-1963).
Major patches are already in and MAPREDUCE-5870 (making MR also to use
app-priority) is in final review stages. This feature is also tested
in-house. With 2.8.0 alpha, I feel we can see more use case based testing
on same.
A documentation JIRA is already raised for App Priority, and I will mark it
for 2.8 so that it will be done before the RC cut.
Thank You
Sunil
On Thu, Nov 12, 2015 at 2:41 AM Vinod Vavilapalli <vi...@hortonworks.com>
wrote:
> I’ll let others comment on specific features.
>
> Regarding the 3.x vs 2.x point, as I noted before on other threads, given
> all the incompatibilities in trunk it will be ways off before users can run
> their production workloads on a 3.x release. Therefore, as I was proposing
> before, we should continue the 2.x lines, but soon get started on rolling
> out a release candidate based off trunk.
>
> Like with 2.8, I’d like to go back and prepare some notes on trunk’s
> content so we can objectively discuss about it.
>
> +Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Karthik Kambatla <ka...@cloudera.com>.
Did we consider cutting a branch-3 that borrows relatively compatible
patches from trunk to run the 3.x line? That said, I would like for us to
really tighten our compatibility policies and actually stick to them
starting the next major release.
On Wed, Nov 11, 2015 at 1:11 PM, Vinod Vavilapalli <vi...@hortonworks.com>
wrote:
> I’ll let others comment on specific features.
>
> Regarding the 3.x vs 2.x point, as I noted before on other threads, given
> all the incompatibilities in trunk it will be ways off before users can run
> their production workloads on a 3.x release. Therefore, as I was proposing
> before, we should continue the 2.x lines, but soon get started on rolling
> out a release candidate based off trunk.
>
> Like with 2.8, I’d like to go back and prepare some notes on trunk’s
> content so we can objectively discuss about it.
>
> +Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 11, 2015, at 1:11 PM, Vinod Vavilapalli <vi...@hortonworks.com> wrote:
>
> I’ll let others comment on specific features.
>
> Regarding the 3.x vs 2.x point, as I noted before on other threads, given all the incompatibilities in trunk it will be ways off before users can run their production workloads on a 3.x release.
[citation needed]
Seriously. Back that statement up especially in light of there having been more incompatibilities in all the 2.x releases combined than in 3.x.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 11, 2015, at 1:11 PM, Vinod Vavilapalli <vi...@hortonworks.com> wrote:
>
> I’ll let others comment on specific features.
>
> Regarding the 3.x vs 2.x point, as I noted before on other threads, given all the incompatibilities in trunk it will be ways off before users can run their production workloads on a 3.x release.
[citation needed]
Seriously. Back that statement up especially in light of there having been more incompatibilities in all the 2.x releases combined than in 3.x.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 11, 2015, at 1:11 PM, Vinod Vavilapalli <vi...@hortonworks.com> wrote:
>
> I’ll let others comment on specific features.
>
> Regarding the 3.x vs 2.x point, as I noted before on other threads, given all the incompatibilities in trunk it will be ways off before users can run their production workloads on a 3.x release.
[citation needed]
Seriously. Back that statement up especially in light of there having been more incompatibilities in all the 2.x releases combined than in 3.x.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Sunil Govind <su...@gmail.com>.
Thank you Vinod for starting this discussion.
+1 for getting a beta/alpha version for Application Priority (YARN-1963).
Major patches are already in and MAPREDUCE-5870 (making MR also to use
app-priority) is in final review stages. This feature is also tested
in-house. With 2.8.0 alpha, I feel we can see more use case based testing
on same.
A documentation JIRA is already raised for App Priority, and I will mark it
for 2.8 so that it will be done before the RC cut.
Thank You
Sunil
On Thu, Nov 12, 2015 at 2:41 AM Vinod Vavilapalli <vi...@hortonworks.com>
wrote:
> I’ll let others comment on specific features.
>
> Regarding the 3.x vs 2.x point, as I noted before on other threads, given
> all the incompatibilities in trunk it will be ways off before users can run
> their production workloads on a 3.x release. Therefore, as I was proposing
> before, we should continue the 2.x lines, but soon get started on rolling
> out a release candidate based off trunk.
>
> Like with 2.8, I’d like to go back and prepare some notes on trunk’s
> content so we can objectively discuss about it.
>
> +Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Karthik Kambatla <ka...@cloudera.com>.
Did we consider cutting a branch-3 that borrows relatively compatible
patches from trunk to run the 3.x line? That said, I would like for us to
really tighten our compatibility policies and actually stick to them
starting the next major release.
On Wed, Nov 11, 2015 at 1:11 PM, Vinod Vavilapalli <vi...@hortonworks.com>
wrote:
> I’ll let others comment on specific features.
>
> Regarding the 3.x vs 2.x point, as I noted before on other threads, given
> all the incompatibilities in trunk it will be ways off before users can run
> their production workloads on a 3.x release. Therefore, as I was proposing
> before, we should continue the 2.x lines, but soon get started on rolling
> out a release candidate based off trunk.
>
> Like with 2.8, I’d like to go back and prepare some notes on trunk’s
> content so we can objectively discuss about it.
>
> +Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Sunil Govind <su...@gmail.com>.
Thank you Vinod for starting this discussion.
+1 for getting a beta/alpha version for Application Priority (YARN-1963).
Major patches are already in and MAPREDUCE-5870 (making MR also to use
app-priority) is in final review stages. This feature is also tested
in-house. With 2.8.0 alpha, I feel we can see more use case based testing
on same.
A documentation JIRA is already raised for App Priority, and I will mark it
for 2.8 so that it will be done before the RC cut.
Thank You
Sunil
On Thu, Nov 12, 2015 at 2:41 AM Vinod Vavilapalli <vi...@hortonworks.com>
wrote:
> I’ll let others comment on specific features.
>
> Regarding the 3.x vs 2.x point, as I noted before on other threads, given
> all the incompatibilities in trunk it will be ways off before users can run
> their production workloads on a 3.x release. Therefore, as I was proposing
> before, we should continue the 2.x lines, but soon get started on rolling
> out a release candidate based off trunk.
>
> Like with 2.8, I’d like to go back and prepare some notes on trunk’s
> content so we can objectively discuss about it.
>
> +Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Vavilapalli <vi...@hortonworks.com>.
I’ll let others comment on specific features.
Regarding the 3.x vs 2.x point, as I noted before on other threads, given all the incompatibilities in trunk it will be ways off before users can run their production workloads on a 3.x release. Therefore, as I was proposing before, we should continue the 2.x lines, but soon get started on rolling out a release candidate based off trunk.
Like with 2.8, I’d like to go back and prepare some notes on trunk’s content so we can objectively discuss about it.
+Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Vavilapalli <vi...@hortonworks.com>.
I’ll let others comment on specific features.
Regarding the 3.x vs 2.x point, as I noted before on other threads, given all the incompatibilities in trunk it will be ways off before users can run their production workloads on a 3.x release. Therefore, as I was proposing before, we should continue the 2.x lines, but soon get started on rolling out a release candidate based off trunk.
Like with 2.8, I’d like to go back and prepare some notes on trunk’s content so we can objectively discuss about it.
+Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Vavilapalli <vi...@hortonworks.com>.
I’ll let others comment on specific features.
Regarding the 3.x vs 2.x point, as I noted before on other threads, given all the incompatibilities in trunk it will be ways off before users can run their production workloads on a 3.x release. Therefore, as I was proposing before, we should continue the 2.x lines, but soon get started on rolling out a release candidate based off trunk.
Like with 2.8, I’d like to go back and prepare some notes on trunk’s content so we can objectively discuss about it.
+Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Vavilapalli <vi...@hortonworks.com>.
I’ll let others comment on specific features.
Regarding the 3.x vs 2.x point, as I noted before on other threads, given all the incompatibilities in trunk it will be ways off before users can run their production workloads on a 3.x release. Therefore, as I was proposing before, we should continue the 2.x lines, but soon get started on rolling out a release candidate based off trunk.
Like with 2.8, I’d like to go back and prepare some notes on trunk’s content so we can objectively discuss about it.
+Vinod
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 11, 2015, at 12:13 PM, Vinod Vavilapalli <vi...@hortonworks.com> wrote:
>
> — HDFS-6200 Create a separate jar for hdfs-client: Compatible improvement - no dimension of alpha/betaness here.
IMO: this feels like a massive break in backwards compatibility. Anyone who is looking for specific methods in specific jars are going to have a bad time. Also, it seems as though every week a new issue crops up that is related to this change. Is Slider still having problems with it? The reasoning “well, the pom sets the dependencies so it’s ok” feels like an *extremely weak* reason this wasn’t marked incompatible— it basically makes the assumption that everyone recompiles for every minor release.
> — Compatibility tools to catch backwards, forwards compatibility issues at patch submission, release times. Some of it is captured at YARN-3292. This also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
There has been talk in the past about adding Java ACC support to Yetus.
> Thoughts?
I’d rather see efforts on 3.x than another disastrous 2.x release. The track record is not good. At least a new major will signify that danger looms ahead. We’re already treating 2.x minor releases as effectively major (see the list of incompatible JIRAs) so what different does it make if we do 2.x vs. 3.x anyway?
>
> Thanks
> +Vinod
> PS:As you may have noted above, this time around, I want to do something that we’ve always wanted to do, but never explicitly did. I’m calling out readiness of each feature as they stand today so we can inform our users better of what they can start relying on in production clusters.
… except some of these changes are so deep reaching that even if you don’t use the feature, you’re still impacted by it ...
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Li Lu <ll...@hortonworks.com>.
On Nov 11, 2015, at 12:13, Vinod Vavilapalli <vi...@hortonworks.com>> wrote:
— YARN Timeline Service v1.5 - YARN-4233: A short term bridge before YARN-2928 comes around. I think this should go in given the tremendous activity recently.
+1, let’s target ATS v1.5 work to 2.8. Most critical patches will be ready for review this week. YARN-4234 also addresses the versioning problem for ATS v1, v1.5, and v2, which will bridge the gap between v1 and v2.
— YARN Timeline Service Next generation: YARN-2928: Lots of momentum, but clearly a work in progress. Two options here
— If it is safe to ship it into 2.8 in a disable manner, we can get the early code into trunk and all the way int o2.8.
Let’s review all changes in YARN-2928 branch and evaluate their impact on the upstream code. Right now in YARN-2928 branch we’ve modified RM/NM/Distributed shell and MR. Before we make a decision on this I think we need to review their effects so that ATS v2 (in its alpha mode) is not affecting existing parts.
— If it is not safe, it organically rolls over into 2.9
Sure.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 11, 2015, at 12:13 PM, Vinod Vavilapalli <vi...@hortonworks.com> wrote:
>
> — HDFS-6200 Create a separate jar for hdfs-client: Compatible improvement - no dimension of alpha/betaness here.
IMO: this feels like a massive break in backwards compatibility. Anyone who is looking for specific methods in specific jars are going to have a bad time. Also, it seems as though every week a new issue crops up that is related to this change. Is Slider still having problems with it? The reasoning “well, the pom sets the dependencies so it’s ok” feels like an *extremely weak* reason this wasn’t marked incompatible— it basically makes the assumption that everyone recompiles for every minor release.
> — Compatibility tools to catch backwards, forwards compatibility issues at patch submission, release times. Some of it is captured at YARN-3292. This also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
There has been talk in the past about adding Java ACC support to Yetus.
> Thoughts?
I’d rather see efforts on 3.x than another disastrous 2.x release. The track record is not good. At least a new major will signify that danger looms ahead. We’re already treating 2.x minor releases as effectively major (see the list of incompatible JIRAs) so what different does it make if we do 2.x vs. 3.x anyway?
>
> Thanks
> +Vinod
> PS:As you may have noted above, this time around, I want to do something that we’ve always wanted to do, but never explicitly did. I’m calling out readiness of each feature as they stand today so we can inform our users better of what they can start relying on in production clusters.
… except some of these changes are so deep reaching that even if you don’t use the feature, you’re still impacted by it ...
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Karthik Kambatla <ka...@cloudera.com>.
I am really against the notion of calling x.y.0 releases alpha/beta; it is
very confusing. If we think a release is alpha/beta quality, why not
release it as x.y.0-alpha or x.y.0-beta, and follow it up eventually with
x.y.0 GA.
I am in favor of labeling any of the newer features to be of alpha/beta
quality.
SharedCache is another close to done feature.
On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vinodkv@hortonworks.com
> wrote:
> Agreed on not mixing this with major release discussions.
>
> Okay, I just finished my review of 2.8 content.
>
> A quick summary follows.
>
> Current state of originally planned items
>
> - Nearly Done / Done and so need to close down quickly
> — Support *both* JDK7 and JDK8 runtimes HADOOP-11090
> — Supporting non-exclusive node-labels: YARN-3214: Done, can push as
> an alpha / beta feature
> — Support priorities across applications within the same queue
> YARN-1963: Can push as an alpha / beta feature
>
> - Definitely have to move out into 2.9 and beyond
> — Early work for disk and network isolation in YARN: YARN-2139,
> YARN-2140: Early noise, some critical pieces designed, done but not a lot
> of movement of late
> — Classpath isolation for downstream clients HADOOP-11656: Lots of
> chatter a while ago, not much movement of late
> — Support for Erasure Codes in HDFS HDFS-7285<
> https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the
> interest of stability / bake-in
>
> Non-planned features that went into 2.8.0
>
> — The overall list of new features:
> https://issues.apache.org/jira/issues/?filter=12333994
> — HDFS-6200 Create a separate jar for hdfs-client: Compatible
> improvement - no dimension of alpha/betaness here.
> — HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
> — Stability improvements ready to use:
> — HDFS-8008 Support client-side back off when the datanodes are
> congested
> — HDFS-8009 Signal congestion on the DataNode
> — YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> most of it anyways. Can push as an alpha feature.
> — YARN-1197 Support changing resources of an allocated container: Can
> push as an alpha/beta feature
>
> Items in progress to think about in 2 weeks
>
> — YARN Timeline Service v1.5 - YARN-4233: A short term bridge before
> YARN-2928 comes around. I think this should go in given the tremendous
> activity recently.
> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
> but clearly a work in progress. Two options here
> — If it is safe to ship it into 2.8 in a disable manner, we can
> get the early code into trunk and all the way int o2.8.
> — If it is not safe, it organically rolls over into 2.9
> — Compatibility tools to catch backwards, forwards compatibility
> issues at patch submission, release times. Some of it is captured at
> YARN-3292. This also involves resurrecting jdiff
> (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
>
> This is my plan of action for now in terms of the release itself
>
> - Cut a branch about two weeks from now
> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> - As with 2.7.x series, the first release will still be called as early /
> alpha release in the interest of
> — gaining downstream adoption
> — wider testing,
> — yet reserving our right to fix any inadvertent incompatibilities
> introduced.
>
> If we can get answers on “Items to think about now” during this and next
> week, we will overall be in good shape.
>
> Thoughts?
>
> Thanks
> +Vinod
> PS:As you may have noted above, this time around, I want to do something
> that we’ve always wanted to do, but never explicitly did. I’m calling out
> readiness of each feature as they stand today so we can inform our users
> better of what they can start relying on in production clusters.
>
>
> On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cmccabe@apache.org<mailto:
> cmccabe@apache.org>> wrote:
>
> I think it makes sense to have a 2.8 release since there are a
> tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
> release seems like something we should consider separately since it
> would not have the same compatibility guarantees as a 2.8 release.
> There's a pretty big delta between trunk and 2.8 as well.
>
> cheers,
> Colin
>
> On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cdouglas@apache.org
> <ma...@apache.org>> wrote:
> With two active sustaining branches (2.6, 2.7), what would you think
> of releasing trunk as 3.x instead of pushing 2.8? There are many new
> features (EC, Y1197, etc.), and trunk could be the source of several
> alpha/beta releases before we fork the 3.x line. -C
>
> On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
> <vi...@hortonworks.com>> wrote:
> As you may have noted, 2.8.0 got completely derailed what with 2.7.x and
> the unusually long 2.6.1 release.
>
> With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2
> and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.
>
> I’ll do a quick survey of where the individual features are and the amount
> of content already present in 2.8 and kick-start 2.8.0 process again.
>
> +Vinod
>
>
> On Apr 21, 2015, at 2:39 PM, vinodkv@apache.org<ma...@apache.org>
> wrote:
>
> With 2.7.0 out of the way, and with more maintenance releases to stabilize
> it, I propose we start thinking about 2.8.0.
>
> Here's my first cut of the proposal, will update the Roadmap wiki.
> - Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
> - Compatibility tools to catch backwards, forwards compatibility issues at
> patch submission, release times. Some of it is captured at YARN-3292. This
> also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310)
> and/or investing in new tools.
> - HADOOP-11656 Classpath isolation for downstream clients
> - Support for Erasure Codes in HDFS HDFS-7285
> - Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
> - YARN Timeline Service Next generation: YARN-2928. At least branch-merge
> + early peek.
> - Supporting non-exclusive node-labels: YARN-3214
>
> I'm experimenting with more agile 2.7.x releases and would like to
> continue the same by volunteering as the RM for 2.8.x too.
>
> Given the long time we took with 2.7.0, the timeline I am looking at is
> 8-12 weeks. We can pick as many features as they finish along and make a
> more predictable releases instead of holding up releases for ever.
>
> Thoughts?
>
> Thanks
> +Vinod
>
>
>
>
RE: [DISCUSS] Looking to a 2.8.0 release
Posted by Carlo Curino <cc...@microsoft.com>.
Hi Vinod,
Beside the HA work Subru mentioned (YARN-2573), we are working on a few more improvements of the reservation system
under umbrella YARN-2572.
A few patches are already checked in in 2.8 (e.g., YARN-3454, YARN-4248, YARN-4358), these are a combination of tooling
and performance improvements (all transparent to users that are not using reservations, and backward compatible improvements
for folks that are using reservations). We are trying to get as much of this finished by the 2.8 release cut, however we should
not hold up the release for this too much (e.g., YARN-4359 is being coded as we speak and might not make it).
Please let me know whether this sounds good, and whether it is fine to keep committing these patches (YARN-4340, YARN-4420,
YARN-4360, YARN-4359) as they become ready to 2.8 until is finalized.
Cheers,
Carlo
-----Original Message-----
From: Subramaniam V K [mailto:subru.vk@gmail.com]
Sent: Wednesday, November 25, 2015 11:47 AM
To: yarn-dev@hadoop.apache.org
Cc: Hadoop Common <co...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org
Subject: Re: [DISCUSS] Looking to a 2.8.0 release
Hi Vinod,
Thanks for driving this. Can you add YARN-2573 which includes the work done to integrate ReservationSystem with the RM failover mechanism to your list.
This can be reviewed and committed (branch-2) also about a month back.
Cheers,
Subru
On Wed, Nov 25, 2015 at 11:37 AM, Vinod Kumar Vavilapalli < vinodkv@apache.org> wrote:
> I think we’ve converged at a high level w.r.t 2.8. And as I just sent
> out an email, I updated the Roadmap wiki reflecting the same:
> https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fwiki.
> apache.org%2fhadoop%2fRoadmap&data=01%7c01%7cCCURINO%40exchange.micros
> oft.com%7c3bc4b2cea9ee42aa858f08d2f5d132e1%7c72f988bf86f141af91ab2d7cd
> 011db47%7c1&sdata=duyyuWIqj1o%2fgZLenhz%2bx3g4QqLMIpSZiAxC55nHa7Q%3d
>
> I plan to create a 2.8 branch EOD today.
>
> The goal for all of us should be to restrict improvements & fixes to
> only
> (a) the feature-set documented under 2.8 in the RoadMap wiki and (b)
> other minor features that are already in 2.8.
>
> Thanks
> +Vinod
>
>
> > On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
> >
> > - Cut a branch about two weeks from now
> > - Do an RC mid next month (leaving ~4weeks since branch-cut)
> > - As with 2.7.x series, the first release will still be called as
> > early
> / alpha release in the interest of
> > — gaining downstream adoption
> > — wider testing,
> > — yet reserving our right to fix any inadvertent
> > incompatibilities
> introduced.
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Subramaniam V K <su...@gmail.com>.
Hi Vinod,
Thanks for driving this. Can you add YARN-2573 which includes the work done
to integrate ReservationSystem with the RM failover mechanism to your list.
This can be reviewed and committed (branch-2) also about a month back.
Cheers,
Subru
On Wed, Nov 25, 2015 at 11:37 AM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:
> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out
> an email, I updated the Roadmap wiki reflecting the same:
> https://wiki.apache.org/hadoop/Roadmap
>
> I plan to create a 2.8 branch EOD today.
>
> The goal for all of us should be to restrict improvements & fixes to only
> (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other
> minor features that are already in 2.8.
>
> Thanks
> +Vinod
>
>
> > On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
> >
> > - Cut a branch about two weeks from now
> > - Do an RC mid next month (leaving ~4weeks since branch-cut)
> > - As with 2.7.x series, the first release will still be called as early
> / alpha release in the interest of
> > — gaining downstream adoption
> > — wider testing,
> > — yet reserving our right to fix any inadvertent incompatibilities
> introduced.
>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
After some googling, I discovered we need the sign profile to make deploy
work:
mvn deploy -Psign -DskipTests -DskipShading
Updated HowToRelease accordingly.
On Fri, Jan 20, 2017 at 11:15 AM, Andrew Wang <an...@cloudera.com>
wrote:
> What I meant is that you can mvn deploy the specific missing files to fix
> up the repo state. Though we should also figure out why the deploy plugin
> didn't pick up the signature files the first time.
>
> Best,
> Andrew
>
> On Fri, Jan 20, 2017 at 11:12 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Yes. I did maven deploy in root directory before close the staging
>> repository. If this is the only suspect, I can drop the repository and do
>> mvn deploy again.
>>
>>
>> Thanks,
>>
>>
>> Junping
>> ------------------------------
>> *From:* Andrew Wang <an...@cloudera.com>
>> *Sent:* Friday, January 20, 2017 10:48 AM
>> *To:* Junping Du
>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>
>> *Subject:* Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> You can check the error message by clicking on it, a bunch like this:
>>
>> Missing Signature: '/org/apache/hadoop/hadoop-map
>> reduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc'
>> does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
>>
>> Did you maven deploy all the signature files?
>>
>> On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com> wrote:
>>
>>> Hi,
>>> I have successfully built the release bit on branch-2.8.0 by
>>> following: https://wiki.apache.org/hadoop/HowToRelease step by step.
>>> However, when try to close the "Staging Repositories" at Nexus page (
>>> https://repository.apache.org/#stagingRepositories), I found our
>>> repository - orgapachehadoop-1051 cannot be closed due to some signature
>>> validation failed and one Apache rule failed. I never met this problem
>>> before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent
>>> changes on release process/tools (docker based)? Any ideas or thoughts on
>>> how to fix the problem?
>>>
>>> Thanks,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Thursday, January 19, 2017 6:46 PM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Cc: Varun Vasudev
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> According to Varun's offline email, the security fixes has landed on
>>> branch-2, 2.8 and 2.8.0 branch.
>>> I was kicking off a new RC build (RC1), and will publish it for vote
>>> soon. In the mean time, please mark fix version as 2.8.1 for any new
>>> commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at
>>> this moment. Thanks!
>>>
>>> Cheers,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Wednesday, January 18, 2017 3:26 PM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Cc: Varun Vasudev
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Hi folks,
>>> In the passed one or two weeks, we found some new blockers get
>>> coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM
>>> restart with work preserving enabled) and YARN-6072 (RM unable to start in
>>> secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I
>>> fixed YARN-6068), and I was starting the RC build process since early this
>>> week. As we have significant build tools/process change (docker based)
>>> since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with
>>> it and finally get a successful build on 2.8.0-RC0 last night.
>>> I already push RC0 tag into public which is prerequisite step
>>> before RC voting. However, in the mean while, I was pinged by Varun
>>> Vasudev that there are a known vulnerability issues on container_executor
>>> get identified and discussed in hadoop security email threads - looks like
>>> YARN-5704 fixed part of it, but left part - the privilege escalation via
>>> /proc/self/environ is not fixed yet. So most likely, I have to withdraw our
>>> 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait
>>> this issue get fixed to prepare a new release candidate. As RC0 tag cannot
>>> be reverted after push into apache, our next release candidate will start
>>> from RC1.
>>> As I mentioned in early email, 2.8.0 is a very big release (2300+
>>> commits since 2.7.3) and I am glad that we are almost there. Thanks
>>> everyone for being patient and contributing our release work. Please let me
>>> know if you have more comments or suggestions.
>>>
>>> Thanks,
>>>
>>> Junping
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Wednesday, January 04, 2017 1:41 AM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Hi all Hadoopers,
>>> I just commit YARN-3866 which is the last blocker for branch-2.8,
>>> so since tomorrow I will kick off process to prepare the first RC for our
>>> 2.8.0 release. This release will include at least 2374 commits that never
>>> landed before in previous 2.x releases. Please check
>>> https://s.apache.org/RW5k for details. There are also 352 fixes marked
>>> in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (
>>> https://s.apache.org/RKti) that I need to double check all are landed
>>> in branch-2.8 and fix the versions before kicking off the first RC.
>>> Our progress is slightly behind my estimation weeks ago. However,
>>> considering we just go through holidays and Jenkins cleanup issues linger
>>> on for several projects (YARN, etc.), our achievement here is still great!
>>> Thanks everyone for keep calm and carry on the help for the release. Kudos
>>> to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on
>>> blockers during this time. Also, special thanks to Vinod and Andrew for
>>> sharing knowledge and practice (include scripts for auto version check) for
>>> releasing effort.
>>> Will try to prepare RC0 of 2.8 release for vote within this week.
>>> Stay tuned!
>>>
>>> Thanks,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Wednesday, December 07, 2016 11:31 AM
>>> To: Akira Ajisaka; common-dev@hadoop.apache.org;
>>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>>> yarn-dev@hadoop.apache.org
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well
>>> for several runs before last weekend, but it get failed for latest several
>>> runs, probably affected by recently Jenkins down.
>>> However, from my recent manual kick off, it seems to be good again:
>>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/.
>>> I will keep an eye for today's nightly run to see if it back to normal.
>>>
>>>
>>> Thanks,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Akira Ajisaka <aj...@apache.org>
>>> Sent: Wednesday, December 07, 2016 12:12 AM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Thanks Junping and Andrew!
>>>
>>> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
>>> compare 2.8 with 2.7.3.
>>>
>>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-tr
>>> unk-JACC/24/artifact/target/compat-check/report.html
>>>
>>> Regards,
>>> Akira
>>>
>>> On 2016/12/02 2:08, Junping Du wrote:
>>> > Thanks Andrew! That's also a nice suggestion. I already create a
>>> similar job: https://builds.apache.org/view
>>> /H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs
>>> manually. Will monitor incompatible status from there.
>>> >
>>> >
>>> > Thanks,
>>> >
>>> >
>>> > Junping
>>> >
>>> >
>>> > ________________________________
>>> > From: Andrew Wang <an...@cloudera.com>
>>> > Sent: Wednesday, November 30, 2016 4:18 PM
>>> > To: Junping Du
>>> > Cc: Sangjin Lee; common-dev@hadoop.apache.org;
>>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>>> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
>>> sjlee@twitter.com
>>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>> >
>>> > I recommend giving the JACC report another look. I set up a
>>> parameterized jenkins job which you can trigger manually for branch-2.8:
>>> >
>>> > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>>> >
>>> > On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jdu@hortonworks.com
>>> <ma...@hortonworks.com>> wrote:
>>> > Hi Sangjin and all,
>>> >
>>> > That sounds good. If anyone have priority fix to backport ,
>>> please nominate it by following this email thread or ping me on JIRA
>>> directly. And I agree that we should keep in mind that our bar for 2.8
>>> release should be high now as we want to move quickly on this release.
>>> Non-critical fixes can wait for next one.
>>> >
>>> > Any other comments and suggestions?
>>> >
>>> >
>>> > Thanks,
>>> >
>>> >
>>> > Junping
>>> >
>>> >
>>> > ________________________________
>>> > From: sjlee0@gmail.com<ma...@gmail.com> <sjlee0@gmail.com
>>> <ma...@gmail.com>> on behalf of Sangjin Lee <sjlee@apache.org
>>> <ma...@apache.org>>
>>> > Sent: Wednesday, November 30, 2016 3:24 PM
>>> > To: Junping Du
>>> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod
>>> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<mailto:sjlee
>>> @twitter.com>
>>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>> >
>>> > Thanks for picking this up Junping!
>>> >
>>> > I heard on email threads and offline discussions that there may be
>>> interests in porting some fixes from branch-2 to branch-2.8. I still feel
>>> that the bar should be pretty high to do that, but it might be good to have
>>> folks suggest some critical fixes that need to be released sooner. Thoughts?
>>> >
>>> > On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com
>>> <ma...@hortonworks.com><mailto:jdu@hortonworks.com<mailto:jdu@hor
>>> tonworks.com>>> wrote:
>>> > Hi folks,
>>> > Per asked by Vinod offline, I would like to continue the effort
>>> to push 2.8 release out in next several weeks. The plan is as following:
>>> > - We only have 5 blockers and 3 critical issues (
>>> https://s.apache.org/6kwx) so far, and most of them are in good
>>> progress now. I put them all on apache ciwiki (
>>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release)
>>> and will do daily monitor and update on these issues in next couple of
>>> weeks.
>>> > - When all blocker/critical issues are gone (target EOD is Dec.
>>> 10), I will push out all major/minor issues that is open. So if you are
>>> working on some major issues and target for release in 2.8, please keep
>>> track of our release status and your work progress.
>>> > - I will create RC immediately once our 2.8 target issues are
>>> clean (especially for blocker and critical issues) for voting process.
>>> >
>>> > I hope we can make it within year 2016, best before Xmas holiday.
>>> And I need help from all of you. :)
>>> >
>>> >
>>> > Thanks,
>>> >
>>> > Junping
>>> >
>>> > ________________________________________
>>> > From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
>>> odkv@apache.org><ma...@apache.org>>>
>>> > Sent: Monday, July 25, 2016 2:57 PM
>>> > To: Jian He
>>> > Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.
>>> apache.org><mailto:mapreduce-dev@hadoop.apache.org<mailto:ma
>>> preduce-dev@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<mailto:
>>> yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto:
>>> yarn-dev@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<mailto:
>>> hdfs-dev@hadoop.apache.org><mailto:hdfs-dev@hadoop.apache.org<mailto:
>>> hdfs-dev@hadoop.apache.org>>; common-dev@hadoop.apache.org<mailto:
>>> common-dev@hadoop.apache.org><mailto:common-dev@hadoop.apache.org
>>> <ma...@hadoop.apache.org>>
>>> > Subject: Re: [Release thread] 2.8.0 release activities
>>> >
>>> > Thanks for all that great help moving this release forward, Jian,
>>> Sangjin, Wangda et al! Appreciate it.
>>> >
>>> > The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0
>>> last week.
>>> >
>>> > I only see 9 blocker / critical fixes pending for 2.8.0 -
>>> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>>> >
>>> > Thanks
>>> > +Vinod
>>> >
>>> >> On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
>>> e@hortonworks.com><ma...@hortonworks.com>>>
>>> wrote:
>>> >>
>>> >> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
>>> attention for 2.8
>>> >>
>>> >> MAPREDUCE-6288
>>> >> YARN-1815
>>> >> YARN-4685
>>> >> YARN-4844
>>> >>
>>> >> The rest are either improvements or long-standing issues and does not
>>> qualify release blocker, IMO.
>>> >> I think we’ll try to get these 4 jiras in asap. The rest will be on
>>> best effort, resolve as much as possible and move them out if not resolved
>>> in time.
>>> >>
>>> >> Jian
>>> >>
>>> >> On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:
>>> wheeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>>> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>>> ><ma...@gmail.com>>>> wrote:
>>> >>
>>> >> Sounds good to me :).
>>> >>
>>> >> Jian and I have looked at all existing 2.8.0 blockers and criticals
>>> today.
>>> >> To me more than half of MR/YARN blockers/criticals of 2.8 should be
>>> moved
>>> >> out. Left comments on these JIRAs asked original owners, plan to
>>> update
>>> >> target version of these JIRAs early next week.
>>> >>
>>> >> Will keep this thread updated.
>>> >>
>>> >> Thanks,
>>> >> Wangda
>>> >>
>>> >>
>>> >> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org
>>> <ma...@apache.org><mailto:sjlee@apache.org<mailto:
>>> sjlee@apache.org>><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>>> ><ma...@apache.org>>>> wrote:
>>> >>
>>> >> How about this? I'll review the HADOOP/HDFS bugs in that list to come
>>> up
>>> >> with true blockers for 2.8.0 or JIRAs that are close to being ready.
>>> I'll
>>> >> report the list here. Then folks can chime in if you agree
>>> >>
>>> >> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>> >>
>>> >> Thanks,
>>> >> Sangjin
>>> >>
>>> >> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
>>> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>>> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>>> ><ma...@gmail.com>>>> wrote:
>>> >>
>>> >> +1, we should close such staled JIRAs to avoid doing unnecessary
>>> checks
>>> >> for
>>> >> every releases.
>>> >>
>>> >> I'm working on reviewing YARN/MR critical/blocker patches currently,
>>> it
>>> >> gonna very helpful if someone else can help with reviewing Common/HDFS
>>> >> JIRAs.
>>> >>
>>> >> Thanks,
>>> >> Wangda
>>> >>
>>> >>
>>> >> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org
>>> <ma...@apache.org><mailto:sjlee@apache.org<mailto:
>>> sjlee@apache.org>><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>>> ><ma...@apache.org>>>> wrote:
>>> >>
>>> >> Where do we stand in terms of closing out blocker/critical issues for
>>> >> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>>> >> https://issues.apache.org/jira/issues/?filter=12334985
>>> >>
>>> >> But I see a lot of JIRAs with no patches or very stale patches. It
>>> >> would be
>>> >> a good exercise to come up with the list of JIRAs that we need to
>>> block
>>> >> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>> >>
>>> >> Thanks,
>>> >> Sangjin
>>> >>
>>> >> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <
>>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>>> stevel@hortonworks.com<ma...@hortonworks.com>><mailto:
>>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>>> stevel@hortonworks.com<ma...@hortonworks.com>>>
>>> >>
>>> >> wrote:
>>> >>
>>> >>
>>> >> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>>> >> vinodkv@apache.org<ma...@apache.org><mailto:vinodkv
>>> @apache.org<ma...@apache.org>><mailto:vinodkv@apache.org
>>> <ma...@apache.org><mailto:vinodkv@apache.org<mailto:
>>> vinodkv@apache.org>>>>
>>> >> wrote:
>>> >>
>>> >> We are not converging - there’s still 58 more. I need help from the
>>> >> community in addressing / review 2.8.0 blockers. If folks can start
>>> >> with
>>> >> reviewing Patch available tickets, that’ll be great.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> I'm still doing the s3a stuff, other people testing and reviewing this
>>> >> stuff welcome.
>>> >>
>>> >> in particular, I could do with others playing with this patch of mine,
>>> >> which adds counters and things into S3a, based on the azure
>>> >> instrumentation
>>> >>
>>> >> https://issues.apache.org/jira/browse/HADOOP-13028
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
>>> yarn-dev-unsubscribe@hadoop.apache.org><mailto:
>>> yarn-dev-unsubscribe@hadoop.apache.org<mailto:yarn-dev-
>>> unsubscribe@hadoop.apache.org>>
>>> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>> <ma...@hadoop.apache.org><mailto:yarn-dev-he
>>> lp@hadoop.apache.org<ma...@hadoop.apache.org>>
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
>>> <ma...@hadoop.apache.org><mailto:
>>> mapreduce-dev-unsubscribe@hadoop.apache.org<mailto:
>>> mapreduce-dev-unsubscribe@hadoop.apache.org>>
>>> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>>> <ma...@hadoop.apache.org><mailto:m
>>> apreduce-dev-help@hadoop.apache.org<mailto:mapreduce-dev-
>>> help@hadoop.apache.org>>
>>> >
>>> >
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>>
>>>
>>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
After some googling, I discovered we need the sign profile to make deploy
work:
mvn deploy -Psign -DskipTests -DskipShading
Updated HowToRelease accordingly.
On Fri, Jan 20, 2017 at 11:15 AM, Andrew Wang <an...@cloudera.com>
wrote:
> What I meant is that you can mvn deploy the specific missing files to fix
> up the repo state. Though we should also figure out why the deploy plugin
> didn't pick up the signature files the first time.
>
> Best,
> Andrew
>
> On Fri, Jan 20, 2017 at 11:12 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Yes. I did maven deploy in root directory before close the staging
>> repository. If this is the only suspect, I can drop the repository and do
>> mvn deploy again.
>>
>>
>> Thanks,
>>
>>
>> Junping
>> ------------------------------
>> *From:* Andrew Wang <an...@cloudera.com>
>> *Sent:* Friday, January 20, 2017 10:48 AM
>> *To:* Junping Du
>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>
>> *Subject:* Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> You can check the error message by clicking on it, a bunch like this:
>>
>> Missing Signature: '/org/apache/hadoop/hadoop-map
>> reduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc'
>> does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
>>
>> Did you maven deploy all the signature files?
>>
>> On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com> wrote:
>>
>>> Hi,
>>> I have successfully built the release bit on branch-2.8.0 by
>>> following: https://wiki.apache.org/hadoop/HowToRelease step by step.
>>> However, when try to close the "Staging Repositories" at Nexus page (
>>> https://repository.apache.org/#stagingRepositories), I found our
>>> repository - orgapachehadoop-1051 cannot be closed due to some signature
>>> validation failed and one Apache rule failed. I never met this problem
>>> before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent
>>> changes on release process/tools (docker based)? Any ideas or thoughts on
>>> how to fix the problem?
>>>
>>> Thanks,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Thursday, January 19, 2017 6:46 PM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Cc: Varun Vasudev
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> According to Varun's offline email, the security fixes has landed on
>>> branch-2, 2.8 and 2.8.0 branch.
>>> I was kicking off a new RC build (RC1), and will publish it for vote
>>> soon. In the mean time, please mark fix version as 2.8.1 for any new
>>> commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at
>>> this moment. Thanks!
>>>
>>> Cheers,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Wednesday, January 18, 2017 3:26 PM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Cc: Varun Vasudev
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Hi folks,
>>> In the passed one or two weeks, we found some new blockers get
>>> coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM
>>> restart with work preserving enabled) and YARN-6072 (RM unable to start in
>>> secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I
>>> fixed YARN-6068), and I was starting the RC build process since early this
>>> week. As we have significant build tools/process change (docker based)
>>> since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with
>>> it and finally get a successful build on 2.8.0-RC0 last night.
>>> I already push RC0 tag into public which is prerequisite step
>>> before RC voting. However, in the mean while, I was pinged by Varun
>>> Vasudev that there are a known vulnerability issues on container_executor
>>> get identified and discussed in hadoop security email threads - looks like
>>> YARN-5704 fixed part of it, but left part - the privilege escalation via
>>> /proc/self/environ is not fixed yet. So most likely, I have to withdraw our
>>> 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait
>>> this issue get fixed to prepare a new release candidate. As RC0 tag cannot
>>> be reverted after push into apache, our next release candidate will start
>>> from RC1.
>>> As I mentioned in early email, 2.8.0 is a very big release (2300+
>>> commits since 2.7.3) and I am glad that we are almost there. Thanks
>>> everyone for being patient and contributing our release work. Please let me
>>> know if you have more comments or suggestions.
>>>
>>> Thanks,
>>>
>>> Junping
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Wednesday, January 04, 2017 1:41 AM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Hi all Hadoopers,
>>> I just commit YARN-3866 which is the last blocker for branch-2.8,
>>> so since tomorrow I will kick off process to prepare the first RC for our
>>> 2.8.0 release. This release will include at least 2374 commits that never
>>> landed before in previous 2.x releases. Please check
>>> https://s.apache.org/RW5k for details. There are also 352 fixes marked
>>> in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (
>>> https://s.apache.org/RKti) that I need to double check all are landed
>>> in branch-2.8 and fix the versions before kicking off the first RC.
>>> Our progress is slightly behind my estimation weeks ago. However,
>>> considering we just go through holidays and Jenkins cleanup issues linger
>>> on for several projects (YARN, etc.), our achievement here is still great!
>>> Thanks everyone for keep calm and carry on the help for the release. Kudos
>>> to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on
>>> blockers during this time. Also, special thanks to Vinod and Andrew for
>>> sharing knowledge and practice (include scripts for auto version check) for
>>> releasing effort.
>>> Will try to prepare RC0 of 2.8 release for vote within this week.
>>> Stay tuned!
>>>
>>> Thanks,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Wednesday, December 07, 2016 11:31 AM
>>> To: Akira Ajisaka; common-dev@hadoop.apache.org;
>>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>>> yarn-dev@hadoop.apache.org
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well
>>> for several runs before last weekend, but it get failed for latest several
>>> runs, probably affected by recently Jenkins down.
>>> However, from my recent manual kick off, it seems to be good again:
>>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/.
>>> I will keep an eye for today's nightly run to see if it back to normal.
>>>
>>>
>>> Thanks,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Akira Ajisaka <aj...@apache.org>
>>> Sent: Wednesday, December 07, 2016 12:12 AM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Thanks Junping and Andrew!
>>>
>>> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
>>> compare 2.8 with 2.7.3.
>>>
>>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-tr
>>> unk-JACC/24/artifact/target/compat-check/report.html
>>>
>>> Regards,
>>> Akira
>>>
>>> On 2016/12/02 2:08, Junping Du wrote:
>>> > Thanks Andrew! That's also a nice suggestion. I already create a
>>> similar job: https://builds.apache.org/view
>>> /H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs
>>> manually. Will monitor incompatible status from there.
>>> >
>>> >
>>> > Thanks,
>>> >
>>> >
>>> > Junping
>>> >
>>> >
>>> > ________________________________
>>> > From: Andrew Wang <an...@cloudera.com>
>>> > Sent: Wednesday, November 30, 2016 4:18 PM
>>> > To: Junping Du
>>> > Cc: Sangjin Lee; common-dev@hadoop.apache.org;
>>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>>> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
>>> sjlee@twitter.com
>>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>> >
>>> > I recommend giving the JACC report another look. I set up a
>>> parameterized jenkins job which you can trigger manually for branch-2.8:
>>> >
>>> > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>>> >
>>> > On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jdu@hortonworks.com
>>> <ma...@hortonworks.com>> wrote:
>>> > Hi Sangjin and all,
>>> >
>>> > That sounds good. If anyone have priority fix to backport ,
>>> please nominate it by following this email thread or ping me on JIRA
>>> directly. And I agree that we should keep in mind that our bar for 2.8
>>> release should be high now as we want to move quickly on this release.
>>> Non-critical fixes can wait for next one.
>>> >
>>> > Any other comments and suggestions?
>>> >
>>> >
>>> > Thanks,
>>> >
>>> >
>>> > Junping
>>> >
>>> >
>>> > ________________________________
>>> > From: sjlee0@gmail.com<ma...@gmail.com> <sjlee0@gmail.com
>>> <ma...@gmail.com>> on behalf of Sangjin Lee <sjlee@apache.org
>>> <ma...@apache.org>>
>>> > Sent: Wednesday, November 30, 2016 3:24 PM
>>> > To: Junping Du
>>> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod
>>> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<mailto:sjlee
>>> @twitter.com>
>>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>> >
>>> > Thanks for picking this up Junping!
>>> >
>>> > I heard on email threads and offline discussions that there may be
>>> interests in porting some fixes from branch-2 to branch-2.8. I still feel
>>> that the bar should be pretty high to do that, but it might be good to have
>>> folks suggest some critical fixes that need to be released sooner. Thoughts?
>>> >
>>> > On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com
>>> <ma...@hortonworks.com><mailto:jdu@hortonworks.com<mailto:jdu@hor
>>> tonworks.com>>> wrote:
>>> > Hi folks,
>>> > Per asked by Vinod offline, I would like to continue the effort
>>> to push 2.8 release out in next several weeks. The plan is as following:
>>> > - We only have 5 blockers and 3 critical issues (
>>> https://s.apache.org/6kwx) so far, and most of them are in good
>>> progress now. I put them all on apache ciwiki (
>>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release)
>>> and will do daily monitor and update on these issues in next couple of
>>> weeks.
>>> > - When all blocker/critical issues are gone (target EOD is Dec.
>>> 10), I will push out all major/minor issues that is open. So if you are
>>> working on some major issues and target for release in 2.8, please keep
>>> track of our release status and your work progress.
>>> > - I will create RC immediately once our 2.8 target issues are
>>> clean (especially for blocker and critical issues) for voting process.
>>> >
>>> > I hope we can make it within year 2016, best before Xmas holiday.
>>> And I need help from all of you. :)
>>> >
>>> >
>>> > Thanks,
>>> >
>>> > Junping
>>> >
>>> > ________________________________________
>>> > From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
>>> odkv@apache.org><ma...@apache.org>>>
>>> > Sent: Monday, July 25, 2016 2:57 PM
>>> > To: Jian He
>>> > Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.
>>> apache.org><mailto:mapreduce-dev@hadoop.apache.org<mailto:ma
>>> preduce-dev@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<mailto:
>>> yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto:
>>> yarn-dev@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<mailto:
>>> hdfs-dev@hadoop.apache.org><mailto:hdfs-dev@hadoop.apache.org<mailto:
>>> hdfs-dev@hadoop.apache.org>>; common-dev@hadoop.apache.org<mailto:
>>> common-dev@hadoop.apache.org><mailto:common-dev@hadoop.apache.org
>>> <ma...@hadoop.apache.org>>
>>> > Subject: Re: [Release thread] 2.8.0 release activities
>>> >
>>> > Thanks for all that great help moving this release forward, Jian,
>>> Sangjin, Wangda et al! Appreciate it.
>>> >
>>> > The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0
>>> last week.
>>> >
>>> > I only see 9 blocker / critical fixes pending for 2.8.0 -
>>> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>>> >
>>> > Thanks
>>> > +Vinod
>>> >
>>> >> On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
>>> e@hortonworks.com><ma...@hortonworks.com>>>
>>> wrote:
>>> >>
>>> >> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
>>> attention for 2.8
>>> >>
>>> >> MAPREDUCE-6288
>>> >> YARN-1815
>>> >> YARN-4685
>>> >> YARN-4844
>>> >>
>>> >> The rest are either improvements or long-standing issues and does not
>>> qualify release blocker, IMO.
>>> >> I think we’ll try to get these 4 jiras in asap. The rest will be on
>>> best effort, resolve as much as possible and move them out if not resolved
>>> in time.
>>> >>
>>> >> Jian
>>> >>
>>> >> On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:
>>> wheeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>>> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>>> ><ma...@gmail.com>>>> wrote:
>>> >>
>>> >> Sounds good to me :).
>>> >>
>>> >> Jian and I have looked at all existing 2.8.0 blockers and criticals
>>> today.
>>> >> To me more than half of MR/YARN blockers/criticals of 2.8 should be
>>> moved
>>> >> out. Left comments on these JIRAs asked original owners, plan to
>>> update
>>> >> target version of these JIRAs early next week.
>>> >>
>>> >> Will keep this thread updated.
>>> >>
>>> >> Thanks,
>>> >> Wangda
>>> >>
>>> >>
>>> >> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org
>>> <ma...@apache.org><mailto:sjlee@apache.org<mailto:
>>> sjlee@apache.org>><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>>> ><ma...@apache.org>>>> wrote:
>>> >>
>>> >> How about this? I'll review the HADOOP/HDFS bugs in that list to come
>>> up
>>> >> with true blockers for 2.8.0 or JIRAs that are close to being ready.
>>> I'll
>>> >> report the list here. Then folks can chime in if you agree
>>> >>
>>> >> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>> >>
>>> >> Thanks,
>>> >> Sangjin
>>> >>
>>> >> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
>>> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>>> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>>> ><ma...@gmail.com>>>> wrote:
>>> >>
>>> >> +1, we should close such staled JIRAs to avoid doing unnecessary
>>> checks
>>> >> for
>>> >> every releases.
>>> >>
>>> >> I'm working on reviewing YARN/MR critical/blocker patches currently,
>>> it
>>> >> gonna very helpful if someone else can help with reviewing Common/HDFS
>>> >> JIRAs.
>>> >>
>>> >> Thanks,
>>> >> Wangda
>>> >>
>>> >>
>>> >> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org
>>> <ma...@apache.org><mailto:sjlee@apache.org<mailto:
>>> sjlee@apache.org>><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>>> ><ma...@apache.org>>>> wrote:
>>> >>
>>> >> Where do we stand in terms of closing out blocker/critical issues for
>>> >> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>>> >> https://issues.apache.org/jira/issues/?filter=12334985
>>> >>
>>> >> But I see a lot of JIRAs with no patches or very stale patches. It
>>> >> would be
>>> >> a good exercise to come up with the list of JIRAs that we need to
>>> block
>>> >> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>> >>
>>> >> Thanks,
>>> >> Sangjin
>>> >>
>>> >> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <
>>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>>> stevel@hortonworks.com<ma...@hortonworks.com>><mailto:
>>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>>> stevel@hortonworks.com<ma...@hortonworks.com>>>
>>> >>
>>> >> wrote:
>>> >>
>>> >>
>>> >> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>>> >> vinodkv@apache.org<ma...@apache.org><mailto:vinodkv
>>> @apache.org<ma...@apache.org>><mailto:vinodkv@apache.org
>>> <ma...@apache.org><mailto:vinodkv@apache.org<mailto:
>>> vinodkv@apache.org>>>>
>>> >> wrote:
>>> >>
>>> >> We are not converging - there’s still 58 more. I need help from the
>>> >> community in addressing / review 2.8.0 blockers. If folks can start
>>> >> with
>>> >> reviewing Patch available tickets, that’ll be great.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> I'm still doing the s3a stuff, other people testing and reviewing this
>>> >> stuff welcome.
>>> >>
>>> >> in particular, I could do with others playing with this patch of mine,
>>> >> which adds counters and things into S3a, based on the azure
>>> >> instrumentation
>>> >>
>>> >> https://issues.apache.org/jira/browse/HADOOP-13028
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
>>> yarn-dev-unsubscribe@hadoop.apache.org><mailto:
>>> yarn-dev-unsubscribe@hadoop.apache.org<mailto:yarn-dev-
>>> unsubscribe@hadoop.apache.org>>
>>> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>> <ma...@hadoop.apache.org><mailto:yarn-dev-he
>>> lp@hadoop.apache.org<ma...@hadoop.apache.org>>
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
>>> <ma...@hadoop.apache.org><mailto:
>>> mapreduce-dev-unsubscribe@hadoop.apache.org<mailto:
>>> mapreduce-dev-unsubscribe@hadoop.apache.org>>
>>> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>>> <ma...@hadoop.apache.org><mailto:m
>>> apreduce-dev-help@hadoop.apache.org<mailto:mapreduce-dev-
>>> help@hadoop.apache.org>>
>>> >
>>> >
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>>
>>>
>>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
After some googling, I discovered we need the sign profile to make deploy
work:
mvn deploy -Psign -DskipTests -DskipShading
Updated HowToRelease accordingly.
On Fri, Jan 20, 2017 at 11:15 AM, Andrew Wang <an...@cloudera.com>
wrote:
> What I meant is that you can mvn deploy the specific missing files to fix
> up the repo state. Though we should also figure out why the deploy plugin
> didn't pick up the signature files the first time.
>
> Best,
> Andrew
>
> On Fri, Jan 20, 2017 at 11:12 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Yes. I did maven deploy in root directory before close the staging
>> repository. If this is the only suspect, I can drop the repository and do
>> mvn deploy again.
>>
>>
>> Thanks,
>>
>>
>> Junping
>> ------------------------------
>> *From:* Andrew Wang <an...@cloudera.com>
>> *Sent:* Friday, January 20, 2017 10:48 AM
>> *To:* Junping Du
>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>
>> *Subject:* Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> You can check the error message by clicking on it, a bunch like this:
>>
>> Missing Signature: '/org/apache/hadoop/hadoop-map
>> reduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc'
>> does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
>>
>> Did you maven deploy all the signature files?
>>
>> On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com> wrote:
>>
>>> Hi,
>>> I have successfully built the release bit on branch-2.8.0 by
>>> following: https://wiki.apache.org/hadoop/HowToRelease step by step.
>>> However, when try to close the "Staging Repositories" at Nexus page (
>>> https://repository.apache.org/#stagingRepositories), I found our
>>> repository - orgapachehadoop-1051 cannot be closed due to some signature
>>> validation failed and one Apache rule failed. I never met this problem
>>> before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent
>>> changes on release process/tools (docker based)? Any ideas or thoughts on
>>> how to fix the problem?
>>>
>>> Thanks,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Thursday, January 19, 2017 6:46 PM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Cc: Varun Vasudev
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> According to Varun's offline email, the security fixes has landed on
>>> branch-2, 2.8 and 2.8.0 branch.
>>> I was kicking off a new RC build (RC1), and will publish it for vote
>>> soon. In the mean time, please mark fix version as 2.8.1 for any new
>>> commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at
>>> this moment. Thanks!
>>>
>>> Cheers,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Wednesday, January 18, 2017 3:26 PM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Cc: Varun Vasudev
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Hi folks,
>>> In the passed one or two weeks, we found some new blockers get
>>> coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM
>>> restart with work preserving enabled) and YARN-6072 (RM unable to start in
>>> secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I
>>> fixed YARN-6068), and I was starting the RC build process since early this
>>> week. As we have significant build tools/process change (docker based)
>>> since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with
>>> it and finally get a successful build on 2.8.0-RC0 last night.
>>> I already push RC0 tag into public which is prerequisite step
>>> before RC voting. However, in the mean while, I was pinged by Varun
>>> Vasudev that there are a known vulnerability issues on container_executor
>>> get identified and discussed in hadoop security email threads - looks like
>>> YARN-5704 fixed part of it, but left part - the privilege escalation via
>>> /proc/self/environ is not fixed yet. So most likely, I have to withdraw our
>>> 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait
>>> this issue get fixed to prepare a new release candidate. As RC0 tag cannot
>>> be reverted after push into apache, our next release candidate will start
>>> from RC1.
>>> As I mentioned in early email, 2.8.0 is a very big release (2300+
>>> commits since 2.7.3) and I am glad that we are almost there. Thanks
>>> everyone for being patient and contributing our release work. Please let me
>>> know if you have more comments or suggestions.
>>>
>>> Thanks,
>>>
>>> Junping
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Wednesday, January 04, 2017 1:41 AM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Hi all Hadoopers,
>>> I just commit YARN-3866 which is the last blocker for branch-2.8,
>>> so since tomorrow I will kick off process to prepare the first RC for our
>>> 2.8.0 release. This release will include at least 2374 commits that never
>>> landed before in previous 2.x releases. Please check
>>> https://s.apache.org/RW5k for details. There are also 352 fixes marked
>>> in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (
>>> https://s.apache.org/RKti) that I need to double check all are landed
>>> in branch-2.8 and fix the versions before kicking off the first RC.
>>> Our progress is slightly behind my estimation weeks ago. However,
>>> considering we just go through holidays and Jenkins cleanup issues linger
>>> on for several projects (YARN, etc.), our achievement here is still great!
>>> Thanks everyone for keep calm and carry on the help for the release. Kudos
>>> to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on
>>> blockers during this time. Also, special thanks to Vinod and Andrew for
>>> sharing knowledge and practice (include scripts for auto version check) for
>>> releasing effort.
>>> Will try to prepare RC0 of 2.8 release for vote within this week.
>>> Stay tuned!
>>>
>>> Thanks,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Wednesday, December 07, 2016 11:31 AM
>>> To: Akira Ajisaka; common-dev@hadoop.apache.org;
>>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>>> yarn-dev@hadoop.apache.org
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well
>>> for several runs before last weekend, but it get failed for latest several
>>> runs, probably affected by recently Jenkins down.
>>> However, from my recent manual kick off, it seems to be good again:
>>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/.
>>> I will keep an eye for today's nightly run to see if it back to normal.
>>>
>>>
>>> Thanks,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Akira Ajisaka <aj...@apache.org>
>>> Sent: Wednesday, December 07, 2016 12:12 AM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Thanks Junping and Andrew!
>>>
>>> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
>>> compare 2.8 with 2.7.3.
>>>
>>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-tr
>>> unk-JACC/24/artifact/target/compat-check/report.html
>>>
>>> Regards,
>>> Akira
>>>
>>> On 2016/12/02 2:08, Junping Du wrote:
>>> > Thanks Andrew! That's also a nice suggestion. I already create a
>>> similar job: https://builds.apache.org/view
>>> /H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs
>>> manually. Will monitor incompatible status from there.
>>> >
>>> >
>>> > Thanks,
>>> >
>>> >
>>> > Junping
>>> >
>>> >
>>> > ________________________________
>>> > From: Andrew Wang <an...@cloudera.com>
>>> > Sent: Wednesday, November 30, 2016 4:18 PM
>>> > To: Junping Du
>>> > Cc: Sangjin Lee; common-dev@hadoop.apache.org;
>>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>>> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
>>> sjlee@twitter.com
>>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>> >
>>> > I recommend giving the JACC report another look. I set up a
>>> parameterized jenkins job which you can trigger manually for branch-2.8:
>>> >
>>> > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>>> >
>>> > On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jdu@hortonworks.com
>>> <ma...@hortonworks.com>> wrote:
>>> > Hi Sangjin and all,
>>> >
>>> > That sounds good. If anyone have priority fix to backport ,
>>> please nominate it by following this email thread or ping me on JIRA
>>> directly. And I agree that we should keep in mind that our bar for 2.8
>>> release should be high now as we want to move quickly on this release.
>>> Non-critical fixes can wait for next one.
>>> >
>>> > Any other comments and suggestions?
>>> >
>>> >
>>> > Thanks,
>>> >
>>> >
>>> > Junping
>>> >
>>> >
>>> > ________________________________
>>> > From: sjlee0@gmail.com<ma...@gmail.com> <sjlee0@gmail.com
>>> <ma...@gmail.com>> on behalf of Sangjin Lee <sjlee@apache.org
>>> <ma...@apache.org>>
>>> > Sent: Wednesday, November 30, 2016 3:24 PM
>>> > To: Junping Du
>>> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod
>>> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<mailto:sjlee
>>> @twitter.com>
>>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>> >
>>> > Thanks for picking this up Junping!
>>> >
>>> > I heard on email threads and offline discussions that there may be
>>> interests in porting some fixes from branch-2 to branch-2.8. I still feel
>>> that the bar should be pretty high to do that, but it might be good to have
>>> folks suggest some critical fixes that need to be released sooner. Thoughts?
>>> >
>>> > On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com
>>> <ma...@hortonworks.com><mailto:jdu@hortonworks.com<mailto:jdu@hor
>>> tonworks.com>>> wrote:
>>> > Hi folks,
>>> > Per asked by Vinod offline, I would like to continue the effort
>>> to push 2.8 release out in next several weeks. The plan is as following:
>>> > - We only have 5 blockers and 3 critical issues (
>>> https://s.apache.org/6kwx) so far, and most of them are in good
>>> progress now. I put them all on apache ciwiki (
>>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release)
>>> and will do daily monitor and update on these issues in next couple of
>>> weeks.
>>> > - When all blocker/critical issues are gone (target EOD is Dec.
>>> 10), I will push out all major/minor issues that is open. So if you are
>>> working on some major issues and target for release in 2.8, please keep
>>> track of our release status and your work progress.
>>> > - I will create RC immediately once our 2.8 target issues are
>>> clean (especially for blocker and critical issues) for voting process.
>>> >
>>> > I hope we can make it within year 2016, best before Xmas holiday.
>>> And I need help from all of you. :)
>>> >
>>> >
>>> > Thanks,
>>> >
>>> > Junping
>>> >
>>> > ________________________________________
>>> > From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
>>> odkv@apache.org><ma...@apache.org>>>
>>> > Sent: Monday, July 25, 2016 2:57 PM
>>> > To: Jian He
>>> > Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.
>>> apache.org><mailto:mapreduce-dev@hadoop.apache.org<mailto:ma
>>> preduce-dev@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<mailto:
>>> yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto:
>>> yarn-dev@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<mailto:
>>> hdfs-dev@hadoop.apache.org><mailto:hdfs-dev@hadoop.apache.org<mailto:
>>> hdfs-dev@hadoop.apache.org>>; common-dev@hadoop.apache.org<mailto:
>>> common-dev@hadoop.apache.org><mailto:common-dev@hadoop.apache.org
>>> <ma...@hadoop.apache.org>>
>>> > Subject: Re: [Release thread] 2.8.0 release activities
>>> >
>>> > Thanks for all that great help moving this release forward, Jian,
>>> Sangjin, Wangda et al! Appreciate it.
>>> >
>>> > The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0
>>> last week.
>>> >
>>> > I only see 9 blocker / critical fixes pending for 2.8.0 -
>>> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>>> >
>>> > Thanks
>>> > +Vinod
>>> >
>>> >> On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
>>> e@hortonworks.com><ma...@hortonworks.com>>>
>>> wrote:
>>> >>
>>> >> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
>>> attention for 2.8
>>> >>
>>> >> MAPREDUCE-6288
>>> >> YARN-1815
>>> >> YARN-4685
>>> >> YARN-4844
>>> >>
>>> >> The rest are either improvements or long-standing issues and does not
>>> qualify release blocker, IMO.
>>> >> I think we’ll try to get these 4 jiras in asap. The rest will be on
>>> best effort, resolve as much as possible and move them out if not resolved
>>> in time.
>>> >>
>>> >> Jian
>>> >>
>>> >> On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:
>>> wheeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>>> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>>> ><ma...@gmail.com>>>> wrote:
>>> >>
>>> >> Sounds good to me :).
>>> >>
>>> >> Jian and I have looked at all existing 2.8.0 blockers and criticals
>>> today.
>>> >> To me more than half of MR/YARN blockers/criticals of 2.8 should be
>>> moved
>>> >> out. Left comments on these JIRAs asked original owners, plan to
>>> update
>>> >> target version of these JIRAs early next week.
>>> >>
>>> >> Will keep this thread updated.
>>> >>
>>> >> Thanks,
>>> >> Wangda
>>> >>
>>> >>
>>> >> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org
>>> <ma...@apache.org><mailto:sjlee@apache.org<mailto:
>>> sjlee@apache.org>><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>>> ><ma...@apache.org>>>> wrote:
>>> >>
>>> >> How about this? I'll review the HADOOP/HDFS bugs in that list to come
>>> up
>>> >> with true blockers for 2.8.0 or JIRAs that are close to being ready.
>>> I'll
>>> >> report the list here. Then folks can chime in if you agree
>>> >>
>>> >> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>> >>
>>> >> Thanks,
>>> >> Sangjin
>>> >>
>>> >> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
>>> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>>> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>>> ><ma...@gmail.com>>>> wrote:
>>> >>
>>> >> +1, we should close such staled JIRAs to avoid doing unnecessary
>>> checks
>>> >> for
>>> >> every releases.
>>> >>
>>> >> I'm working on reviewing YARN/MR critical/blocker patches currently,
>>> it
>>> >> gonna very helpful if someone else can help with reviewing Common/HDFS
>>> >> JIRAs.
>>> >>
>>> >> Thanks,
>>> >> Wangda
>>> >>
>>> >>
>>> >> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org
>>> <ma...@apache.org><mailto:sjlee@apache.org<mailto:
>>> sjlee@apache.org>><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>>> ><ma...@apache.org>>>> wrote:
>>> >>
>>> >> Where do we stand in terms of closing out blocker/critical issues for
>>> >> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>>> >> https://issues.apache.org/jira/issues/?filter=12334985
>>> >>
>>> >> But I see a lot of JIRAs with no patches or very stale patches. It
>>> >> would be
>>> >> a good exercise to come up with the list of JIRAs that we need to
>>> block
>>> >> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>> >>
>>> >> Thanks,
>>> >> Sangjin
>>> >>
>>> >> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <
>>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>>> stevel@hortonworks.com<ma...@hortonworks.com>><mailto:
>>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>>> stevel@hortonworks.com<ma...@hortonworks.com>>>
>>> >>
>>> >> wrote:
>>> >>
>>> >>
>>> >> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>>> >> vinodkv@apache.org<ma...@apache.org><mailto:vinodkv
>>> @apache.org<ma...@apache.org>><mailto:vinodkv@apache.org
>>> <ma...@apache.org><mailto:vinodkv@apache.org<mailto:
>>> vinodkv@apache.org>>>>
>>> >> wrote:
>>> >>
>>> >> We are not converging - there’s still 58 more. I need help from the
>>> >> community in addressing / review 2.8.0 blockers. If folks can start
>>> >> with
>>> >> reviewing Patch available tickets, that’ll be great.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> I'm still doing the s3a stuff, other people testing and reviewing this
>>> >> stuff welcome.
>>> >>
>>> >> in particular, I could do with others playing with this patch of mine,
>>> >> which adds counters and things into S3a, based on the azure
>>> >> instrumentation
>>> >>
>>> >> https://issues.apache.org/jira/browse/HADOOP-13028
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
>>> yarn-dev-unsubscribe@hadoop.apache.org><mailto:
>>> yarn-dev-unsubscribe@hadoop.apache.org<mailto:yarn-dev-
>>> unsubscribe@hadoop.apache.org>>
>>> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>> <ma...@hadoop.apache.org><mailto:yarn-dev-he
>>> lp@hadoop.apache.org<ma...@hadoop.apache.org>>
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
>>> <ma...@hadoop.apache.org><mailto:
>>> mapreduce-dev-unsubscribe@hadoop.apache.org<mailto:
>>> mapreduce-dev-unsubscribe@hadoop.apache.org>>
>>> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>>> <ma...@hadoop.apache.org><mailto:m
>>> apreduce-dev-help@hadoop.apache.org<mailto:mapreduce-dev-
>>> help@hadoop.apache.org>>
>>> >
>>> >
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>>
>>>
>>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
After some googling, I discovered we need the sign profile to make deploy
work:
mvn deploy -Psign -DskipTests -DskipShading
Updated HowToRelease accordingly.
On Fri, Jan 20, 2017 at 11:15 AM, Andrew Wang <an...@cloudera.com>
wrote:
> What I meant is that you can mvn deploy the specific missing files to fix
> up the repo state. Though we should also figure out why the deploy plugin
> didn't pick up the signature files the first time.
>
> Best,
> Andrew
>
> On Fri, Jan 20, 2017 at 11:12 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Yes. I did maven deploy in root directory before close the staging
>> repository. If this is the only suspect, I can drop the repository and do
>> mvn deploy again.
>>
>>
>> Thanks,
>>
>>
>> Junping
>> ------------------------------
>> *From:* Andrew Wang <an...@cloudera.com>
>> *Sent:* Friday, January 20, 2017 10:48 AM
>> *To:* Junping Du
>> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>
>> *Subject:* Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> You can check the error message by clicking on it, a bunch like this:
>>
>> Missing Signature: '/org/apache/hadoop/hadoop-map
>> reduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc'
>> does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
>>
>> Did you maven deploy all the signature files?
>>
>> On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com> wrote:
>>
>>> Hi,
>>> I have successfully built the release bit on branch-2.8.0 by
>>> following: https://wiki.apache.org/hadoop/HowToRelease step by step.
>>> However, when try to close the "Staging Repositories" at Nexus page (
>>> https://repository.apache.org/#stagingRepositories), I found our
>>> repository - orgapachehadoop-1051 cannot be closed due to some signature
>>> validation failed and one Apache rule failed. I never met this problem
>>> before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent
>>> changes on release process/tools (docker based)? Any ideas or thoughts on
>>> how to fix the problem?
>>>
>>> Thanks,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Thursday, January 19, 2017 6:46 PM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Cc: Varun Vasudev
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> According to Varun's offline email, the security fixes has landed on
>>> branch-2, 2.8 and 2.8.0 branch.
>>> I was kicking off a new RC build (RC1), and will publish it for vote
>>> soon. In the mean time, please mark fix version as 2.8.1 for any new
>>> commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at
>>> this moment. Thanks!
>>>
>>> Cheers,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Wednesday, January 18, 2017 3:26 PM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Cc: Varun Vasudev
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Hi folks,
>>> In the passed one or two weeks, we found some new blockers get
>>> coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM
>>> restart with work preserving enabled) and YARN-6072 (RM unable to start in
>>> secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I
>>> fixed YARN-6068), and I was starting the RC build process since early this
>>> week. As we have significant build tools/process change (docker based)
>>> since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with
>>> it and finally get a successful build on 2.8.0-RC0 last night.
>>> I already push RC0 tag into public which is prerequisite step
>>> before RC voting. However, in the mean while, I was pinged by Varun
>>> Vasudev that there are a known vulnerability issues on container_executor
>>> get identified and discussed in hadoop security email threads - looks like
>>> YARN-5704 fixed part of it, but left part - the privilege escalation via
>>> /proc/self/environ is not fixed yet. So most likely, I have to withdraw our
>>> 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait
>>> this issue get fixed to prepare a new release candidate. As RC0 tag cannot
>>> be reverted after push into apache, our next release candidate will start
>>> from RC1.
>>> As I mentioned in early email, 2.8.0 is a very big release (2300+
>>> commits since 2.7.3) and I am glad that we are almost there. Thanks
>>> everyone for being patient and contributing our release work. Please let me
>>> know if you have more comments or suggestions.
>>>
>>> Thanks,
>>>
>>> Junping
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Wednesday, January 04, 2017 1:41 AM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Hi all Hadoopers,
>>> I just commit YARN-3866 which is the last blocker for branch-2.8,
>>> so since tomorrow I will kick off process to prepare the first RC for our
>>> 2.8.0 release. This release will include at least 2374 commits that never
>>> landed before in previous 2.x releases. Please check
>>> https://s.apache.org/RW5k for details. There are also 352 fixes marked
>>> in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (
>>> https://s.apache.org/RKti) that I need to double check all are landed
>>> in branch-2.8 and fix the versions before kicking off the first RC.
>>> Our progress is slightly behind my estimation weeks ago. However,
>>> considering we just go through holidays and Jenkins cleanup issues linger
>>> on for several projects (YARN, etc.), our achievement here is still great!
>>> Thanks everyone for keep calm and carry on the help for the release. Kudos
>>> to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on
>>> blockers during this time. Also, special thanks to Vinod and Andrew for
>>> sharing knowledge and practice (include scripts for auto version check) for
>>> releasing effort.
>>> Will try to prepare RC0 of 2.8 release for vote within this week.
>>> Stay tuned!
>>>
>>> Thanks,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Junping Du <jd...@hortonworks.com>
>>> Sent: Wednesday, December 07, 2016 11:31 AM
>>> To: Akira Ajisaka; common-dev@hadoop.apache.org;
>>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>>> yarn-dev@hadoop.apache.org
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well
>>> for several runs before last weekend, but it get failed for latest several
>>> runs, probably affected by recently Jenkins down.
>>> However, from my recent manual kick off, it seems to be good again:
>>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/.
>>> I will keep an eye for today's nightly run to see if it back to normal.
>>>
>>>
>>> Thanks,
>>>
>>> Junping
>>>
>>> ________________________________________
>>> From: Akira Ajisaka <aj...@apache.org>
>>> Sent: Wednesday, December 07, 2016 12:12 AM
>>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>>
>>> Thanks Junping and Andrew!
>>>
>>> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
>>> compare 2.8 with 2.7.3.
>>>
>>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-tr
>>> unk-JACC/24/artifact/target/compat-check/report.html
>>>
>>> Regards,
>>> Akira
>>>
>>> On 2016/12/02 2:08, Junping Du wrote:
>>> > Thanks Andrew! That's also a nice suggestion. I already create a
>>> similar job: https://builds.apache.org/view
>>> /H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs
>>> manually. Will monitor incompatible status from there.
>>> >
>>> >
>>> > Thanks,
>>> >
>>> >
>>> > Junping
>>> >
>>> >
>>> > ________________________________
>>> > From: Andrew Wang <an...@cloudera.com>
>>> > Sent: Wednesday, November 30, 2016 4:18 PM
>>> > To: Junping Du
>>> > Cc: Sangjin Lee; common-dev@hadoop.apache.org;
>>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>>> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
>>> sjlee@twitter.com
>>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>> >
>>> > I recommend giving the JACC report another look. I set up a
>>> parameterized jenkins job which you can trigger manually for branch-2.8:
>>> >
>>> > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>>> >
>>> > On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jdu@hortonworks.com
>>> <ma...@hortonworks.com>> wrote:
>>> > Hi Sangjin and all,
>>> >
>>> > That sounds good. If anyone have priority fix to backport ,
>>> please nominate it by following this email thread or ping me on JIRA
>>> directly. And I agree that we should keep in mind that our bar for 2.8
>>> release should be high now as we want to move quickly on this release.
>>> Non-critical fixes can wait for next one.
>>> >
>>> > Any other comments and suggestions?
>>> >
>>> >
>>> > Thanks,
>>> >
>>> >
>>> > Junping
>>> >
>>> >
>>> > ________________________________
>>> > From: sjlee0@gmail.com<ma...@gmail.com> <sjlee0@gmail.com
>>> <ma...@gmail.com>> on behalf of Sangjin Lee <sjlee@apache.org
>>> <ma...@apache.org>>
>>> > Sent: Wednesday, November 30, 2016 3:24 PM
>>> > To: Junping Du
>>> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod
>>> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<mailto:sjlee
>>> @twitter.com>
>>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>> >
>>> > Thanks for picking this up Junping!
>>> >
>>> > I heard on email threads and offline discussions that there may be
>>> interests in porting some fixes from branch-2 to branch-2.8. I still feel
>>> that the bar should be pretty high to do that, but it might be good to have
>>> folks suggest some critical fixes that need to be released sooner. Thoughts?
>>> >
>>> > On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com
>>> <ma...@hortonworks.com><mailto:jdu@hortonworks.com<mailto:jdu@hor
>>> tonworks.com>>> wrote:
>>> > Hi folks,
>>> > Per asked by Vinod offline, I would like to continue the effort
>>> to push 2.8 release out in next several weeks. The plan is as following:
>>> > - We only have 5 blockers and 3 critical issues (
>>> https://s.apache.org/6kwx) so far, and most of them are in good
>>> progress now. I put them all on apache ciwiki (
>>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release)
>>> and will do daily monitor and update on these issues in next couple of
>>> weeks.
>>> > - When all blocker/critical issues are gone (target EOD is Dec.
>>> 10), I will push out all major/minor issues that is open. So if you are
>>> working on some major issues and target for release in 2.8, please keep
>>> track of our release status and your work progress.
>>> > - I will create RC immediately once our 2.8 target issues are
>>> clean (especially for blocker and critical issues) for voting process.
>>> >
>>> > I hope we can make it within year 2016, best before Xmas holiday.
>>> And I need help from all of you. :)
>>> >
>>> >
>>> > Thanks,
>>> >
>>> > Junping
>>> >
>>> > ________________________________________
>>> > From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
>>> odkv@apache.org><ma...@apache.org>>>
>>> > Sent: Monday, July 25, 2016 2:57 PM
>>> > To: Jian He
>>> > Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.
>>> apache.org><mailto:mapreduce-dev@hadoop.apache.org<mailto:ma
>>> preduce-dev@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<mailto:
>>> yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto:
>>> yarn-dev@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<mailto:
>>> hdfs-dev@hadoop.apache.org><mailto:hdfs-dev@hadoop.apache.org<mailto:
>>> hdfs-dev@hadoop.apache.org>>; common-dev@hadoop.apache.org<mailto:
>>> common-dev@hadoop.apache.org><mailto:common-dev@hadoop.apache.org
>>> <ma...@hadoop.apache.org>>
>>> > Subject: Re: [Release thread] 2.8.0 release activities
>>> >
>>> > Thanks for all that great help moving this release forward, Jian,
>>> Sangjin, Wangda et al! Appreciate it.
>>> >
>>> > The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0
>>> last week.
>>> >
>>> > I only see 9 blocker / critical fixes pending for 2.8.0 -
>>> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>>> >
>>> > Thanks
>>> > +Vinod
>>> >
>>> >> On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
>>> e@hortonworks.com><ma...@hortonworks.com>>>
>>> wrote:
>>> >>
>>> >> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
>>> attention for 2.8
>>> >>
>>> >> MAPREDUCE-6288
>>> >> YARN-1815
>>> >> YARN-4685
>>> >> YARN-4844
>>> >>
>>> >> The rest are either improvements or long-standing issues and does not
>>> qualify release blocker, IMO.
>>> >> I think we’ll try to get these 4 jiras in asap. The rest will be on
>>> best effort, resolve as much as possible and move them out if not resolved
>>> in time.
>>> >>
>>> >> Jian
>>> >>
>>> >> On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:
>>> wheeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>>> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>>> ><ma...@gmail.com>>>> wrote:
>>> >>
>>> >> Sounds good to me :).
>>> >>
>>> >> Jian and I have looked at all existing 2.8.0 blockers and criticals
>>> today.
>>> >> To me more than half of MR/YARN blockers/criticals of 2.8 should be
>>> moved
>>> >> out. Left comments on these JIRAs asked original owners, plan to
>>> update
>>> >> target version of these JIRAs early next week.
>>> >>
>>> >> Will keep this thread updated.
>>> >>
>>> >> Thanks,
>>> >> Wangda
>>> >>
>>> >>
>>> >> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org
>>> <ma...@apache.org><mailto:sjlee@apache.org<mailto:
>>> sjlee@apache.org>><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>>> ><ma...@apache.org>>>> wrote:
>>> >>
>>> >> How about this? I'll review the HADOOP/HDFS bugs in that list to come
>>> up
>>> >> with true blockers for 2.8.0 or JIRAs that are close to being ready.
>>> I'll
>>> >> report the list here. Then folks can chime in if you agree
>>> >>
>>> >> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>> >>
>>> >> Thanks,
>>> >> Sangjin
>>> >>
>>> >> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
>>> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>>> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>>> ><ma...@gmail.com>>>> wrote:
>>> >>
>>> >> +1, we should close such staled JIRAs to avoid doing unnecessary
>>> checks
>>> >> for
>>> >> every releases.
>>> >>
>>> >> I'm working on reviewing YARN/MR critical/blocker patches currently,
>>> it
>>> >> gonna very helpful if someone else can help with reviewing Common/HDFS
>>> >> JIRAs.
>>> >>
>>> >> Thanks,
>>> >> Wangda
>>> >>
>>> >>
>>> >> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org
>>> <ma...@apache.org><mailto:sjlee@apache.org<mailto:
>>> sjlee@apache.org>><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>>> ><ma...@apache.org>>>> wrote:
>>> >>
>>> >> Where do we stand in terms of closing out blocker/critical issues for
>>> >> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>>> >> https://issues.apache.org/jira/issues/?filter=12334985
>>> >>
>>> >> But I see a lot of JIRAs with no patches or very stale patches. It
>>> >> would be
>>> >> a good exercise to come up with the list of JIRAs that we need to
>>> block
>>> >> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>> >>
>>> >> Thanks,
>>> >> Sangjin
>>> >>
>>> >> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <
>>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>>> stevel@hortonworks.com<ma...@hortonworks.com>><mailto:
>>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>>> stevel@hortonworks.com<ma...@hortonworks.com>>>
>>> >>
>>> >> wrote:
>>> >>
>>> >>
>>> >> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>>> >> vinodkv@apache.org<ma...@apache.org><mailto:vinodkv
>>> @apache.org<ma...@apache.org>><mailto:vinodkv@apache.org
>>> <ma...@apache.org><mailto:vinodkv@apache.org<mailto:
>>> vinodkv@apache.org>>>>
>>> >> wrote:
>>> >>
>>> >> We are not converging - there’s still 58 more. I need help from the
>>> >> community in addressing / review 2.8.0 blockers. If folks can start
>>> >> with
>>> >> reviewing Patch available tickets, that’ll be great.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> I'm still doing the s3a stuff, other people testing and reviewing this
>>> >> stuff welcome.
>>> >>
>>> >> in particular, I could do with others playing with this patch of mine,
>>> >> which adds counters and things into S3a, based on the azure
>>> >> instrumentation
>>> >>
>>> >> https://issues.apache.org/jira/browse/HADOOP-13028
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
>>> yarn-dev-unsubscribe@hadoop.apache.org><mailto:
>>> yarn-dev-unsubscribe@hadoop.apache.org<mailto:yarn-dev-
>>> unsubscribe@hadoop.apache.org>>
>>> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>> <ma...@hadoop.apache.org><mailto:yarn-dev-he
>>> lp@hadoop.apache.org<ma...@hadoop.apache.org>>
>>> >
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
>>> <ma...@hadoop.apache.org><mailto:
>>> mapreduce-dev-unsubscribe@hadoop.apache.org<mailto:
>>> mapreduce-dev-unsubscribe@hadoop.apache.org>>
>>> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>>> <ma...@hadoop.apache.org><mailto:m
>>> apreduce-dev-help@hadoop.apache.org<mailto:mapreduce-dev-
>>> help@hadoop.apache.org>>
>>> >
>>> >
>>> >
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>>
>>>
>>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
What I meant is that you can mvn deploy the specific missing files to fix
up the repo state. Though we should also figure out why the deploy plugin
didn't pick up the signature files the first time.
Best,
Andrew
On Fri, Jan 20, 2017 at 11:12 AM, Junping Du <jd...@hortonworks.com> wrote:
> Yes. I did maven deploy in root directory before close the staging
> repository. If this is the only suspect, I can drop the repository and do
> mvn deploy again.
>
>
> Thanks,
>
>
> Junping
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>
> *Sent:* Friday, January 20, 2017 10:48 AM
> *To:* Junping Du
> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>
> *Subject:* Re: [Continued] [Release thread] 2.8.0 release activities
>
> You can check the error message by clicking on it, a bunch like this:
>
> Missing Signature: '/org/apache/hadoop/hadoop-
> mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc'
> does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
>
> Did you maven deploy all the signature files?
>
> On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Hi,
>> I have successfully built the release bit on branch-2.8.0 by
>> following: https://wiki.apache.org/hadoop/HowToRelease step by step.
>> However, when try to close the "Staging Repositories" at Nexus page (
>> https://repository.apache.org/#stagingRepositories), I found our
>> repository - orgapachehadoop-1051 cannot be closed due to some signature
>> validation failed and one Apache rule failed. I never met this problem
>> before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent
>> changes on release process/tools (docker based)? Any ideas or thoughts on
>> how to fix the problem?
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Thursday, January 19, 2017 6:46 PM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Cc: Varun Vasudev
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> According to Varun's offline email, the security fixes has landed on
>> branch-2, 2.8 and 2.8.0 branch.
>> I was kicking off a new RC build (RC1), and will publish it for vote
>> soon. In the mean time, please mark fix version as 2.8.1 for any new
>> commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at
>> this moment. Thanks!
>>
>> Cheers,
>>
>> Junping
>>
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Wednesday, January 18, 2017 3:26 PM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Cc: Varun Vasudev
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Hi folks,
>> In the passed one or two weeks, we found some new blockers get
>> coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM
>> restart with work preserving enabled) and YARN-6072 (RM unable to start in
>> secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I
>> fixed YARN-6068), and I was starting the RC build process since early this
>> week. As we have significant build tools/process change (docker based)
>> since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with
>> it and finally get a successful build on 2.8.0-RC0 last night.
>> I already push RC0 tag into public which is prerequisite step before
>> RC voting. However, in the mean while, I was pinged by Varun Vasudev that
>> there are a known vulnerability issues on container_executor get identified
>> and discussed in hadoop security email threads - looks like YARN-5704 fixed
>> part of it, but left part - the privilege escalation via /proc/self/environ
>> is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although
>> I haven't announced it public for vote yet. I will wait this issue get
>> fixed to prepare a new release candidate. As RC0 tag cannot be reverted
>> after push into apache, our next release candidate will start from RC1.
>> As I mentioned in early email, 2.8.0 is a very big release (2300+
>> commits since 2.7.3) and I am glad that we are almost there. Thanks
>> everyone for being patient and contributing our release work. Please let me
>> know if you have more comments or suggestions.
>>
>> Thanks,
>>
>> Junping
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Wednesday, January 04, 2017 1:41 AM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Hi all Hadoopers,
>> I just commit YARN-3866 which is the last blocker for branch-2.8, so
>> since tomorrow I will kick off process to prepare the first RC for our
>> 2.8.0 release. This release will include at least 2374 commits that never
>> landed before in previous 2.x releases. Please check
>> https://s.apache.org/RW5k for details. There are also 352 fixes marked
>> in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (
>> https://s.apache.org/RKti) that I need to double check all are landed in
>> branch-2.8 and fix the versions before kicking off the first RC.
>> Our progress is slightly behind my estimation weeks ago. However,
>> considering we just go through holidays and Jenkins cleanup issues linger
>> on for several projects (YARN, etc.), our achievement here is still great!
>> Thanks everyone for keep calm and carry on the help for the release. Kudos
>> to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on
>> blockers during this time. Also, special thanks to Vinod and Andrew for
>> sharing knowledge and practice (include scripts for auto version check) for
>> releasing effort.
>> Will try to prepare RC0 of 2.8 release for vote within this week.
>> Stay tuned!
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Wednesday, December 07, 2016 11:31 AM
>> To: Akira Ajisaka; common-dev@hadoop.apache.org;
>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well
>> for several runs before last weekend, but it get failed for latest several
>> runs, probably affected by recently Jenkins down.
>> However, from my recent manual kick off, it seems to be good again:
>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I
>> will keep an eye for today's nightly run to see if it back to normal.
>>
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Akira Ajisaka <aj...@apache.org>
>> Sent: Wednesday, December 07, 2016 12:12 AM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks Junping and Andrew!
>>
>> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
>> compare 2.8 with 2.7.3.
>>
>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-
>> trunk-JACC/24/artifact/target/compat-check/report.html
>>
>> Regards,
>> Akira
>>
>> On 2016/12/02 2:08, Junping Du wrote:
>> > Thanks Andrew! That's also a nice suggestion. I already create a
>> similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.
>> 8-JACC/ for 2.8 and kick off several runs manually. Will monitor
>> incompatible status from there.
>> >
>> >
>> > Thanks,
>> >
>> >
>> > Junping
>> >
>> >
>> > ________________________________
>> > From: Andrew Wang <an...@cloudera.com>
>> > Sent: Wednesday, November 30, 2016 4:18 PM
>> > To: Junping Du
>> > Cc: Sangjin Lee; common-dev@hadoop.apache.org;
>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
>> sjlee@twitter.com
>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>> >
>> > I recommend giving the JACC report another look. I set up a
>> parameterized jenkins job which you can trigger manually for branch-2.8:
>> >
>> > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>> >
>> > On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jdu@hortonworks.com
>> <ma...@hortonworks.com>> wrote:
>> > Hi Sangjin and all,
>> >
>> > That sounds good. If anyone have priority fix to backport , please
>> nominate it by following this email thread or ping me on JIRA directly. And
>> I agree that we should keep in mind that our bar for 2.8 release should be
>> high now as we want to move quickly on this release. Non-critical fixes can
>> wait for next one.
>> >
>> > Any other comments and suggestions?
>> >
>> >
>> > Thanks,
>> >
>> >
>> > Junping
>> >
>> >
>> > ________________________________
>> > From: sjlee0@gmail.com<ma...@gmail.com> <sjlee0@gmail.com
>> <ma...@gmail.com>> on behalf of Sangjin Lee <sjlee@apache.org
>> <ma...@apache.org>>
>> > Sent: Wednesday, November 30, 2016 3:24 PM
>> > To: Junping Du
>> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod
>> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<mailto:sjlee
>> @twitter.com>
>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>> >
>> > Thanks for picking this up Junping!
>> >
>> > I heard on email threads and offline discussions that there may be
>> interests in porting some fixes from branch-2 to branch-2.8. I still feel
>> that the bar should be pretty high to do that, but it might be good to have
>> folks suggest some critical fixes that need to be released sooner. Thoughts?
>> >
>> > On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com
>> <ma...@hortonworks.com><mailto:jdu@hortonworks.com<mailto:jdu@hor
>> tonworks.com>>> wrote:
>> > Hi folks,
>> > Per asked by Vinod offline, I would like to continue the effort to
>> push 2.8 release out in next several weeks. The plan is as following:
>> > - We only have 5 blockers and 3 critical issues (
>> https://s.apache.org/6kwx) so far, and most of them are in good progress
>> now. I put them all on apache ciwiki ( https://cwiki.apache.org/confl
>> uence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and
>> update on these issues in next couple of weeks.
>> > - When all blocker/critical issues are gone (target EOD is Dec.
>> 10), I will push out all major/minor issues that is open. So if you are
>> working on some major issues and target for release in 2.8, please keep
>> track of our release status and your work progress.
>> > - I will create RC immediately once our 2.8 target issues are
>> clean (especially for blocker and critical issues) for voting process.
>> >
>> > I hope we can make it within year 2016, best before Xmas holiday.
>> And I need help from all of you. :)
>> >
>> >
>> > Thanks,
>> >
>> > Junping
>> >
>> > ________________________________________
>> > From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
>> odkv@apache.org><ma...@apache.org>>>
>> > Sent: Monday, July 25, 2016 2:57 PM
>> > To: Jian He
>> > Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.
>> apache.org><mailto:mapreduce-dev@hadoop.apache.org<mailto:m
>> apreduce-dev@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<mailto:
>> yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto:
>> yarn-dev@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<mailto:
>> hdfs-dev@hadoop.apache.org><mailto:hdfs-dev@hadoop.apache.org<mailto:
>> hdfs-dev@hadoop.apache.org>>; common-dev@hadoop.apache.org<mailto:
>> common-dev@hadoop.apache.org><mailto:common-dev@hadoop.apache.org<mailto:
>> common-dev@hadoop.apache.org>>
>> > Subject: Re: [Release thread] 2.8.0 release activities
>> >
>> > Thanks for all that great help moving this release forward, Jian,
>> Sangjin, Wangda et al! Appreciate it.
>> >
>> > The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0
>> last week.
>> >
>> > I only see 9 blocker / critical fixes pending for 2.8.0 -
>> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>> >
>> > Thanks
>> > +Vinod
>> >
>> >> On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
>> e@hortonworks.com><ma...@hortonworks.com>>>
>> wrote:
>> >>
>> >> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
>> attention for 2.8
>> >>
>> >> MAPREDUCE-6288
>> >> YARN-1815
>> >> YARN-4685
>> >> YARN-4844
>> >>
>> >> The rest are either improvements or long-standing issues and does not
>> qualify release blocker, IMO.
>> >> I think we’ll try to get these 4 jiras in asap. The rest will be on
>> best effort, resolve as much as possible and move them out if not resolved
>> in time.
>> >>
>> >> Jian
>> >>
>> >> On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
>> eeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>> >><ma...@gmail.com><mailto:
>> wheeleast@gmail.com<ma...@gmail.com>>>> wrote:
>> >>
>> >> Sounds good to me :).
>> >>
>> >> Jian and I have looked at all existing 2.8.0 blockers and criticals
>> today.
>> >> To me more than half of MR/YARN blockers/criticals of 2.8 should be
>> moved
>> >> out. Left comments on these JIRAs asked original owners, plan to update
>> >> target version of these JIRAs early next week.
>> >>
>> >> Will keep this thread updated.
>> >>
>> >> Thanks,
>> >> Wangda
>> >>
>> >>
>> >> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>> >><ma...@apache.org><mailto:
>> sjlee@apache.org<ma...@apache.org>>>> wrote:
>> >>
>> >> How about this? I'll review the HADOOP/HDFS bugs in that list to come
>> up
>> >> with true blockers for 2.8.0 or JIRAs that are close to being ready.
>> I'll
>> >> report the list here. Then folks can chime in if you agree
>> >>
>> >> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>> >>
>> >> Thanks,
>> >> Sangjin
>> >>
>> >> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
>> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>> ><ma...@gmail.com>>>> wrote:
>> >>
>> >> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> >> for
>> >> every releases.
>> >>
>> >> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> >> gonna very helpful if someone else can help with reviewing Common/HDFS
>> >> JIRAs.
>> >>
>> >> Thanks,
>> >> Wangda
>> >>
>> >>
>> >> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>> >><ma...@apache.org><mailto:
>> sjlee@apache.org<ma...@apache.org>>>> wrote:
>> >>
>> >> Where do we stand in terms of closing out blocker/critical issues for
>> >> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> >> https://issues.apache.org/jira/issues/?filter=12334985
>> >>
>> >> But I see a lot of JIRAs with no patches or very stale patches. It
>> >> would be
>> >> a good exercise to come up with the list of JIRAs that we need to block
>> >> 2.8.0 for and focus our attention on closing them out. Thoughts?
>> >>
>> >> Thanks,
>> >> Sangjin
>> >>
>> >> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <
>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>> stevel@hortonworks.com<ma...@hortonworks.com>><mailto:
>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>> stevel@hortonworks.com<ma...@hortonworks.com>>>
>> >>
>> >> wrote:
>> >>
>> >>
>> >> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> >> vinodkv@apache.org<ma...@apache.org><mailto:vinodkv
>> @apache.org<ma...@apache.org>><mailto:vinodkv@apache.org<mailto:
>> vinodkv@apache.org><mailto:vinodkv@apache.org<mailto:vinodkv@apache.org
>> >>>>
>> >> wrote:
>> >>
>> >> We are not converging - there’s still 58 more. I need help from the
>> >> community in addressing / review 2.8.0 blockers. If folks can start
>> >> with
>> >> reviewing Patch available tickets, that’ll be great.
>> >>
>> >>
>> >>
>> >>
>> >> I'm still doing the s3a stuff, other people testing and reviewing this
>> >> stuff welcome.
>> >>
>> >> in particular, I could do with others playing with this patch of mine,
>> >> which adds counters and things into S3a, based on the azure
>> >> instrumentation
>> >>
>> >> https://issues.apache.org/jira/browse/HADOOP-13028
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
>> yarn-dev-unsubscribe@hadoop.apache.org><mailto:yarn-dev-unsubscribe@ha
>> doop.apache.org<ma...@hadoop.apache.org>>
>> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>> <ma...@hadoop.apache.org><mailto:yarn-dev-
>> help@hadoop.apache.org<ma...@hadoop.apache.org>>
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
>> <ma...@hadoop.apache.org><mailto:
>> mapreduce-dev-unsubscribe@hadoop.apache.org<mailto:
>> mapreduce-dev-unsubscribe@hadoop.apache.org>>
>> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>> <ma...@hadoop.apache.org><mailto:
>> mapreduce-dev-help@hadoop.apache.org<mailto:mapreduce-
>> dev-help@hadoop.apache.org>>
>> >
>> >
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>
>>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
If you ran mvn clean at any point in your repo between create-release and mvn deploy, you'll need to start at running create-release again. create-release leaves things in a state that mvn deploy should be ready to go, with no clean necessary.
> On Jan 20, 2017, at 11:12 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> Yes. I did maven deploy in root directory before close the staging repository. If this is the only suspect, I can drop the repository and do mvn deploy again.
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Friday, January 20, 2017 10:48 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> You can check the error message by clicking on it, a bunch like this:
>
> Missing Signature: '/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc' does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
>
> Did you maven deploy all the signature files?
>
> On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi,
> I have successfully built the release bit on branch-2.8.0 by following: https://wiki.apache.org/hadoop/HowToRelease step by step. However, when try to close the "Staging Repositories" at Nexus page (https://repository.apache.org/#stagingRepositories), I found our repository - orgapachehadoop-1051 cannot be closed due to some signature validation failed and one Apache rule failed. I never met this problem before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent changes on release process/tools (docker based)? Any ideas or thoughts on how to fix the problem?
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Thursday, January 19, 2017 6:46 PM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
> I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
>
> Cheers,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Wednesday, January 18, 2017 3:26 PM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi folks,
> In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
> I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
> As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
>
> Thanks,
>
> Junping
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Wednesday, January 04, 2017 1:41 AM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi all Hadoopers,
> I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
> Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
> Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Wednesday, December 07, 2016 11:31 AM
> To: Akira Ajisaka; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
> However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Akira Ajisaka <aj...@apache.org>>
> Sent: Wednesday, December 07, 2016 12:12 AM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Junping and Andrew!
>
> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
> compare 2.8 with 2.7.3.
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
>
> Regards,
> Akira
>
> On 2016/12/02 2:08, Junping Du wrote:
>> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>>
>> ________________________________
>> From: Andrew Wang <an...@cloudera.com>>
>> Sent: Wednesday, November 30, 2016 4:18 PM
>> To: Junping Du
>> Cc: Sangjin Lee; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>>
>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>>
>> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>>> wrote:
>> Hi Sangjin and all,
>>
>> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>>
>> Any other comments and suggestions?
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>>
>> ________________________________
>> From: sjlee0@gmail.com<ma...@gmail.com>> <sj...@gmail.com>>> on behalf of Sangjin Lee <sj...@apache.org>>>
>> Sent: Wednesday, November 30, 2016 3:24 PM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>>
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks for picking this up Junping!
>>
>> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>>
>> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>>> wrote:
>> Hi folks,
>> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
>> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
>> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
>> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>>
>> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>>
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>>
>> Sent: Monday, July 25, 2016 2:57 PM
>> To: Jian He
>> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>
>> Subject: Re: [Release thread] 2.8.0 release activities
>>
>> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>>
>> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>>
>> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>>
>> Thanks
>> +Vinod
>>
>>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>>> wrote:
>>>
>>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>>
>>> MAPREDUCE-6288
>>> YARN-1815
>>> YARN-4685
>>> YARN-4844
>>>
>>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>>
>>> Jian
>>>
>>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>>
>>> Sounds good to me :).
>>>
>>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>>> out. Left comments on these JIRAs asked original owners, plan to update
>>> target version of these JIRAs early next week.
>>>
>>> Will keep this thread updated.
>>>
>>> Thanks,
>>> Wangda
>>>
>>>
>>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>>
>>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>>> report the list here. Then folks can chime in if you agree
>>>
>>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>>
>>> Thanks,
>>> Sangjin
>>>
>>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>>
>>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>>> for
>>> every releases.
>>>
>>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>>> gonna very helpful if someone else can help with reviewing Common/HDFS
>>> JIRAs.
>>>
>>> Thanks,
>>> Wangda
>>>
>>>
>>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>>
>>> Where do we stand in terms of closing out blocker/critical issues for
>>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>>> https://issues.apache.org/jira/issues/?filter=12334985
>>>
>>> But I see a lot of JIRAs with no patches or very stale patches. It
>>> would be
>>> a good exercise to come up with the list of JIRAs that we need to block
>>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>>
>>> Thanks,
>>> Sangjin
>>>
>>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>>
>>>
>>> wrote:
>>>
>>>
>>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>>> vinodkv@apache.org<ma...@apache.org>>>>>
>>> wrote:
>>>
>>> We are not converging - there’s still 58 more. I need help from the
>>> community in addressing / review 2.8.0 blockers. If folks can start
>>> with
>>> reviewing Patch available tickets, that’ll be great.
>>>
>>>
>>>
>>>
>>> I'm still doing the s3a stuff, other people testing and reviewing this
>>> stuff welcome.
>>>
>>> in particular, I could do with others playing with this patch of mine,
>>> which adds counters and things into S3a, based on the azure
>>> instrumentation
>>>
>>> https://issues.apache.org/jira/browse/HADOOP-13028
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
>> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>>
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
If you ran mvn clean at any point in your repo between create-release and mvn deploy, you'll need to start at running create-release again. create-release leaves things in a state that mvn deploy should be ready to go, with no clean necessary.
> On Jan 20, 2017, at 11:12 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> Yes. I did maven deploy in root directory before close the staging repository. If this is the only suspect, I can drop the repository and do mvn deploy again.
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Friday, January 20, 2017 10:48 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> You can check the error message by clicking on it, a bunch like this:
>
> Missing Signature: '/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc' does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
>
> Did you maven deploy all the signature files?
>
> On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi,
> I have successfully built the release bit on branch-2.8.0 by following: https://wiki.apache.org/hadoop/HowToRelease step by step. However, when try to close the "Staging Repositories" at Nexus page (https://repository.apache.org/#stagingRepositories), I found our repository - orgapachehadoop-1051 cannot be closed due to some signature validation failed and one Apache rule failed. I never met this problem before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent changes on release process/tools (docker based)? Any ideas or thoughts on how to fix the problem?
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Thursday, January 19, 2017 6:46 PM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
> I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
>
> Cheers,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Wednesday, January 18, 2017 3:26 PM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi folks,
> In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
> I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
> As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
>
> Thanks,
>
> Junping
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Wednesday, January 04, 2017 1:41 AM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi all Hadoopers,
> I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
> Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
> Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Wednesday, December 07, 2016 11:31 AM
> To: Akira Ajisaka; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
> However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Akira Ajisaka <aj...@apache.org>>
> Sent: Wednesday, December 07, 2016 12:12 AM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Junping and Andrew!
>
> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
> compare 2.8 with 2.7.3.
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
>
> Regards,
> Akira
>
> On 2016/12/02 2:08, Junping Du wrote:
>> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>>
>> ________________________________
>> From: Andrew Wang <an...@cloudera.com>>
>> Sent: Wednesday, November 30, 2016 4:18 PM
>> To: Junping Du
>> Cc: Sangjin Lee; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>>
>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>>
>> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>>> wrote:
>> Hi Sangjin and all,
>>
>> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>>
>> Any other comments and suggestions?
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>>
>> ________________________________
>> From: sjlee0@gmail.com<ma...@gmail.com>> <sj...@gmail.com>>> on behalf of Sangjin Lee <sj...@apache.org>>>
>> Sent: Wednesday, November 30, 2016 3:24 PM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>>
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks for picking this up Junping!
>>
>> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>>
>> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>>> wrote:
>> Hi folks,
>> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
>> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
>> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
>> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>>
>> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>>
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>>
>> Sent: Monday, July 25, 2016 2:57 PM
>> To: Jian He
>> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>
>> Subject: Re: [Release thread] 2.8.0 release activities
>>
>> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>>
>> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>>
>> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>>
>> Thanks
>> +Vinod
>>
>>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>>> wrote:
>>>
>>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>>
>>> MAPREDUCE-6288
>>> YARN-1815
>>> YARN-4685
>>> YARN-4844
>>>
>>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>>
>>> Jian
>>>
>>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>>
>>> Sounds good to me :).
>>>
>>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>>> out. Left comments on these JIRAs asked original owners, plan to update
>>> target version of these JIRAs early next week.
>>>
>>> Will keep this thread updated.
>>>
>>> Thanks,
>>> Wangda
>>>
>>>
>>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>>
>>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>>> report the list here. Then folks can chime in if you agree
>>>
>>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>>
>>> Thanks,
>>> Sangjin
>>>
>>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>>
>>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>>> for
>>> every releases.
>>>
>>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>>> gonna very helpful if someone else can help with reviewing Common/HDFS
>>> JIRAs.
>>>
>>> Thanks,
>>> Wangda
>>>
>>>
>>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>>
>>> Where do we stand in terms of closing out blocker/critical issues for
>>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>>> https://issues.apache.org/jira/issues/?filter=12334985
>>>
>>> But I see a lot of JIRAs with no patches or very stale patches. It
>>> would be
>>> a good exercise to come up with the list of JIRAs that we need to block
>>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>>
>>> Thanks,
>>> Sangjin
>>>
>>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>>
>>>
>>> wrote:
>>>
>>>
>>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>>> vinodkv@apache.org<ma...@apache.org>>>>>
>>> wrote:
>>>
>>> We are not converging - there’s still 58 more. I need help from the
>>> community in addressing / review 2.8.0 blockers. If folks can start
>>> with
>>> reviewing Patch available tickets, that’ll be great.
>>>
>>>
>>>
>>>
>>> I'm still doing the s3a stuff, other people testing and reviewing this
>>> stuff welcome.
>>>
>>> in particular, I could do with others playing with this patch of mine,
>>> which adds counters and things into S3a, based on the azure
>>> instrumentation
>>>
>>> https://issues.apache.org/jira/browse/HADOOP-13028
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
>> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>>
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
If you ran mvn clean at any point in your repo between create-release and mvn deploy, you'll need to start at running create-release again. create-release leaves things in a state that mvn deploy should be ready to go, with no clean necessary.
> On Jan 20, 2017, at 11:12 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> Yes. I did maven deploy in root directory before close the staging repository. If this is the only suspect, I can drop the repository and do mvn deploy again.
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Friday, January 20, 2017 10:48 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> You can check the error message by clicking on it, a bunch like this:
>
> Missing Signature: '/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc' does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
>
> Did you maven deploy all the signature files?
>
> On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi,
> I have successfully built the release bit on branch-2.8.0 by following: https://wiki.apache.org/hadoop/HowToRelease step by step. However, when try to close the "Staging Repositories" at Nexus page (https://repository.apache.org/#stagingRepositories), I found our repository - orgapachehadoop-1051 cannot be closed due to some signature validation failed and one Apache rule failed. I never met this problem before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent changes on release process/tools (docker based)? Any ideas or thoughts on how to fix the problem?
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Thursday, January 19, 2017 6:46 PM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
> I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
>
> Cheers,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Wednesday, January 18, 2017 3:26 PM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi folks,
> In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
> I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
> As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
>
> Thanks,
>
> Junping
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Wednesday, January 04, 2017 1:41 AM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi all Hadoopers,
> I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
> Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
> Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Wednesday, December 07, 2016 11:31 AM
> To: Akira Ajisaka; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
> However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Akira Ajisaka <aj...@apache.org>>
> Sent: Wednesday, December 07, 2016 12:12 AM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Junping and Andrew!
>
> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
> compare 2.8 with 2.7.3.
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
>
> Regards,
> Akira
>
> On 2016/12/02 2:08, Junping Du wrote:
>> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>>
>> ________________________________
>> From: Andrew Wang <an...@cloudera.com>>
>> Sent: Wednesday, November 30, 2016 4:18 PM
>> To: Junping Du
>> Cc: Sangjin Lee; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>>
>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>>
>> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>>> wrote:
>> Hi Sangjin and all,
>>
>> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>>
>> Any other comments and suggestions?
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>>
>> ________________________________
>> From: sjlee0@gmail.com<ma...@gmail.com>> <sj...@gmail.com>>> on behalf of Sangjin Lee <sj...@apache.org>>>
>> Sent: Wednesday, November 30, 2016 3:24 PM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>>
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks for picking this up Junping!
>>
>> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>>
>> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>>> wrote:
>> Hi folks,
>> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
>> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
>> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
>> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>>
>> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>>
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>>
>> Sent: Monday, July 25, 2016 2:57 PM
>> To: Jian He
>> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>
>> Subject: Re: [Release thread] 2.8.0 release activities
>>
>> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>>
>> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>>
>> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>>
>> Thanks
>> +Vinod
>>
>>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>>> wrote:
>>>
>>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>>
>>> MAPREDUCE-6288
>>> YARN-1815
>>> YARN-4685
>>> YARN-4844
>>>
>>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>>
>>> Jian
>>>
>>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>>
>>> Sounds good to me :).
>>>
>>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>>> out. Left comments on these JIRAs asked original owners, plan to update
>>> target version of these JIRAs early next week.
>>>
>>> Will keep this thread updated.
>>>
>>> Thanks,
>>> Wangda
>>>
>>>
>>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>>
>>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>>> report the list here. Then folks can chime in if you agree
>>>
>>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>>
>>> Thanks,
>>> Sangjin
>>>
>>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>>
>>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>>> for
>>> every releases.
>>>
>>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>>> gonna very helpful if someone else can help with reviewing Common/HDFS
>>> JIRAs.
>>>
>>> Thanks,
>>> Wangda
>>>
>>>
>>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>>
>>> Where do we stand in terms of closing out blocker/critical issues for
>>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>>> https://issues.apache.org/jira/issues/?filter=12334985
>>>
>>> But I see a lot of JIRAs with no patches or very stale patches. It
>>> would be
>>> a good exercise to come up with the list of JIRAs that we need to block
>>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>>
>>> Thanks,
>>> Sangjin
>>>
>>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>>
>>>
>>> wrote:
>>>
>>>
>>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>>> vinodkv@apache.org<ma...@apache.org>>>>>
>>> wrote:
>>>
>>> We are not converging - there’s still 58 more. I need help from the
>>> community in addressing / review 2.8.0 blockers. If folks can start
>>> with
>>> reviewing Patch available tickets, that’ll be great.
>>>
>>>
>>>
>>>
>>> I'm still doing the s3a stuff, other people testing and reviewing this
>>> stuff welcome.
>>>
>>> in particular, I could do with others playing with this patch of mine,
>>> which adds counters and things into S3a, based on the azure
>>> instrumentation
>>>
>>> https://issues.apache.org/jira/browse/HADOOP-13028
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
>> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>>
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Allen Wittenauer <aw...@effectivemachines.com>.
If you ran mvn clean at any point in your repo between create-release and mvn deploy, you'll need to start at running create-release again. create-release leaves things in a state that mvn deploy should be ready to go, with no clean necessary.
> On Jan 20, 2017, at 11:12 AM, Junping Du <jd...@hortonworks.com> wrote:
>
> Yes. I did maven deploy in root directory before close the staging repository. If this is the only suspect, I can drop the repository and do mvn deploy again.
>
>
> Thanks,
>
>
> Junping
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Friday, January 20, 2017 10:48 AM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> You can check the error message by clicking on it, a bunch like this:
>
> Missing Signature: '/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc' does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
>
> Did you maven deploy all the signature files?
>
> On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi,
> I have successfully built the release bit on branch-2.8.0 by following: https://wiki.apache.org/hadoop/HowToRelease step by step. However, when try to close the "Staging Repositories" at Nexus page (https://repository.apache.org/#stagingRepositories), I found our repository - orgapachehadoop-1051 cannot be closed due to some signature validation failed and one Apache rule failed. I never met this problem before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent changes on release process/tools (docker based)? Any ideas or thoughts on how to fix the problem?
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Thursday, January 19, 2017 6:46 PM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
> I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
>
> Cheers,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Wednesday, January 18, 2017 3:26 PM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi folks,
> In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
> I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
> As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
>
> Thanks,
>
> Junping
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Wednesday, January 04, 2017 1:41 AM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi all Hadoopers,
> I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
> Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
> Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>>
> Sent: Wednesday, December 07, 2016 11:31 AM
> To: Akira Ajisaka; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
> However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Akira Ajisaka <aj...@apache.org>>
> Sent: Wednesday, December 07, 2016 12:12 AM
> To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Junping and Andrew!
>
> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
> compare 2.8 with 2.7.3.
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
>
> Regards,
> Akira
>
> On 2016/12/02 2:08, Junping Du wrote:
>> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>>
>> ________________________________
>> From: Andrew Wang <an...@cloudera.com>>
>> Sent: Wednesday, November 30, 2016 4:18 PM
>> To: Junping Du
>> Cc: Sangjin Lee; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>>
>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>>
>> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>>> wrote:
>> Hi Sangjin and all,
>>
>> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>>
>> Any other comments and suggestions?
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>>
>> ________________________________
>> From: sjlee0@gmail.com<ma...@gmail.com>> <sj...@gmail.com>>> on behalf of Sangjin Lee <sj...@apache.org>>>
>> Sent: Wednesday, November 30, 2016 3:24 PM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>>
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks for picking this up Junping!
>>
>> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>>
>> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>>> wrote:
>> Hi folks,
>> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
>> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
>> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
>> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>>
>> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>>
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>>
>> Sent: Monday, July 25, 2016 2:57 PM
>> To: Jian He
>> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>
>> Subject: Re: [Release thread] 2.8.0 release activities
>>
>> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>>
>> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>>
>> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>>
>> Thanks
>> +Vinod
>>
>>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>>> wrote:
>>>
>>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>>
>>> MAPREDUCE-6288
>>> YARN-1815
>>> YARN-4685
>>> YARN-4844
>>>
>>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>>
>>> Jian
>>>
>>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>>
>>> Sounds good to me :).
>>>
>>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>>> out. Left comments on these JIRAs asked original owners, plan to update
>>> target version of these JIRAs early next week.
>>>
>>> Will keep this thread updated.
>>>
>>> Thanks,
>>> Wangda
>>>
>>>
>>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>>
>>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>>> report the list here. Then folks can chime in if you agree
>>>
>>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>>
>>> Thanks,
>>> Sangjin
>>>
>>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>>
>>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>>> for
>>> every releases.
>>>
>>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>>> gonna very helpful if someone else can help with reviewing Common/HDFS
>>> JIRAs.
>>>
>>> Thanks,
>>> Wangda
>>>
>>>
>>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>>
>>> Where do we stand in terms of closing out blocker/critical issues for
>>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>>> https://issues.apache.org/jira/issues/?filter=12334985
>>>
>>> But I see a lot of JIRAs with no patches or very stale patches. It
>>> would be
>>> a good exercise to come up with the list of JIRAs that we need to block
>>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>>
>>> Thanks,
>>> Sangjin
>>>
>>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>>
>>>
>>> wrote:
>>>
>>>
>>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>>> vinodkv@apache.org<ma...@apache.org>>>>>
>>> wrote:
>>>
>>> We are not converging - there’s still 58 more. I need help from the
>>> community in addressing / review 2.8.0 blockers. If folks can start
>>> with
>>> reviewing Patch available tickets, that’ll be great.
>>>
>>>
>>>
>>>
>>> I'm still doing the s3a stuff, other people testing and reviewing this
>>> stuff welcome.
>>>
>>> in particular, I could do with others playing with this patch of mine,
>>> which adds counters and things into S3a, based on the azure
>>> instrumentation
>>>
>>> https://issues.apache.org/jira/browse/HADOOP-13028
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
>> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>>
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
What I meant is that you can mvn deploy the specific missing files to fix
up the repo state. Though we should also figure out why the deploy plugin
didn't pick up the signature files the first time.
Best,
Andrew
On Fri, Jan 20, 2017 at 11:12 AM, Junping Du <jd...@hortonworks.com> wrote:
> Yes. I did maven deploy in root directory before close the staging
> repository. If this is the only suspect, I can drop the repository and do
> mvn deploy again.
>
>
> Thanks,
>
>
> Junping
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>
> *Sent:* Friday, January 20, 2017 10:48 AM
> *To:* Junping Du
> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>
> *Subject:* Re: [Continued] [Release thread] 2.8.0 release activities
>
> You can check the error message by clicking on it, a bunch like this:
>
> Missing Signature: '/org/apache/hadoop/hadoop-
> mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc'
> does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
>
> Did you maven deploy all the signature files?
>
> On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Hi,
>> I have successfully built the release bit on branch-2.8.0 by
>> following: https://wiki.apache.org/hadoop/HowToRelease step by step.
>> However, when try to close the "Staging Repositories" at Nexus page (
>> https://repository.apache.org/#stagingRepositories), I found our
>> repository - orgapachehadoop-1051 cannot be closed due to some signature
>> validation failed and one Apache rule failed. I never met this problem
>> before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent
>> changes on release process/tools (docker based)? Any ideas or thoughts on
>> how to fix the problem?
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Thursday, January 19, 2017 6:46 PM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Cc: Varun Vasudev
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> According to Varun's offline email, the security fixes has landed on
>> branch-2, 2.8 and 2.8.0 branch.
>> I was kicking off a new RC build (RC1), and will publish it for vote
>> soon. In the mean time, please mark fix version as 2.8.1 for any new
>> commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at
>> this moment. Thanks!
>>
>> Cheers,
>>
>> Junping
>>
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Wednesday, January 18, 2017 3:26 PM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Cc: Varun Vasudev
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Hi folks,
>> In the passed one or two weeks, we found some new blockers get
>> coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM
>> restart with work preserving enabled) and YARN-6072 (RM unable to start in
>> secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I
>> fixed YARN-6068), and I was starting the RC build process since early this
>> week. As we have significant build tools/process change (docker based)
>> since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with
>> it and finally get a successful build on 2.8.0-RC0 last night.
>> I already push RC0 tag into public which is prerequisite step before
>> RC voting. However, in the mean while, I was pinged by Varun Vasudev that
>> there are a known vulnerability issues on container_executor get identified
>> and discussed in hadoop security email threads - looks like YARN-5704 fixed
>> part of it, but left part - the privilege escalation via /proc/self/environ
>> is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although
>> I haven't announced it public for vote yet. I will wait this issue get
>> fixed to prepare a new release candidate. As RC0 tag cannot be reverted
>> after push into apache, our next release candidate will start from RC1.
>> As I mentioned in early email, 2.8.0 is a very big release (2300+
>> commits since 2.7.3) and I am glad that we are almost there. Thanks
>> everyone for being patient and contributing our release work. Please let me
>> know if you have more comments or suggestions.
>>
>> Thanks,
>>
>> Junping
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Wednesday, January 04, 2017 1:41 AM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Hi all Hadoopers,
>> I just commit YARN-3866 which is the last blocker for branch-2.8, so
>> since tomorrow I will kick off process to prepare the first RC for our
>> 2.8.0 release. This release will include at least 2374 commits that never
>> landed before in previous 2.x releases. Please check
>> https://s.apache.org/RW5k for details. There are also 352 fixes marked
>> in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (
>> https://s.apache.org/RKti) that I need to double check all are landed in
>> branch-2.8 and fix the versions before kicking off the first RC.
>> Our progress is slightly behind my estimation weeks ago. However,
>> considering we just go through holidays and Jenkins cleanup issues linger
>> on for several projects (YARN, etc.), our achievement here is still great!
>> Thanks everyone for keep calm and carry on the help for the release. Kudos
>> to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on
>> blockers during this time. Also, special thanks to Vinod and Andrew for
>> sharing knowledge and practice (include scripts for auto version check) for
>> releasing effort.
>> Will try to prepare RC0 of 2.8 release for vote within this week.
>> Stay tuned!
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Wednesday, December 07, 2016 11:31 AM
>> To: Akira Ajisaka; common-dev@hadoop.apache.org;
>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well
>> for several runs before last weekend, but it get failed for latest several
>> runs, probably affected by recently Jenkins down.
>> However, from my recent manual kick off, it seems to be good again:
>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I
>> will keep an eye for today's nightly run to see if it back to normal.
>>
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Akira Ajisaka <aj...@apache.org>
>> Sent: Wednesday, December 07, 2016 12:12 AM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks Junping and Andrew!
>>
>> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
>> compare 2.8 with 2.7.3.
>>
>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-
>> trunk-JACC/24/artifact/target/compat-check/report.html
>>
>> Regards,
>> Akira
>>
>> On 2016/12/02 2:08, Junping Du wrote:
>> > Thanks Andrew! That's also a nice suggestion. I already create a
>> similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.
>> 8-JACC/ for 2.8 and kick off several runs manually. Will monitor
>> incompatible status from there.
>> >
>> >
>> > Thanks,
>> >
>> >
>> > Junping
>> >
>> >
>> > ________________________________
>> > From: Andrew Wang <an...@cloudera.com>
>> > Sent: Wednesday, November 30, 2016 4:18 PM
>> > To: Junping Du
>> > Cc: Sangjin Lee; common-dev@hadoop.apache.org;
>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
>> sjlee@twitter.com
>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>> >
>> > I recommend giving the JACC report another look. I set up a
>> parameterized jenkins job which you can trigger manually for branch-2.8:
>> >
>> > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>> >
>> > On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jdu@hortonworks.com
>> <ma...@hortonworks.com>> wrote:
>> > Hi Sangjin and all,
>> >
>> > That sounds good. If anyone have priority fix to backport , please
>> nominate it by following this email thread or ping me on JIRA directly. And
>> I agree that we should keep in mind that our bar for 2.8 release should be
>> high now as we want to move quickly on this release. Non-critical fixes can
>> wait for next one.
>> >
>> > Any other comments and suggestions?
>> >
>> >
>> > Thanks,
>> >
>> >
>> > Junping
>> >
>> >
>> > ________________________________
>> > From: sjlee0@gmail.com<ma...@gmail.com> <sjlee0@gmail.com
>> <ma...@gmail.com>> on behalf of Sangjin Lee <sjlee@apache.org
>> <ma...@apache.org>>
>> > Sent: Wednesday, November 30, 2016 3:24 PM
>> > To: Junping Du
>> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod
>> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<mailto:sjlee
>> @twitter.com>
>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>> >
>> > Thanks for picking this up Junping!
>> >
>> > I heard on email threads and offline discussions that there may be
>> interests in porting some fixes from branch-2 to branch-2.8. I still feel
>> that the bar should be pretty high to do that, but it might be good to have
>> folks suggest some critical fixes that need to be released sooner. Thoughts?
>> >
>> > On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com
>> <ma...@hortonworks.com><mailto:jdu@hortonworks.com<mailto:jdu@hor
>> tonworks.com>>> wrote:
>> > Hi folks,
>> > Per asked by Vinod offline, I would like to continue the effort to
>> push 2.8 release out in next several weeks. The plan is as following:
>> > - We only have 5 blockers and 3 critical issues (
>> https://s.apache.org/6kwx) so far, and most of them are in good progress
>> now. I put them all on apache ciwiki ( https://cwiki.apache.org/confl
>> uence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and
>> update on these issues in next couple of weeks.
>> > - When all blocker/critical issues are gone (target EOD is Dec.
>> 10), I will push out all major/minor issues that is open. So if you are
>> working on some major issues and target for release in 2.8, please keep
>> track of our release status and your work progress.
>> > - I will create RC immediately once our 2.8 target issues are
>> clean (especially for blocker and critical issues) for voting process.
>> >
>> > I hope we can make it within year 2016, best before Xmas holiday.
>> And I need help from all of you. :)
>> >
>> >
>> > Thanks,
>> >
>> > Junping
>> >
>> > ________________________________________
>> > From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
>> odkv@apache.org><ma...@apache.org>>>
>> > Sent: Monday, July 25, 2016 2:57 PM
>> > To: Jian He
>> > Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.
>> apache.org><mailto:mapreduce-dev@hadoop.apache.org<mailto:m
>> apreduce-dev@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<mailto:
>> yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto:
>> yarn-dev@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<mailto:
>> hdfs-dev@hadoop.apache.org><mailto:hdfs-dev@hadoop.apache.org<mailto:
>> hdfs-dev@hadoop.apache.org>>; common-dev@hadoop.apache.org<mailto:
>> common-dev@hadoop.apache.org><mailto:common-dev@hadoop.apache.org<mailto:
>> common-dev@hadoop.apache.org>>
>> > Subject: Re: [Release thread] 2.8.0 release activities
>> >
>> > Thanks for all that great help moving this release forward, Jian,
>> Sangjin, Wangda et al! Appreciate it.
>> >
>> > The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0
>> last week.
>> >
>> > I only see 9 blocker / critical fixes pending for 2.8.0 -
>> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>> >
>> > Thanks
>> > +Vinod
>> >
>> >> On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
>> e@hortonworks.com><ma...@hortonworks.com>>>
>> wrote:
>> >>
>> >> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
>> attention for 2.8
>> >>
>> >> MAPREDUCE-6288
>> >> YARN-1815
>> >> YARN-4685
>> >> YARN-4844
>> >>
>> >> The rest are either improvements or long-standing issues and does not
>> qualify release blocker, IMO.
>> >> I think we’ll try to get these 4 jiras in asap. The rest will be on
>> best effort, resolve as much as possible and move them out if not resolved
>> in time.
>> >>
>> >> Jian
>> >>
>> >> On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
>> eeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>> >><ma...@gmail.com><mailto:
>> wheeleast@gmail.com<ma...@gmail.com>>>> wrote:
>> >>
>> >> Sounds good to me :).
>> >>
>> >> Jian and I have looked at all existing 2.8.0 blockers and criticals
>> today.
>> >> To me more than half of MR/YARN blockers/criticals of 2.8 should be
>> moved
>> >> out. Left comments on these JIRAs asked original owners, plan to update
>> >> target version of these JIRAs early next week.
>> >>
>> >> Will keep this thread updated.
>> >>
>> >> Thanks,
>> >> Wangda
>> >>
>> >>
>> >> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>> >><ma...@apache.org><mailto:
>> sjlee@apache.org<ma...@apache.org>>>> wrote:
>> >>
>> >> How about this? I'll review the HADOOP/HDFS bugs in that list to come
>> up
>> >> with true blockers for 2.8.0 or JIRAs that are close to being ready.
>> I'll
>> >> report the list here. Then folks can chime in if you agree
>> >>
>> >> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>> >>
>> >> Thanks,
>> >> Sangjin
>> >>
>> >> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
>> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>> ><ma...@gmail.com>>>> wrote:
>> >>
>> >> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> >> for
>> >> every releases.
>> >>
>> >> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> >> gonna very helpful if someone else can help with reviewing Common/HDFS
>> >> JIRAs.
>> >>
>> >> Thanks,
>> >> Wangda
>> >>
>> >>
>> >> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>> >><ma...@apache.org><mailto:
>> sjlee@apache.org<ma...@apache.org>>>> wrote:
>> >>
>> >> Where do we stand in terms of closing out blocker/critical issues for
>> >> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> >> https://issues.apache.org/jira/issues/?filter=12334985
>> >>
>> >> But I see a lot of JIRAs with no patches or very stale patches. It
>> >> would be
>> >> a good exercise to come up with the list of JIRAs that we need to block
>> >> 2.8.0 for and focus our attention on closing them out. Thoughts?
>> >>
>> >> Thanks,
>> >> Sangjin
>> >>
>> >> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <
>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>> stevel@hortonworks.com<ma...@hortonworks.com>><mailto:
>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>> stevel@hortonworks.com<ma...@hortonworks.com>>>
>> >>
>> >> wrote:
>> >>
>> >>
>> >> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> >> vinodkv@apache.org<ma...@apache.org><mailto:vinodkv
>> @apache.org<ma...@apache.org>><mailto:vinodkv@apache.org<mailto:
>> vinodkv@apache.org><mailto:vinodkv@apache.org<mailto:vinodkv@apache.org
>> >>>>
>> >> wrote:
>> >>
>> >> We are not converging - there’s still 58 more. I need help from the
>> >> community in addressing / review 2.8.0 blockers. If folks can start
>> >> with
>> >> reviewing Patch available tickets, that’ll be great.
>> >>
>> >>
>> >>
>> >>
>> >> I'm still doing the s3a stuff, other people testing and reviewing this
>> >> stuff welcome.
>> >>
>> >> in particular, I could do with others playing with this patch of mine,
>> >> which adds counters and things into S3a, based on the azure
>> >> instrumentation
>> >>
>> >> https://issues.apache.org/jira/browse/HADOOP-13028
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
>> yarn-dev-unsubscribe@hadoop.apache.org><mailto:yarn-dev-unsubscribe@ha
>> doop.apache.org<ma...@hadoop.apache.org>>
>> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>> <ma...@hadoop.apache.org><mailto:yarn-dev-
>> help@hadoop.apache.org<ma...@hadoop.apache.org>>
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
>> <ma...@hadoop.apache.org><mailto:
>> mapreduce-dev-unsubscribe@hadoop.apache.org<mailto:
>> mapreduce-dev-unsubscribe@hadoop.apache.org>>
>> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>> <ma...@hadoop.apache.org><mailto:
>> mapreduce-dev-help@hadoop.apache.org<mailto:mapreduce-
>> dev-help@hadoop.apache.org>>
>> >
>> >
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>
>>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
What I meant is that you can mvn deploy the specific missing files to fix
up the repo state. Though we should also figure out why the deploy plugin
didn't pick up the signature files the first time.
Best,
Andrew
On Fri, Jan 20, 2017 at 11:12 AM, Junping Du <jd...@hortonworks.com> wrote:
> Yes. I did maven deploy in root directory before close the staging
> repository. If this is the only suspect, I can drop the repository and do
> mvn deploy again.
>
>
> Thanks,
>
>
> Junping
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>
> *Sent:* Friday, January 20, 2017 10:48 AM
> *To:* Junping Du
> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>
> *Subject:* Re: [Continued] [Release thread] 2.8.0 release activities
>
> You can check the error message by clicking on it, a bunch like this:
>
> Missing Signature: '/org/apache/hadoop/hadoop-
> mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc'
> does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
>
> Did you maven deploy all the signature files?
>
> On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Hi,
>> I have successfully built the release bit on branch-2.8.0 by
>> following: https://wiki.apache.org/hadoop/HowToRelease step by step.
>> However, when try to close the "Staging Repositories" at Nexus page (
>> https://repository.apache.org/#stagingRepositories), I found our
>> repository - orgapachehadoop-1051 cannot be closed due to some signature
>> validation failed and one Apache rule failed. I never met this problem
>> before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent
>> changes on release process/tools (docker based)? Any ideas or thoughts on
>> how to fix the problem?
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Thursday, January 19, 2017 6:46 PM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Cc: Varun Vasudev
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> According to Varun's offline email, the security fixes has landed on
>> branch-2, 2.8 and 2.8.0 branch.
>> I was kicking off a new RC build (RC1), and will publish it for vote
>> soon. In the mean time, please mark fix version as 2.8.1 for any new
>> commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at
>> this moment. Thanks!
>>
>> Cheers,
>>
>> Junping
>>
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Wednesday, January 18, 2017 3:26 PM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Cc: Varun Vasudev
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Hi folks,
>> In the passed one or two weeks, we found some new blockers get
>> coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM
>> restart with work preserving enabled) and YARN-6072 (RM unable to start in
>> secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I
>> fixed YARN-6068), and I was starting the RC build process since early this
>> week. As we have significant build tools/process change (docker based)
>> since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with
>> it and finally get a successful build on 2.8.0-RC0 last night.
>> I already push RC0 tag into public which is prerequisite step before
>> RC voting. However, in the mean while, I was pinged by Varun Vasudev that
>> there are a known vulnerability issues on container_executor get identified
>> and discussed in hadoop security email threads - looks like YARN-5704 fixed
>> part of it, but left part - the privilege escalation via /proc/self/environ
>> is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although
>> I haven't announced it public for vote yet. I will wait this issue get
>> fixed to prepare a new release candidate. As RC0 tag cannot be reverted
>> after push into apache, our next release candidate will start from RC1.
>> As I mentioned in early email, 2.8.0 is a very big release (2300+
>> commits since 2.7.3) and I am glad that we are almost there. Thanks
>> everyone for being patient and contributing our release work. Please let me
>> know if you have more comments or suggestions.
>>
>> Thanks,
>>
>> Junping
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Wednesday, January 04, 2017 1:41 AM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Hi all Hadoopers,
>> I just commit YARN-3866 which is the last blocker for branch-2.8, so
>> since tomorrow I will kick off process to prepare the first RC for our
>> 2.8.0 release. This release will include at least 2374 commits that never
>> landed before in previous 2.x releases. Please check
>> https://s.apache.org/RW5k for details. There are also 352 fixes marked
>> in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (
>> https://s.apache.org/RKti) that I need to double check all are landed in
>> branch-2.8 and fix the versions before kicking off the first RC.
>> Our progress is slightly behind my estimation weeks ago. However,
>> considering we just go through holidays and Jenkins cleanup issues linger
>> on for several projects (YARN, etc.), our achievement here is still great!
>> Thanks everyone for keep calm and carry on the help for the release. Kudos
>> to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on
>> blockers during this time. Also, special thanks to Vinod and Andrew for
>> sharing knowledge and practice (include scripts for auto version check) for
>> releasing effort.
>> Will try to prepare RC0 of 2.8 release for vote within this week.
>> Stay tuned!
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Wednesday, December 07, 2016 11:31 AM
>> To: Akira Ajisaka; common-dev@hadoop.apache.org;
>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well
>> for several runs before last weekend, but it get failed for latest several
>> runs, probably affected by recently Jenkins down.
>> However, from my recent manual kick off, it seems to be good again:
>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I
>> will keep an eye for today's nightly run to see if it back to normal.
>>
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Akira Ajisaka <aj...@apache.org>
>> Sent: Wednesday, December 07, 2016 12:12 AM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks Junping and Andrew!
>>
>> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
>> compare 2.8 with 2.7.3.
>>
>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-
>> trunk-JACC/24/artifact/target/compat-check/report.html
>>
>> Regards,
>> Akira
>>
>> On 2016/12/02 2:08, Junping Du wrote:
>> > Thanks Andrew! That's also a nice suggestion. I already create a
>> similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.
>> 8-JACC/ for 2.8 and kick off several runs manually. Will monitor
>> incompatible status from there.
>> >
>> >
>> > Thanks,
>> >
>> >
>> > Junping
>> >
>> >
>> > ________________________________
>> > From: Andrew Wang <an...@cloudera.com>
>> > Sent: Wednesday, November 30, 2016 4:18 PM
>> > To: Junping Du
>> > Cc: Sangjin Lee; common-dev@hadoop.apache.org;
>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
>> sjlee@twitter.com
>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>> >
>> > I recommend giving the JACC report another look. I set up a
>> parameterized jenkins job which you can trigger manually for branch-2.8:
>> >
>> > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>> >
>> > On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jdu@hortonworks.com
>> <ma...@hortonworks.com>> wrote:
>> > Hi Sangjin and all,
>> >
>> > That sounds good. If anyone have priority fix to backport , please
>> nominate it by following this email thread or ping me on JIRA directly. And
>> I agree that we should keep in mind that our bar for 2.8 release should be
>> high now as we want to move quickly on this release. Non-critical fixes can
>> wait for next one.
>> >
>> > Any other comments and suggestions?
>> >
>> >
>> > Thanks,
>> >
>> >
>> > Junping
>> >
>> >
>> > ________________________________
>> > From: sjlee0@gmail.com<ma...@gmail.com> <sjlee0@gmail.com
>> <ma...@gmail.com>> on behalf of Sangjin Lee <sjlee@apache.org
>> <ma...@apache.org>>
>> > Sent: Wednesday, November 30, 2016 3:24 PM
>> > To: Junping Du
>> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod
>> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<mailto:sjlee
>> @twitter.com>
>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>> >
>> > Thanks for picking this up Junping!
>> >
>> > I heard on email threads and offline discussions that there may be
>> interests in porting some fixes from branch-2 to branch-2.8. I still feel
>> that the bar should be pretty high to do that, but it might be good to have
>> folks suggest some critical fixes that need to be released sooner. Thoughts?
>> >
>> > On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com
>> <ma...@hortonworks.com><mailto:jdu@hortonworks.com<mailto:jdu@hor
>> tonworks.com>>> wrote:
>> > Hi folks,
>> > Per asked by Vinod offline, I would like to continue the effort to
>> push 2.8 release out in next several weeks. The plan is as following:
>> > - We only have 5 blockers and 3 critical issues (
>> https://s.apache.org/6kwx) so far, and most of them are in good progress
>> now. I put them all on apache ciwiki ( https://cwiki.apache.org/confl
>> uence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and
>> update on these issues in next couple of weeks.
>> > - When all blocker/critical issues are gone (target EOD is Dec.
>> 10), I will push out all major/minor issues that is open. So if you are
>> working on some major issues and target for release in 2.8, please keep
>> track of our release status and your work progress.
>> > - I will create RC immediately once our 2.8 target issues are
>> clean (especially for blocker and critical issues) for voting process.
>> >
>> > I hope we can make it within year 2016, best before Xmas holiday.
>> And I need help from all of you. :)
>> >
>> >
>> > Thanks,
>> >
>> > Junping
>> >
>> > ________________________________________
>> > From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
>> odkv@apache.org><ma...@apache.org>>>
>> > Sent: Monday, July 25, 2016 2:57 PM
>> > To: Jian He
>> > Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.
>> apache.org><mailto:mapreduce-dev@hadoop.apache.org<mailto:m
>> apreduce-dev@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<mailto:
>> yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto:
>> yarn-dev@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<mailto:
>> hdfs-dev@hadoop.apache.org><mailto:hdfs-dev@hadoop.apache.org<mailto:
>> hdfs-dev@hadoop.apache.org>>; common-dev@hadoop.apache.org<mailto:
>> common-dev@hadoop.apache.org><mailto:common-dev@hadoop.apache.org<mailto:
>> common-dev@hadoop.apache.org>>
>> > Subject: Re: [Release thread] 2.8.0 release activities
>> >
>> > Thanks for all that great help moving this release forward, Jian,
>> Sangjin, Wangda et al! Appreciate it.
>> >
>> > The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0
>> last week.
>> >
>> > I only see 9 blocker / critical fixes pending for 2.8.0 -
>> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>> >
>> > Thanks
>> > +Vinod
>> >
>> >> On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
>> e@hortonworks.com><ma...@hortonworks.com>>>
>> wrote:
>> >>
>> >> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
>> attention for 2.8
>> >>
>> >> MAPREDUCE-6288
>> >> YARN-1815
>> >> YARN-4685
>> >> YARN-4844
>> >>
>> >> The rest are either improvements or long-standing issues and does not
>> qualify release blocker, IMO.
>> >> I think we’ll try to get these 4 jiras in asap. The rest will be on
>> best effort, resolve as much as possible and move them out if not resolved
>> in time.
>> >>
>> >> Jian
>> >>
>> >> On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
>> eeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>> >><ma...@gmail.com><mailto:
>> wheeleast@gmail.com<ma...@gmail.com>>>> wrote:
>> >>
>> >> Sounds good to me :).
>> >>
>> >> Jian and I have looked at all existing 2.8.0 blockers and criticals
>> today.
>> >> To me more than half of MR/YARN blockers/criticals of 2.8 should be
>> moved
>> >> out. Left comments on these JIRAs asked original owners, plan to update
>> >> target version of these JIRAs early next week.
>> >>
>> >> Will keep this thread updated.
>> >>
>> >> Thanks,
>> >> Wangda
>> >>
>> >>
>> >> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>> >><ma...@apache.org><mailto:
>> sjlee@apache.org<ma...@apache.org>>>> wrote:
>> >>
>> >> How about this? I'll review the HADOOP/HDFS bugs in that list to come
>> up
>> >> with true blockers for 2.8.0 or JIRAs that are close to being ready.
>> I'll
>> >> report the list here. Then folks can chime in if you agree
>> >>
>> >> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>> >>
>> >> Thanks,
>> >> Sangjin
>> >>
>> >> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
>> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>> ><ma...@gmail.com>>>> wrote:
>> >>
>> >> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> >> for
>> >> every releases.
>> >>
>> >> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> >> gonna very helpful if someone else can help with reviewing Common/HDFS
>> >> JIRAs.
>> >>
>> >> Thanks,
>> >> Wangda
>> >>
>> >>
>> >> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>> >><ma...@apache.org><mailto:
>> sjlee@apache.org<ma...@apache.org>>>> wrote:
>> >>
>> >> Where do we stand in terms of closing out blocker/critical issues for
>> >> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> >> https://issues.apache.org/jira/issues/?filter=12334985
>> >>
>> >> But I see a lot of JIRAs with no patches or very stale patches. It
>> >> would be
>> >> a good exercise to come up with the list of JIRAs that we need to block
>> >> 2.8.0 for and focus our attention on closing them out. Thoughts?
>> >>
>> >> Thanks,
>> >> Sangjin
>> >>
>> >> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <
>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>> stevel@hortonworks.com<ma...@hortonworks.com>><mailto:
>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>> stevel@hortonworks.com<ma...@hortonworks.com>>>
>> >>
>> >> wrote:
>> >>
>> >>
>> >> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> >> vinodkv@apache.org<ma...@apache.org><mailto:vinodkv
>> @apache.org<ma...@apache.org>><mailto:vinodkv@apache.org<mailto:
>> vinodkv@apache.org><mailto:vinodkv@apache.org<mailto:vinodkv@apache.org
>> >>>>
>> >> wrote:
>> >>
>> >> We are not converging - there’s still 58 more. I need help from the
>> >> community in addressing / review 2.8.0 blockers. If folks can start
>> >> with
>> >> reviewing Patch available tickets, that’ll be great.
>> >>
>> >>
>> >>
>> >>
>> >> I'm still doing the s3a stuff, other people testing and reviewing this
>> >> stuff welcome.
>> >>
>> >> in particular, I could do with others playing with this patch of mine,
>> >> which adds counters and things into S3a, based on the azure
>> >> instrumentation
>> >>
>> >> https://issues.apache.org/jira/browse/HADOOP-13028
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
>> yarn-dev-unsubscribe@hadoop.apache.org><mailto:yarn-dev-unsubscribe@ha
>> doop.apache.org<ma...@hadoop.apache.org>>
>> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>> <ma...@hadoop.apache.org><mailto:yarn-dev-
>> help@hadoop.apache.org<ma...@hadoop.apache.org>>
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
>> <ma...@hadoop.apache.org><mailto:
>> mapreduce-dev-unsubscribe@hadoop.apache.org<mailto:
>> mapreduce-dev-unsubscribe@hadoop.apache.org>>
>> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>> <ma...@hadoop.apache.org><mailto:
>> mapreduce-dev-help@hadoop.apache.org<mailto:mapreduce-
>> dev-help@hadoop.apache.org>>
>> >
>> >
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>
>>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
What I meant is that you can mvn deploy the specific missing files to fix
up the repo state. Though we should also figure out why the deploy plugin
didn't pick up the signature files the first time.
Best,
Andrew
On Fri, Jan 20, 2017 at 11:12 AM, Junping Du <jd...@hortonworks.com> wrote:
> Yes. I did maven deploy in root directory before close the staging
> repository. If this is the only suspect, I can drop the repository and do
> mvn deploy again.
>
>
> Thanks,
>
>
> Junping
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>
> *Sent:* Friday, January 20, 2017 10:48 AM
> *To:* Junping Du
> *Cc:* common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>
> *Subject:* Re: [Continued] [Release thread] 2.8.0 release activities
>
> You can check the error message by clicking on it, a bunch like this:
>
> Missing Signature: '/org/apache/hadoop/hadoop-
> mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc'
> does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
>
> Did you maven deploy all the signature files?
>
> On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Hi,
>> I have successfully built the release bit on branch-2.8.0 by
>> following: https://wiki.apache.org/hadoop/HowToRelease step by step.
>> However, when try to close the "Staging Repositories" at Nexus page (
>> https://repository.apache.org/#stagingRepositories), I found our
>> repository - orgapachehadoop-1051 cannot be closed due to some signature
>> validation failed and one Apache rule failed. I never met this problem
>> before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent
>> changes on release process/tools (docker based)? Any ideas or thoughts on
>> how to fix the problem?
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Thursday, January 19, 2017 6:46 PM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Cc: Varun Vasudev
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> According to Varun's offline email, the security fixes has landed on
>> branch-2, 2.8 and 2.8.0 branch.
>> I was kicking off a new RC build (RC1), and will publish it for vote
>> soon. In the mean time, please mark fix version as 2.8.1 for any new
>> commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at
>> this moment. Thanks!
>>
>> Cheers,
>>
>> Junping
>>
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Wednesday, January 18, 2017 3:26 PM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Cc: Varun Vasudev
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Hi folks,
>> In the passed one or two weeks, we found some new blockers get
>> coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM
>> restart with work preserving enabled) and YARN-6072 (RM unable to start in
>> secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I
>> fixed YARN-6068), and I was starting the RC build process since early this
>> week. As we have significant build tools/process change (docker based)
>> since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with
>> it and finally get a successful build on 2.8.0-RC0 last night.
>> I already push RC0 tag into public which is prerequisite step before
>> RC voting. However, in the mean while, I was pinged by Varun Vasudev that
>> there are a known vulnerability issues on container_executor get identified
>> and discussed in hadoop security email threads - looks like YARN-5704 fixed
>> part of it, but left part - the privilege escalation via /proc/self/environ
>> is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although
>> I haven't announced it public for vote yet. I will wait this issue get
>> fixed to prepare a new release candidate. As RC0 tag cannot be reverted
>> after push into apache, our next release candidate will start from RC1.
>> As I mentioned in early email, 2.8.0 is a very big release (2300+
>> commits since 2.7.3) and I am glad that we are almost there. Thanks
>> everyone for being patient and contributing our release work. Please let me
>> know if you have more comments or suggestions.
>>
>> Thanks,
>>
>> Junping
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Wednesday, January 04, 2017 1:41 AM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Hi all Hadoopers,
>> I just commit YARN-3866 which is the last blocker for branch-2.8, so
>> since tomorrow I will kick off process to prepare the first RC for our
>> 2.8.0 release. This release will include at least 2374 commits that never
>> landed before in previous 2.x releases. Please check
>> https://s.apache.org/RW5k for details. There are also 352 fixes marked
>> in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (
>> https://s.apache.org/RKti) that I need to double check all are landed in
>> branch-2.8 and fix the versions before kicking off the first RC.
>> Our progress is slightly behind my estimation weeks ago. However,
>> considering we just go through holidays and Jenkins cleanup issues linger
>> on for several projects (YARN, etc.), our achievement here is still great!
>> Thanks everyone for keep calm and carry on the help for the release. Kudos
>> to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on
>> blockers during this time. Also, special thanks to Vinod and Andrew for
>> sharing knowledge and practice (include scripts for auto version check) for
>> releasing effort.
>> Will try to prepare RC0 of 2.8 release for vote within this week.
>> Stay tuned!
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Junping Du <jd...@hortonworks.com>
>> Sent: Wednesday, December 07, 2016 11:31 AM
>> To: Akira Ajisaka; common-dev@hadoop.apache.org;
>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well
>> for several runs before last weekend, but it get failed for latest several
>> runs, probably affected by recently Jenkins down.
>> However, from my recent manual kick off, it seems to be good again:
>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I
>> will keep an eye for today's nightly run to see if it back to normal.
>>
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Akira Ajisaka <aj...@apache.org>
>> Sent: Wednesday, December 07, 2016 12:12 AM
>> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks Junping and Andrew!
>>
>> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
>> compare 2.8 with 2.7.3.
>>
>> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-
>> trunk-JACC/24/artifact/target/compat-check/report.html
>>
>> Regards,
>> Akira
>>
>> On 2016/12/02 2:08, Junping Du wrote:
>> > Thanks Andrew! That's also a nice suggestion. I already create a
>> similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.
>> 8-JACC/ for 2.8 and kick off several runs manually. Will monitor
>> incompatible status from there.
>> >
>> >
>> > Thanks,
>> >
>> >
>> > Junping
>> >
>> >
>> > ________________________________
>> > From: Andrew Wang <an...@cloudera.com>
>> > Sent: Wednesday, November 30, 2016 4:18 PM
>> > To: Junping Du
>> > Cc: Sangjin Lee; common-dev@hadoop.apache.org;
>> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
>> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
>> sjlee@twitter.com
>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>> >
>> > I recommend giving the JACC report another look. I set up a
>> parameterized jenkins job which you can trigger manually for branch-2.8:
>> >
>> > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>> >
>> > On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jdu@hortonworks.com
>> <ma...@hortonworks.com>> wrote:
>> > Hi Sangjin and all,
>> >
>> > That sounds good. If anyone have priority fix to backport , please
>> nominate it by following this email thread or ping me on JIRA directly. And
>> I agree that we should keep in mind that our bar for 2.8 release should be
>> high now as we want to move quickly on this release. Non-critical fixes can
>> wait for next one.
>> >
>> > Any other comments and suggestions?
>> >
>> >
>> > Thanks,
>> >
>> >
>> > Junping
>> >
>> >
>> > ________________________________
>> > From: sjlee0@gmail.com<ma...@gmail.com> <sjlee0@gmail.com
>> <ma...@gmail.com>> on behalf of Sangjin Lee <sjlee@apache.org
>> <ma...@apache.org>>
>> > Sent: Wednesday, November 30, 2016 3:24 PM
>> > To: Junping Du
>> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod
>> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<mailto:sjlee
>> @twitter.com>
>> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>> >
>> > Thanks for picking this up Junping!
>> >
>> > I heard on email threads and offline discussions that there may be
>> interests in porting some fixes from branch-2 to branch-2.8. I still feel
>> that the bar should be pretty high to do that, but it might be good to have
>> folks suggest some critical fixes that need to be released sooner. Thoughts?
>> >
>> > On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com
>> <ma...@hortonworks.com><mailto:jdu@hortonworks.com<mailto:jdu@hor
>> tonworks.com>>> wrote:
>> > Hi folks,
>> > Per asked by Vinod offline, I would like to continue the effort to
>> push 2.8 release out in next several weeks. The plan is as following:
>> > - We only have 5 blockers and 3 critical issues (
>> https://s.apache.org/6kwx) so far, and most of them are in good progress
>> now. I put them all on apache ciwiki ( https://cwiki.apache.org/confl
>> uence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and
>> update on these issues in next couple of weeks.
>> > - When all blocker/critical issues are gone (target EOD is Dec.
>> 10), I will push out all major/minor issues that is open. So if you are
>> working on some major issues and target for release in 2.8, please keep
>> track of our release status and your work progress.
>> > - I will create RC immediately once our 2.8 target issues are
>> clean (especially for blocker and critical issues) for voting process.
>> >
>> > I hope we can make it within year 2016, best before Xmas holiday.
>> And I need help from all of you. :)
>> >
>> >
>> > Thanks,
>> >
>> > Junping
>> >
>> > ________________________________________
>> > From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
>> odkv@apache.org><ma...@apache.org>>>
>> > Sent: Monday, July 25, 2016 2:57 PM
>> > To: Jian He
>> > Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.
>> apache.org><mailto:mapreduce-dev@hadoop.apache.org<mailto:m
>> apreduce-dev@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<mailto:
>> yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto:
>> yarn-dev@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<mailto:
>> hdfs-dev@hadoop.apache.org><mailto:hdfs-dev@hadoop.apache.org<mailto:
>> hdfs-dev@hadoop.apache.org>>; common-dev@hadoop.apache.org<mailto:
>> common-dev@hadoop.apache.org><mailto:common-dev@hadoop.apache.org<mailto:
>> common-dev@hadoop.apache.org>>
>> > Subject: Re: [Release thread] 2.8.0 release activities
>> >
>> > Thanks for all that great help moving this release forward, Jian,
>> Sangjin, Wangda et al! Appreciate it.
>> >
>> > The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0
>> last week.
>> >
>> > I only see 9 blocker / critical fixes pending for 2.8.0 -
>> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>> >
>> > Thanks
>> > +Vinod
>> >
>> >> On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
>> e@hortonworks.com><ma...@hortonworks.com>>>
>> wrote:
>> >>
>> >> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
>> attention for 2.8
>> >>
>> >> MAPREDUCE-6288
>> >> YARN-1815
>> >> YARN-4685
>> >> YARN-4844
>> >>
>> >> The rest are either improvements or long-standing issues and does not
>> qualify release blocker, IMO.
>> >> I think we’ll try to get these 4 jiras in asap. The rest will be on
>> best effort, resolve as much as possible and move them out if not resolved
>> in time.
>> >>
>> >> Jian
>> >>
>> >> On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
>> eeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>> >><ma...@gmail.com><mailto:
>> wheeleast@gmail.com<ma...@gmail.com>>>> wrote:
>> >>
>> >> Sounds good to me :).
>> >>
>> >> Jian and I have looked at all existing 2.8.0 blockers and criticals
>> today.
>> >> To me more than half of MR/YARN blockers/criticals of 2.8 should be
>> moved
>> >> out. Left comments on these JIRAs asked original owners, plan to update
>> >> target version of these JIRAs early next week.
>> >>
>> >> Will keep this thread updated.
>> >>
>> >> Thanks,
>> >> Wangda
>> >>
>> >>
>> >> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>> >><ma...@apache.org><mailto:
>> sjlee@apache.org<ma...@apache.org>>>> wrote:
>> >>
>> >> How about this? I'll review the HADOOP/HDFS bugs in that list to come
>> up
>> >> with true blockers for 2.8.0 or JIRAs that are close to being ready.
>> I'll
>> >> report the list here. Then folks can chime in if you agree
>> >>
>> >> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>> >>
>> >> Thanks,
>> >> Sangjin
>> >>
>> >> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
>> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
>> ><ma...@gmail.com>>>> wrote:
>> >>
>> >> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> >> for
>> >> every releases.
>> >>
>> >> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> >> gonna very helpful if someone else can help with reviewing Common/HDFS
>> >> JIRAs.
>> >>
>> >> Thanks,
>> >> Wangda
>> >>
>> >>
>> >> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
>> >><ma...@apache.org><mailto:
>> sjlee@apache.org<ma...@apache.org>>>> wrote:
>> >>
>> >> Where do we stand in terms of closing out blocker/critical issues for
>> >> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> >> https://issues.apache.org/jira/issues/?filter=12334985
>> >>
>> >> But I see a lot of JIRAs with no patches or very stale patches. It
>> >> would be
>> >> a good exercise to come up with the list of JIRAs that we need to block
>> >> 2.8.0 for and focus our attention on closing them out. Thoughts?
>> >>
>> >> Thanks,
>> >> Sangjin
>> >>
>> >> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <
>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>> stevel@hortonworks.com<ma...@hortonworks.com>><mailto:
>> stevel@hortonworks.com<ma...@hortonworks.com><mailto:
>> stevel@hortonworks.com<ma...@hortonworks.com>>>
>> >>
>> >> wrote:
>> >>
>> >>
>> >> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> >> vinodkv@apache.org<ma...@apache.org><mailto:vinodkv
>> @apache.org<ma...@apache.org>><mailto:vinodkv@apache.org<mailto:
>> vinodkv@apache.org><mailto:vinodkv@apache.org<mailto:vinodkv@apache.org
>> >>>>
>> >> wrote:
>> >>
>> >> We are not converging - there’s still 58 more. I need help from the
>> >> community in addressing / review 2.8.0 blockers. If folks can start
>> >> with
>> >> reviewing Patch available tickets, that’ll be great.
>> >>
>> >>
>> >>
>> >>
>> >> I'm still doing the s3a stuff, other people testing and reviewing this
>> >> stuff welcome.
>> >>
>> >> in particular, I could do with others playing with this patch of mine,
>> >> which adds counters and things into S3a, based on the azure
>> >> instrumentation
>> >>
>> >> https://issues.apache.org/jira/browse/HADOOP-13028
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
>> yarn-dev-unsubscribe@hadoop.apache.org><mailto:yarn-dev-unsubscribe@ha
>> doop.apache.org<ma...@hadoop.apache.org>>
>> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>> <ma...@hadoop.apache.org><mailto:yarn-dev-
>> help@hadoop.apache.org<ma...@hadoop.apache.org>>
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
>> <ma...@hadoop.apache.org><mailto:
>> mapreduce-dev-unsubscribe@hadoop.apache.org<mailto:
>> mapreduce-dev-unsubscribe@hadoop.apache.org>>
>> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>> <ma...@hadoop.apache.org><mailto:
>> mapreduce-dev-help@hadoop.apache.org<mailto:mapreduce-
>> dev-help@hadoop.apache.org>>
>> >
>> >
>> >
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>>
>>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Yes. I did maven deploy in root directory before close the staging repository. If this is the only suspect, I can drop the repository and do mvn deploy again.
Thanks,
Junping
________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Friday, January 20, 2017 10:48 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
You can check the error message by clicking on it, a bunch like this:
Missing Signature: '/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc' does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
Did you maven deploy all the signature files?
On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi,
I have successfully built the release bit on branch-2.8.0 by following: https://wiki.apache.org/hadoop/HowToRelease step by step. However, when try to close the "Staging Repositories" at Nexus page (https://repository.apache.org/#stagingRepositories), I found our repository - orgapachehadoop-1051 cannot be closed due to some signature validation failed and one Apache rule failed. I never met this problem before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent changes on release process/tools (docker based)? Any ideas or thoughts on how to fix the problem?
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Thursday, January 19, 2017 6:46 PM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
Cheers,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Wednesday, January 18, 2017 3:26 PM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com>> <sj...@gmail.com>>> on behalf of Sangjin Lee <sj...@apache.org>>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Yes. I did maven deploy in root directory before close the staging repository. If this is the only suspect, I can drop the repository and do mvn deploy again.
Thanks,
Junping
________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Friday, January 20, 2017 10:48 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
You can check the error message by clicking on it, a bunch like this:
Missing Signature: '/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc' does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
Did you maven deploy all the signature files?
On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi,
I have successfully built the release bit on branch-2.8.0 by following: https://wiki.apache.org/hadoop/HowToRelease step by step. However, when try to close the "Staging Repositories" at Nexus page (https://repository.apache.org/#stagingRepositories), I found our repository - orgapachehadoop-1051 cannot be closed due to some signature validation failed and one Apache rule failed. I never met this problem before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent changes on release process/tools (docker based)? Any ideas or thoughts on how to fix the problem?
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Thursday, January 19, 2017 6:46 PM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
Cheers,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Wednesday, January 18, 2017 3:26 PM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com>> <sj...@gmail.com>>> on behalf of Sangjin Lee <sj...@apache.org>>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Yes. I did maven deploy in root directory before close the staging repository. If this is the only suspect, I can drop the repository and do mvn deploy again.
Thanks,
Junping
________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Friday, January 20, 2017 10:48 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
You can check the error message by clicking on it, a bunch like this:
Missing Signature: '/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc' does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
Did you maven deploy all the signature files?
On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi,
I have successfully built the release bit on branch-2.8.0 by following: https://wiki.apache.org/hadoop/HowToRelease step by step. However, when try to close the "Staging Repositories" at Nexus page (https://repository.apache.org/#stagingRepositories), I found our repository - orgapachehadoop-1051 cannot be closed due to some signature validation failed and one Apache rule failed. I never met this problem before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent changes on release process/tools (docker based)? Any ideas or thoughts on how to fix the problem?
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Thursday, January 19, 2017 6:46 PM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
Cheers,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Wednesday, January 18, 2017 3:26 PM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com>> <sj...@gmail.com>>> on behalf of Sangjin Lee <sj...@apache.org>>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Yes. I did maven deploy in root directory before close the staging repository. If this is the only suspect, I can drop the repository and do mvn deploy again.
Thanks,
Junping
________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Friday, January 20, 2017 10:48 AM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
You can check the error message by clicking on it, a bunch like this:
Missing Signature: '/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc' does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
Did you maven deploy all the signature files?
On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com>> wrote:
Hi,
I have successfully built the release bit on branch-2.8.0 by following: https://wiki.apache.org/hadoop/HowToRelease step by step. However, when try to close the "Staging Repositories" at Nexus page (https://repository.apache.org/#stagingRepositories), I found our repository - orgapachehadoop-1051 cannot be closed due to some signature validation failed and one Apache rule failed. I never met this problem before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent changes on release process/tools (docker based)? Any ideas or thoughts on how to fix the problem?
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Thursday, January 19, 2017 6:46 PM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
Cheers,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Wednesday, January 18, 2017 3:26 PM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com>> <sj...@gmail.com>>> on behalf of Sangjin Lee <sj...@apache.org>>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
You can check the error message by clicking on it, a bunch like this:
Missing Signature:
'/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc'
does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
Did you maven deploy all the signature files?
On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com> wrote:
> Hi,
> I have successfully built the release bit on branch-2.8.0 by
> following: https://wiki.apache.org/hadoop/HowToRelease step by step.
> However, when try to close the "Staging Repositories" at Nexus page (
> https://repository.apache.org/#stagingRepositories), I found our
> repository - orgapachehadoop-1051 cannot be closed due to some signature
> validation failed and one Apache rule failed. I never met this problem
> before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent
> changes on release process/tools (docker based)? Any ideas or thoughts on
> how to fix the problem?
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Thursday, January 19, 2017 6:46 PM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> According to Varun's offline email, the security fixes has landed on
> branch-2, 2.8 and 2.8.0 branch.
> I was kicking off a new RC build (RC1), and will publish it for vote soon.
> In the mean time, please mark fix version as 2.8.1 for any new commits
> landed on branch-2.8, and don't commit anything to branch-2.8.0 at this
> moment. Thanks!
>
> Cheers,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Wednesday, January 18, 2017 3:26 PM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi folks,
> In the passed one or two weeks, we found some new blockers get coming
> on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart
> with work preserving enabled) and YARN-6072 (RM unable to start in secure
> mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed
> YARN-6068), and I was starting the RC build process since early this week.
> As we have significant build tools/process change (docker based) since 2.8
> comparing with 2.6/2.7, it takes me a while to get familiar with it and
> finally get a successful build on 2.8.0-RC0 last night.
> I already push RC0 tag into public which is prerequisite step before
> RC voting. However, in the mean while, I was pinged by Varun Vasudev that
> there are a known vulnerability issues on container_executor get identified
> and discussed in hadoop security email threads - looks like YARN-5704 fixed
> part of it, but left part - the privilege escalation via /proc/self/environ
> is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although
> I haven't announced it public for vote yet. I will wait this issue get
> fixed to prepare a new release candidate. As RC0 tag cannot be reverted
> after push into apache, our next release candidate will start from RC1.
> As I mentioned in early email, 2.8.0 is a very big release (2300+
> commits since 2.7.3) and I am glad that we are almost there. Thanks
> everyone for being patient and contributing our release work. Please let me
> know if you have more comments or suggestions.
>
> Thanks,
>
> Junping
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Wednesday, January 04, 2017 1:41 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi all Hadoopers,
> I just commit YARN-3866 which is the last blocker for branch-2.8, so
> since tomorrow I will kick off process to prepare the first RC for our
> 2.8.0 release. This release will include at least 2374 commits that never
> landed before in previous 2.x releases. Please check
> https://s.apache.org/RW5k for details. There are also 352 fixes marked in
> 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti)
> that I need to double check all are landed in branch-2.8 and fix the
> versions before kicking off the first RC.
> Our progress is slightly behind my estimation weeks ago. However,
> considering we just go through holidays and Jenkins cleanup issues linger
> on for several projects (YARN, etc.), our achievement here is still great!
> Thanks everyone for keep calm and carry on the help for the release. Kudos
> to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on
> blockers during this time. Also, special thanks to Vinod and Andrew for
> sharing knowledge and practice (include scripts for auto version check) for
> releasing effort.
> Will try to prepare RC0 of 2.8 release for vote within this week.
> Stay tuned!
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Wednesday, December 07, 2016 11:31 AM
> To: Akira Ajisaka; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for
> several runs before last weekend, but it get failed for latest several
> runs, probably affected by recently Jenkins down.
> However, from my recent manual kick off, it seems to be good again:
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I
> will keep an eye for today's nightly run to see if it back to normal.
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Akira Ajisaka <aj...@apache.org>
> Sent: Wednesday, December 07, 2016 12:12 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Junping and Andrew!
>
> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
> compare 2.8 with 2.7.3.
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/
> Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
>
> Regards,
> Akira
>
> On 2016/12/02 2:08, Junping Du wrote:
> > Thanks Andrew! That's also a nice suggestion. I already create a similar
> job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/
> for 2.8 and kick off several runs manually. Will monitor incompatible
> status from there.
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Andrew Wang <an...@cloudera.com>
> > Sent: Wednesday, November 30, 2016 4:18 PM
> > To: Junping Du
> > Cc: Sangjin Lee; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
> sjlee@twitter.com
> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
> >
> > I recommend giving the JACC report another look. I set up a
> parameterized jenkins job which you can trigger manually for branch-2.8:
> >
> > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
> >
> > On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com>> wrote:
> > Hi Sangjin and all,
> >
> > That sounds good. If anyone have priority fix to backport , please
> nominate it by following this email thread or ping me on JIRA directly. And
> I agree that we should keep in mind that our bar for 2.8 release should be
> high now as we want to move quickly on this release. Non-critical fixes can
> wait for next one.
> >
> > Any other comments and suggestions?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: sjlee0@gmail.com<ma...@gmail.com> <sjlee0@gmail.com
> <ma...@gmail.com>> on behalf of Sangjin Lee <sjlee@apache.org
> <ma...@apache.org>>
> > Sent: Wednesday, November 30, 2016 3:24 PM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod
> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<mailto:sjlee
> @twitter.com>
> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
> >
> > Thanks for picking this up Junping!
> >
> > I heard on email threads and offline discussions that there may be
> interests in porting some fixes from branch-2 to branch-2.8. I still feel
> that the bar should be pretty high to do that, but it might be good to have
> folks suggest some critical fixes that need to be released sooner. Thoughts?
> >
> > On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com><ma...@hortonworks.com>>>
> wrote:
> > Hi folks,
> > Per asked by Vinod offline, I would like to continue the effort to
> push 2.8 release out in next several weeks. The plan is as following:
> > - We only have 5 blockers and 3 critical issues (
> https://s.apache.org/6kwx) so far, and most of them are in good progress
> now. I put them all on apache ciwiki ( https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor
> and update on these issues in next couple of weeks.
> > - When all blocker/critical issues are gone (target EOD is Dec.
> 10), I will push out all major/minor issues that is open. So if you are
> working on some major issues and target for release in 2.8, please keep
> track of our release status and your work progress.
> > - I will create RC immediately once our 2.8 target issues are clean
> (especially for blocker and critical issues) for voting process.
> >
> > I hope we can make it within year 2016, best before Xmas holiday.
> And I need help from all of you. :)
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
> odkv@apache.org><ma...@apache.org>>>
> > Sent: Monday, July 25, 2016 2:57 PM
> > To: Jian He
> > Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@
> hadoop.apache.org><mailto:mapreduce-dev@hadoop.apache.org<mailto:
> mapreduce-dev@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<mailto:
> yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto:
> yarn-dev@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<mailto:
> hdfs-dev@hadoop.apache.org><mailto:hdfs-dev@hadoop.apache.org<mailto:
> hdfs-dev@hadoop.apache.org>>; common-dev@hadoop.apache.org<mailto:
> common-dev@hadoop.apache.org><mailto:common-dev@hadoop.apache.org<mailto:
> common-dev@hadoop.apache.org>>
> > Subject: Re: [Release thread] 2.8.0 release activities
> >
> > Thanks for all that great help moving this release forward, Jian,
> Sangjin, Wangda et al! Appreciate it.
> >
> > The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0
> last week.
> >
> > I only see 9 blocker / critical fixes pending for 2.8.0 -
> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
> >
> > Thanks
> > +Vinod
> >
> >> On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
> e@hortonworks.com><ma...@hortonworks.com>>>
> wrote:
> >>
> >> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
> attention for 2.8
> >>
> >> MAPREDUCE-6288
> >> YARN-1815
> >> YARN-4685
> >> YARN-4844
> >>
> >> The rest are either improvements or long-standing issues and does not
> qualify release blocker, IMO.
> >> I think we’ll try to get these 4 jiras in asap. The rest will be on
> best effort, resolve as much as possible and move them out if not resolved
> in time.
> >>
> >> Jian
> >>
> >> On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
> eeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
> >><ma...@gmail.com><mailto:
> wheeleast@gmail.com<ma...@gmail.com>>>> wrote:
> >>
> >> Sounds good to me :).
> >>
> >> Jian and I have looked at all existing 2.8.0 blockers and criticals
> today.
> >> To me more than half of MR/YARN blockers/criticals of 2.8 should be
> moved
> >> out. Left comments on these JIRAs asked original owners, plan to update
> >> target version of these JIRAs early next week.
> >>
> >> Will keep this thread updated.
> >>
> >> Thanks,
> >> Wangda
> >>
> >>
> >> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
> >><ma...@apache.org><mailto:sjlee@
> apache.org<ma...@apache.org>>>> wrote:
> >>
> >> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> >> with true blockers for 2.8.0 or JIRAs that are close to being ready.
> I'll
> >> report the list here. Then folks can chime in if you agree
> >>
> >> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
> >>
> >> Thanks,
> >> Sangjin
> >>
> >> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
> ><ma...@gmail.com>>>> wrote:
> >>
> >> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> >> for
> >> every releases.
> >>
> >> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> >> gonna very helpful if someone else can help with reviewing Common/HDFS
> >> JIRAs.
> >>
> >> Thanks,
> >> Wangda
> >>
> >>
> >> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
> >><ma...@apache.org><mailto:sjlee@
> apache.org<ma...@apache.org>>>> wrote:
> >>
> >> Where do we stand in terms of closing out blocker/critical issues for
> >> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> >> https://issues.apache.org/jira/issues/?filter=12334985
> >>
> >> But I see a lot of JIRAs with no patches or very stale patches. It
> >> would be
> >> a good exercise to come up with the list of JIRAs that we need to block
> >> 2.8.0 for and focus our attention on closing them out. Thoughts?
> >>
> >> Thanks,
> >> Sangjin
> >>
> >> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com
> <ma...@hortonworks.com><mailto:stevel@hortonworks.com<mailto:
> stevel@hortonworks.com>><mailto:stevel@hortonworks.com<mailto:
> stevel@hortonworks.com><mailto:stevel@hortonworks.com<mailto:stevel@
> hortonworks.com>>>
> >>
> >> wrote:
> >>
> >>
> >> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> >> vinodkv@apache.org<ma...@apache.org><mailto:vinodkv@apache.org
> <ma...@apache.org>><mailto:vinodkv@apache.org<mailto:vinodkv@
> apache.org><ma...@apache.org>>>>
> >> wrote:
> >>
> >> We are not converging - there’s still 58 more. I need help from the
> >> community in addressing / review 2.8.0 blockers. If folks can start
> >> with
> >> reviewing Patch available tickets, that’ll be great.
> >>
> >>
> >>
> >>
> >> I'm still doing the s3a stuff, other people testing and reviewing this
> >> stuff welcome.
> >>
> >> in particular, I could do with others playing with this patch of mine,
> >> which adds counters and things into S3a, based on the azure
> >> instrumentation
> >>
> >> https://issues.apache.org/jira/browse/HADOOP-13028
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
> yarn-dev-unsubscribe@hadoop.apache.org><mailto:yarn-dev-unsubscribe@
> hadoop.apache.org<ma...@hadoop.apache.org>>
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
> yarn-dev-help@hadoop.apache.org><mailto:yarn-dev-help@hadoop.apache.org<
> mailto:yarn-dev-help@hadoop.apache.org>>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org><mailto:mapreduce-
> dev-unsubscribe@hadoop.apache.org<mailto:mapreduce-dev-
> unsubscribe@hadoop.apache.org>>
> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org><mailto:mapreduce-dev-help@
> hadoop.apache.org<ma...@hadoop.apache.org>>
> >
> >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
You can check the error message by clicking on it, a bunch like this:
Missing Signature:
'/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc'
does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
Did you maven deploy all the signature files?
On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com> wrote:
> Hi,
> I have successfully built the release bit on branch-2.8.0 by
> following: https://wiki.apache.org/hadoop/HowToRelease step by step.
> However, when try to close the "Staging Repositories" at Nexus page (
> https://repository.apache.org/#stagingRepositories), I found our
> repository - orgapachehadoop-1051 cannot be closed due to some signature
> validation failed and one Apache rule failed. I never met this problem
> before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent
> changes on release process/tools (docker based)? Any ideas or thoughts on
> how to fix the problem?
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Thursday, January 19, 2017 6:46 PM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> According to Varun's offline email, the security fixes has landed on
> branch-2, 2.8 and 2.8.0 branch.
> I was kicking off a new RC build (RC1), and will publish it for vote soon.
> In the mean time, please mark fix version as 2.8.1 for any new commits
> landed on branch-2.8, and don't commit anything to branch-2.8.0 at this
> moment. Thanks!
>
> Cheers,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Wednesday, January 18, 2017 3:26 PM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi folks,
> In the passed one or two weeks, we found some new blockers get coming
> on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart
> with work preserving enabled) and YARN-6072 (RM unable to start in secure
> mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed
> YARN-6068), and I was starting the RC build process since early this week.
> As we have significant build tools/process change (docker based) since 2.8
> comparing with 2.6/2.7, it takes me a while to get familiar with it and
> finally get a successful build on 2.8.0-RC0 last night.
> I already push RC0 tag into public which is prerequisite step before
> RC voting. However, in the mean while, I was pinged by Varun Vasudev that
> there are a known vulnerability issues on container_executor get identified
> and discussed in hadoop security email threads - looks like YARN-5704 fixed
> part of it, but left part - the privilege escalation via /proc/self/environ
> is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although
> I haven't announced it public for vote yet. I will wait this issue get
> fixed to prepare a new release candidate. As RC0 tag cannot be reverted
> after push into apache, our next release candidate will start from RC1.
> As I mentioned in early email, 2.8.0 is a very big release (2300+
> commits since 2.7.3) and I am glad that we are almost there. Thanks
> everyone for being patient and contributing our release work. Please let me
> know if you have more comments or suggestions.
>
> Thanks,
>
> Junping
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Wednesday, January 04, 2017 1:41 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi all Hadoopers,
> I just commit YARN-3866 which is the last blocker for branch-2.8, so
> since tomorrow I will kick off process to prepare the first RC for our
> 2.8.0 release. This release will include at least 2374 commits that never
> landed before in previous 2.x releases. Please check
> https://s.apache.org/RW5k for details. There are also 352 fixes marked in
> 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti)
> that I need to double check all are landed in branch-2.8 and fix the
> versions before kicking off the first RC.
> Our progress is slightly behind my estimation weeks ago. However,
> considering we just go through holidays and Jenkins cleanup issues linger
> on for several projects (YARN, etc.), our achievement here is still great!
> Thanks everyone for keep calm and carry on the help for the release. Kudos
> to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on
> blockers during this time. Also, special thanks to Vinod and Andrew for
> sharing knowledge and practice (include scripts for auto version check) for
> releasing effort.
> Will try to prepare RC0 of 2.8 release for vote within this week.
> Stay tuned!
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Wednesday, December 07, 2016 11:31 AM
> To: Akira Ajisaka; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for
> several runs before last weekend, but it get failed for latest several
> runs, probably affected by recently Jenkins down.
> However, from my recent manual kick off, it seems to be good again:
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I
> will keep an eye for today's nightly run to see if it back to normal.
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Akira Ajisaka <aj...@apache.org>
> Sent: Wednesday, December 07, 2016 12:12 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Junping and Andrew!
>
> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
> compare 2.8 with 2.7.3.
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/
> Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
>
> Regards,
> Akira
>
> On 2016/12/02 2:08, Junping Du wrote:
> > Thanks Andrew! That's also a nice suggestion. I already create a similar
> job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/
> for 2.8 and kick off several runs manually. Will monitor incompatible
> status from there.
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Andrew Wang <an...@cloudera.com>
> > Sent: Wednesday, November 30, 2016 4:18 PM
> > To: Junping Du
> > Cc: Sangjin Lee; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
> sjlee@twitter.com
> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
> >
> > I recommend giving the JACC report another look. I set up a
> parameterized jenkins job which you can trigger manually for branch-2.8:
> >
> > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
> >
> > On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com>> wrote:
> > Hi Sangjin and all,
> >
> > That sounds good. If anyone have priority fix to backport , please
> nominate it by following this email thread or ping me on JIRA directly. And
> I agree that we should keep in mind that our bar for 2.8 release should be
> high now as we want to move quickly on this release. Non-critical fixes can
> wait for next one.
> >
> > Any other comments and suggestions?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: sjlee0@gmail.com<ma...@gmail.com> <sjlee0@gmail.com
> <ma...@gmail.com>> on behalf of Sangjin Lee <sjlee@apache.org
> <ma...@apache.org>>
> > Sent: Wednesday, November 30, 2016 3:24 PM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod
> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<mailto:sjlee
> @twitter.com>
> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
> >
> > Thanks for picking this up Junping!
> >
> > I heard on email threads and offline discussions that there may be
> interests in porting some fixes from branch-2 to branch-2.8. I still feel
> that the bar should be pretty high to do that, but it might be good to have
> folks suggest some critical fixes that need to be released sooner. Thoughts?
> >
> > On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com><ma...@hortonworks.com>>>
> wrote:
> > Hi folks,
> > Per asked by Vinod offline, I would like to continue the effort to
> push 2.8 release out in next several weeks. The plan is as following:
> > - We only have 5 blockers and 3 critical issues (
> https://s.apache.org/6kwx) so far, and most of them are in good progress
> now. I put them all on apache ciwiki ( https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor
> and update on these issues in next couple of weeks.
> > - When all blocker/critical issues are gone (target EOD is Dec.
> 10), I will push out all major/minor issues that is open. So if you are
> working on some major issues and target for release in 2.8, please keep
> track of our release status and your work progress.
> > - I will create RC immediately once our 2.8 target issues are clean
> (especially for blocker and critical issues) for voting process.
> >
> > I hope we can make it within year 2016, best before Xmas holiday.
> And I need help from all of you. :)
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
> odkv@apache.org><ma...@apache.org>>>
> > Sent: Monday, July 25, 2016 2:57 PM
> > To: Jian He
> > Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@
> hadoop.apache.org><mailto:mapreduce-dev@hadoop.apache.org<mailto:
> mapreduce-dev@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<mailto:
> yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto:
> yarn-dev@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<mailto:
> hdfs-dev@hadoop.apache.org><mailto:hdfs-dev@hadoop.apache.org<mailto:
> hdfs-dev@hadoop.apache.org>>; common-dev@hadoop.apache.org<mailto:
> common-dev@hadoop.apache.org><mailto:common-dev@hadoop.apache.org<mailto:
> common-dev@hadoop.apache.org>>
> > Subject: Re: [Release thread] 2.8.0 release activities
> >
> > Thanks for all that great help moving this release forward, Jian,
> Sangjin, Wangda et al! Appreciate it.
> >
> > The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0
> last week.
> >
> > I only see 9 blocker / critical fixes pending for 2.8.0 -
> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
> >
> > Thanks
> > +Vinod
> >
> >> On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
> e@hortonworks.com><ma...@hortonworks.com>>>
> wrote:
> >>
> >> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
> attention for 2.8
> >>
> >> MAPREDUCE-6288
> >> YARN-1815
> >> YARN-4685
> >> YARN-4844
> >>
> >> The rest are either improvements or long-standing issues and does not
> qualify release blocker, IMO.
> >> I think we’ll try to get these 4 jiras in asap. The rest will be on
> best effort, resolve as much as possible and move them out if not resolved
> in time.
> >>
> >> Jian
> >>
> >> On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
> eeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
> >><ma...@gmail.com><mailto:
> wheeleast@gmail.com<ma...@gmail.com>>>> wrote:
> >>
> >> Sounds good to me :).
> >>
> >> Jian and I have looked at all existing 2.8.0 blockers and criticals
> today.
> >> To me more than half of MR/YARN blockers/criticals of 2.8 should be
> moved
> >> out. Left comments on these JIRAs asked original owners, plan to update
> >> target version of these JIRAs early next week.
> >>
> >> Will keep this thread updated.
> >>
> >> Thanks,
> >> Wangda
> >>
> >>
> >> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
> >><ma...@apache.org><mailto:sjlee@
> apache.org<ma...@apache.org>>>> wrote:
> >>
> >> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> >> with true blockers for 2.8.0 or JIRAs that are close to being ready.
> I'll
> >> report the list here. Then folks can chime in if you agree
> >>
> >> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
> >>
> >> Thanks,
> >> Sangjin
> >>
> >> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
> ><ma...@gmail.com>>>> wrote:
> >>
> >> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> >> for
> >> every releases.
> >>
> >> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> >> gonna very helpful if someone else can help with reviewing Common/HDFS
> >> JIRAs.
> >>
> >> Thanks,
> >> Wangda
> >>
> >>
> >> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
> >><ma...@apache.org><mailto:sjlee@
> apache.org<ma...@apache.org>>>> wrote:
> >>
> >> Where do we stand in terms of closing out blocker/critical issues for
> >> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> >> https://issues.apache.org/jira/issues/?filter=12334985
> >>
> >> But I see a lot of JIRAs with no patches or very stale patches. It
> >> would be
> >> a good exercise to come up with the list of JIRAs that we need to block
> >> 2.8.0 for and focus our attention on closing them out. Thoughts?
> >>
> >> Thanks,
> >> Sangjin
> >>
> >> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com
> <ma...@hortonworks.com><mailto:stevel@hortonworks.com<mailto:
> stevel@hortonworks.com>><mailto:stevel@hortonworks.com<mailto:
> stevel@hortonworks.com><mailto:stevel@hortonworks.com<mailto:stevel@
> hortonworks.com>>>
> >>
> >> wrote:
> >>
> >>
> >> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> >> vinodkv@apache.org<ma...@apache.org><mailto:vinodkv@apache.org
> <ma...@apache.org>><mailto:vinodkv@apache.org<mailto:vinodkv@
> apache.org><ma...@apache.org>>>>
> >> wrote:
> >>
> >> We are not converging - there’s still 58 more. I need help from the
> >> community in addressing / review 2.8.0 blockers. If folks can start
> >> with
> >> reviewing Patch available tickets, that’ll be great.
> >>
> >>
> >>
> >>
> >> I'm still doing the s3a stuff, other people testing and reviewing this
> >> stuff welcome.
> >>
> >> in particular, I could do with others playing with this patch of mine,
> >> which adds counters and things into S3a, based on the azure
> >> instrumentation
> >>
> >> https://issues.apache.org/jira/browse/HADOOP-13028
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
> yarn-dev-unsubscribe@hadoop.apache.org><mailto:yarn-dev-unsubscribe@
> hadoop.apache.org<ma...@hadoop.apache.org>>
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
> yarn-dev-help@hadoop.apache.org><mailto:yarn-dev-help@hadoop.apache.org<
> mailto:yarn-dev-help@hadoop.apache.org>>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org><mailto:mapreduce-
> dev-unsubscribe@hadoop.apache.org<mailto:mapreduce-dev-
> unsubscribe@hadoop.apache.org>>
> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org><mailto:mapreduce-dev-help@
> hadoop.apache.org<ma...@hadoop.apache.org>>
> >
> >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
You can check the error message by clicking on it, a bunch like this:
Missing Signature:
'/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc'
does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
Did you maven deploy all the signature files?
On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com> wrote:
> Hi,
> I have successfully built the release bit on branch-2.8.0 by
> following: https://wiki.apache.org/hadoop/HowToRelease step by step.
> However, when try to close the "Staging Repositories" at Nexus page (
> https://repository.apache.org/#stagingRepositories), I found our
> repository - orgapachehadoop-1051 cannot be closed due to some signature
> validation failed and one Apache rule failed. I never met this problem
> before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent
> changes on release process/tools (docker based)? Any ideas or thoughts on
> how to fix the problem?
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Thursday, January 19, 2017 6:46 PM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> According to Varun's offline email, the security fixes has landed on
> branch-2, 2.8 and 2.8.0 branch.
> I was kicking off a new RC build (RC1), and will publish it for vote soon.
> In the mean time, please mark fix version as 2.8.1 for any new commits
> landed on branch-2.8, and don't commit anything to branch-2.8.0 at this
> moment. Thanks!
>
> Cheers,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Wednesday, January 18, 2017 3:26 PM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi folks,
> In the passed one or two weeks, we found some new blockers get coming
> on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart
> with work preserving enabled) and YARN-6072 (RM unable to start in secure
> mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed
> YARN-6068), and I was starting the RC build process since early this week.
> As we have significant build tools/process change (docker based) since 2.8
> comparing with 2.6/2.7, it takes me a while to get familiar with it and
> finally get a successful build on 2.8.0-RC0 last night.
> I already push RC0 tag into public which is prerequisite step before
> RC voting. However, in the mean while, I was pinged by Varun Vasudev that
> there are a known vulnerability issues on container_executor get identified
> and discussed in hadoop security email threads - looks like YARN-5704 fixed
> part of it, but left part - the privilege escalation via /proc/self/environ
> is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although
> I haven't announced it public for vote yet. I will wait this issue get
> fixed to prepare a new release candidate. As RC0 tag cannot be reverted
> after push into apache, our next release candidate will start from RC1.
> As I mentioned in early email, 2.8.0 is a very big release (2300+
> commits since 2.7.3) and I am glad that we are almost there. Thanks
> everyone for being patient and contributing our release work. Please let me
> know if you have more comments or suggestions.
>
> Thanks,
>
> Junping
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Wednesday, January 04, 2017 1:41 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi all Hadoopers,
> I just commit YARN-3866 which is the last blocker for branch-2.8, so
> since tomorrow I will kick off process to prepare the first RC for our
> 2.8.0 release. This release will include at least 2374 commits that never
> landed before in previous 2.x releases. Please check
> https://s.apache.org/RW5k for details. There are also 352 fixes marked in
> 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti)
> that I need to double check all are landed in branch-2.8 and fix the
> versions before kicking off the first RC.
> Our progress is slightly behind my estimation weeks ago. However,
> considering we just go through holidays and Jenkins cleanup issues linger
> on for several projects (YARN, etc.), our achievement here is still great!
> Thanks everyone for keep calm and carry on the help for the release. Kudos
> to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on
> blockers during this time. Also, special thanks to Vinod and Andrew for
> sharing knowledge and practice (include scripts for auto version check) for
> releasing effort.
> Will try to prepare RC0 of 2.8 release for vote within this week.
> Stay tuned!
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Wednesday, December 07, 2016 11:31 AM
> To: Akira Ajisaka; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for
> several runs before last weekend, but it get failed for latest several
> runs, probably affected by recently Jenkins down.
> However, from my recent manual kick off, it seems to be good again:
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I
> will keep an eye for today's nightly run to see if it back to normal.
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Akira Ajisaka <aj...@apache.org>
> Sent: Wednesday, December 07, 2016 12:12 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Junping and Andrew!
>
> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
> compare 2.8 with 2.7.3.
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/
> Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
>
> Regards,
> Akira
>
> On 2016/12/02 2:08, Junping Du wrote:
> > Thanks Andrew! That's also a nice suggestion. I already create a similar
> job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/
> for 2.8 and kick off several runs manually. Will monitor incompatible
> status from there.
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Andrew Wang <an...@cloudera.com>
> > Sent: Wednesday, November 30, 2016 4:18 PM
> > To: Junping Du
> > Cc: Sangjin Lee; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
> sjlee@twitter.com
> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
> >
> > I recommend giving the JACC report another look. I set up a
> parameterized jenkins job which you can trigger manually for branch-2.8:
> >
> > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
> >
> > On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com>> wrote:
> > Hi Sangjin and all,
> >
> > That sounds good. If anyone have priority fix to backport , please
> nominate it by following this email thread or ping me on JIRA directly. And
> I agree that we should keep in mind that our bar for 2.8 release should be
> high now as we want to move quickly on this release. Non-critical fixes can
> wait for next one.
> >
> > Any other comments and suggestions?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: sjlee0@gmail.com<ma...@gmail.com> <sjlee0@gmail.com
> <ma...@gmail.com>> on behalf of Sangjin Lee <sjlee@apache.org
> <ma...@apache.org>>
> > Sent: Wednesday, November 30, 2016 3:24 PM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod
> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<mailto:sjlee
> @twitter.com>
> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
> >
> > Thanks for picking this up Junping!
> >
> > I heard on email threads and offline discussions that there may be
> interests in porting some fixes from branch-2 to branch-2.8. I still feel
> that the bar should be pretty high to do that, but it might be good to have
> folks suggest some critical fixes that need to be released sooner. Thoughts?
> >
> > On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com><ma...@hortonworks.com>>>
> wrote:
> > Hi folks,
> > Per asked by Vinod offline, I would like to continue the effort to
> push 2.8 release out in next several weeks. The plan is as following:
> > - We only have 5 blockers and 3 critical issues (
> https://s.apache.org/6kwx) so far, and most of them are in good progress
> now. I put them all on apache ciwiki ( https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor
> and update on these issues in next couple of weeks.
> > - When all blocker/critical issues are gone (target EOD is Dec.
> 10), I will push out all major/minor issues that is open. So if you are
> working on some major issues and target for release in 2.8, please keep
> track of our release status and your work progress.
> > - I will create RC immediately once our 2.8 target issues are clean
> (especially for blocker and critical issues) for voting process.
> >
> > I hope we can make it within year 2016, best before Xmas holiday.
> And I need help from all of you. :)
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
> odkv@apache.org><ma...@apache.org>>>
> > Sent: Monday, July 25, 2016 2:57 PM
> > To: Jian He
> > Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@
> hadoop.apache.org><mailto:mapreduce-dev@hadoop.apache.org<mailto:
> mapreduce-dev@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<mailto:
> yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto:
> yarn-dev@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<mailto:
> hdfs-dev@hadoop.apache.org><mailto:hdfs-dev@hadoop.apache.org<mailto:
> hdfs-dev@hadoop.apache.org>>; common-dev@hadoop.apache.org<mailto:
> common-dev@hadoop.apache.org><mailto:common-dev@hadoop.apache.org<mailto:
> common-dev@hadoop.apache.org>>
> > Subject: Re: [Release thread] 2.8.0 release activities
> >
> > Thanks for all that great help moving this release forward, Jian,
> Sangjin, Wangda et al! Appreciate it.
> >
> > The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0
> last week.
> >
> > I only see 9 blocker / critical fixes pending for 2.8.0 -
> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
> >
> > Thanks
> > +Vinod
> >
> >> On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
> e@hortonworks.com><ma...@hortonworks.com>>>
> wrote:
> >>
> >> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
> attention for 2.8
> >>
> >> MAPREDUCE-6288
> >> YARN-1815
> >> YARN-4685
> >> YARN-4844
> >>
> >> The rest are either improvements or long-standing issues and does not
> qualify release blocker, IMO.
> >> I think we’ll try to get these 4 jiras in asap. The rest will be on
> best effort, resolve as much as possible and move them out if not resolved
> in time.
> >>
> >> Jian
> >>
> >> On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
> eeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
> >><ma...@gmail.com><mailto:
> wheeleast@gmail.com<ma...@gmail.com>>>> wrote:
> >>
> >> Sounds good to me :).
> >>
> >> Jian and I have looked at all existing 2.8.0 blockers and criticals
> today.
> >> To me more than half of MR/YARN blockers/criticals of 2.8 should be
> moved
> >> out. Left comments on these JIRAs asked original owners, plan to update
> >> target version of these JIRAs early next week.
> >>
> >> Will keep this thread updated.
> >>
> >> Thanks,
> >> Wangda
> >>
> >>
> >> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
> >><ma...@apache.org><mailto:sjlee@
> apache.org<ma...@apache.org>>>> wrote:
> >>
> >> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> >> with true blockers for 2.8.0 or JIRAs that are close to being ready.
> I'll
> >> report the list here. Then folks can chime in if you agree
> >>
> >> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
> >>
> >> Thanks,
> >> Sangjin
> >>
> >> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
> ><ma...@gmail.com>>>> wrote:
> >>
> >> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> >> for
> >> every releases.
> >>
> >> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> >> gonna very helpful if someone else can help with reviewing Common/HDFS
> >> JIRAs.
> >>
> >> Thanks,
> >> Wangda
> >>
> >>
> >> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
> >><ma...@apache.org><mailto:sjlee@
> apache.org<ma...@apache.org>>>> wrote:
> >>
> >> Where do we stand in terms of closing out blocker/critical issues for
> >> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> >> https://issues.apache.org/jira/issues/?filter=12334985
> >>
> >> But I see a lot of JIRAs with no patches or very stale patches. It
> >> would be
> >> a good exercise to come up with the list of JIRAs that we need to block
> >> 2.8.0 for and focus our attention on closing them out. Thoughts?
> >>
> >> Thanks,
> >> Sangjin
> >>
> >> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com
> <ma...@hortonworks.com><mailto:stevel@hortonworks.com<mailto:
> stevel@hortonworks.com>><mailto:stevel@hortonworks.com<mailto:
> stevel@hortonworks.com><mailto:stevel@hortonworks.com<mailto:stevel@
> hortonworks.com>>>
> >>
> >> wrote:
> >>
> >>
> >> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> >> vinodkv@apache.org<ma...@apache.org><mailto:vinodkv@apache.org
> <ma...@apache.org>><mailto:vinodkv@apache.org<mailto:vinodkv@
> apache.org><ma...@apache.org>>>>
> >> wrote:
> >>
> >> We are not converging - there’s still 58 more. I need help from the
> >> community in addressing / review 2.8.0 blockers. If folks can start
> >> with
> >> reviewing Patch available tickets, that’ll be great.
> >>
> >>
> >>
> >>
> >> I'm still doing the s3a stuff, other people testing and reviewing this
> >> stuff welcome.
> >>
> >> in particular, I could do with others playing with this patch of mine,
> >> which adds counters and things into S3a, based on the azure
> >> instrumentation
> >>
> >> https://issues.apache.org/jira/browse/HADOOP-13028
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
> yarn-dev-unsubscribe@hadoop.apache.org><mailto:yarn-dev-unsubscribe@
> hadoop.apache.org<ma...@hadoop.apache.org>>
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
> yarn-dev-help@hadoop.apache.org><mailto:yarn-dev-help@hadoop.apache.org<
> mailto:yarn-dev-help@hadoop.apache.org>>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org><mailto:mapreduce-
> dev-unsubscribe@hadoop.apache.org<mailto:mapreduce-dev-
> unsubscribe@hadoop.apache.org>>
> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org><mailto:mapreduce-dev-help@
> hadoop.apache.org<ma...@hadoop.apache.org>>
> >
> >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
You can check the error message by clicking on it, a bunch like this:
Missing Signature:
'/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.8.0/hadoop-mapreduce-client-jobclient-2.8.0-tests.jar.asc'
does not exist for 'hadoop-mapreduce-client-jobclient-2.8.0-tests.jar'.
Did you maven deploy all the signature files?
On Fri, Jan 20, 2017 at 2:04 AM, Junping Du <jd...@hortonworks.com> wrote:
> Hi,
> I have successfully built the release bit on branch-2.8.0 by
> following: https://wiki.apache.org/hadoop/HowToRelease step by step.
> However, when try to close the "Staging Repositories" at Nexus page (
> https://repository.apache.org/#stagingRepositories), I found our
> repository - orgapachehadoop-1051 cannot be closed due to some signature
> validation failed and one Apache rule failed. I never met this problem
> before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent
> changes on release process/tools (docker based)? Any ideas or thoughts on
> how to fix the problem?
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Thursday, January 19, 2017 6:46 PM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> According to Varun's offline email, the security fixes has landed on
> branch-2, 2.8 and 2.8.0 branch.
> I was kicking off a new RC build (RC1), and will publish it for vote soon.
> In the mean time, please mark fix version as 2.8.1 for any new commits
> landed on branch-2.8, and don't commit anything to branch-2.8.0 at this
> moment. Thanks!
>
> Cheers,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Wednesday, January 18, 2017 3:26 PM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Cc: Varun Vasudev
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi folks,
> In the passed one or two weeks, we found some new blockers get coming
> on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart
> with work preserving enabled) and YARN-6072 (RM unable to start in secure
> mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed
> YARN-6068), and I was starting the RC build process since early this week.
> As we have significant build tools/process change (docker based) since 2.8
> comparing with 2.6/2.7, it takes me a while to get familiar with it and
> finally get a successful build on 2.8.0-RC0 last night.
> I already push RC0 tag into public which is prerequisite step before
> RC voting. However, in the mean while, I was pinged by Varun Vasudev that
> there are a known vulnerability issues on container_executor get identified
> and discussed in hadoop security email threads - looks like YARN-5704 fixed
> part of it, but left part - the privilege escalation via /proc/self/environ
> is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although
> I haven't announced it public for vote yet. I will wait this issue get
> fixed to prepare a new release candidate. As RC0 tag cannot be reverted
> after push into apache, our next release candidate will start from RC1.
> As I mentioned in early email, 2.8.0 is a very big release (2300+
> commits since 2.7.3) and I am glad that we are almost there. Thanks
> everyone for being patient and contributing our release work. Please let me
> know if you have more comments or suggestions.
>
> Thanks,
>
> Junping
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Wednesday, January 04, 2017 1:41 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Hi all Hadoopers,
> I just commit YARN-3866 which is the last blocker for branch-2.8, so
> since tomorrow I will kick off process to prepare the first RC for our
> 2.8.0 release. This release will include at least 2374 commits that never
> landed before in previous 2.x releases. Please check
> https://s.apache.org/RW5k for details. There are also 352 fixes marked in
> 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti)
> that I need to double check all are landed in branch-2.8 and fix the
> versions before kicking off the first RC.
> Our progress is slightly behind my estimation weeks ago. However,
> considering we just go through holidays and Jenkins cleanup issues linger
> on for several projects (YARN, etc.), our achievement here is still great!
> Thanks everyone for keep calm and carry on the help for the release. Kudos
> to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on
> blockers during this time. Also, special thanks to Vinod and Andrew for
> sharing knowledge and practice (include scripts for auto version check) for
> releasing effort.
> Will try to prepare RC0 of 2.8 release for vote within this week.
> Stay tuned!
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Junping Du <jd...@hortonworks.com>
> Sent: Wednesday, December 07, 2016 11:31 AM
> To: Akira Ajisaka; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for
> several runs before last weekend, but it get failed for latest several
> runs, probably affected by recently Jenkins down.
> However, from my recent manual kick off, it seems to be good again:
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I
> will keep an eye for today's nightly run to see if it back to normal.
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Akira Ajisaka <aj...@apache.org>
> Sent: Wednesday, December 07, 2016 12:12 AM
> To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks Junping and Andrew!
>
> HADOOP-2.8-JACC is not working well, so I manually kicked a job to
> compare 2.8 with 2.7.3.
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/
> Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
>
> Regards,
> Akira
>
> On 2016/12/02 2:08, Junping Du wrote:
> > Thanks Andrew! That's also a nice suggestion. I already create a similar
> job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/
> for 2.8 and kick off several runs manually. Will monitor incompatible
> status from there.
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: Andrew Wang <an...@cloudera.com>
> > Sent: Wednesday, November 30, 2016 4:18 PM
> > To: Junping Du
> > Cc: Sangjin Lee; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
> sjlee@twitter.com
> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
> >
> > I recommend giving the JACC report another look. I set up a
> parameterized jenkins job which you can trigger manually for branch-2.8:
> >
> > https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
> >
> > On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com>> wrote:
> > Hi Sangjin and all,
> >
> > That sounds good. If anyone have priority fix to backport , please
> nominate it by following this email thread or ping me on JIRA directly. And
> I agree that we should keep in mind that our bar for 2.8 release should be
> high now as we want to move quickly on this release. Non-critical fixes can
> wait for next one.
> >
> > Any other comments and suggestions?
> >
> >
> > Thanks,
> >
> >
> > Junping
> >
> >
> > ________________________________
> > From: sjlee0@gmail.com<ma...@gmail.com> <sjlee0@gmail.com
> <ma...@gmail.com>> on behalf of Sangjin Lee <sjlee@apache.org
> <ma...@apache.org>>
> > Sent: Wednesday, November 30, 2016 3:24 PM
> > To: Junping Du
> > Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod
> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<mailto:sjlee
> @twitter.com>
> > Subject: Re: [Continued] [Release thread] 2.8.0 release activities
> >
> > Thanks for picking this up Junping!
> >
> > I heard on email threads and offline discussions that there may be
> interests in porting some fixes from branch-2 to branch-2.8. I still feel
> that the bar should be pretty high to do that, but it might be good to have
> folks suggest some critical fixes that need to be released sooner. Thoughts?
> >
> > On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com<mailto:
> jdu@hortonworks.com><ma...@hortonworks.com>>>
> wrote:
> > Hi folks,
> > Per asked by Vinod offline, I would like to continue the effort to
> push 2.8 release out in next several weeks. The plan is as following:
> > - We only have 5 blockers and 3 critical issues (
> https://s.apache.org/6kwx) so far, and most of them are in good progress
> now. I put them all on apache ciwiki ( https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor
> and update on these issues in next couple of weeks.
> > - When all blocker/critical issues are gone (target EOD is Dec.
> 10), I will push out all major/minor issues that is open. So if you are
> working on some major issues and target for release in 2.8, please keep
> track of our release status and your work progress.
> > - I will create RC immediately once our 2.8 target issues are clean
> (especially for blocker and critical issues) for voting process.
> >
> > I hope we can make it within year 2016, best before Xmas holiday.
> And I need help from all of you. :)
> >
> >
> > Thanks,
> >
> > Junping
> >
> > ________________________________________
> > From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
> odkv@apache.org><ma...@apache.org>>>
> > Sent: Monday, July 25, 2016 2:57 PM
> > To: Jian He
> > Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@
> hadoop.apache.org><mailto:mapreduce-dev@hadoop.apache.org<mailto:
> mapreduce-dev@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<mailto:
> yarn-dev@hadoop.apache.org><mailto:yarn-dev@hadoop.apache.org<mailto:
> yarn-dev@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<mailto:
> hdfs-dev@hadoop.apache.org><mailto:hdfs-dev@hadoop.apache.org<mailto:
> hdfs-dev@hadoop.apache.org>>; common-dev@hadoop.apache.org<mailto:
> common-dev@hadoop.apache.org><mailto:common-dev@hadoop.apache.org<mailto:
> common-dev@hadoop.apache.org>>
> > Subject: Re: [Release thread] 2.8.0 release activities
> >
> > Thanks for all that great help moving this release forward, Jian,
> Sangjin, Wangda et al! Appreciate it.
> >
> > The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0
> last week.
> >
> > I only see 9 blocker / critical fixes pending for 2.8.0 -
> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
> >
> > Thanks
> > +Vinod
> >
> >> On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
> e@hortonworks.com><ma...@hortonworks.com>>>
> wrote:
> >>
> >> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
> attention for 2.8
> >>
> >> MAPREDUCE-6288
> >> YARN-1815
> >> YARN-4685
> >> YARN-4844
> >>
> >> The rest are either improvements or long-standing issues and does not
> qualify release blocker, IMO.
> >> I think we’ll try to get these 4 jiras in asap. The rest will be on
> best effort, resolve as much as possible and move them out if not resolved
> in time.
> >>
> >> Jian
> >>
> >> On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
> eeleast@gmail.com><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
> >><ma...@gmail.com><mailto:
> wheeleast@gmail.com<ma...@gmail.com>>>> wrote:
> >>
> >> Sounds good to me :).
> >>
> >> Jian and I have looked at all existing 2.8.0 blockers and criticals
> today.
> >> To me more than half of MR/YARN blockers/criticals of 2.8 should be
> moved
> >> out. Left comments on these JIRAs asked original owners, plan to update
> >> target version of these JIRAs early next week.
> >>
> >> Will keep this thread updated.
> >>
> >> Thanks,
> >> Wangda
> >>
> >>
> >> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
> >><ma...@apache.org><mailto:sjlee@
> apache.org<ma...@apache.org>>>> wrote:
> >>
> >> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> >> with true blockers for 2.8.0 or JIRAs that are close to being ready.
> I'll
> >> report the list here. Then folks can chime in if you agree
> >>
> >> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
> >>
> >> Thanks,
> >> Sangjin
> >>
> >> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
> st@gmail.com>><mailto:wheeleast@gmail.com<mailto:wheeleast@gmail.com
> ><ma...@gmail.com>>>> wrote:
> >>
> >> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> >> for
> >> every releases.
> >>
> >> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> >> gonna very helpful if someone else can help with reviewing Common/HDFS
> >> JIRAs.
> >>
> >> Thanks,
> >> Wangda
> >>
> >>
> >> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><mailto:sjlee@apache.org<mailto:sjlee@apache.org
> >><ma...@apache.org><mailto:sjlee@
> apache.org<ma...@apache.org>>>> wrote:
> >>
> >> Where do we stand in terms of closing out blocker/critical issues for
> >> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> >> https://issues.apache.org/jira/issues/?filter=12334985
> >>
> >> But I see a lot of JIRAs with no patches or very stale patches. It
> >> would be
> >> a good exercise to come up with the list of JIRAs that we need to block
> >> 2.8.0 for and focus our attention on closing them out. Thoughts?
> >>
> >> Thanks,
> >> Sangjin
> >>
> >> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com
> <ma...@hortonworks.com><mailto:stevel@hortonworks.com<mailto:
> stevel@hortonworks.com>><mailto:stevel@hortonworks.com<mailto:
> stevel@hortonworks.com><mailto:stevel@hortonworks.com<mailto:stevel@
> hortonworks.com>>>
> >>
> >> wrote:
> >>
> >>
> >> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> >> vinodkv@apache.org<ma...@apache.org><mailto:vinodkv@apache.org
> <ma...@apache.org>><mailto:vinodkv@apache.org<mailto:vinodkv@
> apache.org><ma...@apache.org>>>>
> >> wrote:
> >>
> >> We are not converging - there’s still 58 more. I need help from the
> >> community in addressing / review 2.8.0 blockers. If folks can start
> >> with
> >> reviewing Patch available tickets, that’ll be great.
> >>
> >>
> >>
> >>
> >> I'm still doing the s3a stuff, other people testing and reviewing this
> >> stuff welcome.
> >>
> >> in particular, I could do with others playing with this patch of mine,
> >> which adds counters and things into S3a, based on the azure
> >> instrumentation
> >>
> >> https://issues.apache.org/jira/browse/HADOOP-13028
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
> yarn-dev-unsubscribe@hadoop.apache.org><mailto:yarn-dev-unsubscribe@
> hadoop.apache.org<ma...@hadoop.apache.org>>
> > For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
> yarn-dev-help@hadoop.apache.org><mailto:yarn-dev-help@hadoop.apache.org<
> mailto:yarn-dev-help@hadoop.apache.org>>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org><mailto:mapreduce-
> dev-unsubscribe@hadoop.apache.org<mailto:mapreduce-dev-
> unsubscribe@hadoop.apache.org>>
> > For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org><mailto:mapreduce-dev-help@
> hadoop.apache.org<ma...@hadoop.apache.org>>
> >
> >
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi,
I have successfully built the release bit on branch-2.8.0 by following: https://wiki.apache.org/hadoop/HowToRelease step by step. However, when try to close the "Staging Repositories" at Nexus page (https://repository.apache.org/#stagingRepositories), I found our repository - orgapachehadoop-1051 cannot be closed due to some signature validation failed and one Apache rule failed. I never met this problem before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent changes on release process/tools (docker based)? Any ideas or thoughts on how to fix the problem?
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Thursday, January 19, 2017 6:46 PM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
Cheers,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 18, 2017 3:26 PM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi,
I have successfully built the release bit on branch-2.8.0 by following: https://wiki.apache.org/hadoop/HowToRelease step by step. However, when try to close the "Staging Repositories" at Nexus page (https://repository.apache.org/#stagingRepositories), I found our repository - orgapachehadoop-1051 cannot be closed due to some signature validation failed and one Apache rule failed. I never met this problem before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent changes on release process/tools (docker based)? Any ideas or thoughts on how to fix the problem?
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Thursday, January 19, 2017 6:46 PM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
Cheers,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 18, 2017 3:26 PM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi,
I have successfully built the release bit on branch-2.8.0 by following: https://wiki.apache.org/hadoop/HowToRelease step by step. However, when try to close the "Staging Repositories" at Nexus page (https://repository.apache.org/#stagingRepositories), I found our repository - orgapachehadoop-1051 cannot be closed due to some signature validation failed and one Apache rule failed. I never met this problem before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent changes on release process/tools (docker based)? Any ideas or thoughts on how to fix the problem?
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Thursday, January 19, 2017 6:46 PM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
Cheers,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 18, 2017 3:26 PM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi,
I have successfully built the release bit on branch-2.8.0 by following: https://wiki.apache.org/hadoop/HowToRelease step by step. However, when try to close the "Staging Repositories" at Nexus page (https://repository.apache.org/#stagingRepositories), I found our repository - orgapachehadoop-1051 cannot be closed due to some signature validation failed and one Apache rule failed. I never met this problem before in previous releasing 2.6.3 and 2.6.4. Is this related to our recent changes on release process/tools (docker based)? Any ideas or thoughts on how to fix the problem?
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Thursday, January 19, 2017 6:46 PM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
Cheers,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 18, 2017 3:26 PM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
Cheers,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 18, 2017 3:26 PM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
Cheers,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 18, 2017 3:26 PM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
Cheers,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 18, 2017 3:26 PM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
According to Varun's offline email, the security fixes has landed on branch-2, 2.8 and 2.8.0 branch.
I was kicking off a new RC build (RC1), and will publish it for vote soon. In the mean time, please mark fix version as 2.8.1 for any new commits landed on branch-2.8, and don't commit anything to branch-2.8.0 at this moment. Thanks!
Cheers,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 18, 2017 3:26 PM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Cc: Varun Vasudev
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi folks,
In the passed one or two weeks, we found some new blockers get coming on branch-2.8, like: YARN-6068 (log aggregation get stuck when NM restart with work preserving enabled) and YARN-6072 (RM unable to start in secure mode). Both of them are fixed now (YARN-6072 is fixed by Ajith and I fixed YARN-6068), and I was starting the RC build process since early this week. As we have significant build tools/process change (docker based) since 2.8 comparing with 2.6/2.7, it takes me a while to get familiar with it and finally get a successful build on 2.8.0-RC0 last night.
I already push RC0 tag into public which is prerequisite step before RC voting. However, in the mean while, I was pinged by Varun Vasudev that there are a known vulnerability issues on container_executor get identified and discussed in hadoop security email threads - looks like YARN-5704 fixed part of it, but left part - the privilege escalation via /proc/self/environ is not fixed yet. So most likely, I have to withdraw our 2.8.0 RC0 although I haven't announced it public for vote yet. I will wait this issue get fixed to prepare a new release candidate. As RC0 tag cannot be reverted after push into apache, our next release candidate will start from RC1.
As I mentioned in early email, 2.8.0 is a very big release (2300+ commits since 2.7.3) and I am glad that we are almost there. Thanks everyone for being patient and contributing our release work. Please let me know if you have more comments or suggestions.
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, January 04, 2017 1:41 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi all Hadoopers,
I just commit YARN-3866 which is the last blocker for branch-2.8, so since tomorrow I will kick off process to prepare the first RC for our 2.8.0 release. This release will include at least 2374 commits that never landed before in previous 2.x releases. Please check https://s.apache.org/RW5k for details. There are also 352 fixes marked in 2.7.1, 2.7.2 and 2.7.3 but not marked with 2.8 (https://s.apache.org/RKti) that I need to double check all are landed in branch-2.8 and fix the versions before kicking off the first RC.
Our progress is slightly behind my estimation weeks ago. However, considering we just go through holidays and Jenkins cleanup issues linger on for several projects (YARN, etc.), our achievement here is still great! Thanks everyone for keep calm and carry on the help for the release. Kudos to Wangda, Jian, Akira, Jason, Sangjin, Karthik and all for pushing hard on blockers during this time. Also, special thanks to Vinod and Andrew for sharing knowledge and practice (include scripts for auto version check) for releasing effort.
Will try to prepare RC0 of 2.8 release for vote within this week. Stay tuned!
Thanks,
Junping
________________________________________
From: Junping Du <jd...@hortonworks.com>
Sent: Wednesday, December 07, 2016 11:31 AM
To: Akira Ajisaka; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Akira for reporting this. Actually, HADOOP-2.8-JACC worked well for several runs before last weekend, but it get failed for latest several runs, probably affected by recently Jenkins down.
However, from my recent manual kick off, it seems to be good again: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/9/. I will keep an eye for today's nightly run to see if it back to normal.
Thanks,
Junping
________________________________________
From: Akira Ajisaka <aj...@apache.org>
Sent: Wednesday, December 07, 2016 12:12 AM
To: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - there’s still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, that’ll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Akira Ajisaka <aj...@apache.org>.
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Lets do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think well try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - theres still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, thatll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Akira Ajisaka <aj...@apache.org>.
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Lets do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think well try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - theres still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, thatll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Akira Ajisaka <aj...@apache.org>.
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Lets do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think well try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - theres still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, thatll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
One more thing I forgot, there needs to be a bulk fix version update. Vinod
asked me not to update for 2.8.0 when I updated for 3.0.0-alpha1, but
basically there are a lot of JIRAs in 2.6.x and 2.7.x that also need a
2.8.0 fix version so the changelog diffs correctly with 2.7.0.
I posted the scripts I used to do this for 3.0.0-alpha1. You also should
ask infra to disable JIRA updates temporarily while the script runs:
https://github.com/umbrant/versions
Best,
Andrew
On Thu, Dec 1, 2016 at 9:08 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar
> job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/
> for 2.8 and kick off several runs manually. Will monitor
> incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>
> *Sent:* Wednesday, November 30, 2016 4:18 PM
> *To:* Junping Du
> *Cc:* Sangjin Lee; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
> sjlee@twitter.com
>
> *Subject:* Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized
> jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Hi Sangjin and all,
>>
>> That sounds good. If anyone have priority fix to backport , please
>> nominate it by following this email thread or ping me on JIRA directly. And
>> I agree that we should keep in mind that our bar for 2.8 release should be
>> high now as we want to move quickly on this release. Non-critical fixes can
>> wait for next one.
>>
>> Any other comments and suggestions?
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>>
>> ________________________________
>> From: sjlee0@gmail.com <sj...@gmail.com> on behalf of Sangjin Lee <
>> sjlee@apache.org>
>> Sent: Wednesday, November 30, 2016 3:24 PM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod
>> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks for picking this up Junping!
>>
>> I heard on email threads and offline discussions that there may be
>> interests in porting some fixes from branch-2 to branch-2.8. I still feel
>> that the bar should be pretty high to do that, but it might be good to have
>> folks suggest some critical fixes that need to be released sooner. Thoughts?
>>
>> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com<mailto:
>> jdu@hortonworks.com>> wrote:
>> Hi folks,
>> Per asked by Vinod offline, I would like to continue the effort to
>> push 2.8 release out in next several weeks. The plan is as following:
>> - We only have 5 blockers and 3 critical issues (
>> https://s.apache.org/6kwx) so far, and most of them are in good progress
>> now. I put them all on apache ciwiki ( https://cwiki.apache.org/confl
>> uence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and
>> update on these issues in next couple of weeks.
>> - When all blocker/critical issues are gone (target EOD is Dec. 10),
>> I will push out all major/minor issues that is open. So if you are working
>> on some major issues and target for release in 2.8, please keep track of
>> our release status and your work progress.
>> - I will create RC immediately once our 2.8 target issues are clean
>> (especially for blocker and critical issues) for voting process.
>>
>> I hope we can make it within year 2016, best before Xmas holiday.
>> And I need help from all of you. :)
>>
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
>> odkv@apache.org>>
>> Sent: Monday, July 25, 2016 2:57 PM
>> To: Jian He
>> Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.
>> apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> common-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Re: [Release thread] 2.8.0 release activities
>>
>> Thanks for all that great help moving this release forward, Jian,
>> Sangjin, Wangda et al! Appreciate it.
>>
>> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last
>> week.
>>
>> I only see 9 blocker / critical fixes pending for 2.8.0 -
>> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>>
>> Thanks
>> +Vinod
>>
>> > On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
>> e@hortonworks.com>> wrote:
>> >
>> > For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
>> attention for 2.8
>> >
>> > MAPREDUCE-6288
>> > YARN-1815
>> > YARN-4685
>> > YARN-4844
>> >
>> > The rest are either improvements or long-standing issues and does not
>> qualify release blocker, IMO.
>> > I think we’ll try to get these 4 jiras in asap. The rest will be on
>> best effort, resolve as much as possible and move them out if not resolved
>> in time.
>> >
>> > Jian
>> >
>> > On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
>> eeleast@gmail.com><ma...@gmail.com>>>
>> wrote:
>> >
>> > Sounds good to me :).
>> >
>> > Jian and I have looked at all existing 2.8.0 blockers and criticals
>> today.
>> > To me more than half of MR/YARN blockers/criticals of 2.8 should be
>> moved
>> > out. Left comments on these JIRAs asked original owners, plan to update
>> > target version of these JIRAs early next week.
>> >
>> > Will keep this thread updated.
>> >
>> > Thanks,
>> > Wangda
>> >
>> >
>> > On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><ma...@apache.org>>>
>> wrote:
>> >
>> > How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> > with true blockers for 2.8.0 or JIRAs that are close to being ready.
>> I'll
>> > report the list here. Then folks can chime in if you agree
>> >
>> > Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>> >
>> > Thanks,
>> > Sangjin
>> >
>> > On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
>> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>> st@gmail.com>>> wrote:
>> >
>> > +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> > for
>> > every releases.
>> >
>> > I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> > gonna very helpful if someone else can help with reviewing Common/HDFS
>> > JIRAs.
>> >
>> > Thanks,
>> > Wangda
>> >
>> >
>> > On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><ma...@apache.org>>>
>> wrote:
>> >
>> > Where do we stand in terms of closing out blocker/critical issues for
>> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> > https://issues.apache.org/jira/issues/?filter=12334985
>> >
>> > But I see a lot of JIRAs with no patches or very stale patches. It
>> > would be
>> > a good exercise to come up with the list of JIRAs that we need to block
>> > 2.8.0 for and focus our attention on closing them out. Thoughts?
>> >
>> > Thanks,
>> > Sangjin
>> >
>> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com
>> <ma...@hortonworks.com><mailto:stevel@hortonworks.com<mailto:
>> stevel@hortonworks.com>>
>> >
>> > wrote:
>> >
>> >
>> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> > vinodkv@apache.org<ma...@apache.org><mailto:vinodkv@apache.org
>> <ma...@apache.org>>>
>> > wrote:
>> >
>> > We are not converging - there’s still 58 more. I need help from the
>> > community in addressing / review 2.8.0 blockers. If folks can start
>> > with
>> > reviewing Patch available tickets, that’ll be great.
>> >
>> >
>> >
>> >
>> > I'm still doing the s3a stuff, other people testing and reviewing this
>> > stuff welcome.
>> >
>> > in particular, I could do with others playing with this patch of mine,
>> > which adds counters and things into S3a, based on the azure
>> > instrumentation
>> >
>> > https://issues.apache.org/jira/browse/HADOOP-13028
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
>> yarn-dev-unsubscribe@hadoop.apache.org>
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
>> yarn-dev-help@hadoop.apache.org>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
>> <ma...@hadoop.apache.org>
>> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>> <ma...@hadoop.apache.org>
>>
>>
>>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Akira Ajisaka <aj...@apache.org>.
Thanks Junping and Andrew!
HADOOP-2.8-JACC is not working well, so I manually kicked a job to
compare 2.8 with 2.7.3.
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/24/artifact/target/compat-check/report.html
Regards,
Akira
On 2016/12/02 2:08, Junping Du wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: Andrew Wang <an...@cloudera.com>
> Sent: Wednesday, November 30, 2016 4:18 PM
> To: Junping Du
> Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Lets do this!
>
> Thanks
> +Vinod
>
>> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>>
>> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>>
>> MAPREDUCE-6288
>> YARN-1815
>> YARN-4685
>> YARN-4844
>>
>> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
>> I think well try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>>
>> Jian
>>
>> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> Sounds good to me :).
>>
>> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
>> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
>> out. Left comments on these JIRAs asked original owners, plan to update
>> target version of these JIRAs early next week.
>>
>> Will keep this thread updated.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>>
>> Where do we stand in terms of closing out blocker/critical issues for
>> 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> https://issues.apache.org/jira/issues/?filter=12334985
>>
>> But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> a good exercise to come up with the list of JIRAs that we need to block
>> 2.8.0 for and focus our attention on closing them out. Thoughts?
>>
>> Thanks,
>> Sangjin
>>
>> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>>
>> wrote:
>>
>>
>> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org<ma...@apache.org>>>>
>> wrote:
>>
>> We are not converging - theres still 58 more. I need help from the
>> community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> reviewing Patch available tickets, thatll be great.
>>
>>
>>
>>
>> I'm still doing the s3a stuff, other people testing and reviewing this
>> stuff welcome.
>>
>> in particular, I could do with others playing with this patch of mine,
>> which adds counters and things into S3a, based on the azure
>> instrumentation
>>
>> https://issues.apache.org/jira/browse/HADOOP-13028
>>
>>
>>
>>
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
One more thing I forgot, there needs to be a bulk fix version update. Vinod
asked me not to update for 2.8.0 when I updated for 3.0.0-alpha1, but
basically there are a lot of JIRAs in 2.6.x and 2.7.x that also need a
2.8.0 fix version so the changelog diffs correctly with 2.7.0.
I posted the scripts I used to do this for 3.0.0-alpha1. You also should
ask infra to disable JIRA updates temporarily while the script runs:
https://github.com/umbrant/versions
Best,
Andrew
On Thu, Dec 1, 2016 at 9:08 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar
> job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/
> for 2.8 and kick off several runs manually. Will monitor
> incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>
> *Sent:* Wednesday, November 30, 2016 4:18 PM
> *To:* Junping Du
> *Cc:* Sangjin Lee; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
> sjlee@twitter.com
>
> *Subject:* Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized
> jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Hi Sangjin and all,
>>
>> That sounds good. If anyone have priority fix to backport , please
>> nominate it by following this email thread or ping me on JIRA directly. And
>> I agree that we should keep in mind that our bar for 2.8 release should be
>> high now as we want to move quickly on this release. Non-critical fixes can
>> wait for next one.
>>
>> Any other comments and suggestions?
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>>
>> ________________________________
>> From: sjlee0@gmail.com <sj...@gmail.com> on behalf of Sangjin Lee <
>> sjlee@apache.org>
>> Sent: Wednesday, November 30, 2016 3:24 PM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod
>> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks for picking this up Junping!
>>
>> I heard on email threads and offline discussions that there may be
>> interests in porting some fixes from branch-2 to branch-2.8. I still feel
>> that the bar should be pretty high to do that, but it might be good to have
>> folks suggest some critical fixes that need to be released sooner. Thoughts?
>>
>> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com<mailto:
>> jdu@hortonworks.com>> wrote:
>> Hi folks,
>> Per asked by Vinod offline, I would like to continue the effort to
>> push 2.8 release out in next several weeks. The plan is as following:
>> - We only have 5 blockers and 3 critical issues (
>> https://s.apache.org/6kwx) so far, and most of them are in good progress
>> now. I put them all on apache ciwiki ( https://cwiki.apache.org/confl
>> uence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and
>> update on these issues in next couple of weeks.
>> - When all blocker/critical issues are gone (target EOD is Dec. 10),
>> I will push out all major/minor issues that is open. So if you are working
>> on some major issues and target for release in 2.8, please keep track of
>> our release status and your work progress.
>> - I will create RC immediately once our 2.8 target issues are clean
>> (especially for blocker and critical issues) for voting process.
>>
>> I hope we can make it within year 2016, best before Xmas holiday.
>> And I need help from all of you. :)
>>
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
>> odkv@apache.org>>
>> Sent: Monday, July 25, 2016 2:57 PM
>> To: Jian He
>> Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.
>> apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> common-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Re: [Release thread] 2.8.0 release activities
>>
>> Thanks for all that great help moving this release forward, Jian,
>> Sangjin, Wangda et al! Appreciate it.
>>
>> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last
>> week.
>>
>> I only see 9 blocker / critical fixes pending for 2.8.0 -
>> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>>
>> Thanks
>> +Vinod
>>
>> > On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
>> e@hortonworks.com>> wrote:
>> >
>> > For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
>> attention for 2.8
>> >
>> > MAPREDUCE-6288
>> > YARN-1815
>> > YARN-4685
>> > YARN-4844
>> >
>> > The rest are either improvements or long-standing issues and does not
>> qualify release blocker, IMO.
>> > I think we’ll try to get these 4 jiras in asap. The rest will be on
>> best effort, resolve as much as possible and move them out if not resolved
>> in time.
>> >
>> > Jian
>> >
>> > On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
>> eeleast@gmail.com><ma...@gmail.com>>>
>> wrote:
>> >
>> > Sounds good to me :).
>> >
>> > Jian and I have looked at all existing 2.8.0 blockers and criticals
>> today.
>> > To me more than half of MR/YARN blockers/criticals of 2.8 should be
>> moved
>> > out. Left comments on these JIRAs asked original owners, plan to update
>> > target version of these JIRAs early next week.
>> >
>> > Will keep this thread updated.
>> >
>> > Thanks,
>> > Wangda
>> >
>> >
>> > On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><ma...@apache.org>>>
>> wrote:
>> >
>> > How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> > with true blockers for 2.8.0 or JIRAs that are close to being ready.
>> I'll
>> > report the list here. Then folks can chime in if you agree
>> >
>> > Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>> >
>> > Thanks,
>> > Sangjin
>> >
>> > On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
>> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>> st@gmail.com>>> wrote:
>> >
>> > +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> > for
>> > every releases.
>> >
>> > I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> > gonna very helpful if someone else can help with reviewing Common/HDFS
>> > JIRAs.
>> >
>> > Thanks,
>> > Wangda
>> >
>> >
>> > On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><ma...@apache.org>>>
>> wrote:
>> >
>> > Where do we stand in terms of closing out blocker/critical issues for
>> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> > https://issues.apache.org/jira/issues/?filter=12334985
>> >
>> > But I see a lot of JIRAs with no patches or very stale patches. It
>> > would be
>> > a good exercise to come up with the list of JIRAs that we need to block
>> > 2.8.0 for and focus our attention on closing them out. Thoughts?
>> >
>> > Thanks,
>> > Sangjin
>> >
>> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com
>> <ma...@hortonworks.com><mailto:stevel@hortonworks.com<mailto:
>> stevel@hortonworks.com>>
>> >
>> > wrote:
>> >
>> >
>> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> > vinodkv@apache.org<ma...@apache.org><mailto:vinodkv@apache.org
>> <ma...@apache.org>>>
>> > wrote:
>> >
>> > We are not converging - there’s still 58 more. I need help from the
>> > community in addressing / review 2.8.0 blockers. If folks can start
>> > with
>> > reviewing Patch available tickets, that’ll be great.
>> >
>> >
>> >
>> >
>> > I'm still doing the s3a stuff, other people testing and reviewing this
>> > stuff welcome.
>> >
>> > in particular, I could do with others playing with this patch of mine,
>> > which adds counters and things into S3a, based on the azure
>> > instrumentation
>> >
>> > https://issues.apache.org/jira/browse/HADOOP-13028
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
>> yarn-dev-unsubscribe@hadoop.apache.org>
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
>> yarn-dev-help@hadoop.apache.org>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
>> <ma...@hadoop.apache.org>
>> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>> <ma...@hadoop.apache.org>
>>
>>
>>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
One more thing I forgot, there needs to be a bulk fix version update. Vinod
asked me not to update for 2.8.0 when I updated for 3.0.0-alpha1, but
basically there are a lot of JIRAs in 2.6.x and 2.7.x that also need a
2.8.0 fix version so the changelog diffs correctly with 2.7.0.
I posted the scripts I used to do this for 3.0.0-alpha1. You also should
ask infra to disable JIRA updates temporarily while the script runs:
https://github.com/umbrant/versions
Best,
Andrew
On Thu, Dec 1, 2016 at 9:08 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar
> job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/
> for 2.8 and kick off several runs manually. Will monitor
> incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>
> *Sent:* Wednesday, November 30, 2016 4:18 PM
> *To:* Junping Du
> *Cc:* Sangjin Lee; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
> sjlee@twitter.com
>
> *Subject:* Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized
> jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Hi Sangjin and all,
>>
>> That sounds good. If anyone have priority fix to backport , please
>> nominate it by following this email thread or ping me on JIRA directly. And
>> I agree that we should keep in mind that our bar for 2.8 release should be
>> high now as we want to move quickly on this release. Non-critical fixes can
>> wait for next one.
>>
>> Any other comments and suggestions?
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>>
>> ________________________________
>> From: sjlee0@gmail.com <sj...@gmail.com> on behalf of Sangjin Lee <
>> sjlee@apache.org>
>> Sent: Wednesday, November 30, 2016 3:24 PM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod
>> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks for picking this up Junping!
>>
>> I heard on email threads and offline discussions that there may be
>> interests in porting some fixes from branch-2 to branch-2.8. I still feel
>> that the bar should be pretty high to do that, but it might be good to have
>> folks suggest some critical fixes that need to be released sooner. Thoughts?
>>
>> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com<mailto:
>> jdu@hortonworks.com>> wrote:
>> Hi folks,
>> Per asked by Vinod offline, I would like to continue the effort to
>> push 2.8 release out in next several weeks. The plan is as following:
>> - We only have 5 blockers and 3 critical issues (
>> https://s.apache.org/6kwx) so far, and most of them are in good progress
>> now. I put them all on apache ciwiki ( https://cwiki.apache.org/confl
>> uence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and
>> update on these issues in next couple of weeks.
>> - When all blocker/critical issues are gone (target EOD is Dec. 10),
>> I will push out all major/minor issues that is open. So if you are working
>> on some major issues and target for release in 2.8, please keep track of
>> our release status and your work progress.
>> - I will create RC immediately once our 2.8 target issues are clean
>> (especially for blocker and critical issues) for voting process.
>>
>> I hope we can make it within year 2016, best before Xmas holiday.
>> And I need help from all of you. :)
>>
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
>> odkv@apache.org>>
>> Sent: Monday, July 25, 2016 2:57 PM
>> To: Jian He
>> Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.
>> apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> common-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Re: [Release thread] 2.8.0 release activities
>>
>> Thanks for all that great help moving this release forward, Jian,
>> Sangjin, Wangda et al! Appreciate it.
>>
>> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last
>> week.
>>
>> I only see 9 blocker / critical fixes pending for 2.8.0 -
>> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>>
>> Thanks
>> +Vinod
>>
>> > On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
>> e@hortonworks.com>> wrote:
>> >
>> > For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
>> attention for 2.8
>> >
>> > MAPREDUCE-6288
>> > YARN-1815
>> > YARN-4685
>> > YARN-4844
>> >
>> > The rest are either improvements or long-standing issues and does not
>> qualify release blocker, IMO.
>> > I think we’ll try to get these 4 jiras in asap. The rest will be on
>> best effort, resolve as much as possible and move them out if not resolved
>> in time.
>> >
>> > Jian
>> >
>> > On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
>> eeleast@gmail.com><ma...@gmail.com>>>
>> wrote:
>> >
>> > Sounds good to me :).
>> >
>> > Jian and I have looked at all existing 2.8.0 blockers and criticals
>> today.
>> > To me more than half of MR/YARN blockers/criticals of 2.8 should be
>> moved
>> > out. Left comments on these JIRAs asked original owners, plan to update
>> > target version of these JIRAs early next week.
>> >
>> > Will keep this thread updated.
>> >
>> > Thanks,
>> > Wangda
>> >
>> >
>> > On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><ma...@apache.org>>>
>> wrote:
>> >
>> > How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> > with true blockers for 2.8.0 or JIRAs that are close to being ready.
>> I'll
>> > report the list here. Then folks can chime in if you agree
>> >
>> > Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>> >
>> > Thanks,
>> > Sangjin
>> >
>> > On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
>> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>> st@gmail.com>>> wrote:
>> >
>> > +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> > for
>> > every releases.
>> >
>> > I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> > gonna very helpful if someone else can help with reviewing Common/HDFS
>> > JIRAs.
>> >
>> > Thanks,
>> > Wangda
>> >
>> >
>> > On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><ma...@apache.org>>>
>> wrote:
>> >
>> > Where do we stand in terms of closing out blocker/critical issues for
>> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> > https://issues.apache.org/jira/issues/?filter=12334985
>> >
>> > But I see a lot of JIRAs with no patches or very stale patches. It
>> > would be
>> > a good exercise to come up with the list of JIRAs that we need to block
>> > 2.8.0 for and focus our attention on closing them out. Thoughts?
>> >
>> > Thanks,
>> > Sangjin
>> >
>> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com
>> <ma...@hortonworks.com><mailto:stevel@hortonworks.com<mailto:
>> stevel@hortonworks.com>>
>> >
>> > wrote:
>> >
>> >
>> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> > vinodkv@apache.org<ma...@apache.org><mailto:vinodkv@apache.org
>> <ma...@apache.org>>>
>> > wrote:
>> >
>> > We are not converging - there’s still 58 more. I need help from the
>> > community in addressing / review 2.8.0 blockers. If folks can start
>> > with
>> > reviewing Patch available tickets, that’ll be great.
>> >
>> >
>> >
>> >
>> > I'm still doing the s3a stuff, other people testing and reviewing this
>> > stuff welcome.
>> >
>> > in particular, I could do with others playing with this patch of mine,
>> > which adds counters and things into S3a, based on the azure
>> > instrumentation
>> >
>> > https://issues.apache.org/jira/browse/HADOOP-13028
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
>> yarn-dev-unsubscribe@hadoop.apache.org>
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
>> yarn-dev-help@hadoop.apache.org>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
>> <ma...@hadoop.apache.org>
>> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>> <ma...@hadoop.apache.org>
>>
>>
>>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
One more thing I forgot, there needs to be a bulk fix version update. Vinod
asked me not to update for 2.8.0 when I updated for 3.0.0-alpha1, but
basically there are a lot of JIRAs in 2.6.x and 2.7.x that also need a
2.8.0 fix version so the changelog diffs correctly with 2.7.0.
I posted the scripts I used to do this for 3.0.0-alpha1. You also should
ask infra to disable JIRA updates temporarily while the script runs:
https://github.com/umbrant/versions
Best,
Andrew
On Thu, Dec 1, 2016 at 9:08 AM, Junping Du <jd...@hortonworks.com> wrote:
> Thanks Andrew! That's also a nice suggestion. I already create a similar
> job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/
> for 2.8 and kick off several runs manually. Will monitor
> incompatible status from there.
>
>
> Thanks,
>
>
> Junping
>
>
> ------------------------------
> *From:* Andrew Wang <an...@cloudera.com>
> *Sent:* Wednesday, November 30, 2016 4:18 PM
> *To:* Junping Du
> *Cc:* Sangjin Lee; common-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan;
> sjlee@twitter.com
>
> *Subject:* Re: [Continued] [Release thread] 2.8.0 release activities
>
> I recommend giving the JACC report another look. I set up a parameterized
> jenkins job which you can trigger manually for branch-2.8:
>
> https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
>
> On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com> wrote:
>
>> Hi Sangjin and all,
>>
>> That sounds good. If anyone have priority fix to backport , please
>> nominate it by following this email thread or ping me on JIRA directly. And
>> I agree that we should keep in mind that our bar for 2.8 release should be
>> high now as we want to move quickly on this release. Non-critical fixes can
>> wait for next one.
>>
>> Any other comments and suggestions?
>>
>>
>> Thanks,
>>
>>
>> Junping
>>
>>
>> ________________________________
>> From: sjlee0@gmail.com <sj...@gmail.com> on behalf of Sangjin Lee <
>> sjlee@apache.org>
>> Sent: Wednesday, November 30, 2016 3:24 PM
>> To: Junping Du
>> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
>> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod
>> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
>> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>>
>> Thanks for picking this up Junping!
>>
>> I heard on email threads and offline discussions that there may be
>> interests in porting some fixes from branch-2 to branch-2.8. I still feel
>> that the bar should be pretty high to do that, but it might be good to have
>> folks suggest some critical fixes that need to be released sooner. Thoughts?
>>
>> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com<mailto:
>> jdu@hortonworks.com>> wrote:
>> Hi folks,
>> Per asked by Vinod offline, I would like to continue the effort to
>> push 2.8 release out in next several weeks. The plan is as following:
>> - We only have 5 blockers and 3 critical issues (
>> https://s.apache.org/6kwx) so far, and most of them are in good progress
>> now. I put them all on apache ciwiki ( https://cwiki.apache.org/confl
>> uence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and
>> update on these issues in next couple of weeks.
>> - When all blocker/critical issues are gone (target EOD is Dec. 10),
>> I will push out all major/minor issues that is open. So if you are working
>> on some major issues and target for release in 2.8, please keep track of
>> our release status and your work progress.
>> - I will create RC immediately once our 2.8 target issues are clean
>> (especially for blocker and critical issues) for voting process.
>>
>> I hope we can make it within year 2016, best before Xmas holiday.
>> And I need help from all of you. :)
>>
>>
>> Thanks,
>>
>> Junping
>>
>> ________________________________________
>> From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
>> odkv@apache.org>>
>> Sent: Monday, July 25, 2016 2:57 PM
>> To: Jian He
>> Cc: mapreduce-dev@hadoop.apache.org<mailto:mapreduce-dev@hadoop.
>> apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
>> common-dev@hadoop.apache.org<ma...@hadoop.apache.org>
>> Subject: Re: [Release thread] 2.8.0 release activities
>>
>> Thanks for all that great help moving this release forward, Jian,
>> Sangjin, Wangda et al! Appreciate it.
>>
>> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last
>> week.
>>
>> I only see 9 blocker / critical fixes pending for 2.8.0 -
>> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>>
>> Thanks
>> +Vinod
>>
>> > On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
>> e@hortonworks.com>> wrote:
>> >
>> > For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
>> attention for 2.8
>> >
>> > MAPREDUCE-6288
>> > YARN-1815
>> > YARN-4685
>> > YARN-4844
>> >
>> > The rest are either improvements or long-standing issues and does not
>> qualify release blocker, IMO.
>> > I think we’ll try to get these 4 jiras in asap. The rest will be on
>> best effort, resolve as much as possible and move them out if not resolved
>> in time.
>> >
>> > Jian
>> >
>> > On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
>> eeleast@gmail.com><ma...@gmail.com>>>
>> wrote:
>> >
>> > Sounds good to me :).
>> >
>> > Jian and I have looked at all existing 2.8.0 blockers and criticals
>> today.
>> > To me more than half of MR/YARN blockers/criticals of 2.8 should be
>> moved
>> > out. Left comments on these JIRAs asked original owners, plan to update
>> > target version of these JIRAs early next week.
>> >
>> > Will keep this thread updated.
>> >
>> > Thanks,
>> > Wangda
>> >
>> >
>> > On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><ma...@apache.org>>>
>> wrote:
>> >
>> > How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> > with true blockers for 2.8.0 or JIRAs that are close to being ready.
>> I'll
>> > report the list here. Then folks can chime in if you agree
>> >
>> > Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>> >
>> > Thanks,
>> > Sangjin
>> >
>> > On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com
>> <ma...@gmail.com><mailto:wheeleast@gmail.com<mailto:wheelea
>> st@gmail.com>>> wrote:
>> >
>> > +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> > for
>> > every releases.
>> >
>> > I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> > gonna very helpful if someone else can help with reviewing Common/HDFS
>> > JIRAs.
>> >
>> > Thanks,
>> > Wangda
>> >
>> >
>> > On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
>> sjlee@apache.org><ma...@apache.org>>>
>> wrote:
>> >
>> > Where do we stand in terms of closing out blocker/critical issues for
>> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> > https://issues.apache.org/jira/issues/?filter=12334985
>> >
>> > But I see a lot of JIRAs with no patches or very stale patches. It
>> > would be
>> > a good exercise to come up with the list of JIRAs that we need to block
>> > 2.8.0 for and focus our attention on closing them out. Thoughts?
>> >
>> > Thanks,
>> > Sangjin
>> >
>> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com
>> <ma...@hortonworks.com><mailto:stevel@hortonworks.com<mailto:
>> stevel@hortonworks.com>>
>> >
>> > wrote:
>> >
>> >
>> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> > vinodkv@apache.org<ma...@apache.org><mailto:vinodkv@apache.org
>> <ma...@apache.org>>>
>> > wrote:
>> >
>> > We are not converging - there’s still 58 more. I need help from the
>> > community in addressing / review 2.8.0 blockers. If folks can start
>> > with
>> > reviewing Patch available tickets, that’ll be great.
>> >
>> >
>> >
>> >
>> > I'm still doing the s3a stuff, other people testing and reviewing this
>> > stuff welcome.
>> >
>> > in particular, I could do with others playing with this patch of mine,
>> > which adds counters and things into S3a, based on the azure
>> > instrumentation
>> >
>> > https://issues.apache.org/jira/browse/HADOOP-13028
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
>> yarn-dev-unsubscribe@hadoop.apache.org>
>> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
>> yarn-dev-help@hadoop.apache.org>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
>> <ma...@hadoop.apache.org>
>> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>> <ma...@hadoop.apache.org>
>>
>>
>>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
Thanks,
Junping
________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Wednesday, November 30, 2016 4:18 PM
To: Junping Du
Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
Hi Sangjin and all,
That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
Any other comments and suggestions?
Thanks,
Junping
________________________________
From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
Sent: Wednesday, November 30, 2016 3:24 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks for picking this up Junping!
I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
Hi folks,
Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
- We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
- When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
- I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
Thanks,
Junping
________________________________________
From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
Sent: Monday, July 25, 2016 2:57 PM
To: Jian He
Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
Subject: Re: [Release thread] 2.8.0 release activities
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
Thanks,
Junping
________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Wednesday, November 30, 2016 4:18 PM
To: Junping Du
Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
Hi Sangjin and all,
That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
Any other comments and suggestions?
Thanks,
Junping
________________________________
From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
Sent: Wednesday, November 30, 2016 3:24 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks for picking this up Junping!
I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
Hi folks,
Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
- We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
- When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
- I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
Thanks,
Junping
________________________________________
From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
Sent: Monday, July 25, 2016 2:57 PM
To: Jian He
Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
Subject: Re: [Release thread] 2.8.0 release activities
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
Thanks,
Junping
________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Wednesday, November 30, 2016 4:18 PM
To: Junping Du
Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
Hi Sangjin and all,
That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
Any other comments and suggestions?
Thanks,
Junping
________________________________
From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
Sent: Wednesday, November 30, 2016 3:24 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks for picking this up Junping!
I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
Hi folks,
Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
- We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
- When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
- I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
Thanks,
Junping
________________________________________
From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
Sent: Monday, July 25, 2016 2:57 PM
To: Jian He
Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
Subject: Re: [Release thread] 2.8.0 release activities
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Thanks Andrew! That's also a nice suggestion. I already create a similar job: https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-2.8-JACC/ for 2.8 and kick off several runs manually. Will monitor incompatible status from there.
Thanks,
Junping
________________________________
From: Andrew Wang <an...@cloudera.com>
Sent: Wednesday, November 30, 2016 4:18 PM
To: Junping Du
Cc: Sangjin Lee; common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
I recommend giving the JACC report another look. I set up a parameterized jenkins job which you can trigger manually for branch-2.8:
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com>> wrote:
Hi Sangjin and all,
That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
Any other comments and suggestions?
Thanks,
Junping
________________________________
From: sjlee0@gmail.com<ma...@gmail.com> <sj...@gmail.com>> on behalf of Sangjin Lee <sj...@apache.org>>
Sent: Wednesday, November 30, 2016 3:24 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com<ma...@twitter.com>
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks for picking this up Junping!
I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>>> wrote:
Hi folks,
Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
- We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
- When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
- I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
Thanks,
Junping
________________________________________
From: Vinod Kumar Vavilapalli <vi...@apache.org>>>
Sent: Monday, July 25, 2016 2:57 PM
To: Jian He
Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>>
Subject: Re: [Release thread] 2.8.0 release activities
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>>> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>>
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
I recommend giving the JACC report another look. I set up a parameterized
jenkins job which you can trigger manually for branch-2.8:
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please
> nominate it by following this email thread or ping me on JIRA directly. And
> I agree that we should keep in mind that our bar for 2.8 release should be
> high now as we want to move quickly on this release. Non-critical fixes can
> wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com <sj...@gmail.com> on behalf of Sangjin Lee <
> sjlee@apache.org>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod
> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be
> interests in porting some fixes from branch-2 to branch-2.8. I still feel
> that the bar should be pretty high to do that, but it might be good to have
> folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to
> push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (
> https://s.apache.org/6kwx) so far, and most of them are in good progress
> now. I put them all on apache ciwiki ( https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor
> and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10),
> I will push out all major/minor issues that is open. So if you are working
> on some major issues and target for release in 2.8, please keep track of
> our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean
> (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And
> I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
> odkv@apache.org>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> common-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin,
> Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last
> week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 -
> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
> > On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
> e@hortonworks.com>> wrote:
> >
> > For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
> attention for 2.8
> >
> > MAPREDUCE-6288
> > YARN-1815
> > YARN-4685
> > YARN-4844
> >
> > The rest are either improvements or long-standing issues and does not
> qualify release blocker, IMO.
> > I think we’ll try to get these 4 jiras in asap. The rest will be on best
> effort, resolve as much as possible and move them out if not resolved in
> time.
> >
> > Jian
> >
> > On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
> eeleast@gmail.com><ma...@gmail.com>>>
> wrote:
> >
> > Sounds good to me :).
> >
> > Jian and I have looked at all existing 2.8.0 blockers and criticals
> today.
> > To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> > out. Left comments on these JIRAs asked original owners, plan to update
> > target version of these JIRAs early next week.
> >
> > Will keep this thread updated.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><ma...@apache.org>>>
> wrote:
> >
> > How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> > with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> > report the list here. Then folks can chime in if you agree
> >
> > Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
> >
> > Thanks,
> > Sangjin
> >
> > On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com<mailto:
> wheeleast@gmail.com><ma...@gmail.com>>>
> wrote:
> >
> > +1, we should close such staled JIRAs to avoid doing unnecessary checks
> > for
> > every releases.
> >
> > I'm working on reviewing YARN/MR critical/blocker patches currently, it
> > gonna very helpful if someone else can help with reviewing Common/HDFS
> > JIRAs.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><ma...@apache.org>>>
> wrote:
> >
> > Where do we stand in terms of closing out blocker/critical issues for
> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
> > https://issues.apache.org/jira/issues/?filter=12334985
> >
> > But I see a lot of JIRAs with no patches or very stale patches. It
> > would be
> > a good exercise to come up with the list of JIRAs that we need to block
> > 2.8.0 for and focus our attention on closing them out. Thoughts?
> >
> > Thanks,
> > Sangjin
> >
> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com<
> mailto:stevel@hortonworks.com><mailto:stevel@hortonworks.com<mailto:
> stevel@hortonworks.com>>
> >
> > wrote:
> >
> >
> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> > vinodkv@apache.org<ma...@apache.org><mailto:vinodkv@apache.org
> <ma...@apache.org>>>
> > wrote:
> >
> > We are not converging - there’s still 58 more. I need help from the
> > community in addressing / review 2.8.0 blockers. If folks can start
> > with
> > reviewing Patch available tickets, that’ll be great.
> >
> >
> >
> >
> > I'm still doing the s3a stuff, other people testing and reviewing this
> > stuff welcome.
> >
> > in particular, I could do with others playing with this patch of mine,
> > which adds counters and things into S3a, based on the azure
> > instrumentation
> >
> > https://issues.apache.org/jira/browse/HADOOP-13028
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
> yarn-dev-unsubscribe@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
> yarn-dev-help@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
>
>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
I recommend giving the JACC report another look. I set up a parameterized
jenkins job which you can trigger manually for branch-2.8:
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please
> nominate it by following this email thread or ping me on JIRA directly. And
> I agree that we should keep in mind that our bar for 2.8 release should be
> high now as we want to move quickly on this release. Non-critical fixes can
> wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com <sj...@gmail.com> on behalf of Sangjin Lee <
> sjlee@apache.org>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod
> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be
> interests in porting some fixes from branch-2 to branch-2.8. I still feel
> that the bar should be pretty high to do that, but it might be good to have
> folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to
> push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (
> https://s.apache.org/6kwx) so far, and most of them are in good progress
> now. I put them all on apache ciwiki ( https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor
> and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10),
> I will push out all major/minor issues that is open. So if you are working
> on some major issues and target for release in 2.8, please keep track of
> our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean
> (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And
> I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
> odkv@apache.org>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> common-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin,
> Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last
> week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 -
> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
> > On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
> e@hortonworks.com>> wrote:
> >
> > For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
> attention for 2.8
> >
> > MAPREDUCE-6288
> > YARN-1815
> > YARN-4685
> > YARN-4844
> >
> > The rest are either improvements or long-standing issues and does not
> qualify release blocker, IMO.
> > I think we’ll try to get these 4 jiras in asap. The rest will be on best
> effort, resolve as much as possible and move them out if not resolved in
> time.
> >
> > Jian
> >
> > On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
> eeleast@gmail.com><ma...@gmail.com>>>
> wrote:
> >
> > Sounds good to me :).
> >
> > Jian and I have looked at all existing 2.8.0 blockers and criticals
> today.
> > To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> > out. Left comments on these JIRAs asked original owners, plan to update
> > target version of these JIRAs early next week.
> >
> > Will keep this thread updated.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><ma...@apache.org>>>
> wrote:
> >
> > How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> > with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> > report the list here. Then folks can chime in if you agree
> >
> > Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
> >
> > Thanks,
> > Sangjin
> >
> > On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com<mailto:
> wheeleast@gmail.com><ma...@gmail.com>>>
> wrote:
> >
> > +1, we should close such staled JIRAs to avoid doing unnecessary checks
> > for
> > every releases.
> >
> > I'm working on reviewing YARN/MR critical/blocker patches currently, it
> > gonna very helpful if someone else can help with reviewing Common/HDFS
> > JIRAs.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><ma...@apache.org>>>
> wrote:
> >
> > Where do we stand in terms of closing out blocker/critical issues for
> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
> > https://issues.apache.org/jira/issues/?filter=12334985
> >
> > But I see a lot of JIRAs with no patches or very stale patches. It
> > would be
> > a good exercise to come up with the list of JIRAs that we need to block
> > 2.8.0 for and focus our attention on closing them out. Thoughts?
> >
> > Thanks,
> > Sangjin
> >
> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com<
> mailto:stevel@hortonworks.com><mailto:stevel@hortonworks.com<mailto:
> stevel@hortonworks.com>>
> >
> > wrote:
> >
> >
> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> > vinodkv@apache.org<ma...@apache.org><mailto:vinodkv@apache.org
> <ma...@apache.org>>>
> > wrote:
> >
> > We are not converging - there’s still 58 more. I need help from the
> > community in addressing / review 2.8.0 blockers. If folks can start
> > with
> > reviewing Patch available tickets, that’ll be great.
> >
> >
> >
> >
> > I'm still doing the s3a stuff, other people testing and reviewing this
> > stuff welcome.
> >
> > in particular, I could do with others playing with this patch of mine,
> > which adds counters and things into S3a, based on the azure
> > instrumentation
> >
> > https://issues.apache.org/jira/browse/HADOOP-13028
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
> yarn-dev-unsubscribe@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
> yarn-dev-help@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
>
>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
I recommend giving the JACC report another look. I set up a parameterized
jenkins job which you can trigger manually for branch-2.8:
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please
> nominate it by following this email thread or ping me on JIRA directly. And
> I agree that we should keep in mind that our bar for 2.8 release should be
> high now as we want to move quickly on this release. Non-critical fixes can
> wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com <sj...@gmail.com> on behalf of Sangjin Lee <
> sjlee@apache.org>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod
> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be
> interests in porting some fixes from branch-2 to branch-2.8. I still feel
> that the bar should be pretty high to do that, but it might be good to have
> folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to
> push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (
> https://s.apache.org/6kwx) so far, and most of them are in good progress
> now. I put them all on apache ciwiki ( https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor
> and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10),
> I will push out all major/minor issues that is open. So if you are working
> on some major issues and target for release in 2.8, please keep track of
> our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean
> (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And
> I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
> odkv@apache.org>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> common-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin,
> Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last
> week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 -
> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
> > On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
> e@hortonworks.com>> wrote:
> >
> > For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
> attention for 2.8
> >
> > MAPREDUCE-6288
> > YARN-1815
> > YARN-4685
> > YARN-4844
> >
> > The rest are either improvements or long-standing issues and does not
> qualify release blocker, IMO.
> > I think we’ll try to get these 4 jiras in asap. The rest will be on best
> effort, resolve as much as possible and move them out if not resolved in
> time.
> >
> > Jian
> >
> > On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
> eeleast@gmail.com><ma...@gmail.com>>>
> wrote:
> >
> > Sounds good to me :).
> >
> > Jian and I have looked at all existing 2.8.0 blockers and criticals
> today.
> > To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> > out. Left comments on these JIRAs asked original owners, plan to update
> > target version of these JIRAs early next week.
> >
> > Will keep this thread updated.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><ma...@apache.org>>>
> wrote:
> >
> > How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> > with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> > report the list here. Then folks can chime in if you agree
> >
> > Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
> >
> > Thanks,
> > Sangjin
> >
> > On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com<mailto:
> wheeleast@gmail.com><ma...@gmail.com>>>
> wrote:
> >
> > +1, we should close such staled JIRAs to avoid doing unnecessary checks
> > for
> > every releases.
> >
> > I'm working on reviewing YARN/MR critical/blocker patches currently, it
> > gonna very helpful if someone else can help with reviewing Common/HDFS
> > JIRAs.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><ma...@apache.org>>>
> wrote:
> >
> > Where do we stand in terms of closing out blocker/critical issues for
> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
> > https://issues.apache.org/jira/issues/?filter=12334985
> >
> > But I see a lot of JIRAs with no patches or very stale patches. It
> > would be
> > a good exercise to come up with the list of JIRAs that we need to block
> > 2.8.0 for and focus our attention on closing them out. Thoughts?
> >
> > Thanks,
> > Sangjin
> >
> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com<
> mailto:stevel@hortonworks.com><mailto:stevel@hortonworks.com<mailto:
> stevel@hortonworks.com>>
> >
> > wrote:
> >
> >
> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> > vinodkv@apache.org<ma...@apache.org><mailto:vinodkv@apache.org
> <ma...@apache.org>>>
> > wrote:
> >
> > We are not converging - there’s still 58 more. I need help from the
> > community in addressing / review 2.8.0 blockers. If folks can start
> > with
> > reviewing Patch available tickets, that’ll be great.
> >
> >
> >
> >
> > I'm still doing the s3a stuff, other people testing and reviewing this
> > stuff welcome.
> >
> > in particular, I could do with others playing with this patch of mine,
> > which adds counters and things into S3a, based on the azure
> > instrumentation
> >
> > https://issues.apache.org/jira/browse/HADOOP-13028
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
> yarn-dev-unsubscribe@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
> yarn-dev-help@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
>
>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Andrew Wang <an...@cloudera.com>.
I recommend giving the JACC report another look. I set up a parameterized
jenkins job which you can trigger manually for branch-2.8:
https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-trunk-JACC/
On Wed, Nov 30, 2016 at 4:06 PM, Junping Du <jd...@hortonworks.com> wrote:
> Hi Sangjin and all,
>
> That sounds good. If anyone have priority fix to backport , please
> nominate it by following this email thread or ping me on JIRA directly. And
> I agree that we should keep in mind that our bar for 2.8 release should be
> high now as we want to move quickly on this release. Non-critical fixes can
> wait for next one.
>
> Any other comments and suggestions?
>
>
> Thanks,
>
>
> Junping
>
>
> ________________________________
> From: sjlee0@gmail.com <sj...@gmail.com> on behalf of Sangjin Lee <
> sjlee@apache.org>
> Sent: Wednesday, November 30, 2016 3:24 PM
> To: Junping Du
> Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org;
> mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod
> Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
> Subject: Re: [Continued] [Release thread] 2.8.0 release activities
>
> Thanks for picking this up Junping!
>
> I heard on email threads and offline discussions that there may be
> interests in porting some fixes from branch-2 to branch-2.8. I still feel
> that the bar should be pretty high to do that, but it might be good to have
> folks suggest some critical fixes that need to be released sooner. Thoughts?
>
> On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jdu@hortonworks.com<mailto:jd
> u@hortonworks.com>> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to
> push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (
> https://s.apache.org/6kwx) so far, and most of them are in good progress
> now. I put them all on apache ciwiki ( https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor
> and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10),
> I will push out all major/minor issues that is open. So if you are working
> on some major issues and target for release in 2.8, please keep track of
> our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean
> (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And
> I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vinodkv@apache.org<mailto:vin
> odkv@apache.org>>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>;
> common-dev@hadoop.apache.org<ma...@hadoop.apache.org>
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin,
> Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last
> week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 -
> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
> > On May 11, 2016, at 6:04 PM, Jian He <jhe@hortonworks.com<mailto:jh
> e@hortonworks.com>> wrote:
> >
> > For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
> attention for 2.8
> >
> > MAPREDUCE-6288
> > YARN-1815
> > YARN-4685
> > YARN-4844
> >
> > The rest are either improvements or long-standing issues and does not
> qualify release blocker, IMO.
> > I think we’ll try to get these 4 jiras in asap. The rest will be on best
> effort, resolve as much as possible and move them out if not resolved in
> time.
> >
> > Jian
> >
> > On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
> eeleast@gmail.com><ma...@gmail.com>>>
> wrote:
> >
> > Sounds good to me :).
> >
> > Jian and I have looked at all existing 2.8.0 blockers and criticals
> today.
> > To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> > out. Left comments on these JIRAs asked original owners, plan to update
> > target version of these JIRAs early next week.
> >
> > Will keep this thread updated.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><ma...@apache.org>>>
> wrote:
> >
> > How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> > with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> > report the list here. Then folks can chime in if you agree
> >
> > Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
> >
> > Thanks,
> > Sangjin
> >
> > On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com<mailto:
> wheeleast@gmail.com><ma...@gmail.com>>>
> wrote:
> >
> > +1, we should close such staled JIRAs to avoid doing unnecessary checks
> > for
> > every releases.
> >
> > I'm working on reviewing YARN/MR critical/blocker patches currently, it
> > gonna very helpful if someone else can help with reviewing Common/HDFS
> > JIRAs.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org><ma...@apache.org>>>
> wrote:
> >
> > Where do we stand in terms of closing out blocker/critical issues for
> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
> > https://issues.apache.org/jira/issues/?filter=12334985
> >
> > But I see a lot of JIRAs with no patches or very stale patches. It
> > would be
> > a good exercise to come up with the list of JIRAs that we need to block
> > 2.8.0 for and focus our attention on closing them out. Thoughts?
> >
> > Thanks,
> > Sangjin
> >
> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com<
> mailto:stevel@hortonworks.com><mailto:stevel@hortonworks.com<mailto:
> stevel@hortonworks.com>>
> >
> > wrote:
> >
> >
> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> > vinodkv@apache.org<ma...@apache.org><mailto:vinodkv@apache.org
> <ma...@apache.org>>>
> > wrote:
> >
> > We are not converging - there’s still 58 more. I need help from the
> > community in addressing / review 2.8.0 blockers. If folks can start
> > with
> > reviewing Patch available tickets, that’ll be great.
> >
> >
> >
> >
> > I'm still doing the s3a stuff, other people testing and reviewing this
> > stuff welcome.
> >
> > in particular, I could do with others playing with this patch of mine,
> > which adds counters and things into S3a, based on the azure
> > instrumentation
> >
> > https://issues.apache.org/jira/browse/HADOOP-13028
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<mailto:
> yarn-dev-unsubscribe@hadoop.apache.org>
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<mailto:
> yarn-dev-help@hadoop.apache.org>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> <ma...@hadoop.apache.org>
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
> <ma...@hadoop.apache.org>
>
>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi Sangjin and all,
That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
Any other comments and suggestions?
Thanks,
Junping
________________________________
From: sjlee0@gmail.com <sj...@gmail.com> on behalf of Sangjin Lee <sj...@apache.org>
Sent: Wednesday, November 30, 2016 3:24 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks for picking this up Junping!
I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>> wrote:
Hi folks,
Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
- We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
- When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
- I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
Thanks,
Junping
________________________________________
From: Vinod Kumar Vavilapalli <vi...@apache.org>>
Sent: Monday, July 25, 2016 2:57 PM
To: Jian He
Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Release thread] 2.8.0 release activities
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi Sangjin and all,
That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
Any other comments and suggestions?
Thanks,
Junping
________________________________
From: sjlee0@gmail.com <sj...@gmail.com> on behalf of Sangjin Lee <sj...@apache.org>
Sent: Wednesday, November 30, 2016 3:24 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks for picking this up Junping!
I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>> wrote:
Hi folks,
Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
- We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
- When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
- I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
Thanks,
Junping
________________________________________
From: Vinod Kumar Vavilapalli <vi...@apache.org>>
Sent: Monday, July 25, 2016 2:57 PM
To: Jian He
Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Release thread] 2.8.0 release activities
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi Sangjin and all,
That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
Any other comments and suggestions?
Thanks,
Junping
________________________________
From: sjlee0@gmail.com <sj...@gmail.com> on behalf of Sangjin Lee <sj...@apache.org>
Sent: Wednesday, November 30, 2016 3:24 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks for picking this up Junping!
I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>> wrote:
Hi folks,
Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
- We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
- When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
- I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
Thanks,
Junping
________________________________________
From: Vinod Kumar Vavilapalli <vi...@apache.org>>
Sent: Monday, July 25, 2016 2:57 PM
To: Jian He
Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Release thread] 2.8.0 release activities
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi Sangjin and all,
That sounds good. If anyone have priority fix to backport , please nominate it by following this email thread or ping me on JIRA directly. And I agree that we should keep in mind that our bar for 2.8 release should be high now as we want to move quickly on this release. Non-critical fixes can wait for next one.
Any other comments and suggestions?
Thanks,
Junping
________________________________
From: sjlee0@gmail.com <sj...@gmail.com> on behalf of Sangjin Lee <sj...@apache.org>
Sent: Wednesday, November 30, 2016 3:24 PM
To: Junping Du
Cc: common-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; Vinod Vavilapalli; Jian He; Wangda Tan; sjlee@twitter.com
Subject: Re: [Continued] [Release thread] 2.8.0 release activities
Thanks for picking this up Junping!
I heard on email threads and offline discussions that there may be interests in porting some fixes from branch-2 to branch-2.8. I still feel that the bar should be pretty high to do that, but it might be good to have folks suggest some critical fixes that need to be released sooner. Thoughts?
On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com>> wrote:
Hi folks,
Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
- We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
- When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
- I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
Thanks,
Junping
________________________________________
From: Vinod Kumar Vavilapalli <vi...@apache.org>>
Sent: Monday, July 25, 2016 2:57 PM
To: Jian He
Cc: mapreduce-dev@hadoop.apache.org<ma...@hadoop.apache.org>; yarn-dev@hadoop.apache.org<ma...@hadoop.apache.org>; hdfs-dev@hadoop.apache.org<ma...@hadoop.apache.org>; common-dev@hadoop.apache.org<ma...@hadoop.apache.org>
Subject: Re: [Release thread] 2.8.0 release activities
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com>> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org<ma...@hadoop.apache.org>
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org<ma...@hadoop.apache.org>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
Thanks for picking this up Junping!
I heard on email threads and offline discussions that there may be
interests in porting some fixes from branch-2 to branch-2.8. I still feel
that the bar should be pretty high to do that, but it might be good to have
folks suggest some critical fixes that need to be released sooner. Thoughts?
On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to
> push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (
> https://s.apache.org/6kwx) so far, and most of them are in good progress
> now. I put them all on apache ciwiki ( https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor
> and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10),
> I will push out all major/minor issues that is open. So if you are working
> on some major issues and target for release in 2.8, please keep track of
> our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean
> (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And
> I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin,
> Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last
> week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 -
> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
> > On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com> wrote:
> >
> > For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
> attention for 2.8
> >
> > MAPREDUCE-6288
> > YARN-1815
> > YARN-4685
> > YARN-4844
> >
> > The rest are either improvements or long-standing issues and does not
> qualify release blocker, IMO.
> > I think we’ll try to get these 4 jiras in asap. The rest will be on best
> effort, resolve as much as possible and move them out if not resolved in
> time.
> >
> > Jian
> >
> > On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
> eeleast@gmail.com>> wrote:
> >
> > Sounds good to me :).
> >
> > Jian and I have looked at all existing 2.8.0 blockers and criticals
> today.
> > To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> > out. Left comments on these JIRAs asked original owners, plan to update
> > target version of these JIRAs early next week.
> >
> > Will keep this thread updated.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org>> wrote:
> >
> > How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> > with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> > report the list here. Then folks can chime in if you agree
> >
> > Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
> >
> > Thanks,
> > Sangjin
> >
> > On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com<mailto:
> wheeleast@gmail.com>> wrote:
> >
> > +1, we should close such staled JIRAs to avoid doing unnecessary checks
> > for
> > every releases.
> >
> > I'm working on reviewing YARN/MR critical/blocker patches currently, it
> > gonna very helpful if someone else can help with reviewing Common/HDFS
> > JIRAs.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org>> wrote:
> >
> > Where do we stand in terms of closing out blocker/critical issues for
> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
> > https://issues.apache.org/jira/issues/?filter=12334985
> >
> > But I see a lot of JIRAs with no patches or very stale patches. It
> > would be
> > a good exercise to come up with the list of JIRAs that we need to block
> > 2.8.0 for and focus our attention on closing them out. Thoughts?
> >
> > Thanks,
> > Sangjin
> >
> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com<
> mailto:stevel@hortonworks.com>
> >
> > wrote:
> >
> >
> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> > vinodkv@apache.org<ma...@apache.org>>
> > wrote:
> >
> > We are not converging - there’s still 58 more. I need help from the
> > community in addressing / review 2.8.0 blockers. If folks can start
> > with
> > reviewing Patch available tickets, that’ll be great.
> >
> >
> >
> >
> > I'm still doing the s3a stuff, other people testing and reviewing this
> > stuff welcome.
> >
> > in particular, I could do with others playing with this patch of mine,
> > which adds counters and things into S3a, based on the azure
> > instrumentation
> >
> > https://issues.apache.org/jira/browse/HADOOP-13028
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
Thanks for picking this up Junping!
I heard on email threads and offline discussions that there may be
interests in porting some fixes from branch-2 to branch-2.8. I still feel
that the bar should be pretty high to do that, but it might be good to have
folks suggest some critical fixes that need to be released sooner. Thoughts?
On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to
> push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (
> https://s.apache.org/6kwx) so far, and most of them are in good progress
> now. I put them all on apache ciwiki ( https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor
> and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10),
> I will push out all major/minor issues that is open. So if you are working
> on some major issues and target for release in 2.8, please keep track of
> our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean
> (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And
> I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin,
> Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last
> week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 -
> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
> > On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com> wrote:
> >
> > For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
> attention for 2.8
> >
> > MAPREDUCE-6288
> > YARN-1815
> > YARN-4685
> > YARN-4844
> >
> > The rest are either improvements or long-standing issues and does not
> qualify release blocker, IMO.
> > I think we’ll try to get these 4 jiras in asap. The rest will be on best
> effort, resolve as much as possible and move them out if not resolved in
> time.
> >
> > Jian
> >
> > On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
> eeleast@gmail.com>> wrote:
> >
> > Sounds good to me :).
> >
> > Jian and I have looked at all existing 2.8.0 blockers and criticals
> today.
> > To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> > out. Left comments on these JIRAs asked original owners, plan to update
> > target version of these JIRAs early next week.
> >
> > Will keep this thread updated.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org>> wrote:
> >
> > How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> > with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> > report the list here. Then folks can chime in if you agree
> >
> > Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
> >
> > Thanks,
> > Sangjin
> >
> > On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com<mailto:
> wheeleast@gmail.com>> wrote:
> >
> > +1, we should close such staled JIRAs to avoid doing unnecessary checks
> > for
> > every releases.
> >
> > I'm working on reviewing YARN/MR critical/blocker patches currently, it
> > gonna very helpful if someone else can help with reviewing Common/HDFS
> > JIRAs.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org>> wrote:
> >
> > Where do we stand in terms of closing out blocker/critical issues for
> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
> > https://issues.apache.org/jira/issues/?filter=12334985
> >
> > But I see a lot of JIRAs with no patches or very stale patches. It
> > would be
> > a good exercise to come up with the list of JIRAs that we need to block
> > 2.8.0 for and focus our attention on closing them out. Thoughts?
> >
> > Thanks,
> > Sangjin
> >
> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com<
> mailto:stevel@hortonworks.com>
> >
> > wrote:
> >
> >
> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> > vinodkv@apache.org<ma...@apache.org>>
> > wrote:
> >
> > We are not converging - there’s still 58 more. I need help from the
> > community in addressing / review 2.8.0 blockers. If folks can start
> > with
> > reviewing Patch available tickets, that’ll be great.
> >
> >
> >
> >
> > I'm still doing the s3a stuff, other people testing and reviewing this
> > stuff welcome.
> >
> > in particular, I could do with others playing with this patch of mine,
> > which adds counters and things into S3a, based on the azure
> > instrumentation
> >
> > https://issues.apache.org/jira/browse/HADOOP-13028
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
Thanks for picking this up Junping!
I heard on email threads and offline discussions that there may be
interests in porting some fixes from branch-2 to branch-2.8. I still feel
that the bar should be pretty high to do that, but it might be good to have
folks suggest some critical fixes that need to be released sooner. Thoughts?
On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to
> push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (
> https://s.apache.org/6kwx) so far, and most of them are in good progress
> now. I put them all on apache ciwiki ( https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor
> and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10),
> I will push out all major/minor issues that is open. So if you are working
> on some major issues and target for release in 2.8, please keep track of
> our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean
> (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And
> I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin,
> Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last
> week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 -
> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
> > On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com> wrote:
> >
> > For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
> attention for 2.8
> >
> > MAPREDUCE-6288
> > YARN-1815
> > YARN-4685
> > YARN-4844
> >
> > The rest are either improvements or long-standing issues and does not
> qualify release blocker, IMO.
> > I think we’ll try to get these 4 jiras in asap. The rest will be on best
> effort, resolve as much as possible and move them out if not resolved in
> time.
> >
> > Jian
> >
> > On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
> eeleast@gmail.com>> wrote:
> >
> > Sounds good to me :).
> >
> > Jian and I have looked at all existing 2.8.0 blockers and criticals
> today.
> > To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> > out. Left comments on these JIRAs asked original owners, plan to update
> > target version of these JIRAs early next week.
> >
> > Will keep this thread updated.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org>> wrote:
> >
> > How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> > with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> > report the list here. Then folks can chime in if you agree
> >
> > Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
> >
> > Thanks,
> > Sangjin
> >
> > On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com<mailto:
> wheeleast@gmail.com>> wrote:
> >
> > +1, we should close such staled JIRAs to avoid doing unnecessary checks
> > for
> > every releases.
> >
> > I'm working on reviewing YARN/MR critical/blocker patches currently, it
> > gonna very helpful if someone else can help with reviewing Common/HDFS
> > JIRAs.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org>> wrote:
> >
> > Where do we stand in terms of closing out blocker/critical issues for
> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
> > https://issues.apache.org/jira/issues/?filter=12334985
> >
> > But I see a lot of JIRAs with no patches or very stale patches. It
> > would be
> > a good exercise to come up with the list of JIRAs that we need to block
> > 2.8.0 for and focus our attention on closing them out. Thoughts?
> >
> > Thanks,
> > Sangjin
> >
> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com<
> mailto:stevel@hortonworks.com>
> >
> > wrote:
> >
> >
> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> > vinodkv@apache.org<ma...@apache.org>>
> > wrote:
> >
> > We are not converging - there’s still 58 more. I need help from the
> > community in addressing / review 2.8.0 blockers. If folks can start
> > with
> > reviewing Patch available tickets, that’ll be great.
> >
> >
> >
> >
> > I'm still doing the s3a stuff, other people testing and reviewing this
> > stuff welcome.
> >
> > in particular, I could do with others playing with this patch of mine,
> > which adds counters and things into S3a, based on the azure
> > instrumentation
> >
> > https://issues.apache.org/jira/browse/HADOOP-13028
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>
>
Re: [Continued] [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
Thanks for picking this up Junping!
I heard on email threads and offline discussions that there may be
interests in porting some fixes from branch-2 to branch-2.8. I still feel
that the bar should be pretty high to do that, but it might be good to have
folks suggest some critical fixes that need to be released sooner. Thoughts?
On Tue, Nov 29, 2016 at 8:50 PM, Junping Du <jd...@hortonworks.com> wrote:
> Hi folks,
> Per asked by Vinod offline, I would like to continue the effort to
> push 2.8 release out in next several weeks. The plan is as following:
> - We only have 5 blockers and 3 critical issues (
> https://s.apache.org/6kwx) so far, and most of them are in good progress
> now. I put them all on apache ciwiki ( https://cwiki.apache.org/
> confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor
> and update on these issues in next couple of weeks.
> - When all blocker/critical issues are gone (target EOD is Dec. 10),
> I will push out all major/minor issues that is open. So if you are working
> on some major issues and target for release in 2.8, please keep track of
> our release status and your work progress.
> - I will create RC immediately once our 2.8 target issues are clean
> (especially for blocker and critical issues) for voting process.
>
> I hope we can make it within year 2016, best before Xmas holiday. And
> I need help from all of you. :)
>
>
> Thanks,
>
> Junping
>
> ________________________________________
> From: Vinod Kumar Vavilapalli <vi...@apache.org>
> Sent: Monday, July 25, 2016 2:57 PM
> To: Jian He
> Cc: mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org;
> hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
> Subject: Re: [Release thread] 2.8.0 release activities
>
> Thanks for all that great help moving this release forward, Jian, Sangjin,
> Wangda et al! Appreciate it.
>
> The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last
> week.
>
> I only see 9 blocker / critical fixes pending for 2.8.0 -
> https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
>
> Thanks
> +Vinod
>
> > On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com> wrote:
> >
> > For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs
> attention for 2.8
> >
> > MAPREDUCE-6288
> > YARN-1815
> > YARN-4685
> > YARN-4844
> >
> > The rest are either improvements or long-standing issues and does not
> qualify release blocker, IMO.
> > I think we’ll try to get these 4 jiras in asap. The rest will be on best
> effort, resolve as much as possible and move them out if not resolved in
> time.
> >
> > Jian
> >
> > On May 11, 2016, at 5:37 PM, Wangda Tan <wheeleast@gmail.com<mailto:wh
> eeleast@gmail.com>> wrote:
> >
> > Sounds good to me :).
> >
> > Jian and I have looked at all existing 2.8.0 blockers and criticals
> today.
> > To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> > out. Left comments on these JIRAs asked original owners, plan to update
> > target version of these JIRAs early next week.
> >
> > Will keep this thread updated.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org>> wrote:
> >
> > How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> > with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> > report the list here. Then folks can chime in if you agree
> >
> > Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
> >
> > Thanks,
> > Sangjin
> >
> > On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wheeleast@gmail.com<mailto:
> wheeleast@gmail.com>> wrote:
> >
> > +1, we should close such staled JIRAs to avoid doing unnecessary checks
> > for
> > every releases.
> >
> > I'm working on reviewing YARN/MR critical/blocker patches currently, it
> > gonna very helpful if someone else can help with reviewing Common/HDFS
> > JIRAs.
> >
> > Thanks,
> > Wangda
> >
> >
> > On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sjlee@apache.org<mailto:
> sjlee@apache.org>> wrote:
> >
> > Where do we stand in terms of closing out blocker/critical issues for
> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
> > https://issues.apache.org/jira/issues/?filter=12334985
> >
> > But I see a lot of JIRAs with no patches or very stale patches. It
> > would be
> > a good exercise to come up with the list of JIRAs that we need to block
> > 2.8.0 for and focus our attention on closing them out. Thoughts?
> >
> > Thanks,
> > Sangjin
> >
> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com<
> mailto:stevel@hortonworks.com>
> >
> > wrote:
> >
> >
> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> > vinodkv@apache.org<ma...@apache.org>>
> > wrote:
> >
> > We are not converging - there’s still 58 more. I need help from the
> > community in addressing / review 2.8.0 blockers. If folks can start
> > with
> > reviewing Patch available tickets, that’ll be great.
> >
> >
> >
> >
> > I'm still doing the s3a stuff, other people testing and reviewing this
> > stuff welcome.
> >
> > in particular, I could do with others playing with this patch of mine,
> > which adds counters and things into S3a, based on the azure
> > instrumentation
> >
> > https://issues.apache.org/jira/browse/HADOOP-13028
> >
> >
> >
> >
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
>
>
[Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi folks,
Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
- We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
- When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
- I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
Thanks,
Junping
________________________________________
From: Vinod Kumar Vavilapalli <vi...@apache.org>
Sent: Monday, July 25, 2016 2:57 PM
To: Jian He
Cc: mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
Subject: Re: [Release thread] 2.8.0 release activities
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
[Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi folks,
Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
- We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
- When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
- I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
Thanks,
Junping
________________________________________
From: Vinod Kumar Vavilapalli <vi...@apache.org>
Sent: Monday, July 25, 2016 2:57 PM
To: Jian He
Cc: mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
Subject: Re: [Release thread] 2.8.0 release activities
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
[Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi folks,
Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
- We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
- When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
- I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
Thanks,
Junping
________________________________________
From: Vinod Kumar Vavilapalli <vi...@apache.org>
Sent: Monday, July 25, 2016 2:57 PM
To: Jian He
Cc: mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
Subject: Re: [Release thread] 2.8.0 release activities
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
[Continued] [Release thread] 2.8.0 release activities
Posted by Junping Du <jd...@hortonworks.com>.
Hi folks,
Per asked by Vinod offline, I would like to continue the effort to push 2.8 release out in next several weeks. The plan is as following:
- We only have 5 blockers and 3 critical issues (https://s.apache.org/6kwx) so far, and most of them are in good progress now. I put them all on apache ciwiki ( https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release) and will do daily monitor and update on these issues in next couple of weeks.
- When all blocker/critical issues are gone (target EOD is Dec. 10), I will push out all major/minor issues that is open. So if you are working on some major issues and target for release in 2.8, please keep track of our release status and your work progress.
- I will create RC immediately once our 2.8 target issues are clean (especially for blocker and critical issues) for voting process.
I hope we can make it within year 2016, best before Xmas holiday. And I need help from all of you. :)
Thanks,
Junping
________________________________________
From: Vinod Kumar Vavilapalli <vi...@apache.org>
Sent: Monday, July 25, 2016 2:57 PM
To: Jian He
Cc: mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
Subject: Re: [Release thread] 2.8.0 release activities
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
RE: [Release thread] 2.8.0 release activities
Posted by Rohith Sharma K S <ro...@huawei.com>.
There are few test cases regularly failing in YARN because of HADOOP-12687(issue with DNS). This issue is blocker for releases since YARN test cases do not guarantee quality.
There are some patches ready for this issue. But we doubt will this violate RFC-1535?
I would like to hear opinion from community on this.
Thanks & Regards
Rohith Sharma K S
-----Original Message-----
From: Jian He [mailto:jhe@hortonworks.com]
Sent: 12 May 2016 06:34
To: mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
Subject: Re: [Release thread] 2.8.0 release activities
For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
MAPREDUCE-6288
YARN-1815
YARN-4685
YARN-4844
The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
Jian
On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
Sounds good to me :).
Jian and I have looked at all existing 2.8.0 blockers and criticals today.
To me more than half of MR/YARN blockers/criticals of 2.8 should be moved out. Left comments on these JIRAs asked original owners, plan to update target version of these JIRAs early next week.
Will keep this thread updated.
Thanks,
Wangda
On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
How about this? I'll review the HADOOP/HDFS bugs in that list to come up with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll report the list here. Then folks can chime in if you agree
Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
Thanks,
Sangjin
On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
+1, we should close such staled JIRAs to avoid doing unnecessary checks
for
every releases.
I'm working on reviewing YARN/MR critical/blocker patches currently, it gonna very helpful if someone else can help with reviewing Common/HDFS JIRAs.
Thanks,
Wangda
On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
Where do we stand in terms of closing out blocker/critical issues for 2.8.0? I still see 50 open JIRAs in Vinod's list:
https://issues.apache.org/jira/issues/?filter=12334985
But I see a lot of JIRAs with no patches or very stale patches. It would be a good exercise to come up with the list of JIRAs that we need to block
2.8.0 for and focus our attention on closing them out. Thoughts?
Thanks,
Sangjin
On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
wrote:
On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli < vinodkv@apache.org<ma...@apache.org>>
wrote:
We are not converging - there’s still 58 more. I need help from the community in addressing / review 2.8.0 blockers. If folks can start with reviewing Patch available tickets, that’ll be great.
I'm still doing the s3a stuff, other people testing and reviewing this stuff welcome.
in particular, I could do with others playing with this patch of mine, which adds counters and things into S3a, based on the azure instrumentation
https://issues.apache.org/jira/browse/HADOOP-13028
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
RE: [Release thread] 2.8.0 release activities
Posted by Rohith Sharma K S <ro...@huawei.com>.
There are few test cases regularly failing in YARN because of HADOOP-12687(issue with DNS). This issue is blocker for releases since YARN test cases do not guarantee quality.
There are some patches ready for this issue. But we doubt will this violate RFC-1535?
I would like to hear opinion from community on this.
Thanks & Regards
Rohith Sharma K S
-----Original Message-----
From: Jian He [mailto:jhe@hortonworks.com]
Sent: 12 May 2016 06:34
To: mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
Subject: Re: [Release thread] 2.8.0 release activities
For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
MAPREDUCE-6288
YARN-1815
YARN-4685
YARN-4844
The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
Jian
On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
Sounds good to me :).
Jian and I have looked at all existing 2.8.0 blockers and criticals today.
To me more than half of MR/YARN blockers/criticals of 2.8 should be moved out. Left comments on these JIRAs asked original owners, plan to update target version of these JIRAs early next week.
Will keep this thread updated.
Thanks,
Wangda
On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
How about this? I'll review the HADOOP/HDFS bugs in that list to come up with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll report the list here. Then folks can chime in if you agree
Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
Thanks,
Sangjin
On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
+1, we should close such staled JIRAs to avoid doing unnecessary checks
for
every releases.
I'm working on reviewing YARN/MR critical/blocker patches currently, it gonna very helpful if someone else can help with reviewing Common/HDFS JIRAs.
Thanks,
Wangda
On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
Where do we stand in terms of closing out blocker/critical issues for 2.8.0? I still see 50 open JIRAs in Vinod's list:
https://issues.apache.org/jira/issues/?filter=12334985
But I see a lot of JIRAs with no patches or very stale patches. It would be a good exercise to come up with the list of JIRAs that we need to block
2.8.0 for and focus our attention on closing them out. Thoughts?
Thanks,
Sangjin
On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
wrote:
On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli < vinodkv@apache.org<ma...@apache.org>>
wrote:
We are not converging - there’s still 58 more. I need help from the community in addressing / review 2.8.0 blockers. If folks can start with reviewing Patch available tickets, that’ll be great.
I'm still doing the s3a stuff, other people testing and reviewing this stuff welcome.
in particular, I could do with others playing with this patch of mine, which adds counters and things into S3a, based on the azure instrumentation
https://issues.apache.org/jira/browse/HADOOP-13028
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
RE: [Release thread] 2.8.0 release activities
Posted by Rohith Sharma K S <ro...@huawei.com>.
There are few test cases regularly failing in YARN because of HADOOP-12687(issue with DNS). This issue is blocker for releases since YARN test cases do not guarantee quality.
There are some patches ready for this issue. But we doubt will this violate RFC-1535?
I would like to hear opinion from community on this.
Thanks & Regards
Rohith Sharma K S
-----Original Message-----
From: Jian He [mailto:jhe@hortonworks.com]
Sent: 12 May 2016 06:34
To: mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
Subject: Re: [Release thread] 2.8.0 release activities
For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
MAPREDUCE-6288
YARN-1815
YARN-4685
YARN-4844
The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
Jian
On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
Sounds good to me :).
Jian and I have looked at all existing 2.8.0 blockers and criticals today.
To me more than half of MR/YARN blockers/criticals of 2.8 should be moved out. Left comments on these JIRAs asked original owners, plan to update target version of these JIRAs early next week.
Will keep this thread updated.
Thanks,
Wangda
On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
How about this? I'll review the HADOOP/HDFS bugs in that list to come up with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll report the list here. Then folks can chime in if you agree
Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
Thanks,
Sangjin
On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
+1, we should close such staled JIRAs to avoid doing unnecessary checks
for
every releases.
I'm working on reviewing YARN/MR critical/blocker patches currently, it gonna very helpful if someone else can help with reviewing Common/HDFS JIRAs.
Thanks,
Wangda
On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
Where do we stand in terms of closing out blocker/critical issues for 2.8.0? I still see 50 open JIRAs in Vinod's list:
https://issues.apache.org/jira/issues/?filter=12334985
But I see a lot of JIRAs with no patches or very stale patches. It would be a good exercise to come up with the list of JIRAs that we need to block
2.8.0 for and focus our attention on closing them out. Thoughts?
Thanks,
Sangjin
On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
wrote:
On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli < vinodkv@apache.org<ma...@apache.org>>
wrote:
We are not converging - there’s still 58 more. I need help from the community in addressing / review 2.8.0 blockers. If folks can start with reviewing Patch available tickets, that’ll be great.
I'm still doing the s3a stuff, other people testing and reviewing this stuff welcome.
in particular, I could do with others playing with this patch of mine, which adds counters and things into S3a, based on the azure instrumentation
https://issues.apache.org/jira/browse/HADOOP-13028
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
RE: [Release thread] 2.8.0 release activities
Posted by Rohith Sharma K S <ro...@huawei.com>.
There are few test cases regularly failing in YARN because of HADOOP-12687(issue with DNS). This issue is blocker for releases since YARN test cases do not guarantee quality.
There are some patches ready for this issue. But we doubt will this violate RFC-1535?
I would like to hear opinion from community on this.
Thanks & Regards
Rohith Sharma K S
-----Original Message-----
From: Jian He [mailto:jhe@hortonworks.com]
Sent: 12 May 2016 06:34
To: mapreduce-dev@hadoop.apache.org; yarn-dev@hadoop.apache.org; hdfs-dev@hadoop.apache.org; common-dev@hadoop.apache.org
Subject: Re: [Release thread] 2.8.0 release activities
For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
MAPREDUCE-6288
YARN-1815
YARN-4685
YARN-4844
The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
Jian
On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
Sounds good to me :).
Jian and I have looked at all existing 2.8.0 blockers and criticals today.
To me more than half of MR/YARN blockers/criticals of 2.8 should be moved out. Left comments on these JIRAs asked original owners, plan to update target version of these JIRAs early next week.
Will keep this thread updated.
Thanks,
Wangda
On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
How about this? I'll review the HADOOP/HDFS bugs in that list to come up with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll report the list here. Then folks can chime in if you agree
Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
Thanks,
Sangjin
On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
+1, we should close such staled JIRAs to avoid doing unnecessary checks
for
every releases.
I'm working on reviewing YARN/MR critical/blocker patches currently, it gonna very helpful if someone else can help with reviewing Common/HDFS JIRAs.
Thanks,
Wangda
On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
Where do we stand in terms of closing out blocker/critical issues for 2.8.0? I still see 50 open JIRAs in Vinod's list:
https://issues.apache.org/jira/issues/?filter=12334985
But I see a lot of JIRAs with no patches or very stale patches. It would be a good exercise to come up with the list of JIRAs that we need to block
2.8.0 for and focus our attention on closing them out. Thoughts?
Thanks,
Sangjin
On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
wrote:
On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli < vinodkv@apache.org<ma...@apache.org>>
wrote:
We are not converging - there’s still 58 more. I need help from the community in addressing / review 2.8.0 blockers. If folks can start with reviewing Patch available tickets, that’ll be great.
I'm still doing the s3a stuff, other people testing and reviewing this stuff welcome.
in particular, I could do with others playing with this patch of mine, which adds counters and things into S3a, based on the azure instrumentation
https://issues.apache.org/jira/browse/HADOOP-13028
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Thanks for all that great help moving this release forward, Jian, Sangjin, Wangda et al! Appreciate it.
The License / Notice fix is finally in. And I pushed out a 2.7.3 RC0 last week.
I only see 9 blocker / critical fixes pending for 2.8.0 - https://issues.apache.org/jira/issues/?filter=12334985. Let’s do this!
Thanks
+Vinod
> On May 11, 2016, at 6:04 PM, Jian He <jh...@hortonworks.com> wrote:
>
> For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
>
> MAPREDUCE-6288
> YARN-1815
> YARN-4685
> YARN-4844
>
> The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
> I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
>
> Jian
>
> On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
>
> +1, we should close such staled JIRAs to avoid doing unnecessary checks
> for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
>
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It
> would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
>
> wrote:
>
>
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org<ma...@apache.org>>
> wrote:
>
> We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start
> with
> reviewing Patch available tickets, that’ll be great.
>
>
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure
> instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
>
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [Release thread] 2.8.0 release activities
Posted by Jian He <jh...@hortonworks.com>.
For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
MAPREDUCE-6288
YARN-1815
YARN-4685
YARN-4844
The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
Jian
On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
Sounds good to me :).
Jian and I have looked at all existing 2.8.0 blockers and criticals today.
To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
out. Left comments on these JIRAs asked original owners, plan to update
target version of these JIRAs early next week.
Will keep this thread updated.
Thanks,
Wangda
On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
How about this? I'll review the HADOOP/HDFS bugs in that list to come up
with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
report the list here. Then folks can chime in if you agree
Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
Thanks,
Sangjin
On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
+1, we should close such staled JIRAs to avoid doing unnecessary checks
for
every releases.
I'm working on reviewing YARN/MR critical/blocker patches currently, it
gonna very helpful if someone else can help with reviewing Common/HDFS
JIRAs.
Thanks,
Wangda
On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
Where do we stand in terms of closing out blocker/critical issues for
2.8.0? I still see 50 open JIRAs in Vinod's list:
https://issues.apache.org/jira/issues/?filter=12334985
But I see a lot of JIRAs with no patches or very stale patches. It
would be
a good exercise to come up with the list of JIRAs that we need to block
2.8.0 for and focus our attention on closing them out. Thoughts?
Thanks,
Sangjin
On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
wrote:
On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
vinodkv@apache.org<ma...@apache.org>>
wrote:
We are not converging - there’s still 58 more. I need help from the
community in addressing / review 2.8.0 blockers. If folks can start
with
reviewing Patch available tickets, that’ll be great.
I'm still doing the s3a stuff, other people testing and reviewing this
stuff welcome.
in particular, I could do with others playing with this patch of mine,
which adds counters and things into S3a, based on the azure
instrumentation
https://issues.apache.org/jira/browse/HADOOP-13028
Re: [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
I see the following list of JIRAs from HADOOP and HDFS as blockers for
2.8.0. Other JIRAs are either old issues with little movement or new issues
that don't appear to be as critical/ready.
- HADOOP-12893
- HADOOP-12892
- HADOOP-10940 (patch ready?)
- HADOOP-12971 (can be done relatively quickly?)
- HDFS-7959
- HDFS-7597 (needs review/some more work?)
I would propose moving the rest out of scope for 2.8.0 (23 JIRAs). Let me
know what you think.
On Wed, May 11, 2016 at 5:37 PM, Wangda Tan <wh...@gmail.com> wrote:
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org> wrote:
>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com> wrote:
>>
>>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>>> for
>>> every releases.
>>>
>>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>>> gonna very helpful if someone else can help with reviewing Common/HDFS
>>> JIRAs.
>>>
>>> Thanks,
>>> Wangda
>>>
>>>
>>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>
>>> > Where do we stand in terms of closing out blocker/critical issues for
>>> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
>>> > https://issues.apache.org/jira/issues/?filter=12334985
>>> >
>>> > But I see a lot of JIRAs with no patches or very stale patches. It
>>> would be
>>> > a good exercise to come up with the list of JIRAs that we need to block
>>> > 2.8.0 for and focus our attention on closing them out. Thoughts?
>>> >
>>> > Thanks,
>>> > Sangjin
>>> >
>>> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <
>>> stevel@hortonworks.com>
>>> > wrote:
>>> >
>>> > >
>>> > > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>>> vinodkv@apache.org>
>>> > > wrote:
>>> > > >
>>> > > > We are not converging - there’s still 58 more. I need help from the
>>> > > community in addressing / review 2.8.0 blockers. If folks can start
>>> with
>>> > > reviewing Patch available tickets, that’ll be great.
>>> > > >
>>> > > >
>>> > >
>>> > >
>>> > > I'm still doing the s3a stuff, other people testing and reviewing
>>> this
>>> > > stuff welcome.
>>> > >
>>> > > in particular, I could do with others playing with this patch of
>>> mine,
>>> > > which adds counters and things into S3a, based on the azure
>>> > instrumentation
>>> > >
>>> > > https://issues.apache.org/jira/browse/HADOOP-13028
>>> > >
>>> > >
>>> > >
>>> >
>>>
>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
I see the following list of JIRAs from HADOOP and HDFS as blockers for
2.8.0. Other JIRAs are either old issues with little movement or new issues
that don't appear to be as critical/ready.
- HADOOP-12893
- HADOOP-12892
- HADOOP-10940 (patch ready?)
- HADOOP-12971 (can be done relatively quickly?)
- HDFS-7959
- HDFS-7597 (needs review/some more work?)
I would propose moving the rest out of scope for 2.8.0 (23 JIRAs). Let me
know what you think.
On Wed, May 11, 2016 at 5:37 PM, Wangda Tan <wh...@gmail.com> wrote:
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org> wrote:
>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com> wrote:
>>
>>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>>> for
>>> every releases.
>>>
>>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>>> gonna very helpful if someone else can help with reviewing Common/HDFS
>>> JIRAs.
>>>
>>> Thanks,
>>> Wangda
>>>
>>>
>>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>
>>> > Where do we stand in terms of closing out blocker/critical issues for
>>> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
>>> > https://issues.apache.org/jira/issues/?filter=12334985
>>> >
>>> > But I see a lot of JIRAs with no patches or very stale patches. It
>>> would be
>>> > a good exercise to come up with the list of JIRAs that we need to block
>>> > 2.8.0 for and focus our attention on closing them out. Thoughts?
>>> >
>>> > Thanks,
>>> > Sangjin
>>> >
>>> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <
>>> stevel@hortonworks.com>
>>> > wrote:
>>> >
>>> > >
>>> > > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>>> vinodkv@apache.org>
>>> > > wrote:
>>> > > >
>>> > > > We are not converging - there’s still 58 more. I need help from the
>>> > > community in addressing / review 2.8.0 blockers. If folks can start
>>> with
>>> > > reviewing Patch available tickets, that’ll be great.
>>> > > >
>>> > > >
>>> > >
>>> > >
>>> > > I'm still doing the s3a stuff, other people testing and reviewing
>>> this
>>> > > stuff welcome.
>>> > >
>>> > > in particular, I could do with others playing with this patch of
>>> mine,
>>> > > which adds counters and things into S3a, based on the azure
>>> > instrumentation
>>> > >
>>> > > https://issues.apache.org/jira/browse/HADOOP-13028
>>> > >
>>> > >
>>> > >
>>> >
>>>
>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
I see the following list of JIRAs from HADOOP and HDFS as blockers for
2.8.0. Other JIRAs are either old issues with little movement or new issues
that don't appear to be as critical/ready.
- HADOOP-12893
- HADOOP-12892
- HADOOP-10940 (patch ready?)
- HADOOP-12971 (can be done relatively quickly?)
- HDFS-7959
- HDFS-7597 (needs review/some more work?)
I would propose moving the rest out of scope for 2.8.0 (23 JIRAs). Let me
know what you think.
On Wed, May 11, 2016 at 5:37 PM, Wangda Tan <wh...@gmail.com> wrote:
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org> wrote:
>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com> wrote:
>>
>>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>>> for
>>> every releases.
>>>
>>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>>> gonna very helpful if someone else can help with reviewing Common/HDFS
>>> JIRAs.
>>>
>>> Thanks,
>>> Wangda
>>>
>>>
>>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>
>>> > Where do we stand in terms of closing out blocker/critical issues for
>>> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
>>> > https://issues.apache.org/jira/issues/?filter=12334985
>>> >
>>> > But I see a lot of JIRAs with no patches or very stale patches. It
>>> would be
>>> > a good exercise to come up with the list of JIRAs that we need to block
>>> > 2.8.0 for and focus our attention on closing them out. Thoughts?
>>> >
>>> > Thanks,
>>> > Sangjin
>>> >
>>> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <
>>> stevel@hortonworks.com>
>>> > wrote:
>>> >
>>> > >
>>> > > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>>> vinodkv@apache.org>
>>> > > wrote:
>>> > > >
>>> > > > We are not converging - there’s still 58 more. I need help from the
>>> > > community in addressing / review 2.8.0 blockers. If folks can start
>>> with
>>> > > reviewing Patch available tickets, that’ll be great.
>>> > > >
>>> > > >
>>> > >
>>> > >
>>> > > I'm still doing the s3a stuff, other people testing and reviewing
>>> this
>>> > > stuff welcome.
>>> > >
>>> > > in particular, I could do with others playing with this patch of
>>> mine,
>>> > > which adds counters and things into S3a, based on the azure
>>> > instrumentation
>>> > >
>>> > > https://issues.apache.org/jira/browse/HADOOP-13028
>>> > >
>>> > >
>>> > >
>>> >
>>>
>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
I see the following list of JIRAs from HADOOP and HDFS as blockers for
2.8.0. Other JIRAs are either old issues with little movement or new issues
that don't appear to be as critical/ready.
- HADOOP-12893
- HADOOP-12892
- HADOOP-10940 (patch ready?)
- HADOOP-12971 (can be done relatively quickly?)
- HDFS-7959
- HDFS-7597 (needs review/some more work?)
I would propose moving the rest out of scope for 2.8.0 (23 JIRAs). Let me
know what you think.
On Wed, May 11, 2016 at 5:37 PM, Wangda Tan <wh...@gmail.com> wrote:
> Sounds good to me :).
>
> Jian and I have looked at all existing 2.8.0 blockers and criticals today.
> To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
> out. Left comments on these JIRAs asked original owners, plan to update
> target version of these JIRAs early next week.
>
> Will keep this thread updated.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org> wrote:
>
>> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
>> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
>> report the list here. Then folks can chime in if you agree
>>
>> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>>
>> Thanks,
>> Sangjin
>>
>> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com> wrote:
>>
>>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>>> for
>>> every releases.
>>>
>>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>>> gonna very helpful if someone else can help with reviewing Common/HDFS
>>> JIRAs.
>>>
>>> Thanks,
>>> Wangda
>>>
>>>
>>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
>>>
>>> > Where do we stand in terms of closing out blocker/critical issues for
>>> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
>>> > https://issues.apache.org/jira/issues/?filter=12334985
>>> >
>>> > But I see a lot of JIRAs with no patches or very stale patches. It
>>> would be
>>> > a good exercise to come up with the list of JIRAs that we need to block
>>> > 2.8.0 for and focus our attention on closing them out. Thoughts?
>>> >
>>> > Thanks,
>>> > Sangjin
>>> >
>>> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <
>>> stevel@hortonworks.com>
>>> > wrote:
>>> >
>>> > >
>>> > > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>>> vinodkv@apache.org>
>>> > > wrote:
>>> > > >
>>> > > > We are not converging - there’s still 58 more. I need help from the
>>> > > community in addressing / review 2.8.0 blockers. If folks can start
>>> with
>>> > > reviewing Patch available tickets, that’ll be great.
>>> > > >
>>> > > >
>>> > >
>>> > >
>>> > > I'm still doing the s3a stuff, other people testing and reviewing
>>> this
>>> > > stuff welcome.
>>> > >
>>> > > in particular, I could do with others playing with this patch of
>>> mine,
>>> > > which adds counters and things into S3a, based on the azure
>>> > instrumentation
>>> > >
>>> > > https://issues.apache.org/jira/browse/HADOOP-13028
>>> > >
>>> > >
>>> > >
>>> >
>>>
>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Jian He <jh...@hortonworks.com>.
For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
MAPREDUCE-6288
YARN-1815
YARN-4685
YARN-4844
The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
Jian
On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
Sounds good to me :).
Jian and I have looked at all existing 2.8.0 blockers and criticals today.
To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
out. Left comments on these JIRAs asked original owners, plan to update
target version of these JIRAs early next week.
Will keep this thread updated.
Thanks,
Wangda
On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
How about this? I'll review the HADOOP/HDFS bugs in that list to come up
with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
report the list here. Then folks can chime in if you agree
Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
Thanks,
Sangjin
On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
+1, we should close such staled JIRAs to avoid doing unnecessary checks
for
every releases.
I'm working on reviewing YARN/MR critical/blocker patches currently, it
gonna very helpful if someone else can help with reviewing Common/HDFS
JIRAs.
Thanks,
Wangda
On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
Where do we stand in terms of closing out blocker/critical issues for
2.8.0? I still see 50 open JIRAs in Vinod's list:
https://issues.apache.org/jira/issues/?filter=12334985
But I see a lot of JIRAs with no patches or very stale patches. It
would be
a good exercise to come up with the list of JIRAs that we need to block
2.8.0 for and focus our attention on closing them out. Thoughts?
Thanks,
Sangjin
On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
wrote:
On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
vinodkv@apache.org<ma...@apache.org>>
wrote:
We are not converging - there’s still 58 more. I need help from the
community in addressing / review 2.8.0 blockers. If folks can start
with
reviewing Patch available tickets, that’ll be great.
I'm still doing the s3a stuff, other people testing and reviewing this
stuff welcome.
in particular, I could do with others playing with this patch of mine,
which adds counters and things into S3a, based on the azure
instrumentation
https://issues.apache.org/jira/browse/HADOOP-13028
Re: [Release thread] 2.8.0 release activities
Posted by Jian He <jh...@hortonworks.com>.
For MapReduce/YARN, I closed a few staled ones. Only 4 jiras needs attention for 2.8
MAPREDUCE-6288
YARN-1815
YARN-4685
YARN-4844
The rest are either improvements or long-standing issues and does not qualify release blocker, IMO.
I think we’ll try to get these 4 jiras in asap. The rest will be on best effort, resolve as much as possible and move them out if not resolved in time.
Jian
On May 11, 2016, at 5:37 PM, Wangda Tan <wh...@gmail.com>> wrote:
Sounds good to me :).
Jian and I have looked at all existing 2.8.0 blockers and criticals today.
To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
out. Left comments on these JIRAs asked original owners, plan to update
target version of these JIRAs early next week.
Will keep this thread updated.
Thanks,
Wangda
On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org>> wrote:
How about this? I'll review the HADOOP/HDFS bugs in that list to come up
with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
report the list here. Then folks can chime in if you agree
Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
Thanks,
Sangjin
On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com>> wrote:
+1, we should close such staled JIRAs to avoid doing unnecessary checks
for
every releases.
I'm working on reviewing YARN/MR critical/blocker patches currently, it
gonna very helpful if someone else can help with reviewing Common/HDFS
JIRAs.
Thanks,
Wangda
On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org>> wrote:
Where do we stand in terms of closing out blocker/critical issues for
2.8.0? I still see 50 open JIRAs in Vinod's list:
https://issues.apache.org/jira/issues/?filter=12334985
But I see a lot of JIRAs with no patches or very stale patches. It
would be
a good exercise to come up with the list of JIRAs that we need to block
2.8.0 for and focus our attention on closing them out. Thoughts?
Thanks,
Sangjin
On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
wrote:
On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
vinodkv@apache.org<ma...@apache.org>>
wrote:
We are not converging - there’s still 58 more. I need help from the
community in addressing / review 2.8.0 blockers. If folks can start
with
reviewing Patch available tickets, that’ll be great.
I'm still doing the s3a stuff, other people testing and reviewing this
stuff welcome.
in particular, I could do with others playing with this patch of mine,
which adds counters and things into S3a, based on the azure
instrumentation
https://issues.apache.org/jira/browse/HADOOP-13028
Re: [Release thread] 2.8.0 release activities
Posted by Wangda Tan <wh...@gmail.com>.
Sounds good to me :).
Jian and I have looked at all existing 2.8.0 blockers and criticals today.
To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
out. Left comments on these JIRAs asked original owners, plan to update
target version of these JIRAs early next week.
Will keep this thread updated.
Thanks,
Wangda
On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org> wrote:
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com> wrote:
>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
>>
>> > Where do we stand in terms of closing out blocker/critical issues for
>> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> > https://issues.apache.org/jira/issues/?filter=12334985
>> >
>> > But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> > a good exercise to come up with the list of JIRAs that we need to block
>> > 2.8.0 for and focus our attention on closing them out. Thoughts?
>> >
>> > Thanks,
>> > Sangjin
>> >
>> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com
>> >
>> > wrote:
>> >
>> > >
>> > > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org>
>> > > wrote:
>> > > >
>> > > > We are not converging - there’s still 58 more. I need help from the
>> > > community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> > > reviewing Patch available tickets, that’ll be great.
>> > > >
>> > > >
>> > >
>> > >
>> > > I'm still doing the s3a stuff, other people testing and reviewing this
>> > > stuff welcome.
>> > >
>> > > in particular, I could do with others playing with this patch of mine,
>> > > which adds counters and things into S3a, based on the azure
>> > instrumentation
>> > >
>> > > https://issues.apache.org/jira/browse/HADOOP-13028
>> > >
>> > >
>> > >
>> >
>>
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Wangda Tan <wh...@gmail.com>.
Sounds good to me :).
Jian and I have looked at all existing 2.8.0 blockers and criticals today.
To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
out. Left comments on these JIRAs asked original owners, plan to update
target version of these JIRAs early next week.
Will keep this thread updated.
Thanks,
Wangda
On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org> wrote:
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com> wrote:
>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
>>
>> > Where do we stand in terms of closing out blocker/critical issues for
>> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> > https://issues.apache.org/jira/issues/?filter=12334985
>> >
>> > But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> > a good exercise to come up with the list of JIRAs that we need to block
>> > 2.8.0 for and focus our attention on closing them out. Thoughts?
>> >
>> > Thanks,
>> > Sangjin
>> >
>> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com
>> >
>> > wrote:
>> >
>> > >
>> > > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org>
>> > > wrote:
>> > > >
>> > > > We are not converging - there’s still 58 more. I need help from the
>> > > community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> > > reviewing Patch available tickets, that’ll be great.
>> > > >
>> > > >
>> > >
>> > >
>> > > I'm still doing the s3a stuff, other people testing and reviewing this
>> > > stuff welcome.
>> > >
>> > > in particular, I could do with others playing with this patch of mine,
>> > > which adds counters and things into S3a, based on the azure
>> > instrumentation
>> > >
>> > > https://issues.apache.org/jira/browse/HADOOP-13028
>> > >
>> > >
>> > >
>> >
>>
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Wangda Tan <wh...@gmail.com>.
Sounds good to me :).
Jian and I have looked at all existing 2.8.0 blockers and criticals today.
To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
out. Left comments on these JIRAs asked original owners, plan to update
target version of these JIRAs early next week.
Will keep this thread updated.
Thanks,
Wangda
On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org> wrote:
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com> wrote:
>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
>>
>> > Where do we stand in terms of closing out blocker/critical issues for
>> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> > https://issues.apache.org/jira/issues/?filter=12334985
>> >
>> > But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> > a good exercise to come up with the list of JIRAs that we need to block
>> > 2.8.0 for and focus our attention on closing them out. Thoughts?
>> >
>> > Thanks,
>> > Sangjin
>> >
>> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com
>> >
>> > wrote:
>> >
>> > >
>> > > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org>
>> > > wrote:
>> > > >
>> > > > We are not converging - there’s still 58 more. I need help from the
>> > > community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> > > reviewing Patch available tickets, that’ll be great.
>> > > >
>> > > >
>> > >
>> > >
>> > > I'm still doing the s3a stuff, other people testing and reviewing this
>> > > stuff welcome.
>> > >
>> > > in particular, I could do with others playing with this patch of mine,
>> > > which adds counters and things into S3a, based on the azure
>> > instrumentation
>> > >
>> > > https://issues.apache.org/jira/browse/HADOOP-13028
>> > >
>> > >
>> > >
>> >
>>
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Wangda Tan <wh...@gmail.com>.
Sounds good to me :).
Jian and I have looked at all existing 2.8.0 blockers and criticals today.
To me more than half of MR/YARN blockers/criticals of 2.8 should be moved
out. Left comments on these JIRAs asked original owners, plan to update
target version of these JIRAs early next week.
Will keep this thread updated.
Thanks,
Wangda
On Wed, May 11, 2016 at 5:06 PM, Sangjin Lee <sj...@apache.org> wrote:
> How about this? I'll review the HADOOP/HDFS bugs in that list to come up
> with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
> report the list here. Then folks can chime in if you agree
>
> Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
>
> Thanks,
> Sangjin
>
> On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com> wrote:
>
>> +1, we should close such staled JIRAs to avoid doing unnecessary checks
>> for
>> every releases.
>>
>> I'm working on reviewing YARN/MR critical/blocker patches currently, it
>> gonna very helpful if someone else can help with reviewing Common/HDFS
>> JIRAs.
>>
>> Thanks,
>> Wangda
>>
>>
>> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
>>
>> > Where do we stand in terms of closing out blocker/critical issues for
>> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
>> > https://issues.apache.org/jira/issues/?filter=12334985
>> >
>> > But I see a lot of JIRAs with no patches or very stale patches. It
>> would be
>> > a good exercise to come up with the list of JIRAs that we need to block
>> > 2.8.0 for and focus our attention on closing them out. Thoughts?
>> >
>> > Thanks,
>> > Sangjin
>> >
>> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <stevel@hortonworks.com
>> >
>> > wrote:
>> >
>> > >
>> > > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
>> vinodkv@apache.org>
>> > > wrote:
>> > > >
>> > > > We are not converging - there’s still 58 more. I need help from the
>> > > community in addressing / review 2.8.0 blockers. If folks can start
>> with
>> > > reviewing Patch available tickets, that’ll be great.
>> > > >
>> > > >
>> > >
>> > >
>> > > I'm still doing the s3a stuff, other people testing and reviewing this
>> > > stuff welcome.
>> > >
>> > > in particular, I could do with others playing with this patch of mine,
>> > > which adds counters and things into S3a, based on the azure
>> > instrumentation
>> > >
>> > > https://issues.apache.org/jira/browse/HADOOP-13028
>> > >
>> > >
>> > >
>> >
>>
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
How about this? I'll review the HADOOP/HDFS bugs in that list to come up
with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
report the list here. Then folks can chime in if you agree
Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
Thanks,
Sangjin
On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com> wrote:
> +1, we should close such staled JIRAs to avoid doing unnecessary checks for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
>
> > Where do we stand in terms of closing out blocker/critical issues for
> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
> > https://issues.apache.org/jira/issues/?filter=12334985
> >
> > But I see a lot of JIRAs with no patches or very stale patches. It would
> be
> > a good exercise to come up with the list of JIRAs that we need to block
> > 2.8.0 for and focus our attention on closing them out. Thoughts?
> >
> > Thanks,
> > Sangjin
> >
> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
> > wrote:
> >
> > >
> > > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org>
> > > wrote:
> > > >
> > > > We are not converging - there’s still 58 more. I need help from the
> > > community in addressing / review 2.8.0 blockers. If folks can start
> with
> > > reviewing Patch available tickets, that’ll be great.
> > > >
> > > >
> > >
> > >
> > > I'm still doing the s3a stuff, other people testing and reviewing this
> > > stuff welcome.
> > >
> > > in particular, I could do with others playing with this patch of mine,
> > > which adds counters and things into S3a, based on the azure
> > instrumentation
> > >
> > > https://issues.apache.org/jira/browse/HADOOP-13028
> > >
> > >
> > >
> >
>
Re: [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
How about this? I'll review the HADOOP/HDFS bugs in that list to come up
with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
report the list here. Then folks can chime in if you agree
Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
Thanks,
Sangjin
On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com> wrote:
> +1, we should close such staled JIRAs to avoid doing unnecessary checks for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
>
> > Where do we stand in terms of closing out blocker/critical issues for
> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
> > https://issues.apache.org/jira/issues/?filter=12334985
> >
> > But I see a lot of JIRAs with no patches or very stale patches. It would
> be
> > a good exercise to come up with the list of JIRAs that we need to block
> > 2.8.0 for and focus our attention on closing them out. Thoughts?
> >
> > Thanks,
> > Sangjin
> >
> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
> > wrote:
> >
> > >
> > > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org>
> > > wrote:
> > > >
> > > > We are not converging - there’s still 58 more. I need help from the
> > > community in addressing / review 2.8.0 blockers. If folks can start
> with
> > > reviewing Patch available tickets, that’ll be great.
> > > >
> > > >
> > >
> > >
> > > I'm still doing the s3a stuff, other people testing and reviewing this
> > > stuff welcome.
> > >
> > > in particular, I could do with others playing with this patch of mine,
> > > which adds counters and things into S3a, based on the azure
> > instrumentation
> > >
> > > https://issues.apache.org/jira/browse/HADOOP-13028
> > >
> > >
> > >
> >
>
Re: [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
How about this? I'll review the HADOOP/HDFS bugs in that list to come up
with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
report the list here. Then folks can chime in if you agree
Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
Thanks,
Sangjin
On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com> wrote:
> +1, we should close such staled JIRAs to avoid doing unnecessary checks for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
>
> > Where do we stand in terms of closing out blocker/critical issues for
> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
> > https://issues.apache.org/jira/issues/?filter=12334985
> >
> > But I see a lot of JIRAs with no patches or very stale patches. It would
> be
> > a good exercise to come up with the list of JIRAs that we need to block
> > 2.8.0 for and focus our attention on closing them out. Thoughts?
> >
> > Thanks,
> > Sangjin
> >
> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
> > wrote:
> >
> > >
> > > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org>
> > > wrote:
> > > >
> > > > We are not converging - there’s still 58 more. I need help from the
> > > community in addressing / review 2.8.0 blockers. If folks can start
> with
> > > reviewing Patch available tickets, that’ll be great.
> > > >
> > > >
> > >
> > >
> > > I'm still doing the s3a stuff, other people testing and reviewing this
> > > stuff welcome.
> > >
> > > in particular, I could do with others playing with this patch of mine,
> > > which adds counters and things into S3a, based on the azure
> > instrumentation
> > >
> > > https://issues.apache.org/jira/browse/HADOOP-13028
> > >
> > >
> > >
> >
>
Re: [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
How about this? I'll review the HADOOP/HDFS bugs in that list to come up
with true blockers for 2.8.0 or JIRAs that are close to being ready. I'll
report the list here. Then folks can chime in if you agree
Perhaps Wangda, you can go over the YARN/MR bugs. Sound like a plan?
Thanks,
Sangjin
On Wed, May 11, 2016 at 4:26 PM, Wangda Tan <wh...@gmail.com> wrote:
> +1, we should close such staled JIRAs to avoid doing unnecessary checks for
> every releases.
>
> I'm working on reviewing YARN/MR critical/blocker patches currently, it
> gonna very helpful if someone else can help with reviewing Common/HDFS
> JIRAs.
>
> Thanks,
> Wangda
>
>
> On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
>
> > Where do we stand in terms of closing out blocker/critical issues for
> > 2.8.0? I still see 50 open JIRAs in Vinod's list:
> > https://issues.apache.org/jira/issues/?filter=12334985
> >
> > But I see a lot of JIRAs with no patches or very stale patches. It would
> be
> > a good exercise to come up with the list of JIRAs that we need to block
> > 2.8.0 for and focus our attention on closing them out. Thoughts?
> >
> > Thanks,
> > Sangjin
> >
> > On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
> > wrote:
> >
> > >
> > > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <
> vinodkv@apache.org>
> > > wrote:
> > > >
> > > > We are not converging - there’s still 58 more. I need help from the
> > > community in addressing / review 2.8.0 blockers. If folks can start
> with
> > > reviewing Patch available tickets, that’ll be great.
> > > >
> > > >
> > >
> > >
> > > I'm still doing the s3a stuff, other people testing and reviewing this
> > > stuff welcome.
> > >
> > > in particular, I could do with others playing with this patch of mine,
> > > which adds counters and things into S3a, based on the azure
> > instrumentation
> > >
> > > https://issues.apache.org/jira/browse/HADOOP-13028
> > >
> > >
> > >
> >
>
Re: [Release thread] 2.8.0 release activities
Posted by Wangda Tan <wh...@gmail.com>.
+1, we should close such staled JIRAs to avoid doing unnecessary checks for
every releases.
I'm working on reviewing YARN/MR critical/blocker patches currently, it
gonna very helpful if someone else can help with reviewing Common/HDFS
JIRAs.
Thanks,
Wangda
On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
> wrote:
>
> >
> > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <vi...@apache.org>
> > wrote:
> > >
> > > We are not converging - there’s still 58 more. I need help from the
> > community in addressing / review 2.8.0 blockers. If folks can start with
> > reviewing Patch available tickets, that’ll be great.
> > >
> > >
> >
> >
> > I'm still doing the s3a stuff, other people testing and reviewing this
> > stuff welcome.
> >
> > in particular, I could do with others playing with this patch of mine,
> > which adds counters and things into S3a, based on the azure
> instrumentation
> >
> > https://issues.apache.org/jira/browse/HADOOP-13028
> >
> >
> >
>
Re: [Release thread] 2.8.0 release activities
Posted by Wangda Tan <wh...@gmail.com>.
+1, we should close such staled JIRAs to avoid doing unnecessary checks for
every releases.
I'm working on reviewing YARN/MR critical/blocker patches currently, it
gonna very helpful if someone else can help with reviewing Common/HDFS
JIRAs.
Thanks,
Wangda
On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
> wrote:
>
> >
> > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <vi...@apache.org>
> > wrote:
> > >
> > > We are not converging - there’s still 58 more. I need help from the
> > community in addressing / review 2.8.0 blockers. If folks can start with
> > reviewing Patch available tickets, that’ll be great.
> > >
> > >
> >
> >
> > I'm still doing the s3a stuff, other people testing and reviewing this
> > stuff welcome.
> >
> > in particular, I could do with others playing with this patch of mine,
> > which adds counters and things into S3a, based on the azure
> instrumentation
> >
> > https://issues.apache.org/jira/browse/HADOOP-13028
> >
> >
> >
>
Re: [Release thread] 2.8.0 release activities
Posted by Wangda Tan <wh...@gmail.com>.
+1, we should close such staled JIRAs to avoid doing unnecessary checks for
every releases.
I'm working on reviewing YARN/MR critical/blocker patches currently, it
gonna very helpful if someone else can help with reviewing Common/HDFS
JIRAs.
Thanks,
Wangda
On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
> wrote:
>
> >
> > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <vi...@apache.org>
> > wrote:
> > >
> > > We are not converging - there’s still 58 more. I need help from the
> > community in addressing / review 2.8.0 blockers. If folks can start with
> > reviewing Patch available tickets, that’ll be great.
> > >
> > >
> >
> >
> > I'm still doing the s3a stuff, other people testing and reviewing this
> > stuff welcome.
> >
> > in particular, I could do with others playing with this patch of mine,
> > which adds counters and things into S3a, based on the azure
> instrumentation
> >
> > https://issues.apache.org/jira/browse/HADOOP-13028
> >
> >
> >
>
Re: [Release thread] 2.8.0 release activities
Posted by Wangda Tan <wh...@gmail.com>.
+1, we should close such staled JIRAs to avoid doing unnecessary checks for
every releases.
I'm working on reviewing YARN/MR critical/blocker patches currently, it
gonna very helpful if someone else can help with reviewing Common/HDFS
JIRAs.
Thanks,
Wangda
On Wed, May 11, 2016 at 4:20 PM, Sangjin Lee <sj...@apache.org> wrote:
> Where do we stand in terms of closing out blocker/critical issues for
> 2.8.0? I still see 50 open JIRAs in Vinod's list:
> https://issues.apache.org/jira/issues/?filter=12334985
>
> But I see a lot of JIRAs with no patches or very stale patches. It would be
> a good exercise to come up with the list of JIRAs that we need to block
> 2.8.0 for and focus our attention on closing them out. Thoughts?
>
> Thanks,
> Sangjin
>
> On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
> wrote:
>
> >
> > > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <vi...@apache.org>
> > wrote:
> > >
> > > We are not converging - there’s still 58 more. I need help from the
> > community in addressing / review 2.8.0 blockers. If folks can start with
> > reviewing Patch available tickets, that’ll be great.
> > >
> > >
> >
> >
> > I'm still doing the s3a stuff, other people testing and reviewing this
> > stuff welcome.
> >
> > in particular, I could do with others playing with this patch of mine,
> > which adds counters and things into S3a, based on the azure
> instrumentation
> >
> > https://issues.apache.org/jira/browse/HADOOP-13028
> >
> >
> >
>
Re: [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
Where do we stand in terms of closing out blocker/critical issues for
2.8.0? I still see 50 open JIRAs in Vinod's list:
https://issues.apache.org/jira/issues/?filter=12334985
But I see a lot of JIRAs with no patches or very stale patches. It would be
a good exercise to come up with the list of JIRAs that we need to block
2.8.0 for and focus our attention on closing them out. Thoughts?
Thanks,
Sangjin
On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <vi...@apache.org>
> wrote:
> >
> > We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start with
> reviewing Patch available tickets, that’ll be great.
> >
> >
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
Where do we stand in terms of closing out blocker/critical issues for
2.8.0? I still see 50 open JIRAs in Vinod's list:
https://issues.apache.org/jira/issues/?filter=12334985
But I see a lot of JIRAs with no patches or very stale patches. It would be
a good exercise to come up with the list of JIRAs that we need to block
2.8.0 for and focus our attention on closing them out. Thoughts?
Thanks,
Sangjin
On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <vi...@apache.org>
> wrote:
> >
> > We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start with
> reviewing Patch available tickets, that’ll be great.
> >
> >
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
Where do we stand in terms of closing out blocker/critical issues for
2.8.0? I still see 50 open JIRAs in Vinod's list:
https://issues.apache.org/jira/issues/?filter=12334985
But I see a lot of JIRAs with no patches or very stale patches. It would be
a good exercise to come up with the list of JIRAs that we need to block
2.8.0 for and focus our attention on closing them out. Thoughts?
Thanks,
Sangjin
On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <vi...@apache.org>
> wrote:
> >
> > We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start with
> reviewing Patch available tickets, that’ll be great.
> >
> >
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Sangjin Lee <sj...@apache.org>.
Where do we stand in terms of closing out blocker/critical issues for
2.8.0? I still see 50 open JIRAs in Vinod's list:
https://issues.apache.org/jira/issues/?filter=12334985
But I see a lot of JIRAs with no patches or very stale patches. It would be
a good exercise to come up with the list of JIRAs that we need to block
2.8.0 for and focus our attention on closing them out. Thoughts?
Thanks,
Sangjin
On Sat, Apr 23, 2016 at 5:05 AM, Steve Loughran <st...@hortonworks.com>
wrote:
>
> > On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <vi...@apache.org>
> wrote:
> >
> > We are not converging - there’s still 58 more. I need help from the
> community in addressing / review 2.8.0 blockers. If folks can start with
> reviewing Patch available tickets, that’ll be great.
> >
> >
>
>
> I'm still doing the s3a stuff, other people testing and reviewing this
> stuff welcome.
>
> in particular, I could do with others playing with this patch of mine,
> which adds counters and things into S3a, based on the azure instrumentation
>
> https://issues.apache.org/jira/browse/HADOOP-13028
>
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Steve Loughran <st...@hortonworks.com>.
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> We are not converging - there’s still 58 more. I need help from the community in addressing / review 2.8.0 blockers. If folks can start with reviewing Patch available tickets, that’ll be great.
>
>
I'm still doing the s3a stuff, other people testing and reviewing this stuff welcome.
in particular, I could do with others playing with this patch of mine, which adds counters and things into S3a, based on the azure instrumentation
https://issues.apache.org/jira/browse/HADOOP-13028
Re: [Release thread] 2.8.0 release activities
Posted by Steve Loughran <st...@hortonworks.com>.
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> We are not converging - there’s still 58 more. I need help from the community in addressing / review 2.8.0 blockers. If folks can start with reviewing Patch available tickets, that’ll be great.
>
>
I'm still doing the s3a stuff, other people testing and reviewing this stuff welcome.
in particular, I could do with others playing with this patch of mine, which adds counters and things into S3a, based on the azure instrumentation
https://issues.apache.org/jira/browse/HADOOP-13028
Re: [Release thread] 2.8.0 release activities
Posted by Steve Loughran <st...@hortonworks.com>.
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> We are not converging - there’s still 58 more. I need help from the community in addressing / review 2.8.0 blockers. If folks can start with reviewing Patch available tickets, that’ll be great.
>
>
I'm still doing the s3a stuff, other people testing and reviewing this stuff welcome.
in particular, I could do with others playing with this patch of mine, which adds counters and things into S3a, based on the azure instrumentation
https://issues.apache.org/jira/browse/HADOOP-13028
Re: [Release thread] 2.8.0 release activities
Posted by Steve Loughran <st...@hortonworks.com>.
> On 23 Apr 2016, at 01:24, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> We are not converging - there’s still 58 more. I need help from the community in addressing / review 2.8.0 blockers. If folks can start with reviewing Patch available tickets, that’ll be great.
>
>
I'm still doing the s3a stuff, other people testing and reviewing this stuff welcome.
in particular, I could do with others playing with this patch of mine, which adds counters and things into S3a, based on the azure instrumentation
https://issues.apache.org/jira/browse/HADOOP-13028
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
We are not converging - there’s still 58 more. I need help from the community in addressing / review 2.8.0 blockers. If folks can start with reviewing Patch available tickets, that’ll be great.
Thanks
+Vinod
> On Apr 4, 2016, at 2:16 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
>
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
>
> I’ll start pushing for progress for an RC in a week or two.
>
> Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
>
> Thanks
> +Vinod
>
>> On Feb 9, 2016, at 11:51 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
>>
>> Given that, if you can get them in before a week, we should be good.
>>
>> +Vinod
>>
>>> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>>>
>>> Vinod,
>>>
>>> Thanks for initiating the 2.8 release thread. We are in late review stages
>>> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
>>> ACLs for reservation system), hoping to get them by next week. Any chance
>>> you can put off cutting 2.8 by a week as we are planning to deploy
>>> ReservationSystem and these are critical for that?
>>>
>>> Cheers,
>>> Subru
>>>
>>> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
>>> wrote:
>>>
>>>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>>>> release.
>>>>
>>>> --Chris Nauroth
>>>>
>>>>
>>>>
>>>>
>>>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>>>
>>>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>>>
>>>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>>>
>>>>> wrote:
>>>>>
>>>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>>>
>>>>>> Regards,
>>>>>> Uma
>>>>>>
>>>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>>>> going
>>>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>>>> branch-2.8.
>>>>>>>
>>>>>>> Let me know if anyone has any objections/concerns.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> Branch-2.8 is created.
>>>>>>>>
>>>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>>>> fixes to existing features with a goal of converging on an alpha
>>>>>> release
>>>>>>>> soon.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> Forking threads now in order to track all things related to the
>>>>>>>>> release.
>>>>>>>>>
>>>>>>>>> Creating the branch now.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> +Vinod
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>>>> sent
>>>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>>>
>>>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>>>
>>>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>>>> to
>>>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>>>> and
>>>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> +Vinod
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>>>> early / alpha release in the interest of
>>>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>>>> ‹ wider testing,
>>>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>>>> incompatibilities
>>>>>>>>>>> introduced.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
We are not converging - there’s still 58 more. I need help from the community in addressing / review 2.8.0 blockers. If folks can start with reviewing Patch available tickets, that’ll be great.
Thanks
+Vinod
> On Apr 4, 2016, at 2:16 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
>
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
>
> I’ll start pushing for progress for an RC in a week or two.
>
> Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
>
> Thanks
> +Vinod
>
>> On Feb 9, 2016, at 11:51 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
>>
>> Given that, if you can get them in before a week, we should be good.
>>
>> +Vinod
>>
>>> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>>>
>>> Vinod,
>>>
>>> Thanks for initiating the 2.8 release thread. We are in late review stages
>>> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
>>> ACLs for reservation system), hoping to get them by next week. Any chance
>>> you can put off cutting 2.8 by a week as we are planning to deploy
>>> ReservationSystem and these are critical for that?
>>>
>>> Cheers,
>>> Subru
>>>
>>> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
>>> wrote:
>>>
>>>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>>>> release.
>>>>
>>>> --Chris Nauroth
>>>>
>>>>
>>>>
>>>>
>>>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>>>
>>>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>>>
>>>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>>>
>>>>> wrote:
>>>>>
>>>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>>>
>>>>>> Regards,
>>>>>> Uma
>>>>>>
>>>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>>>> going
>>>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>>>> branch-2.8.
>>>>>>>
>>>>>>> Let me know if anyone has any objections/concerns.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> Branch-2.8 is created.
>>>>>>>>
>>>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>>>> fixes to existing features with a goal of converging on an alpha
>>>>>> release
>>>>>>>> soon.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> Forking threads now in order to track all things related to the
>>>>>>>>> release.
>>>>>>>>>
>>>>>>>>> Creating the branch now.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> +Vinod
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>>>> sent
>>>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>>>
>>>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>>>
>>>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>>>> to
>>>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>>>> and
>>>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> +Vinod
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>>>> early / alpha release in the interest of
>>>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>>>> ‹ wider testing,
>>>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>>>> incompatibilities
>>>>>>>>>>> introduced.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely
busted on branch-2, I figured I have to backport a whole bunch of
patches that are only on trunk, and may be more fixes on top of that *sigh*
To fix this failure, HADOOP-12022 should be backported to branch-2 and
branch-2.8. I reopened the issue and rebased the patch.
Thanks,
Akira
On 4/5/16 06:16, Vinod Kumar Vavilapalli wrote:
> Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
>
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
>
> I’ll start pushing for progress for an RC in a week or two.
>
> Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
>
> Thanks
> +Vinod
>
>> On Feb 9, 2016, at 11:51 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
>>
>> Given that, if you can get them in before a week, we should be good.
>>
>> +Vinod
>>
>>> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>>>
>>> Vinod,
>>>
>>> Thanks for initiating the 2.8 release thread. We are in late review stages
>>> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
>>> ACLs for reservation system), hoping to get them by next week. Any chance
>>> you can put off cutting 2.8 by a week as we are planning to deploy
>>> ReservationSystem and these are critical for that?
>>>
>>> Cheers,
>>> Subru
>>>
>>> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
>>> wrote:
>>>
>>>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>>>> release.
>>>>
>>>> --Chris Nauroth
>>>>
>>>>
>>>>
>>>>
>>>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>>>
>>>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>>>
>>>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>>>
>>>>> wrote:
>>>>>
>>>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>>>
>>>>>> Regards,
>>>>>> Uma
>>>>>>
>>>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>>>> going
>>>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>>>> branch-2.8.
>>>>>>>
>>>>>>> Let me know if anyone has any objections/concerns.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> Branch-2.8 is created.
>>>>>>>>
>>>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>>>> fixes to existing features with a goal of converging on an alpha
>>>>>> release
>>>>>>>> soon.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> Forking threads now in order to track all things related to the
>>>>>>>>> release.
>>>>>>>>>
>>>>>>>>> Creating the branch now.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> +Vinod
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>>>> sent
>>>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>>>
>>>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>>>
>>>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>>>> to
>>>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>>>> and
>>>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> +Vinod
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>>>> early / alpha release in the interest of
>>>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>>>> ‹ wider testing,
>>>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>>>> incompatibilities
>>>>>>>>>>> introduced.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely
busted on branch-2, I figured I have to backport a whole bunch of
patches that are only on trunk, and may be more fixes on top of that *sigh*
To fix this failure, HADOOP-12022 should be backported to branch-2 and
branch-2.8. I reopened the issue and rebased the patch.
Thanks,
Akira
On 4/5/16 06:16, Vinod Kumar Vavilapalli wrote:
> Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
>
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
>
> I’ll start pushing for progress for an RC in a week or two.
>
> Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
>
> Thanks
> +Vinod
>
>> On Feb 9, 2016, at 11:51 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
>>
>> Given that, if you can get them in before a week, we should be good.
>>
>> +Vinod
>>
>>> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>>>
>>> Vinod,
>>>
>>> Thanks for initiating the 2.8 release thread. We are in late review stages
>>> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
>>> ACLs for reservation system), hoping to get them by next week. Any chance
>>> you can put off cutting 2.8 by a week as we are planning to deploy
>>> ReservationSystem and these are critical for that?
>>>
>>> Cheers,
>>> Subru
>>>
>>> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
>>> wrote:
>>>
>>>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>>>> release.
>>>>
>>>> --Chris Nauroth
>>>>
>>>>
>>>>
>>>>
>>>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>>>
>>>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>>>
>>>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>>>
>>>>> wrote:
>>>>>
>>>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>>>
>>>>>> Regards,
>>>>>> Uma
>>>>>>
>>>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>>>> going
>>>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>>>> branch-2.8.
>>>>>>>
>>>>>>> Let me know if anyone has any objections/concerns.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> Branch-2.8 is created.
>>>>>>>>
>>>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>>>> fixes to existing features with a goal of converging on an alpha
>>>>>> release
>>>>>>>> soon.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> Forking threads now in order to track all things related to the
>>>>>>>>> release.
>>>>>>>>>
>>>>>>>>> Creating the branch now.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> +Vinod
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>>>> sent
>>>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>>>
>>>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>>>
>>>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>>>> to
>>>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>>>> and
>>>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> +Vinod
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>>>> early / alpha release in the interest of
>>>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>>>> ‹ wider testing,
>>>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>>>> incompatibilities
>>>>>>>>>>> introduced.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Steve Loughran <st...@hortonworks.com>.
> On 4 Apr 2016, at 22:16, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
>
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
>
> I’ll start pushing for progress for an RC in a week or two.
>
> Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
>
I'm trying to get all the s3a stuff in, hampered by the fact that (a) it's spare time and (b) it now takes 1-2h for all the tests to complete, at least from my house -> AWS ireland.
Anyone else who can help test/review those s3a patches in HADOOP-11694 is very welcome, especially if you test against: different endpoints, different s3 implementations, use different login mechanims
-Steve
Re: [Release thread] 2.8.0 release activities
Posted by Steve Loughran <st...@hortonworks.com>.
> On 4 Apr 2016, at 22:16, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
>
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
>
> I’ll start pushing for progress for an RC in a week or two.
>
> Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
>
I'm trying to get all the s3a stuff in, hampered by the fact that (a) it's spare time and (b) it now takes 1-2h for all the tests to complete, at least from my house -> AWS ireland.
Anyone else who can help test/review those s3a patches in HADOOP-11694 is very welcome, especially if you test against: different endpoints, different s3 implementations, use different login mechanims
-Steve
Re: [Release thread] 2.8.0 release activities
Posted by Steve Loughran <st...@hortonworks.com>.
> On 4 Apr 2016, at 22:16, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
>
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
>
> I’ll start pushing for progress for an RC in a week or two.
>
> Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
>
I'm trying to get all the s3a stuff in, hampered by the fact that (a) it's spare time and (b) it now takes 1-2h for all the tests to complete, at least from my house -> AWS ireland.
Anyone else who can help test/review those s3a patches in HADOOP-11694 is very welcome, especially if you test against: different endpoints, different s3 implementations, use different login mechanims
-Steve
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
We are not converging - there’s still 58 more. I need help from the community in addressing / review 2.8.0 blockers. If folks can start with reviewing Patch available tickets, that’ll be great.
Thanks
+Vinod
> On Apr 4, 2016, at 2:16 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
>
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
>
> I’ll start pushing for progress for an RC in a week or two.
>
> Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
>
> Thanks
> +Vinod
>
>> On Feb 9, 2016, at 11:51 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
>>
>> Given that, if you can get them in before a week, we should be good.
>>
>> +Vinod
>>
>>> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>>>
>>> Vinod,
>>>
>>> Thanks for initiating the 2.8 release thread. We are in late review stages
>>> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
>>> ACLs for reservation system), hoping to get them by next week. Any chance
>>> you can put off cutting 2.8 by a week as we are planning to deploy
>>> ReservationSystem and these are critical for that?
>>>
>>> Cheers,
>>> Subru
>>>
>>> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
>>> wrote:
>>>
>>>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>>>> release.
>>>>
>>>> --Chris Nauroth
>>>>
>>>>
>>>>
>>>>
>>>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>>>
>>>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>>>
>>>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>>>
>>>>> wrote:
>>>>>
>>>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>>>
>>>>>> Regards,
>>>>>> Uma
>>>>>>
>>>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>>>> going
>>>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>>>> branch-2.8.
>>>>>>>
>>>>>>> Let me know if anyone has any objections/concerns.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> Branch-2.8 is created.
>>>>>>>>
>>>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>>>> fixes to existing features with a goal of converging on an alpha
>>>>>> release
>>>>>>>> soon.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> Forking threads now in order to track all things related to the
>>>>>>>>> release.
>>>>>>>>>
>>>>>>>>> Creating the branch now.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> +Vinod
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>>>> sent
>>>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>>>
>>>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>>>
>>>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>>>> to
>>>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>>>> and
>>>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> +Vinod
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>>>> early / alpha release in the interest of
>>>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>>>> ‹ wider testing,
>>>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>>>> incompatibilities
>>>>>>>>>>> introduced.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Steve Loughran <st...@hortonworks.com>.
> On 4 Apr 2016, at 22:16, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
>
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
>
> I’ll start pushing for progress for an RC in a week or two.
>
> Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
>
I'm trying to get all the s3a stuff in, hampered by the fact that (a) it's spare time and (b) it now takes 1-2h for all the tests to complete, at least from my house -> AWS ireland.
Anyone else who can help test/review those s3a patches in HADOOP-11694 is very welcome, especially if you test against: different endpoints, different s3 implementations, use different login mechanims
-Steve
Re: [Release thread] 2.8.0 release activities
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely
busted on branch-2, I figured I have to backport a whole bunch of
patches that are only on trunk, and may be more fixes on top of that *sigh*
To fix this failure, HADOOP-12022 should be backported to branch-2 and
branch-2.8. I reopened the issue and rebased the patch.
Thanks,
Akira
On 4/5/16 06:16, Vinod Kumar Vavilapalli wrote:
> Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
>
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
>
> I’ll start pushing for progress for an RC in a week or two.
>
> Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
>
> Thanks
> +Vinod
>
>> On Feb 9, 2016, at 11:51 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
>>
>> Given that, if you can get them in before a week, we should be good.
>>
>> +Vinod
>>
>>> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>>>
>>> Vinod,
>>>
>>> Thanks for initiating the 2.8 release thread. We are in late review stages
>>> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
>>> ACLs for reservation system), hoping to get them by next week. Any chance
>>> you can put off cutting 2.8 by a week as we are planning to deploy
>>> ReservationSystem and these are critical for that?
>>>
>>> Cheers,
>>> Subru
>>>
>>> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
>>> wrote:
>>>
>>>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>>>> release.
>>>>
>>>> --Chris Nauroth
>>>>
>>>>
>>>>
>>>>
>>>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>>>
>>>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>>>
>>>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>>>
>>>>> wrote:
>>>>>
>>>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>>>
>>>>>> Regards,
>>>>>> Uma
>>>>>>
>>>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>>>> going
>>>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>>>> branch-2.8.
>>>>>>>
>>>>>>> Let me know if anyone has any objections/concerns.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> Branch-2.8 is created.
>>>>>>>>
>>>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>>>> fixes to existing features with a goal of converging on an alpha
>>>>>> release
>>>>>>>> soon.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> Forking threads now in order to track all things related to the
>>>>>>>>> release.
>>>>>>>>>
>>>>>>>>> Creating the branch now.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> +Vinod
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>>>> sent
>>>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>>>
>>>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>>>
>>>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>>>> to
>>>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>>>> and
>>>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> +Vinod
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>>>> early / alpha release in the interest of
>>>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>>>> ‹ wider testing,
>>>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>>>> incompatibilities
>>>>>>>>>>> introduced.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
We are not converging - there’s still 58 more. I need help from the community in addressing / review 2.8.0 blockers. If folks can start with reviewing Patch available tickets, that’ll be great.
Thanks
+Vinod
> On Apr 4, 2016, at 2:16 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
>
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
>
> I’ll start pushing for progress for an RC in a week or two.
>
> Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
>
> Thanks
> +Vinod
>
>> On Feb 9, 2016, at 11:51 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
>>
>> Given that, if you can get them in before a week, we should be good.
>>
>> +Vinod
>>
>>> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>>>
>>> Vinod,
>>>
>>> Thanks for initiating the 2.8 release thread. We are in late review stages
>>> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
>>> ACLs for reservation system), hoping to get them by next week. Any chance
>>> you can put off cutting 2.8 by a week as we are planning to deploy
>>> ReservationSystem and these are critical for that?
>>>
>>> Cheers,
>>> Subru
>>>
>>> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
>>> wrote:
>>>
>>>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>>>> release.
>>>>
>>>> --Chris Nauroth
>>>>
>>>>
>>>>
>>>>
>>>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>>>
>>>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>>>
>>>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>>>
>>>>> wrote:
>>>>>
>>>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>>>
>>>>>> Regards,
>>>>>> Uma
>>>>>>
>>>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>>>> going
>>>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>>>> branch-2.8.
>>>>>>>
>>>>>>> Let me know if anyone has any objections/concerns.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> Branch-2.8 is created.
>>>>>>>>
>>>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>>>> fixes to existing features with a goal of converging on an alpha
>>>>>> release
>>>>>>>> soon.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> Forking threads now in order to track all things related to the
>>>>>>>>> release.
>>>>>>>>>
>>>>>>>>> Creating the branch now.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> +Vinod
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>>>> sent
>>>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>>>
>>>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>>>
>>>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>>>> to
>>>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>>>> and
>>>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> +Vinod
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>>>> early / alpha release in the interest of
>>>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>>>> ‹ wider testing,
>>>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>>>> incompatibilities
>>>>>>>>>>> introduced.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Akira AJISAKA <aj...@oss.nttdata.co.jp>.
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely
busted on branch-2, I figured I have to backport a whole bunch of
patches that are only on trunk, and may be more fixes on top of that *sigh*
To fix this failure, HADOOP-12022 should be backported to branch-2 and
branch-2.8. I reopened the issue and rebased the patch.
Thanks,
Akira
On 4/5/16 06:16, Vinod Kumar Vavilapalli wrote:
> Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
>
> Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
>
> I’ll start pushing for progress for an RC in a week or two.
>
> Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
>
> Thanks
> +Vinod
>
>> On Feb 9, 2016, at 11:51 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
>>
>> Given that, if you can get them in before a week, we should be good.
>>
>> +Vinod
>>
>>> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>>>
>>> Vinod,
>>>
>>> Thanks for initiating the 2.8 release thread. We are in late review stages
>>> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
>>> ACLs for reservation system), hoping to get them by next week. Any chance
>>> you can put off cutting 2.8 by a week as we are planning to deploy
>>> ReservationSystem and these are critical for that?
>>>
>>> Cheers,
>>> Subru
>>>
>>> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
>>> wrote:
>>>
>>>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>>>> release.
>>>>
>>>> --Chris Nauroth
>>>>
>>>>
>>>>
>>>>
>>>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>>>
>>>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>>>
>>>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>>>
>>>>> wrote:
>>>>>
>>>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>>>
>>>>>> Regards,
>>>>>> Uma
>>>>>>
>>>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>>>> going
>>>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>>>> branch-2.8.
>>>>>>>
>>>>>>> Let me know if anyone has any objections/concerns.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> Branch-2.8 is created.
>>>>>>>>
>>>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>>>> fixes to existing features with a goal of converging on an alpha
>>>>>> release
>>>>>>>> soon.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> Forking threads now in order to track all things related to the
>>>>>>>>> release.
>>>>>>>>>
>>>>>>>>> Creating the branch now.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> +Vinod
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>>
>>>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>>>> sent
>>>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>>>
>>>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>>>
>>>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>>>> to
>>>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>>>> and
>>>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>> +Vinod
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>>>> early / alpha release in the interest of
>>>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>>>> ‹ wider testing,
>>>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>>>> incompatibilities
>>>>>>>>>>> introduced.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
I’ll start pushing for progress for an RC in a week or two.
Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
Thanks
+Vinod
> On Feb 9, 2016, at 11:51 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
>
> Given that, if you can get them in before a week, we should be good.
>
> +Vinod
>
>> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>>
>> Vinod,
>>
>> Thanks for initiating the 2.8 release thread. We are in late review stages
>> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
>> ACLs for reservation system), hoping to get them by next week. Any chance
>> you can put off cutting 2.8 by a week as we are planning to deploy
>> ReservationSystem and these are critical for that?
>>
>> Cheers,
>> Subru
>>
>> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
>> wrote:
>>
>>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>>> release.
>>>
>>> --Chris Nauroth
>>>
>>>
>>>
>>>
>>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>>
>>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>>
>>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>>
>>>> wrote:
>>>>
>>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>>
>>>>> Regards,
>>>>> Uma
>>>>>
>>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>>> going
>>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>>> branch-2.8.
>>>>>>
>>>>>> Let me know if anyone has any objections/concerns.
>>>>>>
>>>>>> Thanks
>>>>>> +Vinod
>>>>>>
>>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>>> <vi...@apache.org> wrote:
>>>>>>>
>>>>>>> Branch-2.8 is created.
>>>>>>>
>>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>>> fixes to existing features with a goal of converging on an alpha
>>>>> release
>>>>>>> soon.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> Forking threads now in order to track all things related to the
>>>>>>>> release.
>>>>>>>>
>>>>>>>> Creating the branch now.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>>> sent
>>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>>
>>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>>
>>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>>> to
>>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>>> and
>>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> +Vinod
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>>
>>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>>> early / alpha release in the interest of
>>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>>> ‹ wider testing,
>>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>>> incompatibilities
>>>>>>>>>> introduced.
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
I’ll start pushing for progress for an RC in a week or two.
Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
Thanks
+Vinod
> On Feb 9, 2016, at 11:51 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
>
> Given that, if you can get them in before a week, we should be good.
>
> +Vinod
>
>> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>>
>> Vinod,
>>
>> Thanks for initiating the 2.8 release thread. We are in late review stages
>> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
>> ACLs for reservation system), hoping to get them by next week. Any chance
>> you can put off cutting 2.8 by a week as we are planning to deploy
>> ReservationSystem and these are critical for that?
>>
>> Cheers,
>> Subru
>>
>> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
>> wrote:
>>
>>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>>> release.
>>>
>>> --Chris Nauroth
>>>
>>>
>>>
>>>
>>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>>
>>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>>
>>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>>
>>>> wrote:
>>>>
>>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>>
>>>>> Regards,
>>>>> Uma
>>>>>
>>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>>> going
>>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>>> branch-2.8.
>>>>>>
>>>>>> Let me know if anyone has any objections/concerns.
>>>>>>
>>>>>> Thanks
>>>>>> +Vinod
>>>>>>
>>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>>> <vi...@apache.org> wrote:
>>>>>>>
>>>>>>> Branch-2.8 is created.
>>>>>>>
>>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>>> fixes to existing features with a goal of converging on an alpha
>>>>> release
>>>>>>> soon.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> Forking threads now in order to track all things related to the
>>>>>>>> release.
>>>>>>>>
>>>>>>>> Creating the branch now.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>>> sent
>>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>>
>>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>>
>>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>>> to
>>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>>> and
>>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> +Vinod
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>>
>>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>>> early / alpha release in the interest of
>>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>>> ‹ wider testing,
>>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>>> incompatibilities
>>>>>>>>>> introduced.
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
I’ll start pushing for progress for an RC in a week or two.
Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
Thanks
+Vinod
> On Feb 9, 2016, at 11:51 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
>
> Given that, if you can get them in before a week, we should be good.
>
> +Vinod
>
>> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>>
>> Vinod,
>>
>> Thanks for initiating the 2.8 release thread. We are in late review stages
>> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
>> ACLs for reservation system), hoping to get them by next week. Any chance
>> you can put off cutting 2.8 by a week as we are planning to deploy
>> ReservationSystem and these are critical for that?
>>
>> Cheers,
>> Subru
>>
>> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
>> wrote:
>>
>>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>>> release.
>>>
>>> --Chris Nauroth
>>>
>>>
>>>
>>>
>>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>>
>>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>>
>>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>>
>>>> wrote:
>>>>
>>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>>
>>>>> Regards,
>>>>> Uma
>>>>>
>>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>>> going
>>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>>> branch-2.8.
>>>>>>
>>>>>> Let me know if anyone has any objections/concerns.
>>>>>>
>>>>>> Thanks
>>>>>> +Vinod
>>>>>>
>>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>>> <vi...@apache.org> wrote:
>>>>>>>
>>>>>>> Branch-2.8 is created.
>>>>>>>
>>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>>> fixes to existing features with a goal of converging on an alpha
>>>>> release
>>>>>>> soon.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> Forking threads now in order to track all things related to the
>>>>>>>> release.
>>>>>>>>
>>>>>>>> Creating the branch now.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>>> sent
>>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>>
>>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>>
>>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>>> to
>>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>>> and
>>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> +Vinod
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>>
>>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>>> early / alpha release in the interest of
>>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>>> ‹ wider testing,
>>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>>> incompatibilities
>>>>>>>>>> introduced.
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Here we go again. The blocker / critical tickets ballooned up a lot, I see 64 now! : https://issues.apache.org/jira/issues/?filter=12334985
Also, the docs target (mvn package -Pdocs -DskipTests) is completely busted on branch-2, I figured I have to backport a whole bunch of patches that are only on trunk, and may be more fixes on top of that *sigh*
I’ll start pushing for progress for an RC in a week or two.
Any help towards this, reviewing/committing outstanding patches and contributing to open items is greatly appreciated.
Thanks
+Vinod
> On Feb 9, 2016, at 11:51 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
>
> Given that, if you can get them in before a week, we should be good.
>
> +Vinod
>
>> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>>
>> Vinod,
>>
>> Thanks for initiating the 2.8 release thread. We are in late review stages
>> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
>> ACLs for reservation system), hoping to get them by next week. Any chance
>> you can put off cutting 2.8 by a week as we are planning to deploy
>> ReservationSystem and these are critical for that?
>>
>> Cheers,
>> Subru
>>
>> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
>> wrote:
>>
>>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>>> release.
>>>
>>> --Chris Nauroth
>>>
>>>
>>>
>>>
>>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>>
>>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>>
>>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>>
>>>> wrote:
>>>>
>>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>>
>>>>> Regards,
>>>>> Uma
>>>>>
>>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>>> going
>>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>>> branch-2.8.
>>>>>>
>>>>>> Let me know if anyone has any objections/concerns.
>>>>>>
>>>>>> Thanks
>>>>>> +Vinod
>>>>>>
>>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>>> <vi...@apache.org> wrote:
>>>>>>>
>>>>>>> Branch-2.8 is created.
>>>>>>>
>>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>>> fixes to existing features with a goal of converging on an alpha
>>>>> release
>>>>>>> soon.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> Forking threads now in order to track all things related to the
>>>>>>>> release.
>>>>>>>>
>>>>>>>> Creating the branch now.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>>> sent
>>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>>
>>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>>
>>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>>> to
>>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>>> and
>>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>> +Vinod
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>>
>>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>>> early / alpha release in the interest of
>>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>>> ‹ wider testing,
>>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>>> incompatibilities
>>>>>>>>>> introduced.
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>
>>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
Given that, if you can get them in before a week, we should be good.
+Vinod
> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>
> Vinod,
>
> Thanks for initiating the 2.8 release thread. We are in late review stages
> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
> ACLs for reservation system), hoping to get them by next week. Any chance
> you can put off cutting 2.8 by a week as we are planning to deploy
> ReservationSystem and these are critical for that?
>
> Cheers,
> Subru
>
> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
> wrote:
>
>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>> release.
>>
>> --Chris Nauroth
>>
>>
>>
>>
>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>
>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>
>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>
>>> wrote:
>>>
>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>
>>>> Regards,
>>>> Uma
>>>>
>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>> wrote:
>>>>
>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>> going
>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>> branch-2.8.
>>>>>
>>>>> Let me know if anyone has any objections/concerns.
>>>>>
>>>>> Thanks
>>>>> +Vinod
>>>>>
>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>> <vi...@apache.org> wrote:
>>>>>>
>>>>>> Branch-2.8 is created.
>>>>>>
>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>> fixes to existing features with a goal of converging on an alpha
>>>> release
>>>>>> soon.
>>>>>>
>>>>>> Thanks
>>>>>> +Vinod
>>>>>>
>>>>>>
>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>> <vi...@apache.org> wrote:
>>>>>>>
>>>>>>> Forking threads now in order to track all things related to the
>>>>>>> release.
>>>>>>>
>>>>>>> Creating the branch now.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>> sent
>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>
>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>
>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>> to
>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>> and
>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>
>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>> early / alpha release in the interest of
>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>> ‹ wider testing,
>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>> incompatibilities
>>>>>>>>> introduced.
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>
>>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
Given that, if you can get them in before a week, we should be good.
+Vinod
> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>
> Vinod,
>
> Thanks for initiating the 2.8 release thread. We are in late review stages
> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
> ACLs for reservation system), hoping to get them by next week. Any chance
> you can put off cutting 2.8 by a week as we are planning to deploy
> ReservationSystem and these are critical for that?
>
> Cheers,
> Subru
>
> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
> wrote:
>
>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>> release.
>>
>> --Chris Nauroth
>>
>>
>>
>>
>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>
>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>
>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>
>>> wrote:
>>>
>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>
>>>> Regards,
>>>> Uma
>>>>
>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>> wrote:
>>>>
>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>> going
>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>> branch-2.8.
>>>>>
>>>>> Let me know if anyone has any objections/concerns.
>>>>>
>>>>> Thanks
>>>>> +Vinod
>>>>>
>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>> <vi...@apache.org> wrote:
>>>>>>
>>>>>> Branch-2.8 is created.
>>>>>>
>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>> fixes to existing features with a goal of converging on an alpha
>>>> release
>>>>>> soon.
>>>>>>
>>>>>> Thanks
>>>>>> +Vinod
>>>>>>
>>>>>>
>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>> <vi...@apache.org> wrote:
>>>>>>>
>>>>>>> Forking threads now in order to track all things related to the
>>>>>>> release.
>>>>>>>
>>>>>>> Creating the branch now.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>> sent
>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>
>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>
>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>> to
>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>> and
>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>
>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>> early / alpha release in the interest of
>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>> ‹ wider testing,
>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>> incompatibilities
>>>>>>>>> introduced.
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>
>>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
Given that, if you can get them in before a week, we should be good.
+Vinod
> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>
> Vinod,
>
> Thanks for initiating the 2.8 release thread. We are in late review stages
> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
> ACLs for reservation system), hoping to get them by next week. Any chance
> you can put off cutting 2.8 by a week as we are planning to deploy
> ReservationSystem and these are critical for that?
>
> Cheers,
> Subru
>
> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
> wrote:
>
>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>> release.
>>
>> --Chris Nauroth
>>
>>
>>
>>
>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>
>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>
>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>
>>> wrote:
>>>
>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>
>>>> Regards,
>>>> Uma
>>>>
>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>> wrote:
>>>>
>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>> going
>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>> branch-2.8.
>>>>>
>>>>> Let me know if anyone has any objections/concerns.
>>>>>
>>>>> Thanks
>>>>> +Vinod
>>>>>
>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>> <vi...@apache.org> wrote:
>>>>>>
>>>>>> Branch-2.8 is created.
>>>>>>
>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>> fixes to existing features with a goal of converging on an alpha
>>>> release
>>>>>> soon.
>>>>>>
>>>>>> Thanks
>>>>>> +Vinod
>>>>>>
>>>>>>
>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>> <vi...@apache.org> wrote:
>>>>>>>
>>>>>>> Forking threads now in order to track all things related to the
>>>>>>> release.
>>>>>>>
>>>>>>> Creating the branch now.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>> sent
>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>
>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>
>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>> to
>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>> and
>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>
>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>> early / alpha release in the interest of
>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>> ‹ wider testing,
>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>> incompatibilities
>>>>>>>>> introduced.
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>
>>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Sure. The last time I checked, there were 20 odd blocker/critical tickets too that’ll need some of my time.
Given that, if you can get them in before a week, we should be good.
+Vinod
> On Feb 5, 2016, at 1:19 PM, Subramaniam V K <su...@gmail.com> wrote:
>
> Vinod,
>
> Thanks for initiating the 2.8 release thread. We are in late review stages
> for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
> ACLs for reservation system), hoping to get them by next week. Any chance
> you can put off cutting 2.8 by a week as we are planning to deploy
> ReservationSystem and these are critical for that?
>
> Cheers,
> Subru
>
> On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
> wrote:
>
>> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
>> release.
>>
>> --Chris Nauroth
>>
>>
>>
>>
>> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>>
>>> Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>>> Let us not call it alpha or beta though, it is quite confusing. :)
>>>
>>> On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
>>>
>>> wrote:
>>>
>>>> Thanks Vinod. +1 for 2.8 release start.
>>>>
>>>> Regards,
>>>> Uma
>>>>
>>>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>>> wrote:
>>>>
>>>>> Seems like all the features listed in the Roadmap wiki are in. I¹m
>>>> going
>>>>> to try cutting an RC this weekend for a first/non-stable release off of
>>>>> branch-2.8.
>>>>>
>>>>> Let me know if anyone has any objections/concerns.
>>>>>
>>>>> Thanks
>>>>> +Vinod
>>>>>
>>>>>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>>>>>> <vi...@apache.org> wrote:
>>>>>>
>>>>>> Branch-2.8 is created.
>>>>>>
>>>>>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>>>>> fixes to existing features with a goal of converging on an alpha
>>>> release
>>>>>> soon.
>>>>>>
>>>>>> Thanks
>>>>>> +Vinod
>>>>>>
>>>>>>
>>>>>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>>>>>> <vi...@apache.org> wrote:
>>>>>>>
>>>>>>> Forking threads now in order to track all things related to the
>>>>>>> release.
>>>>>>>
>>>>>>> Creating the branch now.
>>>>>>>
>>>>>>> Thanks
>>>>>>> +Vinod
>>>>>>>
>>>>>>>
>>>>>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>>>>>> <vi...@apache.org> wrote:
>>>>>>>>
>>>>>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>>> sent
>>>>>>>> out an email, I updated the Roadmap wiki reflecting the same:
>>>>>>>> https://wiki.apache.org/hadoop/Roadmap
>>>>>>>> <https://wiki.apache.org/hadoop/Roadmap>
>>>>>>>>
>>>>>>>> I plan to create a 2.8 branch EOD today.
>>>>>>>>
>>>>>>>> The goal for all of us should be to restrict improvements & fixes
>>>> to
>>>>>>>> only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>>> and
>>>>>>>> (b) other minor features that are already in 2.8.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> +Vinod
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>>>>>> <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>>>>>
>>>>>>>>> - Cut a branch about two weeks from now
>>>>>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>>>>> early / alpha release in the interest of
>>>>>>>>> ‹ gaining downstream adoption
>>>>>>>>> ‹ wider testing,
>>>>>>>>> ‹ yet reserving our right to fix any inadvertent
>>>> incompatibilities
>>>>>>>>> introduced.
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>
>>
Re: [Release thread] 2.8.0 release activities
Posted by Subramaniam V K <su...@gmail.com>.
Vinod,
Thanks for initiating the 2.8 release thread. We are in late review stages
for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
ACLs for reservation system), hoping to get them by next week. Any chance
you can put off cutting 2.8 by a week as we are planning to deploy
ReservationSystem and these are critical for that?
Cheers,
Subru
On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
> release.
>
> --Chris Nauroth
>
>
>
>
> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>
> >Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
> >Let us not call it alpha or beta though, it is quite confusing. :)
> >
> >On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
> >
> >wrote:
> >
> >> Thanks Vinod. +1 for 2.8 release start.
> >>
> >> Regards,
> >> Uma
> >>
> >> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
> >>wrote:
> >>
> >> >Seems like all the features listed in the Roadmap wiki are in. I¹m
> >>going
> >> >to try cutting an RC this weekend for a first/non-stable release off of
> >> >branch-2.8.
> >> >
> >> >Let me know if anyone has any objections/concerns.
> >> >
> >> >Thanks
> >> >+Vinod
> >> >
> >> >> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
> >> >><vi...@apache.org> wrote:
> >> >>
> >> >> Branch-2.8 is created.
> >> >>
> >> >> As mentioned before, the goal on branch-2.8 is to put improvements /
> >> >>fixes to existing features with a goal of converging on an alpha
> >>release
> >> >>soon.
> >> >>
> >> >> Thanks
> >> >> +Vinod
> >> >>
> >> >>
> >> >>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
> >> >>><vi...@apache.org> wrote:
> >> >>>
> >> >>> Forking threads now in order to track all things related to the
> >> >>>release.
> >> >>>
> >> >>> Creating the branch now.
> >> >>>
> >> >>> Thanks
> >> >>> +Vinod
> >> >>>
> >> >>>
> >> >>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
> >> >>>><vi...@apache.org> wrote:
> >> >>>>
> >> >>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
> >>sent
> >> >>>>out an email, I updated the Roadmap wiki reflecting the same:
> >> >>>>https://wiki.apache.org/hadoop/Roadmap
> >> >>>><https://wiki.apache.org/hadoop/Roadmap>
> >> >>>>
> >> >>>> I plan to create a 2.8 branch EOD today.
> >> >>>>
> >> >>>> The goal for all of us should be to restrict improvements & fixes
> >>to
> >> >>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki
> >>and
> >> >>>>(b) other minor features that are already in 2.8.
> >> >>>>
> >> >>>> Thanks
> >> >>>> +Vinod
> >> >>>>
> >> >>>>
> >> >>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
> >> >>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
> >> >>>>>
> >> >>>>> - Cut a branch about two weeks from now
> >> >>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> >> >>>>> - As with 2.7.x series, the first release will still be called as
> >> >>>>>early / alpha release in the interest of
> >> >>>>> ‹ gaining downstream adoption
> >> >>>>> ‹ wider testing,
> >> >>>>> ‹ yet reserving our right to fix any inadvertent
> >>incompatibilities
> >> >>>>>introduced.
> >> >>>>
> >> >>>
> >> >>
> >> >
> >>
> >>
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Subramaniam V K <su...@gmail.com>.
Vinod,
Thanks for initiating the 2.8 release thread. We are in late review stages
for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
ACLs for reservation system), hoping to get them by next week. Any chance
you can put off cutting 2.8 by a week as we are planning to deploy
ReservationSystem and these are critical for that?
Cheers,
Subru
On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
> release.
>
> --Chris Nauroth
>
>
>
>
> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>
> >Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
> >Let us not call it alpha or beta though, it is quite confusing. :)
> >
> >On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
> >
> >wrote:
> >
> >> Thanks Vinod. +1 for 2.8 release start.
> >>
> >> Regards,
> >> Uma
> >>
> >> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
> >>wrote:
> >>
> >> >Seems like all the features listed in the Roadmap wiki are in. I¹m
> >>going
> >> >to try cutting an RC this weekend for a first/non-stable release off of
> >> >branch-2.8.
> >> >
> >> >Let me know if anyone has any objections/concerns.
> >> >
> >> >Thanks
> >> >+Vinod
> >> >
> >> >> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
> >> >><vi...@apache.org> wrote:
> >> >>
> >> >> Branch-2.8 is created.
> >> >>
> >> >> As mentioned before, the goal on branch-2.8 is to put improvements /
> >> >>fixes to existing features with a goal of converging on an alpha
> >>release
> >> >>soon.
> >> >>
> >> >> Thanks
> >> >> +Vinod
> >> >>
> >> >>
> >> >>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
> >> >>><vi...@apache.org> wrote:
> >> >>>
> >> >>> Forking threads now in order to track all things related to the
> >> >>>release.
> >> >>>
> >> >>> Creating the branch now.
> >> >>>
> >> >>> Thanks
> >> >>> +Vinod
> >> >>>
> >> >>>
> >> >>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
> >> >>>><vi...@apache.org> wrote:
> >> >>>>
> >> >>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
> >>sent
> >> >>>>out an email, I updated the Roadmap wiki reflecting the same:
> >> >>>>https://wiki.apache.org/hadoop/Roadmap
> >> >>>><https://wiki.apache.org/hadoop/Roadmap>
> >> >>>>
> >> >>>> I plan to create a 2.8 branch EOD today.
> >> >>>>
> >> >>>> The goal for all of us should be to restrict improvements & fixes
> >>to
> >> >>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki
> >>and
> >> >>>>(b) other minor features that are already in 2.8.
> >> >>>>
> >> >>>> Thanks
> >> >>>> +Vinod
> >> >>>>
> >> >>>>
> >> >>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
> >> >>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
> >> >>>>>
> >> >>>>> - Cut a branch about two weeks from now
> >> >>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> >> >>>>> - As with 2.7.x series, the first release will still be called as
> >> >>>>>early / alpha release in the interest of
> >> >>>>> ‹ gaining downstream adoption
> >> >>>>> ‹ wider testing,
> >> >>>>> ‹ yet reserving our right to fix any inadvertent
> >>incompatibilities
> >> >>>>>introduced.
> >> >>>>
> >> >>>
> >> >>
> >> >
> >>
> >>
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Subramaniam V K <su...@gmail.com>.
Vinod,
Thanks for initiating the 2.8 release thread. We are in late review stages
for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
ACLs for reservation system), hoping to get them by next week. Any chance
you can put off cutting 2.8 by a week as we are planning to deploy
ReservationSystem and these are critical for that?
Cheers,
Subru
On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
> release.
>
> --Chris Nauroth
>
>
>
>
> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>
> >Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
> >Let us not call it alpha or beta though, it is quite confusing. :)
> >
> >On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
> >
> >wrote:
> >
> >> Thanks Vinod. +1 for 2.8 release start.
> >>
> >> Regards,
> >> Uma
> >>
> >> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
> >>wrote:
> >>
> >> >Seems like all the features listed in the Roadmap wiki are in. I¹m
> >>going
> >> >to try cutting an RC this weekend for a first/non-stable release off of
> >> >branch-2.8.
> >> >
> >> >Let me know if anyone has any objections/concerns.
> >> >
> >> >Thanks
> >> >+Vinod
> >> >
> >> >> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
> >> >><vi...@apache.org> wrote:
> >> >>
> >> >> Branch-2.8 is created.
> >> >>
> >> >> As mentioned before, the goal on branch-2.8 is to put improvements /
> >> >>fixes to existing features with a goal of converging on an alpha
> >>release
> >> >>soon.
> >> >>
> >> >> Thanks
> >> >> +Vinod
> >> >>
> >> >>
> >> >>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
> >> >>><vi...@apache.org> wrote:
> >> >>>
> >> >>> Forking threads now in order to track all things related to the
> >> >>>release.
> >> >>>
> >> >>> Creating the branch now.
> >> >>>
> >> >>> Thanks
> >> >>> +Vinod
> >> >>>
> >> >>>
> >> >>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
> >> >>>><vi...@apache.org> wrote:
> >> >>>>
> >> >>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
> >>sent
> >> >>>>out an email, I updated the Roadmap wiki reflecting the same:
> >> >>>>https://wiki.apache.org/hadoop/Roadmap
> >> >>>><https://wiki.apache.org/hadoop/Roadmap>
> >> >>>>
> >> >>>> I plan to create a 2.8 branch EOD today.
> >> >>>>
> >> >>>> The goal for all of us should be to restrict improvements & fixes
> >>to
> >> >>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki
> >>and
> >> >>>>(b) other minor features that are already in 2.8.
> >> >>>>
> >> >>>> Thanks
> >> >>>> +Vinod
> >> >>>>
> >> >>>>
> >> >>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
> >> >>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
> >> >>>>>
> >> >>>>> - Cut a branch about two weeks from now
> >> >>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> >> >>>>> - As with 2.7.x series, the first release will still be called as
> >> >>>>>early / alpha release in the interest of
> >> >>>>> ‹ gaining downstream adoption
> >> >>>>> ‹ wider testing,
> >> >>>>> ‹ yet reserving our right to fix any inadvertent
> >>incompatibilities
> >> >>>>>introduced.
> >> >>>>
> >> >>>
> >> >>
> >> >
> >>
> >>
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Subramaniam V K <su...@gmail.com>.
Vinod,
Thanks for initiating the 2.8 release thread. We are in late review stages
for YARN-4420 (Add REST API for listing reservations) and YARN-2575 (Adding
ACLs for reservation system), hoping to get them by next week. Any chance
you can put off cutting 2.8 by a week as we are planning to deploy
ReservationSystem and these are critical for that?
Cheers,
Subru
On Thu, Feb 4, 2016 at 3:17 PM, Chris Nauroth <cn...@hortonworks.com>
wrote:
> FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
> release.
>
> --Chris Nauroth
>
>
>
>
> On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>
> >Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
> >Let us not call it alpha or beta though, it is quite confusing. :)
> >
> >On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <uma.gangumalla@intel.com
> >
> >wrote:
> >
> >> Thanks Vinod. +1 for 2.8 release start.
> >>
> >> Regards,
> >> Uma
> >>
> >> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
> >>wrote:
> >>
> >> >Seems like all the features listed in the Roadmap wiki are in. I¹m
> >>going
> >> >to try cutting an RC this weekend for a first/non-stable release off of
> >> >branch-2.8.
> >> >
> >> >Let me know if anyone has any objections/concerns.
> >> >
> >> >Thanks
> >> >+Vinod
> >> >
> >> >> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
> >> >><vi...@apache.org> wrote:
> >> >>
> >> >> Branch-2.8 is created.
> >> >>
> >> >> As mentioned before, the goal on branch-2.8 is to put improvements /
> >> >>fixes to existing features with a goal of converging on an alpha
> >>release
> >> >>soon.
> >> >>
> >> >> Thanks
> >> >> +Vinod
> >> >>
> >> >>
> >> >>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
> >> >>><vi...@apache.org> wrote:
> >> >>>
> >> >>> Forking threads now in order to track all things related to the
> >> >>>release.
> >> >>>
> >> >>> Creating the branch now.
> >> >>>
> >> >>> Thanks
> >> >>> +Vinod
> >> >>>
> >> >>>
> >> >>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
> >> >>>><vi...@apache.org> wrote:
> >> >>>>
> >> >>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
> >>sent
> >> >>>>out an email, I updated the Roadmap wiki reflecting the same:
> >> >>>>https://wiki.apache.org/hadoop/Roadmap
> >> >>>><https://wiki.apache.org/hadoop/Roadmap>
> >> >>>>
> >> >>>> I plan to create a 2.8 branch EOD today.
> >> >>>>
> >> >>>> The goal for all of us should be to restrict improvements & fixes
> >>to
> >> >>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki
> >>and
> >> >>>>(b) other minor features that are already in 2.8.
> >> >>>>
> >> >>>> Thanks
> >> >>>> +Vinod
> >> >>>>
> >> >>>>
> >> >>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
> >> >>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
> >> >>>>>
> >> >>>>> - Cut a branch about two weeks from now
> >> >>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> >> >>>>> - As with 2.7.x series, the first release will still be called as
> >> >>>>>early / alpha release in the interest of
> >> >>>>> ‹ gaining downstream adoption
> >> >>>>> ‹ wider testing,
> >> >>>>> ‹ yet reserving our right to fix any inadvertent
> >>incompatibilities
> >> >>>>>introduced.
> >> >>>>
> >> >>>
> >> >>
> >> >
> >>
> >>
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Chris Nauroth <cn...@hortonworks.com>.
FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
release.
--Chris Nauroth
On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>Let us not call it alpha or beta though, it is quite confusing. :)
>
>On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <um...@intel.com>
>wrote:
>
>> Thanks Vinod. +1 for 2.8 release start.
>>
>> Regards,
>> Uma
>>
>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>wrote:
>>
>> >Seems like all the features listed in the Roadmap wiki are in. I¹m
>>going
>> >to try cutting an RC this weekend for a first/non-stable release off of
>> >branch-2.8.
>> >
>> >Let me know if anyone has any objections/concerns.
>> >
>> >Thanks
>> >+Vinod
>> >
>> >> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>> >><vi...@apache.org> wrote:
>> >>
>> >> Branch-2.8 is created.
>> >>
>> >> As mentioned before, the goal on branch-2.8 is to put improvements /
>> >>fixes to existing features with a goal of converging on an alpha
>>release
>> >>soon.
>> >>
>> >> Thanks
>> >> +Vinod
>> >>
>> >>
>> >>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>> >>><vi...@apache.org> wrote:
>> >>>
>> >>> Forking threads now in order to track all things related to the
>> >>>release.
>> >>>
>> >>> Creating the branch now.
>> >>>
>> >>> Thanks
>> >>> +Vinod
>> >>>
>> >>>
>> >>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>> >>>><vi...@apache.org> wrote:
>> >>>>
>> >>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>sent
>> >>>>out an email, I updated the Roadmap wiki reflecting the same:
>> >>>>https://wiki.apache.org/hadoop/Roadmap
>> >>>><https://wiki.apache.org/hadoop/Roadmap>
>> >>>>
>> >>>> I plan to create a 2.8 branch EOD today.
>> >>>>
>> >>>> The goal for all of us should be to restrict improvements & fixes
>>to
>> >>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>and
>> >>>>(b) other minor features that are already in 2.8.
>> >>>>
>> >>>> Thanks
>> >>>> +Vinod
>> >>>>
>> >>>>
>> >>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>> >>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>> >>>>>
>> >>>>> - Cut a branch about two weeks from now
>> >>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>> >>>>> - As with 2.7.x series, the first release will still be called as
>> >>>>>early / alpha release in the interest of
>> >>>>> ‹ gaining downstream adoption
>> >>>>> ‹ wider testing,
>> >>>>> ‹ yet reserving our right to fix any inadvertent
>>incompatibilities
>> >>>>>introduced.
>> >>>>
>> >>>
>> >>
>> >
>>
>>
Re: [Release thread] 2.8.0 release activities
Posted by Chris Nauroth <cn...@hortonworks.com>.
FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
release.
--Chris Nauroth
On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>Let us not call it alpha or beta though, it is quite confusing. :)
>
>On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <um...@intel.com>
>wrote:
>
>> Thanks Vinod. +1 for 2.8 release start.
>>
>> Regards,
>> Uma
>>
>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>wrote:
>>
>> >Seems like all the features listed in the Roadmap wiki are in. I¹m
>>going
>> >to try cutting an RC this weekend for a first/non-stable release off of
>> >branch-2.8.
>> >
>> >Let me know if anyone has any objections/concerns.
>> >
>> >Thanks
>> >+Vinod
>> >
>> >> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>> >><vi...@apache.org> wrote:
>> >>
>> >> Branch-2.8 is created.
>> >>
>> >> As mentioned before, the goal on branch-2.8 is to put improvements /
>> >>fixes to existing features with a goal of converging on an alpha
>>release
>> >>soon.
>> >>
>> >> Thanks
>> >> +Vinod
>> >>
>> >>
>> >>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>> >>><vi...@apache.org> wrote:
>> >>>
>> >>> Forking threads now in order to track all things related to the
>> >>>release.
>> >>>
>> >>> Creating the branch now.
>> >>>
>> >>> Thanks
>> >>> +Vinod
>> >>>
>> >>>
>> >>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>> >>>><vi...@apache.org> wrote:
>> >>>>
>> >>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>sent
>> >>>>out an email, I updated the Roadmap wiki reflecting the same:
>> >>>>https://wiki.apache.org/hadoop/Roadmap
>> >>>><https://wiki.apache.org/hadoop/Roadmap>
>> >>>>
>> >>>> I plan to create a 2.8 branch EOD today.
>> >>>>
>> >>>> The goal for all of us should be to restrict improvements & fixes
>>to
>> >>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>and
>> >>>>(b) other minor features that are already in 2.8.
>> >>>>
>> >>>> Thanks
>> >>>> +Vinod
>> >>>>
>> >>>>
>> >>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>> >>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>> >>>>>
>> >>>>> - Cut a branch about two weeks from now
>> >>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>> >>>>> - As with 2.7.x series, the first release will still be called as
>> >>>>>early / alpha release in the interest of
>> >>>>> ‹ gaining downstream adoption
>> >>>>> ‹ wider testing,
>> >>>>> ‹ yet reserving our right to fix any inadvertent
>>incompatibilities
>> >>>>>introduced.
>> >>>>
>> >>>
>> >>
>> >
>>
>>
Re: [Release thread] 2.8.0 release activities
Posted by Chris Nauroth <cn...@hortonworks.com>.
FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
release.
--Chris Nauroth
On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>Let us not call it alpha or beta though, it is quite confusing. :)
>
>On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <um...@intel.com>
>wrote:
>
>> Thanks Vinod. +1 for 2.8 release start.
>>
>> Regards,
>> Uma
>>
>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>wrote:
>>
>> >Seems like all the features listed in the Roadmap wiki are in. I¹m
>>going
>> >to try cutting an RC this weekend for a first/non-stable release off of
>> >branch-2.8.
>> >
>> >Let me know if anyone has any objections/concerns.
>> >
>> >Thanks
>> >+Vinod
>> >
>> >> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>> >><vi...@apache.org> wrote:
>> >>
>> >> Branch-2.8 is created.
>> >>
>> >> As mentioned before, the goal on branch-2.8 is to put improvements /
>> >>fixes to existing features with a goal of converging on an alpha
>>release
>> >>soon.
>> >>
>> >> Thanks
>> >> +Vinod
>> >>
>> >>
>> >>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>> >>><vi...@apache.org> wrote:
>> >>>
>> >>> Forking threads now in order to track all things related to the
>> >>>release.
>> >>>
>> >>> Creating the branch now.
>> >>>
>> >>> Thanks
>> >>> +Vinod
>> >>>
>> >>>
>> >>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>> >>>><vi...@apache.org> wrote:
>> >>>>
>> >>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>sent
>> >>>>out an email, I updated the Roadmap wiki reflecting the same:
>> >>>>https://wiki.apache.org/hadoop/Roadmap
>> >>>><https://wiki.apache.org/hadoop/Roadmap>
>> >>>>
>> >>>> I plan to create a 2.8 branch EOD today.
>> >>>>
>> >>>> The goal for all of us should be to restrict improvements & fixes
>>to
>> >>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>and
>> >>>>(b) other minor features that are already in 2.8.
>> >>>>
>> >>>> Thanks
>> >>>> +Vinod
>> >>>>
>> >>>>
>> >>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>> >>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>> >>>>>
>> >>>>> - Cut a branch about two weeks from now
>> >>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>> >>>>> - As with 2.7.x series, the first release will still be called as
>> >>>>>early / alpha release in the interest of
>> >>>>> ‹ gaining downstream adoption
>> >>>>> ‹ wider testing,
>> >>>>> ‹ yet reserving our right to fix any inadvertent
>>incompatibilities
>> >>>>>introduced.
>> >>>>
>> >>>
>> >>
>> >
>>
>>
Re: [Release thread] 2.8.0 release activities
Posted by Chris Nauroth <cn...@hortonworks.com>.
FYI, I've just needed to raise HDFS-9761 to blocker status for the 2.8.0
release.
--Chris Nauroth
On 2/3/16, 6:19 PM, "Karthik Kambatla" <ka...@cloudera.com> wrote:
>Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
>Let us not call it alpha or beta though, it is quite confusing. :)
>
>On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <um...@intel.com>
>wrote:
>
>> Thanks Vinod. +1 for 2.8 release start.
>>
>> Regards,
>> Uma
>>
>> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org>
>>wrote:
>>
>> >Seems like all the features listed in the Roadmap wiki are in. I¹m
>>going
>> >to try cutting an RC this weekend for a first/non-stable release off of
>> >branch-2.8.
>> >
>> >Let me know if anyone has any objections/concerns.
>> >
>> >Thanks
>> >+Vinod
>> >
>> >> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>> >><vi...@apache.org> wrote:
>> >>
>> >> Branch-2.8 is created.
>> >>
>> >> As mentioned before, the goal on branch-2.8 is to put improvements /
>> >>fixes to existing features with a goal of converging on an alpha
>>release
>> >>soon.
>> >>
>> >> Thanks
>> >> +Vinod
>> >>
>> >>
>> >>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>> >>><vi...@apache.org> wrote:
>> >>>
>> >>> Forking threads now in order to track all things related to the
>> >>>release.
>> >>>
>> >>> Creating the branch now.
>> >>>
>> >>> Thanks
>> >>> +Vinod
>> >>>
>> >>>
>> >>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>> >>>><vi...@apache.org> wrote:
>> >>>>
>> >>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just
>>sent
>> >>>>out an email, I updated the Roadmap wiki reflecting the same:
>> >>>>https://wiki.apache.org/hadoop/Roadmap
>> >>>><https://wiki.apache.org/hadoop/Roadmap>
>> >>>>
>> >>>> I plan to create a 2.8 branch EOD today.
>> >>>>
>> >>>> The goal for all of us should be to restrict improvements & fixes
>>to
>> >>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki
>>and
>> >>>>(b) other minor features that are already in 2.8.
>> >>>>
>> >>>> Thanks
>> >>>> +Vinod
>> >>>>
>> >>>>
>> >>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>> >>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>> >>>>>
>> >>>>> - Cut a branch about two weeks from now
>> >>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>> >>>>> - As with 2.7.x series, the first release will still be called as
>> >>>>>early / alpha release in the interest of
>> >>>>> ‹ gaining downstream adoption
>> >>>>> ‹ wider testing,
>> >>>>> ‹ yet reserving our right to fix any inadvertent
>>incompatibilities
>> >>>>>introduced.
>> >>>>
>> >>>
>> >>
>> >
>>
>>
Re: [Release thread] 2.8.0 release activities
Posted by Karthik Kambatla <ka...@cloudera.com>.
Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
Let us not call it alpha or beta though, it is quite confusing. :)
On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <um...@intel.com>
wrote:
> Thanks Vinod. +1 for 2.8 release start.
>
> Regards,
> Uma
>
> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org> wrote:
>
> >Seems like all the features listed in the Roadmap wiki are in. I¹m going
> >to try cutting an RC this weekend for a first/non-stable release off of
> >branch-2.8.
> >
> >Let me know if anyone has any objections/concerns.
> >
> >Thanks
> >+Vinod
> >
> >> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
> >><vi...@apache.org> wrote:
> >>
> >> Branch-2.8 is created.
> >>
> >> As mentioned before, the goal on branch-2.8 is to put improvements /
> >>fixes to existing features with a goal of converging on an alpha release
> >>soon.
> >>
> >> Thanks
> >> +Vinod
> >>
> >>
> >>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
> >>><vi...@apache.org> wrote:
> >>>
> >>> Forking threads now in order to track all things related to the
> >>>release.
> >>>
> >>> Creating the branch now.
> >>>
> >>> Thanks
> >>> +Vinod
> >>>
> >>>
> >>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
> >>>><vi...@apache.org> wrote:
> >>>>
> >>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just sent
> >>>>out an email, I updated the Roadmap wiki reflecting the same:
> >>>>https://wiki.apache.org/hadoop/Roadmap
> >>>><https://wiki.apache.org/hadoop/Roadmap>
> >>>>
> >>>> I plan to create a 2.8 branch EOD today.
> >>>>
> >>>> The goal for all of us should be to restrict improvements & fixes to
> >>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki and
> >>>>(b) other minor features that are already in 2.8.
> >>>>
> >>>> Thanks
> >>>> +Vinod
> >>>>
> >>>>
> >>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
> >>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
> >>>>>
> >>>>> - Cut a branch about two weeks from now
> >>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> >>>>> - As with 2.7.x series, the first release will still be called as
> >>>>>early / alpha release in the interest of
> >>>>> ‹ gaining downstream adoption
> >>>>> ‹ wider testing,
> >>>>> ‹ yet reserving our right to fix any inadvertent incompatibilities
> >>>>>introduced.
> >>>>
> >>>
> >>
> >
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Karthik Kambatla <ka...@cloudera.com>.
Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
Let us not call it alpha or beta though, it is quite confusing. :)
On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <um...@intel.com>
wrote:
> Thanks Vinod. +1 for 2.8 release start.
>
> Regards,
> Uma
>
> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org> wrote:
>
> >Seems like all the features listed in the Roadmap wiki are in. I¹m going
> >to try cutting an RC this weekend for a first/non-stable release off of
> >branch-2.8.
> >
> >Let me know if anyone has any objections/concerns.
> >
> >Thanks
> >+Vinod
> >
> >> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
> >><vi...@apache.org> wrote:
> >>
> >> Branch-2.8 is created.
> >>
> >> As mentioned before, the goal on branch-2.8 is to put improvements /
> >>fixes to existing features with a goal of converging on an alpha release
> >>soon.
> >>
> >> Thanks
> >> +Vinod
> >>
> >>
> >>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
> >>><vi...@apache.org> wrote:
> >>>
> >>> Forking threads now in order to track all things related to the
> >>>release.
> >>>
> >>> Creating the branch now.
> >>>
> >>> Thanks
> >>> +Vinod
> >>>
> >>>
> >>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
> >>>><vi...@apache.org> wrote:
> >>>>
> >>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just sent
> >>>>out an email, I updated the Roadmap wiki reflecting the same:
> >>>>https://wiki.apache.org/hadoop/Roadmap
> >>>><https://wiki.apache.org/hadoop/Roadmap>
> >>>>
> >>>> I plan to create a 2.8 branch EOD today.
> >>>>
> >>>> The goal for all of us should be to restrict improvements & fixes to
> >>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki and
> >>>>(b) other minor features that are already in 2.8.
> >>>>
> >>>> Thanks
> >>>> +Vinod
> >>>>
> >>>>
> >>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
> >>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
> >>>>>
> >>>>> - Cut a branch about two weeks from now
> >>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> >>>>> - As with 2.7.x series, the first release will still be called as
> >>>>>early / alpha release in the interest of
> >>>>> ‹ gaining downstream adoption
> >>>>> ‹ wider testing,
> >>>>> ‹ yet reserving our right to fix any inadvertent incompatibilities
> >>>>>introduced.
> >>>>
> >>>
> >>
> >
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Karthik Kambatla <ka...@cloudera.com>.
Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
Let us not call it alpha or beta though, it is quite confusing. :)
On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <um...@intel.com>
wrote:
> Thanks Vinod. +1 for 2.8 release start.
>
> Regards,
> Uma
>
> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org> wrote:
>
> >Seems like all the features listed in the Roadmap wiki are in. I¹m going
> >to try cutting an RC this weekend for a first/non-stable release off of
> >branch-2.8.
> >
> >Let me know if anyone has any objections/concerns.
> >
> >Thanks
> >+Vinod
> >
> >> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
> >><vi...@apache.org> wrote:
> >>
> >> Branch-2.8 is created.
> >>
> >> As mentioned before, the goal on branch-2.8 is to put improvements /
> >>fixes to existing features with a goal of converging on an alpha release
> >>soon.
> >>
> >> Thanks
> >> +Vinod
> >>
> >>
> >>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
> >>><vi...@apache.org> wrote:
> >>>
> >>> Forking threads now in order to track all things related to the
> >>>release.
> >>>
> >>> Creating the branch now.
> >>>
> >>> Thanks
> >>> +Vinod
> >>>
> >>>
> >>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
> >>>><vi...@apache.org> wrote:
> >>>>
> >>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just sent
> >>>>out an email, I updated the Roadmap wiki reflecting the same:
> >>>>https://wiki.apache.org/hadoop/Roadmap
> >>>><https://wiki.apache.org/hadoop/Roadmap>
> >>>>
> >>>> I plan to create a 2.8 branch EOD today.
> >>>>
> >>>> The goal for all of us should be to restrict improvements & fixes to
> >>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki and
> >>>>(b) other minor features that are already in 2.8.
> >>>>
> >>>> Thanks
> >>>> +Vinod
> >>>>
> >>>>
> >>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
> >>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
> >>>>>
> >>>>> - Cut a branch about two weeks from now
> >>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> >>>>> - As with 2.7.x series, the first release will still be called as
> >>>>>early / alpha release in the interest of
> >>>>> ‹ gaining downstream adoption
> >>>>> ‹ wider testing,
> >>>>> ‹ yet reserving our right to fix any inadvertent incompatibilities
> >>>>>introduced.
> >>>>
> >>>
> >>
> >
>
>
Re: [Release thread] 2.8.0 release activities
Posted by Karthik Kambatla <ka...@cloudera.com>.
Thanks Vinod. Not labeling 2.8.0 stable sounds perfectly reasonable to me.
Let us not call it alpha or beta though, it is quite confusing. :)
On Wed, Feb 3, 2016 at 8:17 PM, Gangumalla, Uma <um...@intel.com>
wrote:
> Thanks Vinod. +1 for 2.8 release start.
>
> Regards,
> Uma
>
> On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org> wrote:
>
> >Seems like all the features listed in the Roadmap wiki are in. I¹m going
> >to try cutting an RC this weekend for a first/non-stable release off of
> >branch-2.8.
> >
> >Let me know if anyone has any objections/concerns.
> >
> >Thanks
> >+Vinod
> >
> >> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
> >><vi...@apache.org> wrote:
> >>
> >> Branch-2.8 is created.
> >>
> >> As mentioned before, the goal on branch-2.8 is to put improvements /
> >>fixes to existing features with a goal of converging on an alpha release
> >>soon.
> >>
> >> Thanks
> >> +Vinod
> >>
> >>
> >>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
> >>><vi...@apache.org> wrote:
> >>>
> >>> Forking threads now in order to track all things related to the
> >>>release.
> >>>
> >>> Creating the branch now.
> >>>
> >>> Thanks
> >>> +Vinod
> >>>
> >>>
> >>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
> >>>><vi...@apache.org> wrote:
> >>>>
> >>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just sent
> >>>>out an email, I updated the Roadmap wiki reflecting the same:
> >>>>https://wiki.apache.org/hadoop/Roadmap
> >>>><https://wiki.apache.org/hadoop/Roadmap>
> >>>>
> >>>> I plan to create a 2.8 branch EOD today.
> >>>>
> >>>> The goal for all of us should be to restrict improvements & fixes to
> >>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki and
> >>>>(b) other minor features that are already in 2.8.
> >>>>
> >>>> Thanks
> >>>> +Vinod
> >>>>
> >>>>
> >>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
> >>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
> >>>>>
> >>>>> - Cut a branch about two weeks from now
> >>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> >>>>> - As with 2.7.x series, the first release will still be called as
> >>>>>early / alpha release in the interest of
> >>>>> ‹ gaining downstream adoption
> >>>>> ‹ wider testing,
> >>>>> ‹ yet reserving our right to fix any inadvertent incompatibilities
> >>>>>introduced.
> >>>>
> >>>
> >>
> >
>
>
Re: [Release thread] 2.8.0 release activities
Posted by "Gangumalla, Uma" <um...@intel.com>.
Thanks Vinod. +1 for 2.8 release start.
Regards,
Uma
On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org> wrote:
>Seems like all the features listed in the Roadmap wiki are in. I¹m going
>to try cutting an RC this weekend for a first/non-stable release off of
>branch-2.8.
>
>Let me know if anyone has any objections/concerns.
>
>Thanks
>+Vinod
>
>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>><vi...@apache.org> wrote:
>>
>> Branch-2.8 is created.
>>
>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>fixes to existing features with a goal of converging on an alpha release
>>soon.
>>
>> Thanks
>> +Vinod
>>
>>
>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>><vi...@apache.org> wrote:
>>>
>>> Forking threads now in order to track all things related to the
>>>release.
>>>
>>> Creating the branch now.
>>>
>>> Thanks
>>> +Vinod
>>>
>>>
>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>><vi...@apache.org> wrote:
>>>>
>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just sent
>>>>out an email, I updated the Roadmap wiki reflecting the same:
>>>>https://wiki.apache.org/hadoop/Roadmap
>>>><https://wiki.apache.org/hadoop/Roadmap>
>>>>
>>>> I plan to create a 2.8 branch EOD today.
>>>>
>>>> The goal for all of us should be to restrict improvements & fixes to
>>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki and
>>>>(b) other minor features that are already in 2.8.
>>>>
>>>> Thanks
>>>> +Vinod
>>>>
>>>>
>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>
>>>>> - Cut a branch about two weeks from now
>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>early / alpha release in the interest of
>>>>> ‹ gaining downstream adoption
>>>>> ‹ wider testing,
>>>>> ‹ yet reserving our right to fix any inadvertent incompatibilities
>>>>>introduced.
>>>>
>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Steve Loughran <st...@hortonworks.com>.
> On 3 Feb 2016, at 23:53, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Seems like all the features listed in the Roadmap wiki are in. I’m going to try cutting an RC this weekend for a first/non-stable release off of branch-2.8.
>
> Let me know if anyone has any objections/concerns.
>
> Thanks
> +Vinod
>
could anyone take a quick look at https://issues.apache.org/jira/browse/HADOOP-12426 ; adds a new entry point for kerberos diagnostics
1/ doesn't change any existing code , so doesn't break things.
2/ people need this
we can evolve it once in, but getting it in is the first step
Ta,
Steve
Re: [Release thread] 2.8.0 release activities
Posted by "Gangumalla, Uma" <um...@intel.com>.
Thanks Vinod. +1 for 2.8 release start.
Regards,
Uma
On 2/3/16, 3:53 PM, "Vinod Kumar Vavilapalli" <vi...@apache.org> wrote:
>Seems like all the features listed in the Roadmap wiki are in. I¹m going
>to try cutting an RC this weekend for a first/non-stable release off of
>branch-2.8.
>
>Let me know if anyone has any objections/concerns.
>
>Thanks
>+Vinod
>
>> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli
>><vi...@apache.org> wrote:
>>
>> Branch-2.8 is created.
>>
>> As mentioned before, the goal on branch-2.8 is to put improvements /
>>fixes to existing features with a goal of converging on an alpha release
>>soon.
>>
>> Thanks
>> +Vinod
>>
>>
>>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli
>>><vi...@apache.org> wrote:
>>>
>>> Forking threads now in order to track all things related to the
>>>release.
>>>
>>> Creating the branch now.
>>>
>>> Thanks
>>> +Vinod
>>>
>>>
>>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli
>>>><vi...@apache.org> wrote:
>>>>
>>>> I think we¹ve converged at a high level w.r.t 2.8. And as I just sent
>>>>out an email, I updated the Roadmap wiki reflecting the same:
>>>>https://wiki.apache.org/hadoop/Roadmap
>>>><https://wiki.apache.org/hadoop/Roadmap>
>>>>
>>>> I plan to create a 2.8 branch EOD today.
>>>>
>>>> The goal for all of us should be to restrict improvements & fixes to
>>>>only (a) the feature-set documented under 2.8 in the RoadMap wiki and
>>>>(b) other minor features that are already in 2.8.
>>>>
>>>> Thanks
>>>> +Vinod
>>>>
>>>>
>>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli
>>>>><vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>>
>>>>> - Cut a branch about two weeks from now
>>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>>> - As with 2.7.x series, the first release will still be called as
>>>>>early / alpha release in the interest of
>>>>> ‹ gaining downstream adoption
>>>>> ‹ wider testing,
>>>>> ‹ yet reserving our right to fix any inadvertent incompatibilities
>>>>>introduced.
>>>>
>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Seems like all the features listed in the Roadmap wiki are in. I’m going to try cutting an RC this weekend for a first/non-stable release off of branch-2.8.
Let me know if anyone has any objections/concerns.
Thanks
+Vinod
> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Branch-2.8 is created.
>
> As mentioned before, the goal on branch-2.8 is to put improvements / fixes to existing features with a goal of converging on an alpha release soon.
>
> Thanks
> +Vinod
>
>
>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> Forking threads now in order to track all things related to the release.
>>
>> Creating the branch now.
>>
>> Thanks
>> +Vinod
>>
>>
>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>>
>>> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap <https://wiki.apache.org/hadoop/Roadmap>
>>>
>>> I plan to create a 2.8 branch EOD today.
>>>
>>> The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
>>>
>>> Thanks
>>> +Vinod
>>>
>>>
>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>
>>>> - Cut a branch about two weeks from now
>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
>>>> — gaining downstream adoption
>>>> — wider testing,
>>>> — yet reserving our right to fix any inadvertent incompatibilities introduced.
>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Seems like all the features listed in the Roadmap wiki are in. I’m going to try cutting an RC this weekend for a first/non-stable release off of branch-2.8.
Let me know if anyone has any objections/concerns.
Thanks
+Vinod
> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Branch-2.8 is created.
>
> As mentioned before, the goal on branch-2.8 is to put improvements / fixes to existing features with a goal of converging on an alpha release soon.
>
> Thanks
> +Vinod
>
>
>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> Forking threads now in order to track all things related to the release.
>>
>> Creating the branch now.
>>
>> Thanks
>> +Vinod
>>
>>
>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>>
>>> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap <https://wiki.apache.org/hadoop/Roadmap>
>>>
>>> I plan to create a 2.8 branch EOD today.
>>>
>>> The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
>>>
>>> Thanks
>>> +Vinod
>>>
>>>
>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>
>>>> - Cut a branch about two weeks from now
>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
>>>> — gaining downstream adoption
>>>> — wider testing,
>>>> — yet reserving our right to fix any inadvertent incompatibilities introduced.
>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Seems like all the features listed in the Roadmap wiki are in. I’m going to try cutting an RC this weekend for a first/non-stable release off of branch-2.8.
Let me know if anyone has any objections/concerns.
Thanks
+Vinod
> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Branch-2.8 is created.
>
> As mentioned before, the goal on branch-2.8 is to put improvements / fixes to existing features with a goal of converging on an alpha release soon.
>
> Thanks
> +Vinod
>
>
>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> Forking threads now in order to track all things related to the release.
>>
>> Creating the branch now.
>>
>> Thanks
>> +Vinod
>>
>>
>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>>
>>> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap <https://wiki.apache.org/hadoop/Roadmap>
>>>
>>> I plan to create a 2.8 branch EOD today.
>>>
>>> The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
>>>
>>> Thanks
>>> +Vinod
>>>
>>>
>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>
>>>> - Cut a branch about two weeks from now
>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
>>>> — gaining downstream adoption
>>>> — wider testing,
>>>> — yet reserving our right to fix any inadvertent incompatibilities introduced.
>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Seems like all the features listed in the Roadmap wiki are in. I’m going to try cutting an RC this weekend for a first/non-stable release off of branch-2.8.
Let me know if anyone has any objections/concerns.
Thanks
+Vinod
> On Nov 25, 2015, at 5:59 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Branch-2.8 is created.
>
> As mentioned before, the goal on branch-2.8 is to put improvements / fixes to existing features with a goal of converging on an alpha release soon.
>
> Thanks
> +Vinod
>
>
>> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> Forking threads now in order to track all things related to the release.
>>
>> Creating the branch now.
>>
>> Thanks
>> +Vinod
>>
>>
>>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>>
>>> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap <https://wiki.apache.org/hadoop/Roadmap>
>>>
>>> I plan to create a 2.8 branch EOD today.
>>>
>>> The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
>>>
>>> Thanks
>>> +Vinod
>>>
>>>
>>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>>
>>>> - Cut a branch about two weeks from now
>>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>>> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
>>>> — gaining downstream adoption
>>>> — wider testing,
>>>> — yet reserving our right to fix any inadvertent incompatibilities introduced.
>>>
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Branch-2.8 is created.
As mentioned before, the goal on branch-2.8 is to put improvements / fixes to existing features with a goal of converging on an alpha release soon.
Thanks
+Vinod
> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Forking threads now in order to track all things related to the release.
>
> Creating the branch now.
>
> Thanks
> +Vinod
>
>
>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap <https://wiki.apache.org/hadoop/Roadmap>
>>
>> I plan to create a 2.8 branch EOD today.
>>
>> The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
>>
>> Thanks
>> +Vinod
>>
>>
>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>
>>> - Cut a branch about two weeks from now
>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
>>> — gaining downstream adoption
>>> — wider testing,
>>> — yet reserving our right to fix any inadvertent incompatibilities introduced.
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Branch-2.8 is created.
As mentioned before, the goal on branch-2.8 is to put improvements / fixes to existing features with a goal of converging on an alpha release soon.
Thanks
+Vinod
> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Forking threads now in order to track all things related to the release.
>
> Creating the branch now.
>
> Thanks
> +Vinod
>
>
>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap <https://wiki.apache.org/hadoop/Roadmap>
>>
>> I plan to create a 2.8 branch EOD today.
>>
>> The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
>>
>> Thanks
>> +Vinod
>>
>>
>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>
>>> - Cut a branch about two weeks from now
>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
>>> — gaining downstream adoption
>>> — wider testing,
>>> — yet reserving our right to fix any inadvertent incompatibilities introduced.
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Branch-2.8 is created.
As mentioned before, the goal on branch-2.8 is to put improvements / fixes to existing features with a goal of converging on an alpha release soon.
Thanks
+Vinod
> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Forking threads now in order to track all things related to the release.
>
> Creating the branch now.
>
> Thanks
> +Vinod
>
>
>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap <https://wiki.apache.org/hadoop/Roadmap>
>>
>> I plan to create a 2.8 branch EOD today.
>>
>> The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
>>
>> Thanks
>> +Vinod
>>
>>
>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>
>>> - Cut a branch about two weeks from now
>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
>>> — gaining downstream adoption
>>> — wider testing,
>>> — yet reserving our right to fix any inadvertent incompatibilities introduced.
>>
>
Re: [Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Branch-2.8 is created.
As mentioned before, the goal on branch-2.8 is to put improvements / fixes to existing features with a goal of converging on an alpha release soon.
Thanks
+Vinod
> On Nov 25, 2015, at 5:30 PM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> Forking threads now in order to track all things related to the release.
>
> Creating the branch now.
>
> Thanks
> +Vinod
>
>
>> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>>
>> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap <https://wiki.apache.org/hadoop/Roadmap>
>>
>> I plan to create a 2.8 branch EOD today.
>>
>> The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
>>
>> Thanks
>> +Vinod
>>
>>
>>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>>
>>> - Cut a branch about two weeks from now
>>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>>> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
>>> — gaining downstream adoption
>>> — wider testing,
>>> — yet reserving our right to fix any inadvertent incompatibilities introduced.
>>
>
[Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Forking threads now in order to track all things related to the release.
Creating the branch now.
Thanks
+Vinod
> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap <https://wiki.apache.org/hadoop/Roadmap>
>
> I plan to create a 2.8 branch EOD today.
>
> The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
>
> Thanks
> +Vinod
>
>
>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>
>> - Cut a branch about two weeks from now
>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
>> — gaining downstream adoption
>> — wider testing,
>> — yet reserving our right to fix any inadvertent incompatibilities introduced.
>
[Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Forking threads now in order to track all things related to the release.
Creating the branch now.
Thanks
+Vinod
> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap <https://wiki.apache.org/hadoop/Roadmap>
>
> I plan to create a 2.8 branch EOD today.
>
> The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
>
> Thanks
> +Vinod
>
>
>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>
>> - Cut a branch about two weeks from now
>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
>> — gaining downstream adoption
>> — wider testing,
>> — yet reserving our right to fix any inadvertent incompatibilities introduced.
>
[Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Forking threads now in order to track all things related to the release.
Creating the branch now.
Thanks
+Vinod
> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap <https://wiki.apache.org/hadoop/Roadmap>
>
> I plan to create a 2.8 branch EOD today.
>
> The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
>
> Thanks
> +Vinod
>
>
>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>
>> - Cut a branch about two weeks from now
>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
>> — gaining downstream adoption
>> — wider testing,
>> — yet reserving our right to fix any inadvertent incompatibilities introduced.
>
[Release thread] 2.8.0 release activities
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
Forking threads now in order to track all things related to the release.
Creating the branch now.
Thanks
+Vinod
> On Nov 25, 2015, at 11:37 AM, Vinod Kumar Vavilapalli <vi...@apache.org> wrote:
>
> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap <https://wiki.apache.org/hadoop/Roadmap>
>
> I plan to create a 2.8 branch EOD today.
>
> The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
>
> Thanks
> +Vinod
>
>
>> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vinodkv@hortonworks.com <ma...@hortonworks.com>> wrote:
>>
>> - Cut a branch about two weeks from now
>> - Do an RC mid next month (leaving ~4weeks since branch-cut)
>> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
>> — gaining downstream adoption
>> — wider testing,
>> — yet reserving our right to fix any inadvertent incompatibilities introduced.
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Subramaniam V K <su...@gmail.com>.
Hi Vinod,
Thanks for driving this. Can you add YARN-2573 which includes the work done
to integrate ReservationSystem with the RM failover mechanism to your list.
This can be reviewed and committed (branch-2) also about a month back.
Cheers,
Subru
On Wed, Nov 25, 2015 at 11:37 AM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:
> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out
> an email, I updated the Roadmap wiki reflecting the same:
> https://wiki.apache.org/hadoop/Roadmap
>
> I plan to create a 2.8 branch EOD today.
>
> The goal for all of us should be to restrict improvements & fixes to only
> (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other
> minor features that are already in 2.8.
>
> Thanks
> +Vinod
>
>
> > On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
> >
> > - Cut a branch about two weeks from now
> > - Do an RC mid next month (leaving ~4weeks since branch-cut)
> > - As with 2.7.x series, the first release will still be called as early
> / alpha release in the interest of
> > — gaining downstream adoption
> > — wider testing,
> > — yet reserving our right to fix any inadvertent incompatibilities
> introduced.
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Subramaniam V K <su...@gmail.com>.
Hi Vinod,
Thanks for driving this. Can you add YARN-2573 which includes the work done
to integrate ReservationSystem with the RM failover mechanism to your list.
This can be reviewed and committed (branch-2) also about a month back.
Cheers,
Subru
On Wed, Nov 25, 2015 at 11:37 AM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:
> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out
> an email, I updated the Roadmap wiki reflecting the same:
> https://wiki.apache.org/hadoop/Roadmap
>
> I plan to create a 2.8 branch EOD today.
>
> The goal for all of us should be to restrict improvements & fixes to only
> (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other
> minor features that are already in 2.8.
>
> Thanks
> +Vinod
>
>
> > On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
> >
> > - Cut a branch about two weeks from now
> > - Do an RC mid next month (leaving ~4weeks since branch-cut)
> > - As with 2.7.x series, the first release will still be called as early
> / alpha release in the interest of
> > — gaining downstream adoption
> > — wider testing,
> > — yet reserving our right to fix any inadvertent incompatibilities
> introduced.
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Subramaniam V K <su...@gmail.com>.
Hi Vinod,
Thanks for driving this. Can you add YARN-2573 which includes the work done
to integrate ReservationSystem with the RM failover mechanism to your list.
This can be reviewed and committed (branch-2) also about a month back.
Cheers,
Subru
On Wed, Nov 25, 2015 at 11:37 AM, Vinod Kumar Vavilapalli <
vinodkv@apache.org> wrote:
> I think we’ve converged at a high level w.r.t 2.8. And as I just sent out
> an email, I updated the Roadmap wiki reflecting the same:
> https://wiki.apache.org/hadoop/Roadmap
>
> I plan to create a 2.8 branch EOD today.
>
> The goal for all of us should be to restrict improvements & fixes to only
> (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other
> minor features that are already in 2.8.
>
> Thanks
> +Vinod
>
>
> > On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
> >
> > - Cut a branch about two weeks from now
> > - Do an RC mid next month (leaving ~4weeks since branch-cut)
> > - As with 2.7.x series, the first release will still be called as early
> / alpha release in the interest of
> > — gaining downstream adoption
> > — wider testing,
> > — yet reserving our right to fix any inadvertent incompatibilities
> introduced.
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap
I plan to create a 2.8 branch EOD today.
The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
Thanks
+Vinod
> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
>
> - Cut a branch about two weeks from now
> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
> — gaining downstream adoption
> — wider testing,
> — yet reserving our right to fix any inadvertent incompatibilities introduced.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Karthik Kambatla <ka...@cloudera.com>.
I am really against the notion of calling x.y.0 releases alpha/beta; it is
very confusing. If we think a release is alpha/beta quality, why not
release it as x.y.0-alpha or x.y.0-beta, and follow it up eventually with
x.y.0 GA.
I am in favor of labeling any of the newer features to be of alpha/beta
quality.
SharedCache is another close to done feature.
On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vinodkv@hortonworks.com
> wrote:
> Agreed on not mixing this with major release discussions.
>
> Okay, I just finished my review of 2.8 content.
>
> A quick summary follows.
>
> Current state of originally planned items
>
> - Nearly Done / Done and so need to close down quickly
> — Support *both* JDK7 and JDK8 runtimes HADOOP-11090
> — Supporting non-exclusive node-labels: YARN-3214: Done, can push as
> an alpha / beta feature
> — Support priorities across applications within the same queue
> YARN-1963: Can push as an alpha / beta feature
>
> - Definitely have to move out into 2.9 and beyond
> — Early work for disk and network isolation in YARN: YARN-2139,
> YARN-2140: Early noise, some critical pieces designed, done but not a lot
> of movement of late
> — Classpath isolation for downstream clients HADOOP-11656: Lots of
> chatter a while ago, not much movement of late
> — Support for Erasure Codes in HDFS HDFS-7285<
> https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the
> interest of stability / bake-in
>
> Non-planned features that went into 2.8.0
>
> — The overall list of new features:
> https://issues.apache.org/jira/issues/?filter=12333994
> — HDFS-6200 Create a separate jar for hdfs-client: Compatible
> improvement - no dimension of alpha/betaness here.
> — HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
> — Stability improvements ready to use:
> — HDFS-8008 Support client-side back off when the datanodes are
> congested
> — HDFS-8009 Signal congestion on the DataNode
> — YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> most of it anyways. Can push as an alpha feature.
> — YARN-1197 Support changing resources of an allocated container: Can
> push as an alpha/beta feature
>
> Items in progress to think about in 2 weeks
>
> — YARN Timeline Service v1.5 - YARN-4233: A short term bridge before
> YARN-2928 comes around. I think this should go in given the tremendous
> activity recently.
> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
> but clearly a work in progress. Two options here
> — If it is safe to ship it into 2.8 in a disable manner, we can
> get the early code into trunk and all the way int o2.8.
> — If it is not safe, it organically rolls over into 2.9
> — Compatibility tools to catch backwards, forwards compatibility
> issues at patch submission, release times. Some of it is captured at
> YARN-3292. This also involves resurrecting jdiff
> (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
>
> This is my plan of action for now in terms of the release itself
>
> - Cut a branch about two weeks from now
> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> - As with 2.7.x series, the first release will still be called as early /
> alpha release in the interest of
> — gaining downstream adoption
> — wider testing,
> — yet reserving our right to fix any inadvertent incompatibilities
> introduced.
>
> If we can get answers on “Items to think about now” during this and next
> week, we will overall be in good shape.
>
> Thoughts?
>
> Thanks
> +Vinod
> PS:As you may have noted above, this time around, I want to do something
> that we’ve always wanted to do, but never explicitly did. I’m calling out
> readiness of each feature as they stand today so we can inform our users
> better of what they can start relying on in production clusters.
>
>
> On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cmccabe@apache.org<mailto:
> cmccabe@apache.org>> wrote:
>
> I think it makes sense to have a 2.8 release since there are a
> tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
> release seems like something we should consider separately since it
> would not have the same compatibility guarantees as a 2.8 release.
> There's a pretty big delta between trunk and 2.8 as well.
>
> cheers,
> Colin
>
> On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cdouglas@apache.org
> <ma...@apache.org>> wrote:
> With two active sustaining branches (2.6, 2.7), what would you think
> of releasing trunk as 3.x instead of pushing 2.8? There are many new
> features (EC, Y1197, etc.), and trunk could be the source of several
> alpha/beta releases before we fork the 3.x line. -C
>
> On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
> <vi...@hortonworks.com>> wrote:
> As you may have noted, 2.8.0 got completely derailed what with 2.7.x and
> the unusually long 2.6.1 release.
>
> With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2
> and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.
>
> I’ll do a quick survey of where the individual features are and the amount
> of content already present in 2.8 and kick-start 2.8.0 process again.
>
> +Vinod
>
>
> On Apr 21, 2015, at 2:39 PM, vinodkv@apache.org<ma...@apache.org>
> wrote:
>
> With 2.7.0 out of the way, and with more maintenance releases to stabilize
> it, I propose we start thinking about 2.8.0.
>
> Here's my first cut of the proposal, will update the Roadmap wiki.
> - Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
> - Compatibility tools to catch backwards, forwards compatibility issues at
> patch submission, release times. Some of it is captured at YARN-3292. This
> also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310)
> and/or investing in new tools.
> - HADOOP-11656 Classpath isolation for downstream clients
> - Support for Erasure Codes in HDFS HDFS-7285
> - Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
> - YARN Timeline Service Next generation: YARN-2928. At least branch-merge
> + early peek.
> - Supporting non-exclusive node-labels: YARN-3214
>
> I'm experimenting with more agile 2.7.x releases and would like to
> continue the same by volunteering as the RM for 2.8.x too.
>
> Given the long time we took with 2.7.0, the timeline I am looking at is
> 8-12 weeks. We can pick as many features as they finish along and make a
> more predictable releases instead of holding up releases for ever.
>
> Thoughts?
>
> Thanks
> +Vinod
>
>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap
I plan to create a 2.8 branch EOD today.
The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
Thanks
+Vinod
> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
>
> - Cut a branch about two weeks from now
> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
> — gaining downstream adoption
> — wider testing,
> — yet reserving our right to fix any inadvertent incompatibilities introduced.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap
I plan to create a 2.8 branch EOD today.
The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
Thanks
+Vinod
> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
>
> - Cut a branch about two weeks from now
> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
> — gaining downstream adoption
> — wider testing,
> — yet reserving our right to fix any inadvertent incompatibilities introduced.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Sangjin Lee <sj...@gmail.com>.
On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vinodkv@hortonworks.com
> wrote:
> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
> but clearly a work in progress. Two options here
> — If it is safe to ship it into 2.8 in a disable manner, we can
> get the early code into trunk and all the way int o2.8.
> — If it is not safe, it organically rolls over into 2.9
>
I'll review the changes on YARN-2928 to see what impact it has (if any) if
the timeline service v.2 is disabled.
Another condition for it to make 2.8 is whether the branch will be in a
shape in a couple of weeks such that it adds value for folks that want to
test it. Hopefully it will become clearer soon.
Sangjin
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Wangda Tan <wh...@gmail.com>.
Thanks to Vinod for starting this discussion!
+1 to add YARN-1197 (container resizing) to 2.8.0, it is end-to-end tested.
I'd prefer to push it as an Alpha feature before wilder testing.
And also agree to call first release of 2.8 as an Alpha release according
to the number of new features / code changes.
Regards,
Wangda
On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vinodkv@hortonworks.com
> wrote:
> Agreed on not mixing this with major release discussions.
>
> Okay, I just finished my review of 2.8 content.
>
> A quick summary follows.
>
> Current state of originally planned items
>
> - Nearly Done / Done and so need to close down quickly
> — Support *both* JDK7 and JDK8 runtimes HADOOP-11090
> — Supporting non-exclusive node-labels: YARN-3214: Done, can push as
> an alpha / beta feature
> — Support priorities across applications within the same queue
> YARN-1963: Can push as an alpha / beta feature
>
> - Definitely have to move out into 2.9 and beyond
> — Early work for disk and network isolation in YARN: YARN-2139,
> YARN-2140: Early noise, some critical pieces designed, done but not a lot
> of movement of late
> — Classpath isolation for downstream clients HADOOP-11656: Lots of
> chatter a while ago, not much movement of late
> — Support for Erasure Codes in HDFS HDFS-7285<
> https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the
> interest of stability / bake-in
>
> Non-planned features that went into 2.8.0
>
> — The overall list of new features:
> https://issues.apache.org/jira/issues/?filter=12333994
> — HDFS-6200 Create a separate jar for hdfs-client: Compatible
> improvement - no dimension of alpha/betaness here.
> — HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
> — Stability improvements ready to use:
> — HDFS-8008 Support client-side back off when the datanodes are
> congested
> — HDFS-8009 Signal congestion on the DataNode
> — YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> most of it anyways. Can push as an alpha feature.
> — YARN-1197 Support changing resources of an allocated container: Can
> push as an alpha/beta feature
>
> Items in progress to think about in 2 weeks
>
> — YARN Timeline Service v1.5 - YARN-4233: A short term bridge before
> YARN-2928 comes around. I think this should go in given the tremendous
> activity recently.
> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
> but clearly a work in progress. Two options here
> — If it is safe to ship it into 2.8 in a disable manner, we can
> get the early code into trunk and all the way int o2.8.
> — If it is not safe, it organically rolls over into 2.9
> — Compatibility tools to catch backwards, forwards compatibility
> issues at patch submission, release times. Some of it is captured at
> YARN-3292. This also involves resurrecting jdiff
> (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
>
> This is my plan of action for now in terms of the release itself
>
> - Cut a branch about two weeks from now
> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> - As with 2.7.x series, the first release will still be called as early /
> alpha release in the interest of
> — gaining downstream adoption
> — wider testing,
> — yet reserving our right to fix any inadvertent incompatibilities
> introduced.
>
> If we can get answers on “Items to think about now” during this and next
> week, we will overall be in good shape.
>
> Thoughts?
>
> Thanks
> +Vinod
> PS:As you may have noted above, this time around, I want to do something
> that we’ve always wanted to do, but never explicitly did. I’m calling out
> readiness of each feature as they stand today so we can inform our users
> better of what they can start relying on in production clusters.
>
>
> On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cmccabe@apache.org<mailto:
> cmccabe@apache.org>> wrote:
>
> I think it makes sense to have a 2.8 release since there are a
> tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
> release seems like something we should consider separately since it
> would not have the same compatibility guarantees as a 2.8 release.
> There's a pretty big delta between trunk and 2.8 as well.
>
> cheers,
> Colin
>
> On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cdouglas@apache.org
> <ma...@apache.org>> wrote:
> With two active sustaining branches (2.6, 2.7), what would you think
> of releasing trunk as 3.x instead of pushing 2.8? There are many new
> features (EC, Y1197, etc.), and trunk could be the source of several
> alpha/beta releases before we fork the 3.x line. -C
>
> On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
> <vi...@hortonworks.com>> wrote:
> As you may have noted, 2.8.0 got completely derailed what with 2.7.x and
> the unusually long 2.6.1 release.
>
> With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2
> and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.
>
> I’ll do a quick survey of where the individual features are and the amount
> of content already present in 2.8 and kick-start 2.8.0 process again.
>
> +Vinod
>
>
> On Apr 21, 2015, at 2:39 PM, vinodkv@apache.org<ma...@apache.org>
> wrote:
>
> With 2.7.0 out of the way, and with more maintenance releases to stabilize
> it, I propose we start thinking about 2.8.0.
>
> Here's my first cut of the proposal, will update the Roadmap wiki.
> - Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
> - Compatibility tools to catch backwards, forwards compatibility issues at
> patch submission, release times. Some of it is captured at YARN-3292. This
> also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310)
> and/or investing in new tools.
> - HADOOP-11656 Classpath isolation for downstream clients
> - Support for Erasure Codes in HDFS HDFS-7285
> - Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
> - YARN Timeline Service Next generation: YARN-2928. At least branch-merge
> + early peek.
> - Supporting non-exclusive node-labels: YARN-3214
>
> I'm experimenting with more agile 2.7.x releases and would like to
> continue the same by volunteering as the RM for 2.8.x too.
>
> Given the long time we took with 2.7.0, the timeline I am looking at is
> 8-12 weeks. We can pick as many features as they finish along and make a
> more predictable releases instead of holding up releases for ever.
>
> Thoughts?
>
> Thanks
> +Vinod
>
>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Sangjin Lee <sj...@gmail.com>.
On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vinodkv@hortonworks.com
> wrote:
> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
> but clearly a work in progress. Two options here
> — If it is safe to ship it into 2.8 in a disable manner, we can
> get the early code into trunk and all the way int o2.8.
> — If it is not safe, it organically rolls over into 2.9
>
I'll review the changes on YARN-2928 to see what impact it has (if any) if
the timeline service v.2 is disabled.
Another condition for it to make 2.8 is whether the branch will be in a
shape in a couple of weeks such that it adds value for folks that want to
test it. Hopefully it will become clearer soon.
Sangjin
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Wangda Tan <wh...@gmail.com>.
Thanks to Vinod for starting this discussion!
+1 to add YARN-1197 (container resizing) to 2.8.0, it is end-to-end tested.
I'd prefer to push it as an Alpha feature before wilder testing.
And also agree to call first release of 2.8 as an Alpha release according
to the number of new features / code changes.
Regards,
Wangda
On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vinodkv@hortonworks.com
> wrote:
> Agreed on not mixing this with major release discussions.
>
> Okay, I just finished my review of 2.8 content.
>
> A quick summary follows.
>
> Current state of originally planned items
>
> - Nearly Done / Done and so need to close down quickly
> — Support *both* JDK7 and JDK8 runtimes HADOOP-11090
> — Supporting non-exclusive node-labels: YARN-3214: Done, can push as
> an alpha / beta feature
> — Support priorities across applications within the same queue
> YARN-1963: Can push as an alpha / beta feature
>
> - Definitely have to move out into 2.9 and beyond
> — Early work for disk and network isolation in YARN: YARN-2139,
> YARN-2140: Early noise, some critical pieces designed, done but not a lot
> of movement of late
> — Classpath isolation for downstream clients HADOOP-11656: Lots of
> chatter a while ago, not much movement of late
> — Support for Erasure Codes in HDFS HDFS-7285<
> https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the
> interest of stability / bake-in
>
> Non-planned features that went into 2.8.0
>
> — The overall list of new features:
> https://issues.apache.org/jira/issues/?filter=12333994
> — HDFS-6200 Create a separate jar for hdfs-client: Compatible
> improvement - no dimension of alpha/betaness here.
> — HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
> — Stability improvements ready to use:
> — HDFS-8008 Support client-side back off when the datanodes are
> congested
> — HDFS-8009 Signal congestion on the DataNode
> — YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> most of it anyways. Can push as an alpha feature.
> — YARN-1197 Support changing resources of an allocated container: Can
> push as an alpha/beta feature
>
> Items in progress to think about in 2 weeks
>
> — YARN Timeline Service v1.5 - YARN-4233: A short term bridge before
> YARN-2928 comes around. I think this should go in given the tremendous
> activity recently.
> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
> but clearly a work in progress. Two options here
> — If it is safe to ship it into 2.8 in a disable manner, we can
> get the early code into trunk and all the way int o2.8.
> — If it is not safe, it organically rolls over into 2.9
> — Compatibility tools to catch backwards, forwards compatibility
> issues at patch submission, release times. Some of it is captured at
> YARN-3292. This also involves resurrecting jdiff
> (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
>
> This is my plan of action for now in terms of the release itself
>
> - Cut a branch about two weeks from now
> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> - As with 2.7.x series, the first release will still be called as early /
> alpha release in the interest of
> — gaining downstream adoption
> — wider testing,
> — yet reserving our right to fix any inadvertent incompatibilities
> introduced.
>
> If we can get answers on “Items to think about now” during this and next
> week, we will overall be in good shape.
>
> Thoughts?
>
> Thanks
> +Vinod
> PS:As you may have noted above, this time around, I want to do something
> that we’ve always wanted to do, but never explicitly did. I’m calling out
> readiness of each feature as they stand today so we can inform our users
> better of what they can start relying on in production clusters.
>
>
> On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cmccabe@apache.org<mailto:
> cmccabe@apache.org>> wrote:
>
> I think it makes sense to have a 2.8 release since there are a
> tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
> release seems like something we should consider separately since it
> would not have the same compatibility guarantees as a 2.8 release.
> There's a pretty big delta between trunk and 2.8 as well.
>
> cheers,
> Colin
>
> On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cdouglas@apache.org
> <ma...@apache.org>> wrote:
> With two active sustaining branches (2.6, 2.7), what would you think
> of releasing trunk as 3.x instead of pushing 2.8? There are many new
> features (EC, Y1197, etc.), and trunk could be the source of several
> alpha/beta releases before we fork the 3.x line. -C
>
> On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
> <vi...@hortonworks.com>> wrote:
> As you may have noted, 2.8.0 got completely derailed what with 2.7.x and
> the unusually long 2.6.1 release.
>
> With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2
> and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.
>
> I’ll do a quick survey of where the individual features are and the amount
> of content already present in 2.8 and kick-start 2.8.0 process again.
>
> +Vinod
>
>
> On Apr 21, 2015, at 2:39 PM, vinodkv@apache.org<ma...@apache.org>
> wrote:
>
> With 2.7.0 out of the way, and with more maintenance releases to stabilize
> it, I propose we start thinking about 2.8.0.
>
> Here's my first cut of the proposal, will update the Roadmap wiki.
> - Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
> - Compatibility tools to catch backwards, forwards compatibility issues at
> patch submission, release times. Some of it is captured at YARN-3292. This
> also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310)
> and/or investing in new tools.
> - HADOOP-11656 Classpath isolation for downstream clients
> - Support for Erasure Codes in HDFS HDFS-7285
> - Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
> - YARN Timeline Service Next generation: YARN-2928. At least branch-merge
> + early peek.
> - Supporting non-exclusive node-labels: YARN-3214
>
> I'm experimenting with more agile 2.7.x releases and would like to
> continue the same by volunteering as the RM for 2.8.x too.
>
> Given the long time we took with 2.7.0, the timeline I am looking at is
> 8-12 weeks. We can pick as many features as they finish along and make a
> more predictable releases instead of holding up releases for ever.
>
> Thoughts?
>
> Thanks
> +Vinod
>
>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Wangda Tan <wh...@gmail.com>.
Thanks to Vinod for starting this discussion!
+1 to add YARN-1197 (container resizing) to 2.8.0, it is end-to-end tested.
I'd prefer to push it as an Alpha feature before wilder testing.
And also agree to call first release of 2.8 as an Alpha release according
to the number of new features / code changes.
Regards,
Wangda
On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vinodkv@hortonworks.com
> wrote:
> Agreed on not mixing this with major release discussions.
>
> Okay, I just finished my review of 2.8 content.
>
> A quick summary follows.
>
> Current state of originally planned items
>
> - Nearly Done / Done and so need to close down quickly
> — Support *both* JDK7 and JDK8 runtimes HADOOP-11090
> — Supporting non-exclusive node-labels: YARN-3214: Done, can push as
> an alpha / beta feature
> — Support priorities across applications within the same queue
> YARN-1963: Can push as an alpha / beta feature
>
> - Definitely have to move out into 2.9 and beyond
> — Early work for disk and network isolation in YARN: YARN-2139,
> YARN-2140: Early noise, some critical pieces designed, done but not a lot
> of movement of late
> — Classpath isolation for downstream clients HADOOP-11656: Lots of
> chatter a while ago, not much movement of late
> — Support for Erasure Codes in HDFS HDFS-7285<
> https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the
> interest of stability / bake-in
>
> Non-planned features that went into 2.8.0
>
> — The overall list of new features:
> https://issues.apache.org/jira/issues/?filter=12333994
> — HDFS-6200 Create a separate jar for hdfs-client: Compatible
> improvement - no dimension of alpha/betaness here.
> — HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
> — Stability improvements ready to use:
> — HDFS-8008 Support client-side back off when the datanodes are
> congested
> — HDFS-8009 Signal congestion on the DataNode
> — YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> most of it anyways. Can push as an alpha feature.
> — YARN-1197 Support changing resources of an allocated container: Can
> push as an alpha/beta feature
>
> Items in progress to think about in 2 weeks
>
> — YARN Timeline Service v1.5 - YARN-4233: A short term bridge before
> YARN-2928 comes around. I think this should go in given the tremendous
> activity recently.
> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
> but clearly a work in progress. Two options here
> — If it is safe to ship it into 2.8 in a disable manner, we can
> get the early code into trunk and all the way int o2.8.
> — If it is not safe, it organically rolls over into 2.9
> — Compatibility tools to catch backwards, forwards compatibility
> issues at patch submission, release times. Some of it is captured at
> YARN-3292. This also involves resurrecting jdiff
> (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
>
> This is my plan of action for now in terms of the release itself
>
> - Cut a branch about two weeks from now
> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> - As with 2.7.x series, the first release will still be called as early /
> alpha release in the interest of
> — gaining downstream adoption
> — wider testing,
> — yet reserving our right to fix any inadvertent incompatibilities
> introduced.
>
> If we can get answers on “Items to think about now” during this and next
> week, we will overall be in good shape.
>
> Thoughts?
>
> Thanks
> +Vinod
> PS:As you may have noted above, this time around, I want to do something
> that we’ve always wanted to do, but never explicitly did. I’m calling out
> readiness of each feature as they stand today so we can inform our users
> better of what they can start relying on in production clusters.
>
>
> On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cmccabe@apache.org<mailto:
> cmccabe@apache.org>> wrote:
>
> I think it makes sense to have a 2.8 release since there are a
> tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
> release seems like something we should consider separately since it
> would not have the same compatibility guarantees as a 2.8 release.
> There's a pretty big delta between trunk and 2.8 as well.
>
> cheers,
> Colin
>
> On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cdouglas@apache.org
> <ma...@apache.org>> wrote:
> With two active sustaining branches (2.6, 2.7), what would you think
> of releasing trunk as 3.x instead of pushing 2.8? There are many new
> features (EC, Y1197, etc.), and trunk could be the source of several
> alpha/beta releases before we fork the 3.x line. -C
>
> On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
> <vi...@hortonworks.com>> wrote:
> As you may have noted, 2.8.0 got completely derailed what with 2.7.x and
> the unusually long 2.6.1 release.
>
> With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2
> and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.
>
> I’ll do a quick survey of where the individual features are and the amount
> of content already present in 2.8 and kick-start 2.8.0 process again.
>
> +Vinod
>
>
> On Apr 21, 2015, at 2:39 PM, vinodkv@apache.org<ma...@apache.org>
> wrote:
>
> With 2.7.0 out of the way, and with more maintenance releases to stabilize
> it, I propose we start thinking about 2.8.0.
>
> Here's my first cut of the proposal, will update the Roadmap wiki.
> - Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
> - Compatibility tools to catch backwards, forwards compatibility issues at
> patch submission, release times. Some of it is captured at YARN-3292. This
> also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310)
> and/or investing in new tools.
> - HADOOP-11656 Classpath isolation for downstream clients
> - Support for Erasure Codes in HDFS HDFS-7285
> - Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
> - YARN Timeline Service Next generation: YARN-2928. At least branch-merge
> + early peek.
> - Supporting non-exclusive node-labels: YARN-3214
>
> I'm experimenting with more agile 2.7.x releases and would like to
> continue the same by volunteering as the RM for 2.8.x too.
>
> Given the long time we took with 2.7.0, the timeline I am looking at is
> 8-12 weeks. We can pick as many features as they finish along and make a
> more predictable releases instead of holding up releases for ever.
>
> Thoughts?
>
> Thanks
> +Vinod
>
>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Karthik Kambatla <ka...@cloudera.com>.
I am really against the notion of calling x.y.0 releases alpha/beta; it is
very confusing. If we think a release is alpha/beta quality, why not
release it as x.y.0-alpha or x.y.0-beta, and follow it up eventually with
x.y.0 GA.
I am in favor of labeling any of the newer features to be of alpha/beta
quality.
SharedCache is another close to done feature.
On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vinodkv@hortonworks.com
> wrote:
> Agreed on not mixing this with major release discussions.
>
> Okay, I just finished my review of 2.8 content.
>
> A quick summary follows.
>
> Current state of originally planned items
>
> - Nearly Done / Done and so need to close down quickly
> — Support *both* JDK7 and JDK8 runtimes HADOOP-11090
> — Supporting non-exclusive node-labels: YARN-3214: Done, can push as
> an alpha / beta feature
> — Support priorities across applications within the same queue
> YARN-1963: Can push as an alpha / beta feature
>
> - Definitely have to move out into 2.9 and beyond
> — Early work for disk and network isolation in YARN: YARN-2139,
> YARN-2140: Early noise, some critical pieces designed, done but not a lot
> of movement of late
> — Classpath isolation for downstream clients HADOOP-11656: Lots of
> chatter a while ago, not much movement of late
> — Support for Erasure Codes in HDFS HDFS-7285<
> https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the
> interest of stability / bake-in
>
> Non-planned features that went into 2.8.0
>
> — The overall list of new features:
> https://issues.apache.org/jira/issues/?filter=12333994
> — HDFS-6200 Create a separate jar for hdfs-client: Compatible
> improvement - no dimension of alpha/betaness here.
> — HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
> — Stability improvements ready to use:
> — HDFS-8008 Support client-side back off when the datanodes are
> congested
> — HDFS-8009 Signal congestion on the DataNode
> — YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> most of it anyways. Can push as an alpha feature.
> — YARN-1197 Support changing resources of an allocated container: Can
> push as an alpha/beta feature
>
> Items in progress to think about in 2 weeks
>
> — YARN Timeline Service v1.5 - YARN-4233: A short term bridge before
> YARN-2928 comes around. I think this should go in given the tremendous
> activity recently.
> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
> but clearly a work in progress. Two options here
> — If it is safe to ship it into 2.8 in a disable manner, we can
> get the early code into trunk and all the way int o2.8.
> — If it is not safe, it organically rolls over into 2.9
> — Compatibility tools to catch backwards, forwards compatibility
> issues at patch submission, release times. Some of it is captured at
> YARN-3292. This also involves resurrecting jdiff
> (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
>
> This is my plan of action for now in terms of the release itself
>
> - Cut a branch about two weeks from now
> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> - As with 2.7.x series, the first release will still be called as early /
> alpha release in the interest of
> — gaining downstream adoption
> — wider testing,
> — yet reserving our right to fix any inadvertent incompatibilities
> introduced.
>
> If we can get answers on “Items to think about now” during this and next
> week, we will overall be in good shape.
>
> Thoughts?
>
> Thanks
> +Vinod
> PS:As you may have noted above, this time around, I want to do something
> that we’ve always wanted to do, but never explicitly did. I’m calling out
> readiness of each feature as they stand today so we can inform our users
> better of what they can start relying on in production clusters.
>
>
> On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cmccabe@apache.org<mailto:
> cmccabe@apache.org>> wrote:
>
> I think it makes sense to have a 2.8 release since there are a
> tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
> release seems like something we should consider separately since it
> would not have the same compatibility guarantees as a 2.8 release.
> There's a pretty big delta between trunk and 2.8 as well.
>
> cheers,
> Colin
>
> On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cdouglas@apache.org
> <ma...@apache.org>> wrote:
> With two active sustaining branches (2.6, 2.7), what would you think
> of releasing trunk as 3.x instead of pushing 2.8? There are many new
> features (EC, Y1197, etc.), and trunk could be the source of several
> alpha/beta releases before we fork the 3.x line. -C
>
> On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
> <vi...@hortonworks.com>> wrote:
> As you may have noted, 2.8.0 got completely derailed what with 2.7.x and
> the unusually long 2.6.1 release.
>
> With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2
> and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.
>
> I’ll do a quick survey of where the individual features are and the amount
> of content already present in 2.8 and kick-start 2.8.0 process again.
>
> +Vinod
>
>
> On Apr 21, 2015, at 2:39 PM, vinodkv@apache.org<ma...@apache.org>
> wrote:
>
> With 2.7.0 out of the way, and with more maintenance releases to stabilize
> it, I propose we start thinking about 2.8.0.
>
> Here's my first cut of the proposal, will update the Roadmap wiki.
> - Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
> - Compatibility tools to catch backwards, forwards compatibility issues at
> patch submission, release times. Some of it is captured at YARN-3292. This
> also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310)
> and/or investing in new tools.
> - HADOOP-11656 Classpath isolation for downstream clients
> - Support for Erasure Codes in HDFS HDFS-7285
> - Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
> - YARN Timeline Service Next generation: YARN-2928. At least branch-merge
> + early peek.
> - Supporting non-exclusive node-labels: YARN-3214
>
> I'm experimenting with more agile 2.7.x releases and would like to
> continue the same by volunteering as the RM for 2.8.x too.
>
> Given the long time we took with 2.7.0, the timeline I am looking at is
> 8-12 weeks. We can pick as many features as they finish along and make a
> more predictable releases instead of holding up releases for ever.
>
> Thoughts?
>
> Thanks
> +Vinod
>
>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Vinod Kumar Vavilapalli <vi...@apache.org>.
I think we’ve converged at a high level w.r.t 2.8. And as I just sent out an email, I updated the Roadmap wiki reflecting the same: https://wiki.apache.org/hadoop/Roadmap
I plan to create a 2.8 branch EOD today.
The goal for all of us should be to restrict improvements & fixes to only (a) the feature-set documented under 2.8 in the RoadMap wiki and (b) other minor features that are already in 2.8.
Thanks
+Vinod
> On Nov 11, 2015, at 12:13 PM, Vinod Kumar Vavilapalli <vi...@hortonworks.com> wrote:
>
> - Cut a branch about two weeks from now
> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of
> — gaining downstream adoption
> — wider testing,
> — yet reserving our right to fix any inadvertent incompatibilities introduced.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Li Lu <ll...@hortonworks.com>.
On Nov 11, 2015, at 12:13, Vinod Vavilapalli <vi...@hortonworks.com>> wrote:
— YARN Timeline Service v1.5 - YARN-4233: A short term bridge before YARN-2928 comes around. I think this should go in given the tremendous activity recently.
+1, let’s target ATS v1.5 work to 2.8. Most critical patches will be ready for review this week. YARN-4234 also addresses the versioning problem for ATS v1, v1.5, and v2, which will bridge the gap between v1 and v2.
— YARN Timeline Service Next generation: YARN-2928: Lots of momentum, but clearly a work in progress. Two options here
— If it is safe to ship it into 2.8 in a disable manner, we can get the early code into trunk and all the way int o2.8.
Let’s review all changes in YARN-2928 branch and evaluate their impact on the upstream code. Right now in YARN-2928 branch we’ve modified RM/NM/Distributed shell and MR. Before we make a decision on this I think we need to review their effects so that ATS v2 (in its alpha mode) is not affecting existing parts.
— If it is not safe, it organically rolls over into 2.9
Sure.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Li Lu <ll...@hortonworks.com>.
On Nov 11, 2015, at 12:13, Vinod Vavilapalli <vi...@hortonworks.com>> wrote:
— YARN Timeline Service v1.5 - YARN-4233: A short term bridge before YARN-2928 comes around. I think this should go in given the tremendous activity recently.
+1, let’s target ATS v1.5 work to 2.8. Most critical patches will be ready for review this week. YARN-4234 also addresses the versioning problem for ATS v1, v1.5, and v2, which will bridge the gap between v1 and v2.
— YARN Timeline Service Next generation: YARN-2928: Lots of momentum, but clearly a work in progress. Two options here
— If it is safe to ship it into 2.8 in a disable manner, we can get the early code into trunk and all the way int o2.8.
Let’s review all changes in YARN-2928 branch and evaluate their impact on the upstream code. Right now in YARN-2928 branch we’ve modified RM/NM/Distributed shell and MR. Before we make a decision on this I think we need to review their effects so that ATS v2 (in its alpha mode) is not affecting existing parts.
— If it is not safe, it organically rolls over into 2.9
Sure.
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 11, 2015, at 12:13 PM, Vinod Vavilapalli <vi...@hortonworks.com> wrote:
>
> — HDFS-6200 Create a separate jar for hdfs-client: Compatible improvement - no dimension of alpha/betaness here.
IMO: this feels like a massive break in backwards compatibility. Anyone who is looking for specific methods in specific jars are going to have a bad time. Also, it seems as though every week a new issue crops up that is related to this change. Is Slider still having problems with it? The reasoning “well, the pom sets the dependencies so it’s ok” feels like an *extremely weak* reason this wasn’t marked incompatible— it basically makes the assumption that everyone recompiles for every minor release.
> — Compatibility tools to catch backwards, forwards compatibility issues at patch submission, release times. Some of it is captured at YARN-3292. This also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
There has been talk in the past about adding Java ACC support to Yetus.
> Thoughts?
I’d rather see efforts on 3.x than another disastrous 2.x release. The track record is not good. At least a new major will signify that danger looms ahead. We’re already treating 2.x minor releases as effectively major (see the list of incompatible JIRAs) so what different does it make if we do 2.x vs. 3.x anyway?
>
> Thanks
> +Vinod
> PS:As you may have noted above, this time around, I want to do something that we’ve always wanted to do, but never explicitly did. I’m calling out readiness of each feature as they stand today so we can inform our users better of what they can start relying on in production clusters.
… except some of these changes are so deep reaching that even if you don’t use the feature, you’re still impacted by it ...
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Allen Wittenauer <aw...@altiscale.com>.
> On Nov 11, 2015, at 12:13 PM, Vinod Vavilapalli <vi...@hortonworks.com> wrote:
>
> — HDFS-6200 Create a separate jar for hdfs-client: Compatible improvement - no dimension of alpha/betaness here.
IMO: this feels like a massive break in backwards compatibility. Anyone who is looking for specific methods in specific jars are going to have a bad time. Also, it seems as though every week a new issue crops up that is related to this change. Is Slider still having problems with it? The reasoning “well, the pom sets the dependencies so it’s ok” feels like an *extremely weak* reason this wasn’t marked incompatible— it basically makes the assumption that everyone recompiles for every minor release.
> — Compatibility tools to catch backwards, forwards compatibility issues at patch submission, release times. Some of it is captured at YARN-3292. This also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
There has been talk in the past about adding Java ACC support to Yetus.
> Thoughts?
I’d rather see efforts on 3.x than another disastrous 2.x release. The track record is not good. At least a new major will signify that danger looms ahead. We’re already treating 2.x minor releases as effectively major (see the list of incompatible JIRAs) so what different does it make if we do 2.x vs. 3.x anyway?
>
> Thanks
> +Vinod
> PS:As you may have noted above, this time around, I want to do something that we’ve always wanted to do, but never explicitly did. I’m calling out readiness of each feature as they stand today so we can inform our users better of what they can start relying on in production clusters.
… except some of these changes are so deep reaching that even if you don’t use the feature, you’re still impacted by it ...
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Wangda Tan <wh...@gmail.com>.
Thanks to Vinod for starting this discussion!
+1 to add YARN-1197 (container resizing) to 2.8.0, it is end-to-end tested.
I'd prefer to push it as an Alpha feature before wilder testing.
And also agree to call first release of 2.8 as an Alpha release according
to the number of new features / code changes.
Regards,
Wangda
On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vinodkv@hortonworks.com
> wrote:
> Agreed on not mixing this with major release discussions.
>
> Okay, I just finished my review of 2.8 content.
>
> A quick summary follows.
>
> Current state of originally planned items
>
> - Nearly Done / Done and so need to close down quickly
> — Support *both* JDK7 and JDK8 runtimes HADOOP-11090
> — Supporting non-exclusive node-labels: YARN-3214: Done, can push as
> an alpha / beta feature
> — Support priorities across applications within the same queue
> YARN-1963: Can push as an alpha / beta feature
>
> - Definitely have to move out into 2.9 and beyond
> — Early work for disk and network isolation in YARN: YARN-2139,
> YARN-2140: Early noise, some critical pieces designed, done but not a lot
> of movement of late
> — Classpath isolation for downstream clients HADOOP-11656: Lots of
> chatter a while ago, not much movement of late
> — Support for Erasure Codes in HDFS HDFS-7285<
> https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the
> interest of stability / bake-in
>
> Non-planned features that went into 2.8.0
>
> — The overall list of new features:
> https://issues.apache.org/jira/issues/?filter=12333994
> — HDFS-6200 Create a separate jar for hdfs-client: Compatible
> improvement - no dimension of alpha/betaness here.
> — HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
> — Stability improvements ready to use:
> — HDFS-8008 Support client-side back off when the datanodes are
> congested
> — HDFS-8009 Signal congestion on the DataNode
> — YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well
> most of it anyways. Can push as an alpha feature.
> — YARN-1197 Support changing resources of an allocated container: Can
> push as an alpha/beta feature
>
> Items in progress to think about in 2 weeks
>
> — YARN Timeline Service v1.5 - YARN-4233: A short term bridge before
> YARN-2928 comes around. I think this should go in given the tremendous
> activity recently.
> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
> but clearly a work in progress. Two options here
> — If it is safe to ship it into 2.8 in a disable manner, we can
> get the early code into trunk and all the way int o2.8.
> — If it is not safe, it organically rolls over into 2.9
> — Compatibility tools to catch backwards, forwards compatibility
> issues at patch submission, release times. Some of it is captured at
> YARN-3292. This also involves resurrecting jdiff
> (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools.
>
> This is my plan of action for now in terms of the release itself
>
> - Cut a branch about two weeks from now
> - Do an RC mid next month (leaving ~4weeks since branch-cut)
> - As with 2.7.x series, the first release will still be called as early /
> alpha release in the interest of
> — gaining downstream adoption
> — wider testing,
> — yet reserving our right to fix any inadvertent incompatibilities
> introduced.
>
> If we can get answers on “Items to think about now” during this and next
> week, we will overall be in good shape.
>
> Thoughts?
>
> Thanks
> +Vinod
> PS:As you may have noted above, this time around, I want to do something
> that we’ve always wanted to do, but never explicitly did. I’m calling out
> readiness of each feature as they stand today so we can inform our users
> better of what they can start relying on in production clusters.
>
>
> On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cmccabe@apache.org<mailto:
> cmccabe@apache.org>> wrote:
>
> I think it makes sense to have a 2.8 release since there are a
> tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
> release seems like something we should consider separately since it
> would not have the same compatibility guarantees as a 2.8 release.
> There's a pretty big delta between trunk and 2.8 as well.
>
> cheers,
> Colin
>
> On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cdouglas@apache.org
> <ma...@apache.org>> wrote:
> With two active sustaining branches (2.6, 2.7), what would you think
> of releasing trunk as 3.x instead of pushing 2.8? There are many new
> features (EC, Y1197, etc.), and trunk could be the source of several
> alpha/beta releases before we fork the 3.x line. -C
>
> On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
> <vi...@hortonworks.com>> wrote:
> As you may have noted, 2.8.0 got completely derailed what with 2.7.x and
> the unusually long 2.6.1 release.
>
> With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2
> and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.
>
> I’ll do a quick survey of where the individual features are and the amount
> of content already present in 2.8 and kick-start 2.8.0 process again.
>
> +Vinod
>
>
> On Apr 21, 2015, at 2:39 PM, vinodkv@apache.org<ma...@apache.org>
> wrote:
>
> With 2.7.0 out of the way, and with more maintenance releases to stabilize
> it, I propose we start thinking about 2.8.0.
>
> Here's my first cut of the proposal, will update the Roadmap wiki.
> - Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
> - Compatibility tools to catch backwards, forwards compatibility issues at
> patch submission, release times. Some of it is captured at YARN-3292. This
> also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310)
> and/or investing in new tools.
> - HADOOP-11656 Classpath isolation for downstream clients
> - Support for Erasure Codes in HDFS HDFS-7285
> - Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
> - YARN Timeline Service Next generation: YARN-2928. At least branch-merge
> + early peek.
> - Supporting non-exclusive node-labels: YARN-3214
>
> I'm experimenting with more agile 2.7.x releases and would like to
> continue the same by volunteering as the RM for 2.8.x too.
>
> Given the long time we took with 2.7.0, the timeline I am looking at is
> 8-12 weeks. We can pick as many features as they finish along and make a
> more predictable releases instead of holding up releases for ever.
>
> Thoughts?
>
> Thanks
> +Vinod
>
>
>
>
Re: [DISCUSS] Looking to a 2.8.0 release
Posted by Sangjin Lee <sj...@gmail.com>.
On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vinodkv@hortonworks.com
> wrote:
> — YARN Timeline Service Next generation: YARN-2928: Lots of momentum,
> but clearly a work in progress. Two options here
> — If it is safe to ship it into 2.8 in a disable manner, we can
> get the early code into trunk and all the way int o2.8.
> — If it is not safe, it organically rolls over into 2.9
>
I'll review the changes on YARN-2928 to see what impact it has (if any) if
the timeline service v.2 is disabled.
Another condition for it to make 2.8 is whether the branch will be in a
shape in a couple of weeks such that it adds value for folks that want to
test it. Hopefully it will become clearer soon.
Sangjin