You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Arun C Murthy <ac...@hortonworks.com> on 2013/06/26 10:17:55 UTC
[VOTE] Release Apache Hadoop 2.1.0-beta
Folks,
I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
# HDFS Snapshots
# Windows support
# YARN API stabilization
# MapReduce Binary Compatibility with hadoop-1.x
# Substantial amount of integration testing with rest of projects in the ecosystem
The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
The maven artifacts are available via repository.apache.org.
Please try the release and vote; the vote will run for the usual 7 days.
thanks,
Arun
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Roman Shaposhnik <rv...@apache.org>.
On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>
> This release represents a *huge* amount of work done by the community (639 fixes) which
> includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the ecosystem
>
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
+1 (non-binding).
As usual I built the entire Bigtop based on this RC of Hadoop
and ran the battery of smoke tests with secure and unsecure
configuration.
Thanks,
Roman.
P.S. The blocker bug from a previous RC where NN would consistently OOM
seems to have disappeared. I'll update the JIRA accordingly.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Jason Lowe <jl...@yahoo-inc.com>.
I committed MAPREDUCE-5358 to branch-2 but did not commit it to
branch-2.1-beta since it wasn't a blocker and Arun was in the middle of
cutting the release.
Arun, if you feel it's appropriate to put this in branch-2.1-beta feel
free to pull it in or let me know. Thanks!
Jason
On 07/03/2013 02:24 AM, Tsuyoshi OZAWA wrote:
> Hi Arun,
>
> Some bug fixes about MapReduce should be included in next release.
> MAPREDUCE-5221 should be merged because it's a lost feature of MRv2
> compared to MRv1.
> MAPREDUCE-5358 and MAPREDUCE-5335 are very trivial fixes, so it can be
> merged immediately.
>
> # I sent the previous email only to hdfs-dev, so I resending it.
> # I apologize for my mistake.
>
> On Wed, Jun 26, 2013 at 5:17 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>
>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>
>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Jason Lowe <jl...@yahoo-inc.com>.
I committed MAPREDUCE-5358 to branch-2 but did not commit it to
branch-2.1-beta since it wasn't a blocker and Arun was in the middle of
cutting the release.
Arun, if you feel it's appropriate to put this in branch-2.1-beta feel
free to pull it in or let me know. Thanks!
Jason
On 07/03/2013 02:24 AM, Tsuyoshi OZAWA wrote:
> Hi Arun,
>
> Some bug fixes about MapReduce should be included in next release.
> MAPREDUCE-5221 should be merged because it's a lost feature of MRv2
> compared to MRv1.
> MAPREDUCE-5358 and MAPREDUCE-5335 are very trivial fixes, so it can be
> merged immediately.
>
> # I sent the previous email only to hdfs-dev, so I resending it.
> # I apologize for my mistake.
>
> On Wed, Jun 26, 2013 at 5:17 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>
>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>
>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Jason Lowe <jl...@yahoo-inc.com>.
I committed MAPREDUCE-5358 to branch-2 but did not commit it to
branch-2.1-beta since it wasn't a blocker and Arun was in the middle of
cutting the release.
Arun, if you feel it's appropriate to put this in branch-2.1-beta feel
free to pull it in or let me know. Thanks!
Jason
On 07/03/2013 02:24 AM, Tsuyoshi OZAWA wrote:
> Hi Arun,
>
> Some bug fixes about MapReduce should be included in next release.
> MAPREDUCE-5221 should be merged because it's a lost feature of MRv2
> compared to MRv1.
> MAPREDUCE-5358 and MAPREDUCE-5335 are very trivial fixes, so it can be
> merged immediately.
>
> # I sent the previous email only to hdfs-dev, so I resending it.
> # I apologize for my mistake.
>
> On Wed, Jun 26, 2013 at 5:17 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>
>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>
>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Jason Lowe <jl...@yahoo-inc.com>.
I committed MAPREDUCE-5358 to branch-2 but did not commit it to
branch-2.1-beta since it wasn't a blocker and Arun was in the middle of
cutting the release.
Arun, if you feel it's appropriate to put this in branch-2.1-beta feel
free to pull it in or let me know. Thanks!
Jason
On 07/03/2013 02:24 AM, Tsuyoshi OZAWA wrote:
> Hi Arun,
>
> Some bug fixes about MapReduce should be included in next release.
> MAPREDUCE-5221 should be merged because it's a lost feature of MRv2
> compared to MRv1.
> MAPREDUCE-5358 and MAPREDUCE-5335 are very trivial fixes, so it can be
> merged immediately.
>
> # I sent the previous email only to hdfs-dev, so I resending it.
> # I apologize for my mistake.
>
> On Wed, Jun 26, 2013 at 5:17 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>
>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>
>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Tsuyoshi OZAWA <oz...@gmail.com>.
Hi Arun,
Some bug fixes about MapReduce should be included in next release.
MAPREDUCE-5221 should be merged because it's a lost feature of MRv2
compared to MRv1.
MAPREDUCE-5358 and MAPREDUCE-5335 are very trivial fixes, so it can be
merged immediately.
# I sent the previous email only to hdfs-dev, so I resending it.
# I apologize for my mistake.
On Wed, Jun 26, 2013 at 5:17 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>
> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the ecosystem
>
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
--
- Tsuyoshi
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Tsuyoshi OZAWA <oz...@gmail.com>.
Hi Arun,
I think MAPREDUCE-5221 should be merged because it's a missing feature
from MRv1.
Thanks,
Tsuyoshi
On Wed, Jul 3, 2013 at 9:10 AM, Ramya Sunil <ra...@hortonworks.com> wrote:
> -1.
> Some of the cli and distcp system tests which use hftp:// and webhdfs://
> are failing on secure cluster (HDFS-4841 and HDFS-4952/HDFS-4896). This is
> a regression and we need to make sure they work before we call a release.
>
>
> On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
>
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would
>> like to get released.
>>
>> This release represents a *huge* amount of work done by the community (639
>> fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the
>> ecosystem
>>
>> The RC is available at:
>> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here:
>> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Ramya Sunil <ra...@hortonworks.com>.
-1.
Some of the cli and distcp system tests which use hftp:// and webhdfs://
are failing on secure cluster (HDFS-4841 and HDFS-4952/HDFS-4896). This is
a regression and we need to make sure they work before we call a release.
On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would
> like to get released.
>
> This release represents a *huge* amount of work done by the community (639
> fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the
> ecosystem
>
> The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Tsuyoshi OZAWA <oz...@gmail.com>.
Hi Arun,
Some bug fixes about MapReduce should be included in next release.
MAPREDUCE-5221 should be merged because it's a lost feature of MRv2
compared to MRv1.
MAPREDUCE-5358 and MAPREDUCE-5335 are very trivial fixes, so it can be
merged immediately.
# I sent the previous email only to hdfs-dev, so I resending it.
# I apologize for my mistake.
On Wed, Jun 26, 2013 at 5:17 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>
> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the ecosystem
>
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
--
- Tsuyoshi
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Tsuyoshi OZAWA <oz...@gmail.com>.
Hi Arun,
Some bug fixes about MapReduce should be included in next release.
MAPREDUCE-5221 should be merged because it's a lost feature of MRv2
compared to MRv1.
MAPREDUCE-5358 and MAPREDUCE-5335 are very trivial fixes, so it can be
merged immediately.
# I sent the previous email only to hdfs-dev, so I resending it.
# I apologize for my mistake.
On Wed, Jun 26, 2013 at 5:17 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>
> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the ecosystem
>
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
--
- Tsuyoshi
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Azuryy Yu <az...@gmail.com>.
+1 only for HDFS and common.
I compiled using source, now everything I tested is fine.
I configured QJM HA, enabled short circuit read, ran a MRv1 job;
note: my testing cluster using HDFS-2.1.0-beta and common-2.1.0-beta, but
MR is 1.1.2.( I did some change)
On Wed, Jun 26, 2013 at 5:21 PM, Azuryy Yu <az...@gmail.com> wrote:
> Hi Arun,
>
> I downloaded bin-tar and deployed, then I ran a complicated MR jobs, it's
> succeed.
> But when I add the following entries [HDFS-347]:
> <property>
> <name>dfs.client.read.shortcircuit</name>
> <value>true</value>
> </property>
> <property>
> <name>dfs.domain.socket.path</name>
> <value>/home/hadoop/dn_socket</value>
> </property>
>
> I cannot start HDFS, log as following:
>
> 2013-06-26 17:11:42,413 FATAL
> org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
> java.lang.RuntimeException: Although a UNIX domain socket path is
> configured as /home/hadoop/dn_socket, we cannot start a
> localDataXceiverServer because libhadoop cannot be loaded.
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.getDomainPeerServer(DataNode.java:597)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.initDataXceiver(DataNode.java:569)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:740)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:343)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1799)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1714)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1737)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1909)
> at
> org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1930)
> 2013-06-26 17:11:42,425 INFO org.apache.hadoop.util.ExitUtil: Exiting with
> status 1
>
>
>
>
>
>
>
>
>
> On Wed, Jun 26, 2013 at 4:17 PM, Arun C Murthy <ac...@hortonworks.com>wrote:
>
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would
>> like to get released.
>>
>> This release represents a *huge* amount of work done by the community
>> (639 fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the
>> ecosystem
>>
>> The RC is available at:
>> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here:
>> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Azuryy Yu <az...@gmail.com>.
Hi Arun,
I downloaded bin-tar and deployed, then I ran a complicated MR jobs, it's
succeed.
But when I add the following entries [HDFS-347]:
<property>
<name>dfs.client.read.shortcircuit</name>
<value>true</value>
</property>
<property>
<name>dfs.domain.socket.path</name>
<value>/home/hadoop/dn_socket</value>
</property>
I cannot start HDFS, log as following:
2013-06-26 17:11:42,413 FATAL
org.apache.hadoop.hdfs.server.datanode.DataNode: Exception in secureMain
java.lang.RuntimeException: Although a UNIX domain socket path is
configured as /home/hadoop/dn_socket, we cannot start a
localDataXceiverServer because libhadoop cannot be loaded.
at
org.apache.hadoop.hdfs.server.datanode.DataNode.getDomainPeerServer(DataNode.java:597)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.initDataXceiver(DataNode.java:569)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:740)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:343)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1799)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1714)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1737)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1909)
at
org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1930)
2013-06-26 17:11:42,425 INFO org.apache.hadoop.util.ExitUtil: Exiting with
status 1
On Wed, Jun 26, 2013 at 4:17 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would
> like to get released.
>
> This release represents a *huge* amount of work done by the community (639
> fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the
> ecosystem
>
> The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Roman Shaposhnik <rv...@apache.org>.
Hi Arun!
On Fri, Jun 28, 2013 at 2:05 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Thanks Hitesh & Roman, I'll roll RC1 once these are fixed.
Here's the latest update. I've incorporated Collin's patch for HADOOP-9676
and re-ran the tests. A few subtests of TestCLI still fail, but quite a few
pass and on top of that the failing ones no longer make the NN go OOM.
Now, we still need to get to the bottom of why the subset of tests
keep failing, but
at this point I'd feel comfortable committing a change for HADOOP-9676
to branch-2.1 and going from there.
Let me know if I can help some more.
Thanks,
Roman.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Roman Shaposhnik <rv...@apache.org>.
Hi Arun!
On Fri, Jun 28, 2013 at 2:05 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Thanks Hitesh & Roman, I'll roll RC1 once these are fixed.
Here's the latest update. I've incorporated Collin's patch for HADOOP-9676
and re-ran the tests. A few subtests of TestCLI still fail, but quite a few
pass and on top of that the failing ones no longer make the NN go OOM.
Now, we still need to get to the bottom of why the subset of tests
keep failing, but
at this point I'd feel comfortable committing a change for HADOOP-9676
to branch-2.1 and going from there.
Let me know if I can help some more.
Thanks,
Roman.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Arun C Murthy <ac...@hortonworks.com>.
Thanks Hitesh & Roman, I'll roll RC1 once these are fixed.
Arun
On Jun 28, 2013, at 12:25 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> I think it would very useful if somebody more familiar with HDFS
> (Suresh?) could take a look at:
> https://issues.apache.org/jira/browse/HDFS-4940
>
> Suresh was very helpful in unblocking the client side of things, but
> even after I followed his recommendations on the heap size, etc.
> The NN still OOMs it just takes longer.
>
> A quick look at the provided heap dump could help us understand
> whether this is something serious or not.
>
> This is the last bit that blocks the Bigtop side of things as far as
> 2.1 is concerned.
>
> Thanks,
> Roman.
>
> On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>
>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>
>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Arun C Murthy <ac...@hortonworks.com>.
Thanks Hitesh & Roman, I'll roll RC1 once these are fixed.
Arun
On Jun 28, 2013, at 12:25 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> I think it would very useful if somebody more familiar with HDFS
> (Suresh?) could take a look at:
> https://issues.apache.org/jira/browse/HDFS-4940
>
> Suresh was very helpful in unblocking the client side of things, but
> even after I followed his recommendations on the heap size, etc.
> The NN still OOMs it just takes longer.
>
> A quick look at the provided heap dump could help us understand
> whether this is something serious or not.
>
> This is the last bit that blocks the Bigtop side of things as far as
> 2.1 is concerned.
>
> Thanks,
> Roman.
>
> On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>
>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>
>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Arun C Murthy <ac...@hortonworks.com>.
Thanks Hitesh & Roman, I'll roll RC1 once these are fixed.
Arun
On Jun 28, 2013, at 12:25 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> I think it would very useful if somebody more familiar with HDFS
> (Suresh?) could take a look at:
> https://issues.apache.org/jira/browse/HDFS-4940
>
> Suresh was very helpful in unblocking the client side of things, but
> even after I followed his recommendations on the heap size, etc.
> The NN still OOMs it just takes longer.
>
> A quick look at the provided heap dump could help us understand
> whether this is something serious or not.
>
> This is the last bit that blocks the Bigtop side of things as far as
> 2.1 is concerned.
>
> Thanks,
> Roman.
>
> On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>
>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>
>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Arun C Murthy <ac...@hortonworks.com>.
Thanks Hitesh & Roman, I'll roll RC1 once these are fixed.
Arun
On Jun 28, 2013, at 12:25 PM, Roman Shaposhnik <rv...@apache.org> wrote:
> I think it would very useful if somebody more familiar with HDFS
> (Suresh?) could take a look at:
> https://issues.apache.org/jira/browse/HDFS-4940
>
> Suresh was very helpful in unblocking the client side of things, but
> even after I followed his recommendations on the heap size, etc.
> The NN still OOMs it just takes longer.
>
> A quick look at the provided heap dump could help us understand
> whether this is something serious or not.
>
> This is the last bit that blocks the Bigtop side of things as far as
> 2.1 is concerned.
>
> Thanks,
> Roman.
>
> On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>
>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>
>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Roman Shaposhnik <rv...@apache.org>.
I think it would very useful if somebody more familiar with HDFS
(Suresh?) could take a look at:
https://issues.apache.org/jira/browse/HDFS-4940
Suresh was very helpful in unblocking the client side of things, but
even after I followed his recommendations on the heap size, etc.
The NN still OOMs it just takes longer.
A quick look at the provided heap dump could help us understand
whether this is something serious or not.
This is the last bit that blocks the Bigtop side of things as far as
2.1 is concerned.
Thanks,
Roman.
On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>
> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the ecosystem
>
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Steve Loughran <st...@hortonworks.com>.
On 26 June 2013 09:17, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
>
> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
>
did all the poms go up?
[INFO]
------------------------------------------------------------------------
[INFO] Building Hoya: HBase on YARN 0.0.1-SNAPSHOT
[INFO]
------------------------------------------------------------------------
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta.pom
[WARNING] The POM for org.apache.hadoop:hadoop-common:jar:2.1.0-beta is
missing, no dependency information available
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-minicluster/2.1.0-beta/hadoop-minicluster-2.1.0-beta.pom
[WARNING] The POM for org.apache.hadoop:hadoop-minicluster:jar:2.1.0-beta
is missing, no dependency information available
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-hdfs/2.1.0-beta/hadoop-hdfs-2.1.0-beta.pom
[WARNING] The POM for org.apache.hadoop:hadoop-hdfs:jar:2.1.0-beta is
missing, no dependency information available
[WARNING] The POM for org.apache.hadoop:hadoop-common:jar:tests:2.1.0-beta
is missing, no dependency information available
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-yarn-server-common/2.1.0-beta/hadoop-yarn-server-common-2.1.0-beta.pom
[WARNING] The POM for
org.apache.hadoop:hadoop-yarn-server-common:jar:2.1.0-beta is missing, no
dependency information available
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta.jar
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-minicluster/2.1.0-beta/hadoop-minicluster-2.1.0-beta.jar
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-hdfs/2.1.0-beta/hadoop-hdfs-2.1.0-beta.jar
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta-tests.jar
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-yarn-server-common/2.1.0-beta/hadoop-yarn-server-common-2.1.0-beta.jar
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 2.137s
[INFO] Finished at: Wed Jun 26 20:45:22 BST 2013
[INFO] Final Memory: 6M/265M
[INFO]
------------------------------------------------------------------------
[ERROR] Failed to execute goal on project hoya: Could not resolve
dependencies for project org.apache.hadoop.hoya:hoya:jar:0.0.1-SNAPSHOT:
The following artifacts could not be resolved:
org.apache.hadoop:hadoop-common:jar:2.1.0-beta,
org.apache.hadoop:hadoop-minicluster:jar:2.1.0-beta,
org.apache.hadoop:hadoop-hdfs:jar:2.1.0-beta,
org.apache.hadoop:hadoop-common:jar:tests:2.1.0-beta,
org.apache.hadoop:hadoop-yarn-server-common:jar:2.1.0-beta: Could not find
artifact org.apache.hadoop:hadoop-common:jar:2.1.0-beta in central (
http://repo.maven.apache.org/maven2) -> [Help 1]
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Sandy Ryza <sa...@cloudera.com>.
For YARN-791, if we can come to consensus on the correct approach, I can
try to have a patch ASAP.
-Sandy
On Fri, Jun 28, 2013 at 12:03 PM, Hitesh Shah <hi...@apache.org> wrote:
> Hi Arun,
>
> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may
> potentially change the apis. They can implemented in a backward compat
> fashion if committed after 2.1.0. However, this will require adding of
> differently-named apis ( different urls in case of the webservices ) and
> make the current version of the api deprecated and/or obsolete. YARN-818
> which is currently patch available also changes behavior.
>
> Assuming that as soon as 2.1.0 is released, we are to follow a very strict
> backward-compat retaining approach to all user-facing layers (
> api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense
> to try and pull them in and roll out a new RC after they are ready? Perhaps
> Vinod can chime in if he is aware of any other such jiras under YARN-386
> which should be considered compat-related blockers for a 2.1.0 RC.
>
> thanks
> -- Hitesh
>
> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
>
> > Folks,
> >
> > I've created a release candidate (rc0) for hadoop-2.1.0-beta that I
> would like to get released.
> >
> > This release represents a *huge* amount of work done by the community
> (639 fixes) which includes several major advances including:
> > # HDFS Snapshots
> > # Windows support
> > # YARN API stabilization
> > # MapReduce Binary Compatibility with hadoop-1.x
> > # Substantial amount of integration testing with rest of projects in the
> ecosystem
> >
> > The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> > The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
> >
> > The maven artifacts are available via repository.apache.org.
> >
> > Please try the release and vote; the vote will run for the usual 7 days.
> >
> > thanks,
> > Arun
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Roman Shaposhnik <rv...@apache.org>.
On Mon, Jul 29, 2013 at 3:00 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
Great news! I'm back from my vacation (man, those were good 3 weeks!
;-)) and I'll
try to pull the RC into Bigtop and test it ASAP.
Thanks,
Roman.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Karthik Kambatla <ka...@cloudera.com>.
Hi Arun,
Would it be possible to include YARN-1056 in the next RC - it is a
straight-forward config change. I marked it as a blocker for 2.1.0.
Thanks
Karthik
On Thu, Aug 8, 2013 at 8:14 AM, Kihwal Lee <ki...@yahoo-inc.com> wrote:
> Another blocker, HADOOP-9850, has been committed.
>
>
> Kihwal
>
>
> ________________________________
> From: Arun C Murthy <ac...@hortonworks.com>
> To: Daryn Sharp <da...@yahoo-inc.com>
> Cc: "<hd...@hadoop.apache.org>" <hd...@hadoop.apache.org>; "
> mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>; "
> yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; "
> common-dev@hadoop.apache.org" <co...@hadoop.apache.org>
> Sent: Thursday, August 1, 2013 1:30 PM
> Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta
>
>
> Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in -
> I'd appreciate if you could help push the fix in ASAP.
>
> Thanks again!
>
> Arun
>
> On Aug 1, 2013, at 9:38 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
>
> > I broke RPC QOP for integrity and privacy options. :( See blocker
> HADOOP-9816. I think I understand the problem and it shouldn't be hard to
> fix.
> >
> > The bug went unnoticed because sadly there are no unit tests for the QOP
> options, even though it just involves a conf setting.
> >
> > Daryn
> >
> >
> > On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote:
> >
> >> Ok, I think we are close to rc1 now - the last of blockers should be
> committed later today… I'll try and spin RC1 tonight.
> >>
> >> thanks,
> >> Arun
> >>
> >> On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
> >>
> >>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 ..
> This
> >>> bug can easily be reproduced by some HBase tests. I'd like this to be
> >>> considered before we make a beta release. Have spoken about this with
> some
> >>> hdfs folks offline and I am told that it is being worked on.
> >>>
> >>> Thanks
> >>> Devaraj
> >>>
> >>>
> >>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tucu00@gmail.com
> >wrote:
> >>>
> >>>> As I've mentioned in my previous email, if we get YARN-701 in, we
> should
> >>>> also get in the fix for unmanaged AMs in an un-secure setup in
> 2.1.0-beta.
> >>>> Else is a regression of a functionality it is already working.
> >>>>
> >>>> Because of that, to avoid continuing delaying the release, I'm
> suggesting
> >>>> to mention in the release notes the API changes and behavior changes
> that
> >>>> YARN-918 and YARN-701 will bring into the next beta or GA release.
> >>>>
> >>>> thx
> >>>>
> >>>>
> >>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
> >>>> vinodkv@hortonworks.com> wrote:
> >>>>
> >>>>>
> >>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
> >>>>>
> >>>>>> * YARN-701
> >>>>>>
> >>>>>> It should be addressed before a GA release.
> >>>>>>
> >>>>>> Still, as it is this breaks unmanaged AMs and to me
> >>>>>> that would be a blocker for the beta.
> >>>>>>
> >>>>>> YARN-701 and the unmanaged AMs fix should be committed
> >>>>>> in tandem.
> >>>>>>
> >>>>>> * YARN-918
> >>>>>>
> >>>>>> It is a consequence of YARN-701 and depends on it.
> >>>>>
> >>>>>
> >>>>>
> >>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We
> need
> >>>>> both in 2.1.0.
> >>>>>
> >>>>>
> >>>>>
> >>>>>> * YARN-926
> >>>>>>
> >>>>>> It would be nice to have it addressed before GA release.
> >>>>>
> >>>>>
> >>>>> Either ways. I'd get it in sooner than later specifically when we are
> >>>>> trying to replace the old API with the new one.
> >>>>>
> >>>>> Thanks,
> >>>>> +Vino
> >>>>>
> >>>>>
> >>>>
> >>
> >> --
> >> Arun C. Murthy
> >> Hortonworks Inc.
> >> http://hortonworks.com/
> >>
> >>
> >
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Karthik Kambatla <ka...@cloudera.com>.
Hi Arun,
Would it be possible to include YARN-1056 in the next RC - it is a
straight-forward config change. I marked it as a blocker for 2.1.0.
Thanks
Karthik
On Thu, Aug 8, 2013 at 8:14 AM, Kihwal Lee <ki...@yahoo-inc.com> wrote:
> Another blocker, HADOOP-9850, has been committed.
>
>
> Kihwal
>
>
> ________________________________
> From: Arun C Murthy <ac...@hortonworks.com>
> To: Daryn Sharp <da...@yahoo-inc.com>
> Cc: "<hd...@hadoop.apache.org>" <hd...@hadoop.apache.org>; "
> mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>; "
> yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; "
> common-dev@hadoop.apache.org" <co...@hadoop.apache.org>
> Sent: Thursday, August 1, 2013 1:30 PM
> Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta
>
>
> Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in -
> I'd appreciate if you could help push the fix in ASAP.
>
> Thanks again!
>
> Arun
>
> On Aug 1, 2013, at 9:38 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
>
> > I broke RPC QOP for integrity and privacy options. :( See blocker
> HADOOP-9816. I think I understand the problem and it shouldn't be hard to
> fix.
> >
> > The bug went unnoticed because sadly there are no unit tests for the QOP
> options, even though it just involves a conf setting.
> >
> > Daryn
> >
> >
> > On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote:
> >
> >> Ok, I think we are close to rc1 now - the last of blockers should be
> committed later today… I'll try and spin RC1 tonight.
> >>
> >> thanks,
> >> Arun
> >>
> >> On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
> >>
> >>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 ..
> This
> >>> bug can easily be reproduced by some HBase tests. I'd like this to be
> >>> considered before we make a beta release. Have spoken about this with
> some
> >>> hdfs folks offline and I am told that it is being worked on.
> >>>
> >>> Thanks
> >>> Devaraj
> >>>
> >>>
> >>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tucu00@gmail.com
> >wrote:
> >>>
> >>>> As I've mentioned in my previous email, if we get YARN-701 in, we
> should
> >>>> also get in the fix for unmanaged AMs in an un-secure setup in
> 2.1.0-beta.
> >>>> Else is a regression of a functionality it is already working.
> >>>>
> >>>> Because of that, to avoid continuing delaying the release, I'm
> suggesting
> >>>> to mention in the release notes the API changes and behavior changes
> that
> >>>> YARN-918 and YARN-701 will bring into the next beta or GA release.
> >>>>
> >>>> thx
> >>>>
> >>>>
> >>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
> >>>> vinodkv@hortonworks.com> wrote:
> >>>>
> >>>>>
> >>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
> >>>>>
> >>>>>> * YARN-701
> >>>>>>
> >>>>>> It should be addressed before a GA release.
> >>>>>>
> >>>>>> Still, as it is this breaks unmanaged AMs and to me
> >>>>>> that would be a blocker for the beta.
> >>>>>>
> >>>>>> YARN-701 and the unmanaged AMs fix should be committed
> >>>>>> in tandem.
> >>>>>>
> >>>>>> * YARN-918
> >>>>>>
> >>>>>> It is a consequence of YARN-701 and depends on it.
> >>>>>
> >>>>>
> >>>>>
> >>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We
> need
> >>>>> both in 2.1.0.
> >>>>>
> >>>>>
> >>>>>
> >>>>>> * YARN-926
> >>>>>>
> >>>>>> It would be nice to have it addressed before GA release.
> >>>>>
> >>>>>
> >>>>> Either ways. I'd get it in sooner than later specifically when we are
> >>>>> trying to replace the old API with the new one.
> >>>>>
> >>>>> Thanks,
> >>>>> +Vino
> >>>>>
> >>>>>
> >>>>
> >>
> >> --
> >> Arun C. Murthy
> >> Hortonworks Inc.
> >> http://hortonworks.com/
> >>
> >>
> >
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Karthik Kambatla <ka...@cloudera.com>.
Hi Arun,
Would it be possible to include YARN-1056 in the next RC - it is a
straight-forward config change. I marked it as a blocker for 2.1.0.
Thanks
Karthik
On Thu, Aug 8, 2013 at 8:14 AM, Kihwal Lee <ki...@yahoo-inc.com> wrote:
> Another blocker, HADOOP-9850, has been committed.
>
>
> Kihwal
>
>
> ________________________________
> From: Arun C Murthy <ac...@hortonworks.com>
> To: Daryn Sharp <da...@yahoo-inc.com>
> Cc: "<hd...@hadoop.apache.org>" <hd...@hadoop.apache.org>; "
> mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>; "
> yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; "
> common-dev@hadoop.apache.org" <co...@hadoop.apache.org>
> Sent: Thursday, August 1, 2013 1:30 PM
> Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta
>
>
> Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in -
> I'd appreciate if you could help push the fix in ASAP.
>
> Thanks again!
>
> Arun
>
> On Aug 1, 2013, at 9:38 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
>
> > I broke RPC QOP for integrity and privacy options. :( See blocker
> HADOOP-9816. I think I understand the problem and it shouldn't be hard to
> fix.
> >
> > The bug went unnoticed because sadly there are no unit tests for the QOP
> options, even though it just involves a conf setting.
> >
> > Daryn
> >
> >
> > On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote:
> >
> >> Ok, I think we are close to rc1 now - the last of blockers should be
> committed later today… I'll try and spin RC1 tonight.
> >>
> >> thanks,
> >> Arun
> >>
> >> On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
> >>
> >>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 ..
> This
> >>> bug can easily be reproduced by some HBase tests. I'd like this to be
> >>> considered before we make a beta release. Have spoken about this with
> some
> >>> hdfs folks offline and I am told that it is being worked on.
> >>>
> >>> Thanks
> >>> Devaraj
> >>>
> >>>
> >>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tucu00@gmail.com
> >wrote:
> >>>
> >>>> As I've mentioned in my previous email, if we get YARN-701 in, we
> should
> >>>> also get in the fix for unmanaged AMs in an un-secure setup in
> 2.1.0-beta.
> >>>> Else is a regression of a functionality it is already working.
> >>>>
> >>>> Because of that, to avoid continuing delaying the release, I'm
> suggesting
> >>>> to mention in the release notes the API changes and behavior changes
> that
> >>>> YARN-918 and YARN-701 will bring into the next beta or GA release.
> >>>>
> >>>> thx
> >>>>
> >>>>
> >>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
> >>>> vinodkv@hortonworks.com> wrote:
> >>>>
> >>>>>
> >>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
> >>>>>
> >>>>>> * YARN-701
> >>>>>>
> >>>>>> It should be addressed before a GA release.
> >>>>>>
> >>>>>> Still, as it is this breaks unmanaged AMs and to me
> >>>>>> that would be a blocker for the beta.
> >>>>>>
> >>>>>> YARN-701 and the unmanaged AMs fix should be committed
> >>>>>> in tandem.
> >>>>>>
> >>>>>> * YARN-918
> >>>>>>
> >>>>>> It is a consequence of YARN-701 and depends on it.
> >>>>>
> >>>>>
> >>>>>
> >>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We
> need
> >>>>> both in 2.1.0.
> >>>>>
> >>>>>
> >>>>>
> >>>>>> * YARN-926
> >>>>>>
> >>>>>> It would be nice to have it addressed before GA release.
> >>>>>
> >>>>>
> >>>>> Either ways. I'd get it in sooner than later specifically when we are
> >>>>> trying to replace the old API with the new one.
> >>>>>
> >>>>> Thanks,
> >>>>> +Vino
> >>>>>
> >>>>>
> >>>>
> >>
> >> --
> >> Arun C. Murthy
> >> Hortonworks Inc.
> >> http://hortonworks.com/
> >>
> >>
> >
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Kihwal Lee <ki...@yahoo-inc.com>.
Another blocker, HADOOP-9850, has been committed.
Kihwal
________________________________
From: Arun C Murthy <ac...@hortonworks.com>
To: Daryn Sharp <da...@yahoo-inc.com>
Cc: "<hd...@hadoop.apache.org>" <hd...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>; "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>
Sent: Thursday, August 1, 2013 1:30 PM
Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in - I'd appreciate if you could help push the fix in ASAP.
Thanks again!
Arun
On Aug 1, 2013, at 9:38 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
> I broke RPC QOP for integrity and privacy options. :( See blocker HADOOP-9816. I think I understand the problem and it shouldn't be hard to fix.
>
> The bug went unnoticed because sadly there are no unit tests for the QOP options, even though it just involves a conf setting.
>
> Daryn
>
>
> On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote:
>
>> Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
>>
>> thanks,
>> Arun
>>
>> On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
>>
>>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
>>> bug can easily be reproduced by some HBase tests. I'd like this to be
>>> considered before we make a beta release. Have spoken about this with some
>>> hdfs folks offline and I am told that it is being worked on.
>>>
>>> Thanks
>>> Devaraj
>>>
>>>
>>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
>>>
>>>> As I've mentioned in my previous email, if we get YARN-701 in, we should
>>>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
>>>> Else is a regression of a functionality it is already working.
>>>>
>>>> Because of that, to avoid continuing delaying the release, I'm suggesting
>>>> to mention in the release notes the API changes and behavior changes that
>>>> YARN-918 and YARN-701 will bring into the next beta or GA release.
>>>>
>>>> thx
>>>>
>>>>
>>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>>
>>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>>>>>
>>>>>> * YARN-701
>>>>>>
>>>>>> It should be addressed before a GA release.
>>>>>>
>>>>>> Still, as it is this breaks unmanaged AMs and to me
>>>>>> that would be a blocker for the beta.
>>>>>>
>>>>>> YARN-701 and the unmanaged AMs fix should be committed
>>>>>> in tandem.
>>>>>>
>>>>>> * YARN-918
>>>>>>
>>>>>> It is a consequence of YARN-701 and depends on it.
>>>>>
>>>>>
>>>>>
>>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
>>>>> both in 2.1.0.
>>>>>
>>>>>
>>>>>
>>>>>> * YARN-926
>>>>>>
>>>>>> It would be nice to have it addressed before GA release.
>>>>>
>>>>>
>>>>> Either ways. I'd get it in sooner than later specifically when we are
>>>>> trying to replace the old API with the new one.
>>>>>
>>>>> Thanks,
>>>>> +Vino
>>>>>
>>>>>
>>>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Kihwal Lee <ki...@yahoo-inc.com>.
Another blocker, HADOOP-9850, has been committed.
Kihwal
________________________________
From: Arun C Murthy <ac...@hortonworks.com>
To: Daryn Sharp <da...@yahoo-inc.com>
Cc: "<hd...@hadoop.apache.org>" <hd...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>; "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>
Sent: Thursday, August 1, 2013 1:30 PM
Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in - I'd appreciate if you could help push the fix in ASAP.
Thanks again!
Arun
On Aug 1, 2013, at 9:38 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
> I broke RPC QOP for integrity and privacy options. :( See blocker HADOOP-9816. I think I understand the problem and it shouldn't be hard to fix.
>
> The bug went unnoticed because sadly there are no unit tests for the QOP options, even though it just involves a conf setting.
>
> Daryn
>
>
> On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote:
>
>> Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
>>
>> thanks,
>> Arun
>>
>> On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
>>
>>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
>>> bug can easily be reproduced by some HBase tests. I'd like this to be
>>> considered before we make a beta release. Have spoken about this with some
>>> hdfs folks offline and I am told that it is being worked on.
>>>
>>> Thanks
>>> Devaraj
>>>
>>>
>>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
>>>
>>>> As I've mentioned in my previous email, if we get YARN-701 in, we should
>>>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
>>>> Else is a regression of a functionality it is already working.
>>>>
>>>> Because of that, to avoid continuing delaying the release, I'm suggesting
>>>> to mention in the release notes the API changes and behavior changes that
>>>> YARN-918 and YARN-701 will bring into the next beta or GA release.
>>>>
>>>> thx
>>>>
>>>>
>>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>>
>>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>>>>>
>>>>>> * YARN-701
>>>>>>
>>>>>> It should be addressed before a GA release.
>>>>>>
>>>>>> Still, as it is this breaks unmanaged AMs and to me
>>>>>> that would be a blocker for the beta.
>>>>>>
>>>>>> YARN-701 and the unmanaged AMs fix should be committed
>>>>>> in tandem.
>>>>>>
>>>>>> * YARN-918
>>>>>>
>>>>>> It is a consequence of YARN-701 and depends on it.
>>>>>
>>>>>
>>>>>
>>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
>>>>> both in 2.1.0.
>>>>>
>>>>>
>>>>>
>>>>>> * YARN-926
>>>>>>
>>>>>> It would be nice to have it addressed before GA release.
>>>>>
>>>>>
>>>>> Either ways. I'd get it in sooner than later specifically when we are
>>>>> trying to replace the old API with the new one.
>>>>>
>>>>> Thanks,
>>>>> +Vino
>>>>>
>>>>>
>>>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Kihwal Lee <ki...@yahoo-inc.com>.
Another blocker, HADOOP-9850, has been committed.
Kihwal
________________________________
From: Arun C Murthy <ac...@hortonworks.com>
To: Daryn Sharp <da...@yahoo-inc.com>
Cc: "<hd...@hadoop.apache.org>" <hd...@hadoop.apache.org>; "mapreduce-dev@hadoop.apache.org" <ma...@hadoop.apache.org>; "yarn-dev@hadoop.apache.org" <ya...@hadoop.apache.org>; "common-dev@hadoop.apache.org" <co...@hadoop.apache.org>
Sent: Thursday, August 1, 2013 1:30 PM
Subject: Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in - I'd appreciate if you could help push the fix in ASAP.
Thanks again!
Arun
On Aug 1, 2013, at 9:38 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
> I broke RPC QOP for integrity and privacy options. :( See blocker HADOOP-9816. I think I understand the problem and it shouldn't be hard to fix.
>
> The bug went unnoticed because sadly there are no unit tests for the QOP options, even though it just involves a conf setting.
>
> Daryn
>
>
> On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote:
>
>> Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
>>
>> thanks,
>> Arun
>>
>> On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
>>
>>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
>>> bug can easily be reproduced by some HBase tests. I'd like this to be
>>> considered before we make a beta release. Have spoken about this with some
>>> hdfs folks offline and I am told that it is being worked on.
>>>
>>> Thanks
>>> Devaraj
>>>
>>>
>>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
>>>
>>>> As I've mentioned in my previous email, if we get YARN-701 in, we should
>>>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
>>>> Else is a regression of a functionality it is already working.
>>>>
>>>> Because of that, to avoid continuing delaying the release, I'm suggesting
>>>> to mention in the release notes the API changes and behavior changes that
>>>> YARN-918 and YARN-701 will bring into the next beta or GA release.
>>>>
>>>> thx
>>>>
>>>>
>>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>>
>>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>>>>>
>>>>>> * YARN-701
>>>>>>
>>>>>> It should be addressed before a GA release.
>>>>>>
>>>>>> Still, as it is this breaks unmanaged AMs and to me
>>>>>> that would be a blocker for the beta.
>>>>>>
>>>>>> YARN-701 and the unmanaged AMs fix should be committed
>>>>>> in tandem.
>>>>>>
>>>>>> * YARN-918
>>>>>>
>>>>>> It is a consequence of YARN-701 and depends on it.
>>>>>
>>>>>
>>>>>
>>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
>>>>> both in 2.1.0.
>>>>>
>>>>>
>>>>>
>>>>>> * YARN-926
>>>>>>
>>>>>> It would be nice to have it addressed before GA release.
>>>>>
>>>>>
>>>>> Either ways. I'd get it in sooner than later specifically when we are
>>>>> trying to replace the old API with the new one.
>>>>>
>>>>> Thanks,
>>>>> +Vino
>>>>>
>>>>>
>>>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Arun C Murthy <ac...@hortonworks.com>.
Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in - I'd appreciate if you could help push the fix in ASAP.
Thanks again!
Arun
On Aug 1, 2013, at 9:38 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
> I broke RPC QOP for integrity and privacy options. :( See blocker HADOOP-9816. I think I understand the problem and it shouldn't be hard to fix.
>
> The bug went unnoticed because sadly there are no unit tests for the QOP options, even though it just involves a conf setting.
>
> Daryn
>
>
> On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote:
>
>> Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
>>
>> thanks,
>> Arun
>>
>> On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
>>
>>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
>>> bug can easily be reproduced by some HBase tests. I'd like this to be
>>> considered before we make a beta release. Have spoken about this with some
>>> hdfs folks offline and I am told that it is being worked on.
>>>
>>> Thanks
>>> Devaraj
>>>
>>>
>>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
>>>
>>>> As I've mentioned in my previous email, if we get YARN-701 in, we should
>>>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
>>>> Else is a regression of a functionality it is already working.
>>>>
>>>> Because of that, to avoid continuing delaying the release, I'm suggesting
>>>> to mention in the release notes the API changes and behavior changes that
>>>> YARN-918 and YARN-701 will bring into the next beta or GA release.
>>>>
>>>> thx
>>>>
>>>>
>>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>>
>>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>>>>>
>>>>>> * YARN-701
>>>>>>
>>>>>> It should be addressed before a GA release.
>>>>>>
>>>>>> Still, as it is this breaks unmanaged AMs and to me
>>>>>> that would be a blocker for the beta.
>>>>>>
>>>>>> YARN-701 and the unmanaged AMs fix should be committed
>>>>>> in tandem.
>>>>>>
>>>>>> * YARN-918
>>>>>>
>>>>>> It is a consequence of YARN-701 and depends on it.
>>>>>
>>>>>
>>>>>
>>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
>>>>> both in 2.1.0.
>>>>>
>>>>>
>>>>>
>>>>>> * YARN-926
>>>>>>
>>>>>> It would be nice to have it addressed before GA release.
>>>>>
>>>>>
>>>>> Either ways. I'd get it in sooner than later specifically when we are
>>>>> trying to replace the old API with the new one.
>>>>>
>>>>> Thanks,
>>>>> +Vino
>>>>>
>>>>>
>>>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Arun C Murthy <ac...@hortonworks.com>.
Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in - I'd appreciate if you could help push the fix in ASAP.
Thanks again!
Arun
On Aug 1, 2013, at 9:38 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
> I broke RPC QOP for integrity and privacy options. :( See blocker HADOOP-9816. I think I understand the problem and it shouldn't be hard to fix.
>
> The bug went unnoticed because sadly there are no unit tests for the QOP options, even though it just involves a conf setting.
>
> Daryn
>
>
> On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote:
>
>> Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
>>
>> thanks,
>> Arun
>>
>> On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
>>
>>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
>>> bug can easily be reproduced by some HBase tests. I'd like this to be
>>> considered before we make a beta release. Have spoken about this with some
>>> hdfs folks offline and I am told that it is being worked on.
>>>
>>> Thanks
>>> Devaraj
>>>
>>>
>>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
>>>
>>>> As I've mentioned in my previous email, if we get YARN-701 in, we should
>>>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
>>>> Else is a regression of a functionality it is already working.
>>>>
>>>> Because of that, to avoid continuing delaying the release, I'm suggesting
>>>> to mention in the release notes the API changes and behavior changes that
>>>> YARN-918 and YARN-701 will bring into the next beta or GA release.
>>>>
>>>> thx
>>>>
>>>>
>>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>>
>>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>>>>>
>>>>>> * YARN-701
>>>>>>
>>>>>> It should be addressed before a GA release.
>>>>>>
>>>>>> Still, as it is this breaks unmanaged AMs and to me
>>>>>> that would be a blocker for the beta.
>>>>>>
>>>>>> YARN-701 and the unmanaged AMs fix should be committed
>>>>>> in tandem.
>>>>>>
>>>>>> * YARN-918
>>>>>>
>>>>>> It is a consequence of YARN-701 and depends on it.
>>>>>
>>>>>
>>>>>
>>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
>>>>> both in 2.1.0.
>>>>>
>>>>>
>>>>>
>>>>>> * YARN-926
>>>>>>
>>>>>> It would be nice to have it addressed before GA release.
>>>>>
>>>>>
>>>>> Either ways. I'd get it in sooner than later specifically when we are
>>>>> trying to replace the old API with the new one.
>>>>>
>>>>> Thanks,
>>>>> +Vino
>>>>>
>>>>>
>>>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Arun C Murthy <ac...@hortonworks.com>.
Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in - I'd appreciate if you could help push the fix in ASAP.
Thanks again!
Arun
On Aug 1, 2013, at 9:38 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
> I broke RPC QOP for integrity and privacy options. :( See blocker HADOOP-9816. I think I understand the problem and it shouldn't be hard to fix.
>
> The bug went unnoticed because sadly there are no unit tests for the QOP options, even though it just involves a conf setting.
>
> Daryn
>
>
> On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote:
>
>> Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
>>
>> thanks,
>> Arun
>>
>> On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
>>
>>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
>>> bug can easily be reproduced by some HBase tests. I'd like this to be
>>> considered before we make a beta release. Have spoken about this with some
>>> hdfs folks offline and I am told that it is being worked on.
>>>
>>> Thanks
>>> Devaraj
>>>
>>>
>>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
>>>
>>>> As I've mentioned in my previous email, if we get YARN-701 in, we should
>>>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
>>>> Else is a regression of a functionality it is already working.
>>>>
>>>> Because of that, to avoid continuing delaying the release, I'm suggesting
>>>> to mention in the release notes the API changes and behavior changes that
>>>> YARN-918 and YARN-701 will bring into the next beta or GA release.
>>>>
>>>> thx
>>>>
>>>>
>>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>>
>>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>>>>>
>>>>>> * YARN-701
>>>>>>
>>>>>> It should be addressed before a GA release.
>>>>>>
>>>>>> Still, as it is this breaks unmanaged AMs and to me
>>>>>> that would be a blocker for the beta.
>>>>>>
>>>>>> YARN-701 and the unmanaged AMs fix should be committed
>>>>>> in tandem.
>>>>>>
>>>>>> * YARN-918
>>>>>>
>>>>>> It is a consequence of YARN-701 and depends on it.
>>>>>
>>>>>
>>>>>
>>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
>>>>> both in 2.1.0.
>>>>>
>>>>>
>>>>>
>>>>>> * YARN-926
>>>>>>
>>>>>> It would be nice to have it addressed before GA release.
>>>>>
>>>>>
>>>>> Either ways. I'd get it in sooner than later specifically when we are
>>>>> trying to replace the old API with the new one.
>>>>>
>>>>> Thanks,
>>>>> +Vino
>>>>>
>>>>>
>>>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Arun C Murthy <ac...@hortonworks.com>.
Ok, thanks for heads up Daryn. I'll spin an RC2 once HADOOP-9816 gets in - I'd appreciate if you could help push the fix in ASAP.
Thanks again!
Arun
On Aug 1, 2013, at 9:38 AM, Daryn Sharp <da...@yahoo-inc.com> wrote:
> I broke RPC QOP for integrity and privacy options. :( See blocker HADOOP-9816. I think I understand the problem and it shouldn't be hard to fix.
>
> The bug went unnoticed because sadly there are no unit tests for the QOP options, even though it just involves a conf setting.
>
> Daryn
>
>
> On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote:
>
>> Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
>>
>> thanks,
>> Arun
>>
>> On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
>>
>>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
>>> bug can easily be reproduced by some HBase tests. I'd like this to be
>>> considered before we make a beta release. Have spoken about this with some
>>> hdfs folks offline and I am told that it is being worked on.
>>>
>>> Thanks
>>> Devaraj
>>>
>>>
>>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
>>>
>>>> As I've mentioned in my previous email, if we get YARN-701 in, we should
>>>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
>>>> Else is a regression of a functionality it is already working.
>>>>
>>>> Because of that, to avoid continuing delaying the release, I'm suggesting
>>>> to mention in the release notes the API changes and behavior changes that
>>>> YARN-918 and YARN-701 will bring into the next beta or GA release.
>>>>
>>>> thx
>>>>
>>>>
>>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
>>>> vinodkv@hortonworks.com> wrote:
>>>>
>>>>>
>>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>>>>>
>>>>>> * YARN-701
>>>>>>
>>>>>> It should be addressed before a GA release.
>>>>>>
>>>>>> Still, as it is this breaks unmanaged AMs and to me
>>>>>> that would be a blocker for the beta.
>>>>>>
>>>>>> YARN-701 and the unmanaged AMs fix should be committed
>>>>>> in tandem.
>>>>>>
>>>>>> * YARN-918
>>>>>>
>>>>>> It is a consequence of YARN-701 and depends on it.
>>>>>
>>>>>
>>>>>
>>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
>>>>> both in 2.1.0.
>>>>>
>>>>>
>>>>>
>>>>>> * YARN-926
>>>>>>
>>>>>> It would be nice to have it addressed before GA release.
>>>>>
>>>>>
>>>>> Either ways. I'd get it in sooner than later specifically when we are
>>>>> trying to replace the old API with the new one.
>>>>>
>>>>> Thanks,
>>>>> +Vino
>>>>>
>>>>>
>>>>
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Daryn Sharp <da...@yahoo-inc.com>.
I broke RPC QOP for integrity and privacy options. :( See blocker HADOOP-9816. I think I understand the problem and it shouldn't be hard to fix.
The bug went unnoticed because sadly there are no unit tests for the QOP options, even though it just involves a conf setting.
Daryn
On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote:
> Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
>
> thanks,
> Arun
>
> On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
>
>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
>> bug can easily be reproduced by some HBase tests. I'd like this to be
>> considered before we make a beta release. Have spoken about this with some
>> hdfs folks offline and I am told that it is being worked on.
>>
>> Thanks
>> Devaraj
>>
>>
>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
>>
>>> As I've mentioned in my previous email, if we get YARN-701 in, we should
>>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
>>> Else is a regression of a functionality it is already working.
>>>
>>> Because of that, to avoid continuing delaying the release, I'm suggesting
>>> to mention in the release notes the API changes and behavior changes that
>>> YARN-918 and YARN-701 will bring into the next beta or GA release.
>>>
>>> thx
>>>
>>>
>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
>>> vinodkv@hortonworks.com> wrote:
>>>
>>>>
>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>>>>
>>>>> * YARN-701
>>>>>
>>>>> It should be addressed before a GA release.
>>>>>
>>>>> Still, as it is this breaks unmanaged AMs and to me
>>>>> that would be a blocker for the beta.
>>>>>
>>>>> YARN-701 and the unmanaged AMs fix should be committed
>>>>> in tandem.
>>>>>
>>>>> * YARN-918
>>>>>
>>>>> It is a consequence of YARN-701 and depends on it.
>>>>
>>>>
>>>>
>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
>>>> both in 2.1.0.
>>>>
>>>>
>>>>
>>>>> * YARN-926
>>>>>
>>>>> It would be nice to have it addressed before GA release.
>>>>
>>>>
>>>> Either ways. I'd get it in sooner than later specifically when we are
>>>> trying to replace the old API with the new one.
>>>>
>>>> Thanks,
>>>> +Vino
>>>>
>>>>
>>>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Daryn Sharp <da...@yahoo-inc.com>.
I broke RPC QOP for integrity and privacy options. :( See blocker HADOOP-9816. I think I understand the problem and it shouldn't be hard to fix.
The bug went unnoticed because sadly there are no unit tests for the QOP options, even though it just involves a conf setting.
Daryn
On Jul 29, 2013, at 5:00 PM, Arun C Murthy wrote:
> Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
>
> thanks,
> Arun
>
> On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
>
>> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
>> bug can easily be reproduced by some HBase tests. I'd like this to be
>> considered before we make a beta release. Have spoken about this with some
>> hdfs folks offline and I am told that it is being worked on.
>>
>> Thanks
>> Devaraj
>>
>>
>> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
>>
>>> As I've mentioned in my previous email, if we get YARN-701 in, we should
>>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
>>> Else is a regression of a functionality it is already working.
>>>
>>> Because of that, to avoid continuing delaying the release, I'm suggesting
>>> to mention in the release notes the API changes and behavior changes that
>>> YARN-918 and YARN-701 will bring into the next beta or GA release.
>>>
>>> thx
>>>
>>>
>>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
>>> vinodkv@hortonworks.com> wrote:
>>>
>>>>
>>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>>>>
>>>>> * YARN-701
>>>>>
>>>>> It should be addressed before a GA release.
>>>>>
>>>>> Still, as it is this breaks unmanaged AMs and to me
>>>>> that would be a blocker for the beta.
>>>>>
>>>>> YARN-701 and the unmanaged AMs fix should be committed
>>>>> in tandem.
>>>>>
>>>>> * YARN-918
>>>>>
>>>>> It is a consequence of YARN-701 and depends on it.
>>>>
>>>>
>>>>
>>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
>>>> both in 2.1.0.
>>>>
>>>>
>>>>
>>>>> * YARN-926
>>>>>
>>>>> It would be nice to have it addressed before GA release.
>>>>
>>>>
>>>> Either ways. I'd get it in sooner than later specifically when we are
>>>> trying to replace the old API with the new one.
>>>>
>>>> Thanks,
>>>> +Vino
>>>>
>>>>
>>>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Roman Shaposhnik <rv...@apache.org>.
On Mon, Jul 29, 2013 at 3:00 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
Great news! I'm back from my vacation (man, those were good 3 weeks!
;-)) and I'll
try to pull the RC into Bigtop and test it ASAP.
Thanks,
Roman.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Arun C Murthy <ac...@hortonworks.com>.
Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
thanks,
Arun
On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
> bug can easily be reproduced by some HBase tests. I'd like this to be
> considered before we make a beta release. Have spoken about this with some
> hdfs folks offline and I am told that it is being worked on.
>
> Thanks
> Devaraj
>
>
> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
>
>> As I've mentioned in my previous email, if we get YARN-701 in, we should
>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
>> Else is a regression of a functionality it is already working.
>>
>> Because of that, to avoid continuing delaying the release, I'm suggesting
>> to mention in the release notes the API changes and behavior changes that
>> YARN-918 and YARN-701 will bring into the next beta or GA release.
>>
>> thx
>>
>>
>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
>> vinodkv@hortonworks.com> wrote:
>>
>>>
>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>>>
>>>> * YARN-701
>>>>
>>>> It should be addressed before a GA release.
>>>>
>>>> Still, as it is this breaks unmanaged AMs and to me
>>>> that would be a blocker for the beta.
>>>>
>>>> YARN-701 and the unmanaged AMs fix should be committed
>>>> in tandem.
>>>>
>>>> * YARN-918
>>>>
>>>> It is a consequence of YARN-701 and depends on it.
>>>
>>>
>>>
>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
>>> both in 2.1.0.
>>>
>>>
>>>
>>>> * YARN-926
>>>>
>>>> It would be nice to have it addressed before GA release.
>>>
>>>
>>> Either ways. I'd get it in sooner than later specifically when we are
>>> trying to replace the old API with the new one.
>>>
>>> Thanks,
>>> +Vino
>>>
>>>
>>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Arun C Murthy <ac...@hortonworks.com>.
Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
thanks,
Arun
On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
> bug can easily be reproduced by some HBase tests. I'd like this to be
> considered before we make a beta release. Have spoken about this with some
> hdfs folks offline and I am told that it is being worked on.
>
> Thanks
> Devaraj
>
>
> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
>
>> As I've mentioned in my previous email, if we get YARN-701 in, we should
>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
>> Else is a regression of a functionality it is already working.
>>
>> Because of that, to avoid continuing delaying the release, I'm suggesting
>> to mention in the release notes the API changes and behavior changes that
>> YARN-918 and YARN-701 will bring into the next beta or GA release.
>>
>> thx
>>
>>
>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
>> vinodkv@hortonworks.com> wrote:
>>
>>>
>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>>>
>>>> * YARN-701
>>>>
>>>> It should be addressed before a GA release.
>>>>
>>>> Still, as it is this breaks unmanaged AMs and to me
>>>> that would be a blocker for the beta.
>>>>
>>>> YARN-701 and the unmanaged AMs fix should be committed
>>>> in tandem.
>>>>
>>>> * YARN-918
>>>>
>>>> It is a consequence of YARN-701 and depends on it.
>>>
>>>
>>>
>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
>>> both in 2.1.0.
>>>
>>>
>>>
>>>> * YARN-926
>>>>
>>>> It would be nice to have it addressed before GA release.
>>>
>>>
>>> Either ways. I'd get it in sooner than later specifically when we are
>>> trying to replace the old API with the new one.
>>>
>>> Thanks,
>>> +Vino
>>>
>>>
>>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Arun C Murthy <ac...@hortonworks.com>.
Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
thanks,
Arun
On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
> bug can easily be reproduced by some HBase tests. I'd like this to be
> considered before we make a beta release. Have spoken about this with some
> hdfs folks offline and I am told that it is being worked on.
>
> Thanks
> Devaraj
>
>
> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
>
>> As I've mentioned in my previous email, if we get YARN-701 in, we should
>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
>> Else is a regression of a functionality it is already working.
>>
>> Because of that, to avoid continuing delaying the release, I'm suggesting
>> to mention in the release notes the API changes and behavior changes that
>> YARN-918 and YARN-701 will bring into the next beta or GA release.
>>
>> thx
>>
>>
>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
>> vinodkv@hortonworks.com> wrote:
>>
>>>
>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>>>
>>>> * YARN-701
>>>>
>>>> It should be addressed before a GA release.
>>>>
>>>> Still, as it is this breaks unmanaged AMs and to me
>>>> that would be a blocker for the beta.
>>>>
>>>> YARN-701 and the unmanaged AMs fix should be committed
>>>> in tandem.
>>>>
>>>> * YARN-918
>>>>
>>>> It is a consequence of YARN-701 and depends on it.
>>>
>>>
>>>
>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
>>> both in 2.1.0.
>>>
>>>
>>>
>>>> * YARN-926
>>>>
>>>> It would be nice to have it addressed before GA release.
>>>
>>>
>>> Either ways. I'd get it in sooner than later specifically when we are
>>> trying to replace the old API with the new one.
>>>
>>> Thanks,
>>> +Vino
>>>
>>>
>>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Arun C Murthy <ac...@hortonworks.com>.
Ok, I think we are close to rc1 now - the last of blockers should be committed later today… I'll try and spin RC1 tonight.
thanks,
Arun
On Jul 21, 2013, at 12:43 AM, Devaraj Das <dd...@hortonworks.com> wrote:
> I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
> bug can easily be reproduced by some HBase tests. I'd like this to be
> considered before we make a beta release. Have spoken about this with some
> hdfs folks offline and I am told that it is being worked on.
>
> Thanks
> Devaraj
>
>
> On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
>
>> As I've mentioned in my previous email, if we get YARN-701 in, we should
>> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
>> Else is a regression of a functionality it is already working.
>>
>> Because of that, to avoid continuing delaying the release, I'm suggesting
>> to mention in the release notes the API changes and behavior changes that
>> YARN-918 and YARN-701 will bring into the next beta or GA release.
>>
>> thx
>>
>>
>> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
>> vinodkv@hortonworks.com> wrote:
>>
>>>
>>> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>>>
>>>> * YARN-701
>>>>
>>>> It should be addressed before a GA release.
>>>>
>>>> Still, as it is this breaks unmanaged AMs and to me
>>>> that would be a blocker for the beta.
>>>>
>>>> YARN-701 and the unmanaged AMs fix should be committed
>>>> in tandem.
>>>>
>>>> * YARN-918
>>>>
>>>> It is a consequence of YARN-701 and depends on it.
>>>
>>>
>>>
>>> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
>>> both in 2.1.0.
>>>
>>>
>>>
>>>> * YARN-926
>>>>
>>>> It would be nice to have it addressed before GA release.
>>>
>>>
>>> Either ways. I'd get it in sooner than later specifically when we are
>>> trying to replace the old API with the new one.
>>>
>>> Thanks,
>>> +Vino
>>>
>>>
>>
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Devaraj Das <dd...@hortonworks.com>.
I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
bug can easily be reproduced by some HBase tests. I'd like this to be
considered before we make a beta release. Have spoken about this with some
hdfs folks offline and I am told that it is being worked on.
Thanks
Devaraj
On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
> As I've mentioned in my previous email, if we get YARN-701 in, we should
> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
> Else is a regression of a functionality it is already working.
>
> Because of that, to avoid continuing delaying the release, I'm suggesting
> to mention in the release notes the API changes and behavior changes that
> YARN-918 and YARN-701 will bring into the next beta or GA release.
>
> thx
>
>
> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
>
> >
> > On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
> >
> > > * YARN-701
> > >
> > > It should be addressed before a GA release.
> > >
> > > Still, as it is this breaks unmanaged AMs and to me
> > > that would be a blocker for the beta.
> > >
> > > YARN-701 and the unmanaged AMs fix should be committed
> > > in tandem.
> > >
> > > * YARN-918
> > >
> > > It is a consequence of YARN-701 and depends on it.
> >
> >
> >
> > YARN-918 is an API change. And YARN-701 is a behaviour change. We need
> > both in 2.1.0.
> >
> >
> >
> > > * YARN-926
> > >
> > > It would be nice to have it addressed before GA release.
> >
> >
> > Either ways. I'd get it in sooner than later specifically when we are
> > trying to replace the old API with the new one.
> >
> > Thanks,
> > +Vino
> >
> >
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Devaraj Das <dd...@hortonworks.com>.
I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
bug can easily be reproduced by some HBase tests. I'd like this to be
considered before we make a beta release. Have spoken about this with some
hdfs folks offline and I am told that it is being worked on.
Thanks
Devaraj
On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
> As I've mentioned in my previous email, if we get YARN-701 in, we should
> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
> Else is a regression of a functionality it is already working.
>
> Because of that, to avoid continuing delaying the release, I'm suggesting
> to mention in the release notes the API changes and behavior changes that
> YARN-918 and YARN-701 will bring into the next beta or GA release.
>
> thx
>
>
> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
>
> >
> > On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
> >
> > > * YARN-701
> > >
> > > It should be addressed before a GA release.
> > >
> > > Still, as it is this breaks unmanaged AMs and to me
> > > that would be a blocker for the beta.
> > >
> > > YARN-701 and the unmanaged AMs fix should be committed
> > > in tandem.
> > >
> > > * YARN-918
> > >
> > > It is a consequence of YARN-701 and depends on it.
> >
> >
> >
> > YARN-918 is an API change. And YARN-701 is a behaviour change. We need
> > both in 2.1.0.
> >
> >
> >
> > > * YARN-926
> > >
> > > It would be nice to have it addressed before GA release.
> >
> >
> > Either ways. I'd get it in sooner than later specifically when we are
> > trying to replace the old API with the new one.
> >
> > Thanks,
> > +Vino
> >
> >
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Devaraj Das <dd...@hortonworks.com>.
I have just raised https://issues.apache.org/jira/browse/HDFS-5016 .. This
bug can easily be reproduced by some HBase tests. I'd like this to be
considered before we make a beta release. Have spoken about this with some
hdfs folks offline and I am told that it is being worked on.
Thanks
Devaraj
On Wed, Jul 17, 2013 at 4:25 PM, Alejandro Abdelnur <tu...@gmail.com>wrote:
> As I've mentioned in my previous email, if we get YARN-701 in, we should
> also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
> Else is a regression of a functionality it is already working.
>
> Because of that, to avoid continuing delaying the release, I'm suggesting
> to mention in the release notes the API changes and behavior changes that
> YARN-918 and YARN-701 will bring into the next beta or GA release.
>
> thx
>
>
> On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
> vinodkv@hortonworks.com> wrote:
>
> >
> > On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
> >
> > > * YARN-701
> > >
> > > It should be addressed before a GA release.
> > >
> > > Still, as it is this breaks unmanaged AMs and to me
> > > that would be a blocker for the beta.
> > >
> > > YARN-701 and the unmanaged AMs fix should be committed
> > > in tandem.
> > >
> > > * YARN-918
> > >
> > > It is a consequence of YARN-701 and depends on it.
> >
> >
> >
> > YARN-918 is an API change. And YARN-701 is a behaviour change. We need
> > both in 2.1.0.
> >
> >
> >
> > > * YARN-926
> > >
> > > It would be nice to have it addressed before GA release.
> >
> >
> > Either ways. I'd get it in sooner than later specifically when we are
> > trying to replace the old API with the new one.
> >
> > Thanks,
> > +Vino
> >
> >
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Alejandro Abdelnur <tu...@gmail.com>.
As I've mentioned in my previous email, if we get YARN-701 in, we should
also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
Else is a regression of a functionality it is already working.
Because of that, to avoid continuing delaying the release, I'm suggesting
to mention in the release notes the API changes and behavior changes that
YARN-918 and YARN-701 will bring into the next beta or GA release.
thx
On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
>
> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>
> > * YARN-701
> >
> > It should be addressed before a GA release.
> >
> > Still, as it is this breaks unmanaged AMs and to me
> > that would be a blocker for the beta.
> >
> > YARN-701 and the unmanaged AMs fix should be committed
> > in tandem.
> >
> > * YARN-918
> >
> > It is a consequence of YARN-701 and depends on it.
>
>
>
> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
> both in 2.1.0.
>
>
>
> > * YARN-926
> >
> > It would be nice to have it addressed before GA release.
>
>
> Either ways. I'd get it in sooner than later specifically when we are
> trying to replace the old API with the new one.
>
> Thanks,
> +Vino
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Alejandro Abdelnur <tu...@gmail.com>.
As I've mentioned in my previous email, if we get YARN-701 in, we should
also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
Else is a regression of a functionality it is already working.
Because of that, to avoid continuing delaying the release, I'm suggesting
to mention in the release notes the API changes and behavior changes that
YARN-918 and YARN-701 will bring into the next beta or GA release.
thx
On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
>
> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>
> > * YARN-701
> >
> > It should be addressed before a GA release.
> >
> > Still, as it is this breaks unmanaged AMs and to me
> > that would be a blocker for the beta.
> >
> > YARN-701 and the unmanaged AMs fix should be committed
> > in tandem.
> >
> > * YARN-918
> >
> > It is a consequence of YARN-701 and depends on it.
>
>
>
> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
> both in 2.1.0.
>
>
>
> > * YARN-926
> >
> > It would be nice to have it addressed before GA release.
>
>
> Either ways. I'd get it in sooner than later specifically when we are
> trying to replace the old API with the new one.
>
> Thanks,
> +Vino
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Alejandro Abdelnur <tu...@gmail.com>.
As I've mentioned in my previous email, if we get YARN-701 in, we should
also get in the fix for unmanaged AMs in an un-secure setup in 2.1.0-beta.
Else is a regression of a functionality it is already working.
Because of that, to avoid continuing delaying the release, I'm suggesting
to mention in the release notes the API changes and behavior changes that
YARN-918 and YARN-701 will bring into the next beta or GA release.
thx
On Wed, Jul 17, 2013 at 4:14 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
>
> On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
>
> > * YARN-701
> >
> > It should be addressed before a GA release.
> >
> > Still, as it is this breaks unmanaged AMs and to me
> > that would be a blocker for the beta.
> >
> > YARN-701 and the unmanaged AMs fix should be committed
> > in tandem.
> >
> > * YARN-918
> >
> > It is a consequence of YARN-701 and depends on it.
>
>
>
> YARN-918 is an API change. And YARN-701 is a behaviour change. We need
> both in 2.1.0.
>
>
>
> > * YARN-926
> >
> > It would be nice to have it addressed before GA release.
>
>
> Either ways. I'd get it in sooner than later specifically when we are
> trying to replace the old API with the new one.
>
> Thanks,
> +Vino
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
> * YARN-701
>
> It should be addressed before a GA release.
>
> Still, as it is this breaks unmanaged AMs and to me
> that would be a blocker for the beta.
>
> YARN-701 and the unmanaged AMs fix should be committed
> in tandem.
>
> * YARN-918
>
> It is a consequence of YARN-701 and depends on it.
YARN-918 is an API change. And YARN-701 is a behaviour change. We need both in 2.1.0.
> * YARN-926
>
> It would be nice to have it addressed before GA release.
Either ways. I'd get it in sooner than later specifically when we are trying to replace the old API with the new one.
Thanks,
+Vino
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
> * YARN-701
>
> It should be addressed before a GA release.
>
> Still, as it is this breaks unmanaged AMs and to me
> that would be a blocker for the beta.
>
> YARN-701 and the unmanaged AMs fix should be committed
> in tandem.
>
> * YARN-918
>
> It is a consequence of YARN-701 and depends on it.
YARN-918 is an API change. And YARN-701 is a behaviour change. We need both in 2.1.0.
> * YARN-926
>
> It would be nice to have it addressed before GA release.
Either ways. I'd get it in sooner than later specifically when we are trying to replace the old API with the new one.
Thanks,
+Vino
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
> * YARN-701
>
> It should be addressed before a GA release.
>
> Still, as it is this breaks unmanaged AMs and to me
> that would be a blocker for the beta.
>
> YARN-701 and the unmanaged AMs fix should be committed
> in tandem.
>
> * YARN-918
>
> It is a consequence of YARN-701 and depends on it.
YARN-918 is an API change. And YARN-701 is a behaviour change. We need both in 2.1.0.
> * YARN-926
>
> It would be nice to have it addressed before GA release.
Either ways. I'd get it in sooner than later specifically when we are trying to replace the old API with the new one.
Thanks,
+Vino
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
On Jul 17, 2013, at 1:04 PM, Alejandro Abdelnur wrote:
> * YARN-701
>
> It should be addressed before a GA release.
>
> Still, as it is this breaks unmanaged AMs and to me
> that would be a blocker for the beta.
>
> YARN-701 and the unmanaged AMs fix should be committed
> in tandem.
>
> * YARN-918
>
> It is a consequence of YARN-701 and depends on it.
YARN-918 is an API change. And YARN-701 is a behaviour change. We need both in 2.1.0.
> * YARN-926
>
> It would be nice to have it addressed before GA release.
Either ways. I'd get it in sooner than later specifically when we are trying to replace the old API with the new one.
Thanks,
+Vino
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Vinod,
Thanks for reviving this thread.
The current blockers are:
https://issues.apache.org/jira/issues/?jql=project%20in%20(hadoop%2C%20mapreduce%2C%20hdfs%2C%20yarn)%20and%20status%20in%20(open%2C%20'patch%20available')%20and%20priority%20%3D%20blocker%20and%20%22Target%20Version%2Fs%22%20%3D%20%222.1.0-beta%22
By looking at them I don't see they are necessary blockers for a beta
release.
* HADOOP-9688 & HADOOP-9698
They definitely have to be addressed before a GA
release.
* YARN-701
It should be addressed before a GA release.
Still, as it is this breaks unmanaged AMs and to me
that would be a blocker for the beta.
YARN-701 and the unmanaged AMs fix should be committed
in tandem.
* YARN-918
It is a consequence of YARN-701 and depends on it.
* YARN-926
It would be nice to have it addressed before GA release.
We could do a beta with what we have at the moment in branch-2 and have a
special release note indicating API changes coming in the next beta/GA
release as part of YARN-918 & YARN-926.
IMO, we should move forward with the beta release with the current state.
Else we'll continue delaying it and adding more things that break/change
things.
Thanks.
Alejandro
On Wed, Jul 17, 2013 at 12:24 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
>
> Looks like this RC has gone stale and lots of bug fixes went into 2.1 and
> 2.1.0 branches and there are 4-5 outstanding blockers. And from what I see
> in CHANGES.txt files there seems to be a confusion about which branch to
> get in what.
>
> I'm blowing off the current 2.1.0 release branch so that we can create a
> fresh release branch and call voting on that. I'll fix CHANGES.txt entries
> as well as JIRA fix version for bugs committed recently if there are
> inconsistencies.
>
> Let me know if something is amiss while I do this.
>
> Thanks,
> +Vinod
>
> On Jul 3, 2013, at 11:06 AM, Vinod Kumar Vavilapalli wrote:
>
> >
> > We should get these in, looking at them now.
> >
> > Thanks,
> > +Vinod
> >
> > On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote:
> >
> >> Hi Arun,
> >>
> >> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may
> potentially change the apis. They can implemented in a backward compat
> fashion if committed after 2.1.0. However, this will require adding of
> differently-named apis ( different urls in case of the webservices ) and
> make the current version of the api deprecated and/or obsolete. YARN-818
> which is currently patch available also changes behavior.
> >>
> >> Assuming that as soon as 2.1.0 is released, we are to follow a very
> strict backward-compat retaining approach to all user-facing layers (
> api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense
> to try and pull them in and roll out a new RC after they are ready? Perhaps
> Vinod can chime in if he is aware of any other such jiras under YARN-386
> which should be considered compat-related blockers for a 2.1.0 RC.
> >>
> >> thanks
> >> -- Hitesh
> >>
> >> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
> >>
> >>> Folks,
> >>>
> >>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I
> would like to get released.
> >>>
> >>> This release represents a *huge* amount of work done by the community
> (639 fixes) which includes several major advances including:
> >>> # HDFS Snapshots
> >>> # Windows support
> >>> # YARN API stabilization
> >>> # MapReduce Binary Compatibility with hadoop-1.x
> >>> # Substantial amount of integration testing with rest of projects in
> the ecosystem
> >>>
> >>> The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> >>> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
> >>>
> >>> The maven artifacts are available via repository.apache.org.
> >>>
> >>> Please try the release and vote; the vote will run for the usual 7
> days.
> >>>
> >>> thanks,
> >>> Arun
> >>>
> >>> --
> >>> Arun C. Murthy
> >>> Hortonworks Inc.
> >>> http://hortonworks.com/
> >>>
> >>>
> >>
> >
>
>
--
Alejandro
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Vinod,
Thanks for reviving this thread.
The current blockers are:
https://issues.apache.org/jira/issues/?jql=project%20in%20(hadoop%2C%20mapreduce%2C%20hdfs%2C%20yarn)%20and%20status%20in%20(open%2C%20'patch%20available')%20and%20priority%20%3D%20blocker%20and%20%22Target%20Version%2Fs%22%20%3D%20%222.1.0-beta%22
By looking at them I don't see they are necessary blockers for a beta
release.
* HADOOP-9688 & HADOOP-9698
They definitely have to be addressed before a GA
release.
* YARN-701
It should be addressed before a GA release.
Still, as it is this breaks unmanaged AMs and to me
that would be a blocker for the beta.
YARN-701 and the unmanaged AMs fix should be committed
in tandem.
* YARN-918
It is a consequence of YARN-701 and depends on it.
* YARN-926
It would be nice to have it addressed before GA release.
We could do a beta with what we have at the moment in branch-2 and have a
special release note indicating API changes coming in the next beta/GA
release as part of YARN-918 & YARN-926.
IMO, we should move forward with the beta release with the current state.
Else we'll continue delaying it and adding more things that break/change
things.
Thanks.
Alejandro
On Wed, Jul 17, 2013 at 12:24 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
>
> Looks like this RC has gone stale and lots of bug fixes went into 2.1 and
> 2.1.0 branches and there are 4-5 outstanding blockers. And from what I see
> in CHANGES.txt files there seems to be a confusion about which branch to
> get in what.
>
> I'm blowing off the current 2.1.0 release branch so that we can create a
> fresh release branch and call voting on that. I'll fix CHANGES.txt entries
> as well as JIRA fix version for bugs committed recently if there are
> inconsistencies.
>
> Let me know if something is amiss while I do this.
>
> Thanks,
> +Vinod
>
> On Jul 3, 2013, at 11:06 AM, Vinod Kumar Vavilapalli wrote:
>
> >
> > We should get these in, looking at them now.
> >
> > Thanks,
> > +Vinod
> >
> > On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote:
> >
> >> Hi Arun,
> >>
> >> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may
> potentially change the apis. They can implemented in a backward compat
> fashion if committed after 2.1.0. However, this will require adding of
> differently-named apis ( different urls in case of the webservices ) and
> make the current version of the api deprecated and/or obsolete. YARN-818
> which is currently patch available also changes behavior.
> >>
> >> Assuming that as soon as 2.1.0 is released, we are to follow a very
> strict backward-compat retaining approach to all user-facing layers (
> api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense
> to try and pull them in and roll out a new RC after they are ready? Perhaps
> Vinod can chime in if he is aware of any other such jiras under YARN-386
> which should be considered compat-related blockers for a 2.1.0 RC.
> >>
> >> thanks
> >> -- Hitesh
> >>
> >> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
> >>
> >>> Folks,
> >>>
> >>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I
> would like to get released.
> >>>
> >>> This release represents a *huge* amount of work done by the community
> (639 fixes) which includes several major advances including:
> >>> # HDFS Snapshots
> >>> # Windows support
> >>> # YARN API stabilization
> >>> # MapReduce Binary Compatibility with hadoop-1.x
> >>> # Substantial amount of integration testing with rest of projects in
> the ecosystem
> >>>
> >>> The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> >>> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
> >>>
> >>> The maven artifacts are available via repository.apache.org.
> >>>
> >>> Please try the release and vote; the vote will run for the usual 7
> days.
> >>>
> >>> thanks,
> >>> Arun
> >>>
> >>> --
> >>> Arun C. Murthy
> >>> Hortonworks Inc.
> >>> http://hortonworks.com/
> >>>
> >>>
> >>
> >
>
>
--
Alejandro
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Vinod,
Thanks for reviving this thread.
The current blockers are:
https://issues.apache.org/jira/issues/?jql=project%20in%20(hadoop%2C%20mapreduce%2C%20hdfs%2C%20yarn)%20and%20status%20in%20(open%2C%20'patch%20available')%20and%20priority%20%3D%20blocker%20and%20%22Target%20Version%2Fs%22%20%3D%20%222.1.0-beta%22
By looking at them I don't see they are necessary blockers for a beta
release.
* HADOOP-9688 & HADOOP-9698
They definitely have to be addressed before a GA
release.
* YARN-701
It should be addressed before a GA release.
Still, as it is this breaks unmanaged AMs and to me
that would be a blocker for the beta.
YARN-701 and the unmanaged AMs fix should be committed
in tandem.
* YARN-918
It is a consequence of YARN-701 and depends on it.
* YARN-926
It would be nice to have it addressed before GA release.
We could do a beta with what we have at the moment in branch-2 and have a
special release note indicating API changes coming in the next beta/GA
release as part of YARN-918 & YARN-926.
IMO, we should move forward with the beta release with the current state.
Else we'll continue delaying it and adding more things that break/change
things.
Thanks.
Alejandro
On Wed, Jul 17, 2013 at 12:24 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
>
> Looks like this RC has gone stale and lots of bug fixes went into 2.1 and
> 2.1.0 branches and there are 4-5 outstanding blockers. And from what I see
> in CHANGES.txt files there seems to be a confusion about which branch to
> get in what.
>
> I'm blowing off the current 2.1.0 release branch so that we can create a
> fresh release branch and call voting on that. I'll fix CHANGES.txt entries
> as well as JIRA fix version for bugs committed recently if there are
> inconsistencies.
>
> Let me know if something is amiss while I do this.
>
> Thanks,
> +Vinod
>
> On Jul 3, 2013, at 11:06 AM, Vinod Kumar Vavilapalli wrote:
>
> >
> > We should get these in, looking at them now.
> >
> > Thanks,
> > +Vinod
> >
> > On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote:
> >
> >> Hi Arun,
> >>
> >> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may
> potentially change the apis. They can implemented in a backward compat
> fashion if committed after 2.1.0. However, this will require adding of
> differently-named apis ( different urls in case of the webservices ) and
> make the current version of the api deprecated and/or obsolete. YARN-818
> which is currently patch available also changes behavior.
> >>
> >> Assuming that as soon as 2.1.0 is released, we are to follow a very
> strict backward-compat retaining approach to all user-facing layers (
> api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense
> to try and pull them in and roll out a new RC after they are ready? Perhaps
> Vinod can chime in if he is aware of any other such jiras under YARN-386
> which should be considered compat-related blockers for a 2.1.0 RC.
> >>
> >> thanks
> >> -- Hitesh
> >>
> >> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
> >>
> >>> Folks,
> >>>
> >>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I
> would like to get released.
> >>>
> >>> This release represents a *huge* amount of work done by the community
> (639 fixes) which includes several major advances including:
> >>> # HDFS Snapshots
> >>> # Windows support
> >>> # YARN API stabilization
> >>> # MapReduce Binary Compatibility with hadoop-1.x
> >>> # Substantial amount of integration testing with rest of projects in
> the ecosystem
> >>>
> >>> The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> >>> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
> >>>
> >>> The maven artifacts are available via repository.apache.org.
> >>>
> >>> Please try the release and vote; the vote will run for the usual 7
> days.
> >>>
> >>> thanks,
> >>> Arun
> >>>
> >>> --
> >>> Arun C. Murthy
> >>> Hortonworks Inc.
> >>> http://hortonworks.com/
> >>>
> >>>
> >>
> >
>
>
--
Alejandro
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Alejandro Abdelnur <tu...@cloudera.com>.
Vinod,
Thanks for reviving this thread.
The current blockers are:
https://issues.apache.org/jira/issues/?jql=project%20in%20(hadoop%2C%20mapreduce%2C%20hdfs%2C%20yarn)%20and%20status%20in%20(open%2C%20'patch%20available')%20and%20priority%20%3D%20blocker%20and%20%22Target%20Version%2Fs%22%20%3D%20%222.1.0-beta%22
By looking at them I don't see they are necessary blockers for a beta
release.
* HADOOP-9688 & HADOOP-9698
They definitely have to be addressed before a GA
release.
* YARN-701
It should be addressed before a GA release.
Still, as it is this breaks unmanaged AMs and to me
that would be a blocker for the beta.
YARN-701 and the unmanaged AMs fix should be committed
in tandem.
* YARN-918
It is a consequence of YARN-701 and depends on it.
* YARN-926
It would be nice to have it addressed before GA release.
We could do a beta with what we have at the moment in branch-2 and have a
special release note indicating API changes coming in the next beta/GA
release as part of YARN-918 & YARN-926.
IMO, we should move forward with the beta release with the current state.
Else we'll continue delaying it and adding more things that break/change
things.
Thanks.
Alejandro
On Wed, Jul 17, 2013 at 12:24 PM, Vinod Kumar Vavilapalli <
vinodkv@hortonworks.com> wrote:
>
> Looks like this RC has gone stale and lots of bug fixes went into 2.1 and
> 2.1.0 branches and there are 4-5 outstanding blockers. And from what I see
> in CHANGES.txt files there seems to be a confusion about which branch to
> get in what.
>
> I'm blowing off the current 2.1.0 release branch so that we can create a
> fresh release branch and call voting on that. I'll fix CHANGES.txt entries
> as well as JIRA fix version for bugs committed recently if there are
> inconsistencies.
>
> Let me know if something is amiss while I do this.
>
> Thanks,
> +Vinod
>
> On Jul 3, 2013, at 11:06 AM, Vinod Kumar Vavilapalli wrote:
>
> >
> > We should get these in, looking at them now.
> >
> > Thanks,
> > +Vinod
> >
> > On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote:
> >
> >> Hi Arun,
> >>
> >> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may
> potentially change the apis. They can implemented in a backward compat
> fashion if committed after 2.1.0. However, this will require adding of
> differently-named apis ( different urls in case of the webservices ) and
> make the current version of the api deprecated and/or obsolete. YARN-818
> which is currently patch available also changes behavior.
> >>
> >> Assuming that as soon as 2.1.0 is released, we are to follow a very
> strict backward-compat retaining approach to all user-facing layers (
> api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense
> to try and pull them in and roll out a new RC after they are ready? Perhaps
> Vinod can chime in if he is aware of any other such jiras under YARN-386
> which should be considered compat-related blockers for a 2.1.0 RC.
> >>
> >> thanks
> >> -- Hitesh
> >>
> >> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
> >>
> >>> Folks,
> >>>
> >>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I
> would like to get released.
> >>>
> >>> This release represents a *huge* amount of work done by the community
> (639 fixes) which includes several major advances including:
> >>> # HDFS Snapshots
> >>> # Windows support
> >>> # YARN API stabilization
> >>> # MapReduce Binary Compatibility with hadoop-1.x
> >>> # Substantial amount of integration testing with rest of projects in
> the ecosystem
> >>>
> >>> The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> >>> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
> >>>
> >>> The maven artifacts are available via repository.apache.org.
> >>>
> >>> Please try the release and vote; the vote will run for the usual 7
> days.
> >>>
> >>> thanks,
> >>> Arun
> >>>
> >>> --
> >>> Arun C. Murthy
> >>> Hortonworks Inc.
> >>> http://hortonworks.com/
> >>>
> >>>
> >>
> >
>
>
--
Alejandro
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Looks like this RC has gone stale and lots of bug fixes went into 2.1 and 2.1.0 branches and there are 4-5 outstanding blockers. And from what I see in CHANGES.txt files there seems to be a confusion about which branch to get in what.
I'm blowing off the current 2.1.0 release branch so that we can create a fresh release branch and call voting on that. I'll fix CHANGES.txt entries as well as JIRA fix version for bugs committed recently if there are inconsistencies.
Let me know if something is amiss while I do this.
Thanks,
+Vinod
On Jul 3, 2013, at 11:06 AM, Vinod Kumar Vavilapalli wrote:
>
> We should get these in, looking at them now.
>
> Thanks,
> +Vinod
>
> On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote:
>
>> Hi Arun,
>>
>> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and make the current version of the api deprecated and/or obsolete. YARN-818 which is currently patch available also changes behavior.
>>
>> Assuming that as soon as 2.1.0 is released, we are to follow a very strict backward-compat retaining approach to all user-facing layers ( api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to try and pull them in and roll out a new RC after they are ready? Perhaps Vinod can chime in if he is aware of any other such jiras under YARN-386 which should be considered compat-related blockers for a 2.1.0 RC.
>>
>> thanks
>> -- Hitesh
>>
>> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
>>
>>> Folks,
>>>
>>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>>
>>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>>> # HDFS Snapshots
>>> # Windows support
>>> # YARN API stabilization
>>> # MapReduce Binary Compatibility with hadoop-1.x
>>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>>
>>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>>
>>> The maven artifacts are available via repository.apache.org.
>>>
>>> Please try the release and vote; the vote will run for the usual 7 days.
>>>
>>> thanks,
>>> Arun
>>>
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>>
>>>
>>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Looks like this RC has gone stale and lots of bug fixes went into 2.1 and 2.1.0 branches and there are 4-5 outstanding blockers. And from what I see in CHANGES.txt files there seems to be a confusion about which branch to get in what.
I'm blowing off the current 2.1.0 release branch so that we can create a fresh release branch and call voting on that. I'll fix CHANGES.txt entries as well as JIRA fix version for bugs committed recently if there are inconsistencies.
Let me know if something is amiss while I do this.
Thanks,
+Vinod
On Jul 3, 2013, at 11:06 AM, Vinod Kumar Vavilapalli wrote:
>
> We should get these in, looking at them now.
>
> Thanks,
> +Vinod
>
> On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote:
>
>> Hi Arun,
>>
>> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and make the current version of the api deprecated and/or obsolete. YARN-818 which is currently patch available also changes behavior.
>>
>> Assuming that as soon as 2.1.0 is released, we are to follow a very strict backward-compat retaining approach to all user-facing layers ( api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to try and pull them in and roll out a new RC after they are ready? Perhaps Vinod can chime in if he is aware of any other such jiras under YARN-386 which should be considered compat-related blockers for a 2.1.0 RC.
>>
>> thanks
>> -- Hitesh
>>
>> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
>>
>>> Folks,
>>>
>>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>>
>>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>>> # HDFS Snapshots
>>> # Windows support
>>> # YARN API stabilization
>>> # MapReduce Binary Compatibility with hadoop-1.x
>>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>>
>>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>>
>>> The maven artifacts are available via repository.apache.org.
>>>
>>> Please try the release and vote; the vote will run for the usual 7 days.
>>>
>>> thanks,
>>> Arun
>>>
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>>
>>>
>>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Looks like this RC has gone stale and lots of bug fixes went into 2.1 and 2.1.0 branches and there are 4-5 outstanding blockers. And from what I see in CHANGES.txt files there seems to be a confusion about which branch to get in what.
I'm blowing off the current 2.1.0 release branch so that we can create a fresh release branch and call voting on that. I'll fix CHANGES.txt entries as well as JIRA fix version for bugs committed recently if there are inconsistencies.
Let me know if something is amiss while I do this.
Thanks,
+Vinod
On Jul 3, 2013, at 11:06 AM, Vinod Kumar Vavilapalli wrote:
>
> We should get these in, looking at them now.
>
> Thanks,
> +Vinod
>
> On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote:
>
>> Hi Arun,
>>
>> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and make the current version of the api deprecated and/or obsolete. YARN-818 which is currently patch available also changes behavior.
>>
>> Assuming that as soon as 2.1.0 is released, we are to follow a very strict backward-compat retaining approach to all user-facing layers ( api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to try and pull them in and roll out a new RC after they are ready? Perhaps Vinod can chime in if he is aware of any other such jiras under YARN-386 which should be considered compat-related blockers for a 2.1.0 RC.
>>
>> thanks
>> -- Hitesh
>>
>> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
>>
>>> Folks,
>>>
>>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>>
>>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>>> # HDFS Snapshots
>>> # Windows support
>>> # YARN API stabilization
>>> # MapReduce Binary Compatibility with hadoop-1.x
>>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>>
>>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>>
>>> The maven artifacts are available via repository.apache.org.
>>>
>>> Please try the release and vote; the vote will run for the usual 7 days.
>>>
>>> thanks,
>>> Arun
>>>
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>>
>>>
>>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
Looks like this RC has gone stale and lots of bug fixes went into 2.1 and 2.1.0 branches and there are 4-5 outstanding blockers. And from what I see in CHANGES.txt files there seems to be a confusion about which branch to get in what.
I'm blowing off the current 2.1.0 release branch so that we can create a fresh release branch and call voting on that. I'll fix CHANGES.txt entries as well as JIRA fix version for bugs committed recently if there are inconsistencies.
Let me know if something is amiss while I do this.
Thanks,
+Vinod
On Jul 3, 2013, at 11:06 AM, Vinod Kumar Vavilapalli wrote:
>
> We should get these in, looking at them now.
>
> Thanks,
> +Vinod
>
> On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote:
>
>> Hi Arun,
>>
>> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and make the current version of the api deprecated and/or obsolete. YARN-818 which is currently patch available also changes behavior.
>>
>> Assuming that as soon as 2.1.0 is released, we are to follow a very strict backward-compat retaining approach to all user-facing layers ( api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to try and pull them in and roll out a new RC after they are ready? Perhaps Vinod can chime in if he is aware of any other such jiras under YARN-386 which should be considered compat-related blockers for a 2.1.0 RC.
>>
>> thanks
>> -- Hitesh
>>
>> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
>>
>>> Folks,
>>>
>>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>>
>>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>>> # HDFS Snapshots
>>> # Windows support
>>> # YARN API stabilization
>>> # MapReduce Binary Compatibility with hadoop-1.x
>>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>>
>>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>>
>>> The maven artifacts are available via repository.apache.org.
>>>
>>> Please try the release and vote; the vote will run for the usual 7 days.
>>>
>>> thanks,
>>> Arun
>>>
>>> --
>>> Arun C. Murthy
>>> Hortonworks Inc.
>>> http://hortonworks.com/
>>>
>>>
>>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
We should get these in, looking at them now.
Thanks,
+Vinod
On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote:
> Hi Arun,
>
> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and make the current version of the api deprecated and/or obsolete. YARN-818 which is currently patch available also changes behavior.
>
> Assuming that as soon as 2.1.0 is released, we are to follow a very strict backward-compat retaining approach to all user-facing layers ( api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to try and pull them in and roll out a new RC after they are ready? Perhaps Vinod can chime in if he is aware of any other such jiras under YARN-386 which should be considered compat-related blockers for a 2.1.0 RC.
>
> thanks
> -- Hitesh
>
> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
>
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>
>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>
>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
We should get these in, looking at them now.
Thanks,
+Vinod
On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote:
> Hi Arun,
>
> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and make the current version of the api deprecated and/or obsolete. YARN-818 which is currently patch available also changes behavior.
>
> Assuming that as soon as 2.1.0 is released, we are to follow a very strict backward-compat retaining approach to all user-facing layers ( api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to try and pull them in and roll out a new RC after they are ready? Perhaps Vinod can chime in if he is aware of any other such jiras under YARN-386 which should be considered compat-related blockers for a 2.1.0 RC.
>
> thanks
> -- Hitesh
>
> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
>
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>
>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>
>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Sandy Ryza <sa...@cloudera.com>.
For YARN-791, if we can come to consensus on the correct approach, I can
try to have a patch ASAP.
-Sandy
On Fri, Jun 28, 2013 at 12:03 PM, Hitesh Shah <hi...@apache.org> wrote:
> Hi Arun,
>
> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may
> potentially change the apis. They can implemented in a backward compat
> fashion if committed after 2.1.0. However, this will require adding of
> differently-named apis ( different urls in case of the webservices ) and
> make the current version of the api deprecated and/or obsolete. YARN-818
> which is currently patch available also changes behavior.
>
> Assuming that as soon as 2.1.0 is released, we are to follow a very strict
> backward-compat retaining approach to all user-facing layers (
> api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense
> to try and pull them in and roll out a new RC after they are ready? Perhaps
> Vinod can chime in if he is aware of any other such jiras under YARN-386
> which should be considered compat-related blockers for a 2.1.0 RC.
>
> thanks
> -- Hitesh
>
> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
>
> > Folks,
> >
> > I've created a release candidate (rc0) for hadoop-2.1.0-beta that I
> would like to get released.
> >
> > This release represents a *huge* amount of work done by the community
> (639 fixes) which includes several major advances including:
> > # HDFS Snapshots
> > # Windows support
> > # YARN API stabilization
> > # MapReduce Binary Compatibility with hadoop-1.x
> > # Substantial amount of integration testing with rest of projects in the
> ecosystem
> >
> > The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> > The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
> >
> > The maven artifacts are available via repository.apache.org.
> >
> > Please try the release and vote; the vote will run for the usual 7 days.
> >
> > thanks,
> > Arun
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
We should get these in, looking at them now.
Thanks,
+Vinod
On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote:
> Hi Arun,
>
> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and make the current version of the api deprecated and/or obsolete. YARN-818 which is currently patch available also changes behavior.
>
> Assuming that as soon as 2.1.0 is released, we are to follow a very strict backward-compat retaining approach to all user-facing layers ( api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to try and pull them in and roll out a new RC after they are ready? Perhaps Vinod can chime in if he is aware of any other such jiras under YARN-386 which should be considered compat-related blockers for a 2.1.0 RC.
>
> thanks
> -- Hitesh
>
> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
>
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>
>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>
>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Sandy Ryza <sa...@cloudera.com>.
For YARN-791, if we can come to consensus on the correct approach, I can
try to have a patch ASAP.
-Sandy
On Fri, Jun 28, 2013 at 12:03 PM, Hitesh Shah <hi...@apache.org> wrote:
> Hi Arun,
>
> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may
> potentially change the apis. They can implemented in a backward compat
> fashion if committed after 2.1.0. However, this will require adding of
> differently-named apis ( different urls in case of the webservices ) and
> make the current version of the api deprecated and/or obsolete. YARN-818
> which is currently patch available also changes behavior.
>
> Assuming that as soon as 2.1.0 is released, we are to follow a very strict
> backward-compat retaining approach to all user-facing layers (
> api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense
> to try and pull them in and roll out a new RC after they are ready? Perhaps
> Vinod can chime in if he is aware of any other such jiras under YARN-386
> which should be considered compat-related blockers for a 2.1.0 RC.
>
> thanks
> -- Hitesh
>
> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
>
> > Folks,
> >
> > I've created a release candidate (rc0) for hadoop-2.1.0-beta that I
> would like to get released.
> >
> > This release represents a *huge* amount of work done by the community
> (639 fixes) which includes several major advances including:
> > # HDFS Snapshots
> > # Windows support
> > # YARN API stabilization
> > # MapReduce Binary Compatibility with hadoop-1.x
> > # Substantial amount of integration testing with rest of projects in the
> ecosystem
> >
> > The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> > The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
> >
> > The maven artifacts are available via repository.apache.org.
> >
> > Please try the release and vote; the vote will run for the usual 7 days.
> >
> > thanks,
> > Arun
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Vinod Kumar Vavilapalli <vi...@hortonworks.com>.
We should get these in, looking at them now.
Thanks,
+Vinod
On Jun 28, 2013, at 12:03 PM, Hitesh Shah wrote:
> Hi Arun,
>
> From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and make the current version of the api deprecated and/or obsolete. YARN-818 which is currently patch available also changes behavior.
>
> Assuming that as soon as 2.1.0 is released, we are to follow a very strict backward-compat retaining approach to all user-facing layers ( api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to try and pull them in and roll out a new RC after they are ready? Perhaps Vinod can chime in if he is aware of any other such jiras under YARN-386 which should be considered compat-related blockers for a 2.1.0 RC.
>
> thanks
> -- Hitesh
>
> On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
>
>> Folks,
>>
>> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>>
>> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
>> # HDFS Snapshots
>> # Windows support
>> # YARN API stabilization
>> # MapReduce Binary Compatibility with hadoop-1.x
>> # Substantial amount of integration testing with rest of projects in the ecosystem
>>
>> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
>> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>>
>> The maven artifacts are available via repository.apache.org.
>>
>> Please try the release and vote; the vote will run for the usual 7 days.
>>
>> thanks,
>> Arun
>>
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/
>>
>>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Hitesh Shah <hi...@apache.org>.
Hi Arun,
From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and make the current version of the api deprecated and/or obsolete. YARN-818 which is currently patch available also changes behavior.
Assuming that as soon as 2.1.0 is released, we are to follow a very strict backward-compat retaining approach to all user-facing layers ( api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to try and pull them in and roll out a new RC after they are ready? Perhaps Vinod can chime in if he is aware of any other such jiras under YARN-386 which should be considered compat-related blockers for a 2.1.0 RC.
thanks
-- Hitesh
On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>
> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the ecosystem
>
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Tsuyoshi OZAWA <oz...@gmail.com>.
Hi Arun,
Some bug fixes about MapReduce should be included in next release.
MAPREDUCE-5221 should be merged because it's a lost feature of MRv2
compared to MRv1.
MAPREDUCE-5358 and MAPREDUCE-5335 are very trivial fixes, so it can be
merged immediately.
# I sent the previous email only to hdfs-dev, so I resending it.
# I apologize for my mistake.
On Wed, Jun 26, 2013 at 5:17 PM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>
> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the ecosystem
>
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
--
- Tsuyoshi
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Hitesh Shah <hi...@apache.org>.
Hi Arun,
From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and make the current version of the api deprecated and/or obsolete. YARN-818 which is currently patch available also changes behavior.
Assuming that as soon as 2.1.0 is released, we are to follow a very strict backward-compat retaining approach to all user-facing layers ( api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to try and pull them in and roll out a new RC after they are ready? Perhaps Vinod can chime in if he is aware of any other such jiras under YARN-386 which should be considered compat-related blockers for a 2.1.0 RC.
thanks
-- Hitesh
On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>
> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the ecosystem
>
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Roman Shaposhnik <rv...@apache.org>.
I think it would very useful if somebody more familiar with HDFS
(Suresh?) could take a look at:
https://issues.apache.org/jira/browse/HDFS-4940
Suresh was very helpful in unblocking the client side of things, but
even after I followed his recommendations on the heap size, etc.
The NN still OOMs it just takes longer.
A quick look at the provided heap dump could help us understand
whether this is something serious or not.
This is the last bit that blocks the Bigtop side of things as far as
2.1 is concerned.
Thanks,
Roman.
On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>
> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the ecosystem
>
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Roman Shaposhnik <rv...@apache.org>.
On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>
> This release represents a *huge* amount of work done by the community (639 fixes) which
> includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the ecosystem
>
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
+1 (non-binding).
As usual I built the entire Bigtop based on this RC of Hadoop
and ran the battery of smoke tests with secure and unsecure
configuration.
Thanks,
Roman.
P.S. The blocker bug from a previous RC where NN would consistently OOM
seems to have disappeared. I'll update the JIRA accordingly.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Hitesh Shah <hi...@apache.org>.
Hi Arun,
From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and make the current version of the api deprecated and/or obsolete. YARN-818 which is currently patch available also changes behavior.
Assuming that as soon as 2.1.0 is released, we are to follow a very strict backward-compat retaining approach to all user-facing layers ( api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to try and pull them in and roll out a new RC after they are ready? Perhaps Vinod can chime in if he is aware of any other such jiras under YARN-386 which should be considered compat-related blockers for a 2.1.0 RC.
thanks
-- Hitesh
On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>
> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the ecosystem
>
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Ramya Sunil <ra...@hortonworks.com>.
-1.
Some of the cli and distcp system tests which use hftp:// and webhdfs://
are failing on secure cluster (HDFS-4841 and HDFS-4952/HDFS-4896). This is
a regression and we need to make sure they work before we call a release.
On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would
> like to get released.
>
> This release represents a *huge* amount of work done by the community (639
> fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the
> ecosystem
>
> The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Hitesh Shah <hi...@apache.org>.
Hi Arun,
From a YARN perspective, YARN-791 and YARN-727 are 2 jiras that may potentially change the apis. They can implemented in a backward compat fashion if committed after 2.1.0. However, this will require adding of differently-named apis ( different urls in case of the webservices ) and make the current version of the api deprecated and/or obsolete. YARN-818 which is currently patch available also changes behavior.
Assuming that as soon as 2.1.0 is released, we are to follow a very strict backward-compat retaining approach to all user-facing layers ( api/webservices/rpc/... ) in common/hdfs/yarn/mapreduce, does it make sense to try and pull them in and roll out a new RC after they are ready? Perhaps Vinod can chime in if he is aware of any other such jiras under YARN-386 which should be considered compat-related blockers for a 2.1.0 RC.
thanks
-- Hitesh
On Jun 26, 2013, at 1:17 AM, Arun C Murthy wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>
> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the ecosystem
>
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Ramya Sunil <ra...@hortonworks.com>.
-1.
Some of the cli and distcp system tests which use hftp:// and webhdfs://
are failing on secure cluster (HDFS-4841 and HDFS-4952/HDFS-4896). This is
a regression and we need to make sure they work before we call a release.
On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would
> like to get released.
>
> This release represents a *huge* amount of work done by the community (639
> fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the
> ecosystem
>
> The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Steve Loughran <st...@hortonworks.com>.
On 26 June 2013 09:17, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would
> like to get released.
>
> This release represents a *huge* amount of work done by the community (639
> fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the
> ecosystem
>
> The RC is available at:
> http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here:
> http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
>
>
>
I'm going to +1 here because
-we should get the beta out and get feedback on it
-once it is in the maven repositories, it is effectively released: the
artifacts are in the wild. Even if something with the same version name is
pushed out, maven won't update client machines that have pulled it down:
new artifacts with new versions need to go out.
Now, those maven artifacts don't come with complete Poms, meaning I can't
use them in my builds. The Jars come down, but without the POM files the
build breaks
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta.pom
[WARNING] The POM for org.apache.hadoop:hadoop-common:jar:2.1.0-beta is
missing, no dependency information available
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-minicluster/2.1.0-beta/hadoop-minicluster-2.1.0-beta.pom
[WARNING] The POM for org.apache.hadoop:hadoop-minicluster:jar:2.1.0-beta
is missing, no dependency information available
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-hdfs/2.1.0-beta/hadoop-hdfs-2.1.0-beta.pom
[WARNING] The POM for org.apache.hadoop:hadoop-hdfs:jar:2.1.0-beta is
missing, no dependency information available
[WARNING] The POM for org.apache.hadoop:hadoop-common:jar:tests:2.1.0-beta
is missing, no dependency information available
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-yarn-server-common/2.1.0-beta/hadoop-yarn-server-common-2.1.0-beta.pom
[WARNING] The POM for
org.apache.hadoop:hadoop-yarn-server-common:jar:2.1.0-beta is missing, no
dependency information available
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta.jar
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-minicluster/2.1.0-beta/hadoop-minicluster-2.1.0-beta.jar
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-hdfs/2.1.0-beta/hadoop-hdfs-2.1.0-beta.jar
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-common/2.1.0-beta/hadoop-common-2.1.0-beta-tests.jar
Downloading:
http://repo.maven.apache.org/maven2/org/apache/hadoop/hadoop-yarn-server-common/2.1.0-beta/hadoop-yarn-server-common-2.1.0-beta.jar
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
As a result, I can't actually use it -I have to make my own 2.1.1-SNAPSHOT
release not because I want anything newer than the beta, but because those
missing POM files stop me coding against the beta.
I have to view this as a bug in the maven-based release process: somehow
the POM files didn't get published to the apache repos.
1. We need to know what happened here so that the next beta can be used
in downstream builds -and eventually, the final release. Because if those
artifacts don't go up in a way that can be used in maven and ant/ivy
builds, then I do consider it to be broken.
2. That said, I'm not sure that artifacts go to the repo until vote
time, precisely because it is impossible to revoke them once it's happened.
If there is a way of doing a build of the release with the -SNAPSHOT tag,
we could push it out to the apache snapshot repo and make sure it works
there. That is, unless there is some different in -SNAPSHOT releases and
non snapshot ones.
Arun,
1. do you have the log of the bits of the build where the Maven
artifacts were published?
2. do you still have the POM files locally? If you do, we could try to
publish them again.
Finally, who understands that step of the build that pushes the artifacts
to the apache release repo? Does it work for the 0.27.x and 0.20.20x
releases?
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
Posted by Roman Shaposhnik <rv...@apache.org>.
I think it would very useful if somebody more familiar with HDFS
(Suresh?) could take a look at:
https://issues.apache.org/jira/browse/HDFS-4940
Suresh was very helpful in unblocking the client side of things, but
even after I followed his recommendations on the heap size, etc.
The NN still OOMs it just takes longer.
A quick look at the provided heap dump could help us understand
whether this is something serious or not.
This is the last bit that blocks the Bigtop side of things as far as
2.1 is concerned.
Thanks,
Roman.
On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy <ac...@hortonworks.com> wrote:
> Folks,
>
> I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released.
>
> This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including:
> # HDFS Snapshots
> # Windows support
> # YARN API stabilization
> # MapReduce Binary Compatibility with hadoop-1.x
> # Substantial amount of integration testing with rest of projects in the ecosystem
>
> The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/
> The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0
>
> The maven artifacts are available via repository.apache.org.
>
> Please try the release and vote; the vote will run for the usual 7 days.
>
> thanks,
> Arun
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>