You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@crunch.apache.org by Josh Wills <jw...@apache.org> on 2013/12/13 00:43:48 UTC
[VOTE] Release Apache Crunch 0.9.0 RC0
Hello,
This is a call for a vote on releasing the following candidate as Apache
Crunch 0.9.0.
This release contains the following fixes for the following issues and is
the first release of Crunch compiled against Hadoop 2.2.0 and HBase 0.96:
https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313526&version=12325563
Release artifacts:
http://people.apache.org/~jwills/crunch-0.9.0-RC0/
Maven staging repo:
https://repository.apache.org/content/repositories/orgapachecrunch-049/
The tag to be voted upon:
https://git-wip-us.apache.org/repos/asf?p=crunch.git;a=tag;h=8ab9e3db5c845d5df13b16396215e7f314e0f4ee
Crunch's KEYS file:
http://www.apache.org/dist/crunch/KEYS
Thanks,
Josh
Re: [VOTE] Release Apache Crunch 0.9.0 RC0
Posted by Gabriel Reid <ga...@gmail.com>.
On Tue, Dec 17, 2013 at 9:03 PM, Josh Wills <jw...@cloudera.com> wrote:
>
> On Tue, Dec 17, 2013 at 11:55 AM, Gabriel Reid <ga...@gmail.com>
> wrote:
>>
>> I noticed that there are two source distributions: the "default"
>> version, and the hadoop2 version. I'm guessing this is nothing new and
>> I just missed it in the past, but I want to be sure: is this the
>> intention? I thought that we only made two separate binary releases,
>> and a single source release.
>
>
> It isn't new; the mvn release scripts build both the src and bin versions
> for hadoop1 and hadoop2, and I generally only push the default src packages
> to the release dir. I just happened to copy both of them this time; it's no
> problem to remove them from the RC to ensure no one is confused.
>
Thanks for the clarification. I don't think that there's a big need to
remove the hadoop2 src release, I just hadn't noticed it in the past,
which isn't a problem.
>>
>>
>> Related to this, I also saw that the version pointed to by the release
>> tag in git has the 0.x.x-hadoop2 release, which I guess means that
>> there isn't a release tag for the "default" release version. Is that
>> correct, and/or the intention?
>
>
> So I have to do the release runs twice, once w/ -Dcrunch.platform=2 and once
> without, and after I run the first one, I have to run a git reset to do the
> next one. There isn't any source difference that results from the maven
> tagging, it's just a function of the order that I happen to do the mvn
> release runs.
>
Ok, so if I understand correctly, this means RC tag on the 0.x.x
version gets overwritten to point to the 0.x.x-hadoop2 version. I
guess this is nothing to hold up the release, but it does seem like
something that should be improved in the release process somehow
(although I don't know how), because checking out the release tag now
will give you a source tree where the version is 0.x.x-hadoop2 (which
seems a bit weird).
That being said, it's a +1 from me to release. I verified the
signatures, verified contents of tar.gz distributions vs zip
distributions, sucessfully ran full unit and integration tests, and
compared the source distribution to the git tagged version.
- Gabriel
Re: [VOTE] Release Apache Crunch 0.9.0 RC0
Posted by Josh Wills <jw...@cloudera.com>.
On Tue, Dec 17, 2013 at 11:55 AM, Gabriel Reid <ga...@gmail.com>wrote:
> Going through the validation of this and the 0.8.2 RC, I ran into
> something that I want to double-check.
>
> I noticed that there are two source distributions: the "default"
> version, and the hadoop2 version. I'm guessing this is nothing new and
> I just missed it in the past, but I want to be sure: is this the
> intention? I thought that we only made two separate binary releases,
> and a single source release.
>
It isn't new; the mvn release scripts build both the src and bin versions
for hadoop1 and hadoop2, and I generally only push the default src packages
to the release dir. I just happened to copy both of them this time; it's no
problem to remove them from the RC to ensure no one is confused.
>
> Related to this, I also saw that the version pointed to by the release
> tag in git has the 0.x.x-hadoop2 release, which I guess means that
> there isn't a release tag for the "default" release version. Is that
> correct, and/or the intention?
>
So I have to do the release runs twice, once w/ -Dcrunch.platform=2 and
once without, and after I run the first one, I have to run a git reset to
do the next one. There isn't any source difference that results from the
maven tagging, it's just a function of the order that I happen to do the
mvn release runs.
>
> - Gabriel
>
>
> On Fri, Dec 13, 2013 at 12:43 AM, Josh Wills <jw...@apache.org> wrote:
> > Hello,
> >
> > This is a call for a vote on releasing the following candidate as Apache
> > Crunch 0.9.0.
> >
> > This release contains the following fixes for the following issues and is
> > the first release of Crunch compiled against Hadoop 2.2.0 and HBase 0.96:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313526&version=12325563
> >
> > Release artifacts:
> > http://people.apache.org/~jwills/crunch-0.9.0-RC0/
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachecrunch-049/
> >
> > The tag to be voted upon:
> >
> https://git-wip-us.apache.org/repos/asf?p=crunch.git;a=tag;h=8ab9e3db5c845d5df13b16396215e7f314e0f4ee
> >
> > Crunch's KEYS file:
> > http://www.apache.org/dist/crunch/KEYS
> >
> > Thanks,
> > Josh
>
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: [VOTE] Release Apache Crunch 0.9.0 RC0
Posted by Gabriel Reid <ga...@gmail.com>.
Going through the validation of this and the 0.8.2 RC, I ran into
something that I want to double-check.
I noticed that there are two source distributions: the "default"
version, and the hadoop2 version. I'm guessing this is nothing new and
I just missed it in the past, but I want to be sure: is this the
intention? I thought that we only made two separate binary releases,
and a single source release.
Related to this, I also saw that the version pointed to by the release
tag in git has the 0.x.x-hadoop2 release, which I guess means that
there isn't a release tag for the "default" release version. Is that
correct, and/or the intention?
- Gabriel
On Fri, Dec 13, 2013 at 12:43 AM, Josh Wills <jw...@apache.org> wrote:
> Hello,
>
> This is a call for a vote on releasing the following candidate as Apache
> Crunch 0.9.0.
>
> This release contains the following fixes for the following issues and is
> the first release of Crunch compiled against Hadoop 2.2.0 and HBase 0.96:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313526&version=12325563
>
> Release artifacts:
> http://people.apache.org/~jwills/crunch-0.9.0-RC0/
>
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachecrunch-049/
>
> The tag to be voted upon:
> https://git-wip-us.apache.org/repos/asf?p=crunch.git;a=tag;h=8ab9e3db5c845d5df13b16396215e7f314e0f4ee
>
> Crunch's KEYS file:
> http://www.apache.org/dist/crunch/KEYS
>
> Thanks,
> Josh
Re: [VOTE] Release Apache Crunch 0.9.0 RC0
Posted by Brock Noland <br...@cloudera.com>.
+1
Checked sigs/hashes, ran unit tests, and extracted and sanity checked all
tars/zips.
Brock
On Mon, Dec 16, 2013 at 11:17 AM, Tom White <to...@cloudera.com> wrote:
> +1
>
> I checked sigs, ran RAT, looked for spurious binary files, checked the
> source tag against the source distro, and ran all the unit tests
> successfully.
>
> Cheers,
> Tom
>
> On Thu, Dec 12, 2013 at 11:43 PM, Josh Wills <jw...@apache.org> wrote:
> > Hello,
> >
> > This is a call for a vote on releasing the following candidate as Apache
> > Crunch 0.9.0.
> >
> > This release contains the following fixes for the following issues and is
> > the first release of Crunch compiled against Hadoop 2.2.0 and HBase 0.96:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313526&version=12325563
> >
> > Release artifacts:
> > http://people.apache.org/~jwills/crunch-0.9.0-RC0/
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachecrunch-049/
> >
> > The tag to be voted upon:
> >
> https://git-wip-us.apache.org/repos/asf?p=crunch.git;a=tag;h=8ab9e3db5c845d5df13b16396215e7f314e0f4ee
> >
> > Crunch's KEYS file:
> > http://www.apache.org/dist/crunch/KEYS
> >
> > Thanks,
> > Josh
>
--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
Re: [VOTE] Release Apache Crunch 0.9.0 RC0
Posted by Tom White <to...@cloudera.com>.
+1
I checked sigs, ran RAT, looked for spurious binary files, checked the
source tag against the source distro, and ran all the unit tests
successfully.
Cheers,
Tom
On Thu, Dec 12, 2013 at 11:43 PM, Josh Wills <jw...@apache.org> wrote:
> Hello,
>
> This is a call for a vote on releasing the following candidate as Apache
> Crunch 0.9.0.
>
> This release contains the following fixes for the following issues and is
> the first release of Crunch compiled against Hadoop 2.2.0 and HBase 0.96:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313526&version=12325563
>
> Release artifacts:
> http://people.apache.org/~jwills/crunch-0.9.0-RC0/
>
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachecrunch-049/
>
> The tag to be voted upon:
> https://git-wip-us.apache.org/repos/asf?p=crunch.git;a=tag;h=8ab9e3db5c845d5df13b16396215e7f314e0f4ee
>
> Crunch's KEYS file:
> http://www.apache.org/dist/crunch/KEYS
>
> Thanks,
> Josh
Re: [VOTE] Release Apache Crunch 0.9.0 RC0
Posted by Matthias Friedrich <ma...@mafr.de>.
Ah, didn't notice the repository element (thanks, Gabriel!). After
removing my maven mirror settings, the build works and I was able to
confirm that there are in fact no spark libraries in our distribution.
So, I change my -1 to a +0 :)
Regards,
Matthias
On Tuesday, 2013-12-17, Josh Wills wrote:
> On Tue, Dec 17, 2013 at 1:08 AM, Matthias Friedrich <ma...@mafr.de> wrote:
>
> > -1 Sorry, it seems Spark isn't listed in LICENSE, which is required for
> > our convenience binaries.
> >
>
> I don't think Spark is in the convenience binaries-- I just unzipped them,
> and it's not there in lib. IIRC, we only include the dependencies of
> crunch-core in the convenience binaries (like parquet and avro), not the
> other modules (so for example, we don't ship HBase libs in the convenience
> binaries.)
>
>
> >
> > Unfortunately, some artifacts related to Spark don't seem to be available
> > via
> > Maven central (com.typesafe.akka:akka-actor:jar:2.0.5), so the build
> > doesn't
> > work. You probably have them in your repo at Cloudera.
> >
> > Regards,
> > Matthias
> >
> > On Thursday, 2013-12-12, Josh Wills wrote:
> > > Hello,
> > >
> > > This is a call for a vote on releasing the following candidate as Apache
> > > Crunch 0.9.0.
> > >
> > > This release contains the following fixes for the following issues and is
> > > the first release of Crunch compiled against Hadoop 2.2.0 and HBase 0.96:
> > >
> > >
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313526&version=12325563
> > >
> > > Release artifacts:
> > > http://people.apache.org/~jwills/crunch-0.9.0-RC0/
> > >
> > > Maven staging repo:
> > > https://repository.apache.org/content/repositories/orgapachecrunch-049/
> > >
> > > The tag to be voted upon:
> > >
> > https://git-wip-us.apache.org/repos/asf?p=crunch.git;a=tag;h=8ab9e3db5c845d5df13b16396215e7f314e0f4ee
> > >
> > > Crunch's KEYS file:
> > > http://www.apache.org/dist/crunch/KEYS
> > >
> > > Thanks,
> > > Josh
> >
>
>
>
> --
> Director of Data Science
> Cloudera <http://www.cloudera.com>
> Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: [VOTE] Release Apache Crunch 0.9.0 RC0
Posted by Josh Wills <jw...@cloudera.com>.
On Tue, Dec 17, 2013 at 1:08 AM, Matthias Friedrich <ma...@mafr.de> wrote:
> -1 Sorry, it seems Spark isn't listed in LICENSE, which is required for
> our convenience binaries.
>
I don't think Spark is in the convenience binaries-- I just unzipped them,
and it's not there in lib. IIRC, we only include the dependencies of
crunch-core in the convenience binaries (like parquet and avro), not the
other modules (so for example, we don't ship HBase libs in the convenience
binaries.)
>
> Unfortunately, some artifacts related to Spark don't seem to be available
> via
> Maven central (com.typesafe.akka:akka-actor:jar:2.0.5), so the build
> doesn't
> work. You probably have them in your repo at Cloudera.
>
> Regards,
> Matthias
>
> On Thursday, 2013-12-12, Josh Wills wrote:
> > Hello,
> >
> > This is a call for a vote on releasing the following candidate as Apache
> > Crunch 0.9.0.
> >
> > This release contains the following fixes for the following issues and is
> > the first release of Crunch compiled against Hadoop 2.2.0 and HBase 0.96:
> >
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313526&version=12325563
> >
> > Release artifacts:
> > http://people.apache.org/~jwills/crunch-0.9.0-RC0/
> >
> > Maven staging repo:
> > https://repository.apache.org/content/repositories/orgapachecrunch-049/
> >
> > The tag to be voted upon:
> >
> https://git-wip-us.apache.org/repos/asf?p=crunch.git;a=tag;h=8ab9e3db5c845d5df13b16396215e7f314e0f4ee
> >
> > Crunch's KEYS file:
> > http://www.apache.org/dist/crunch/KEYS
> >
> > Thanks,
> > Josh
>
--
Director of Data Science
Cloudera <http://www.cloudera.com>
Twitter: @josh_wills <http://twitter.com/josh_wills>
Re: [VOTE] Release Apache Crunch 0.9.0 RC0
Posted by Gabriel Reid <ga...@gmail.com>.
Sorry I'm only getting onto this now, I've been offline for the past week.
>
> Unfortunately, some artifacts related to Spark don't seem to be available via
> Maven central (com.typesafe.akka:akka-actor:jar:2.0.5), so the build doesn't
> work. You probably have them in your repo at Cloudera.
>
I tried building by clearing out my local repository (and no
settings.xml), and it seems it all worked fine for me. Looking more
closely, it appears that the spark parent pom includes the Akka repo
(among others), which I suppose allows pulling in the akka
dependencies. Any idea why the build doesn't work at all on your end?
- Gabriel
Re: [VOTE] Release Apache Crunch 0.9.0 RC0
Posted by Matthias Friedrich <ma...@mafr.de>.
-1 Sorry, it seems Spark isn't listed in LICENSE, which is required for
our convenience binaries.
Unfortunately, some artifacts related to Spark don't seem to be available via
Maven central (com.typesafe.akka:akka-actor:jar:2.0.5), so the build doesn't
work. You probably have them in your repo at Cloudera.
Regards,
Matthias
On Thursday, 2013-12-12, Josh Wills wrote:
> Hello,
>
> This is a call for a vote on releasing the following candidate as Apache
> Crunch 0.9.0.
>
> This release contains the following fixes for the following issues and is
> the first release of Crunch compiled against Hadoop 2.2.0 and HBase 0.96:
>
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12313526&version=12325563
>
> Release artifacts:
> http://people.apache.org/~jwills/crunch-0.9.0-RC0/
>
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachecrunch-049/
>
> The tag to be voted upon:
> https://git-wip-us.apache.org/repos/asf?p=crunch.git;a=tag;h=8ab9e3db5c845d5df13b16396215e7f314e0f4ee
>
> Crunch's KEYS file:
> http://www.apache.org/dist/crunch/KEYS
>
> Thanks,
> Josh