You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Patrick Wendell <pw...@gmail.com> on 2013/12/11 01:49:09 UTC
[VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Please vote on releasing the following candidate as Apache Spark
(incubating) version 0.8.1.
The tag to be voted on is v0.8.1-incubating (commit b87d31d):
https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
The release files, including signatures, digests, etc can be found at:
http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/pwendell.asc
The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-040/
The documentation corresponding to this release can be found at:
http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
For information about the contents of this release see:
https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
Please vote on releasing this package as Apache Spark 0.8.1-incubating!
The vote is open until Saturday, December 14th at 01:00 UTC and
passes if a majority of at least 3 +1 PPMC votes are cast.
[ ] +1 Release this package as Apache Spark 0.8.1-incubating
[ ] -1 Do not release this package because ...
To learn more about Apache Spark, please see
http://spark.incubator.apache.org/
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Tom Graves <tg...@yahoo.com>.
+1.
Built spark on yarn for both hadoop 0.23 and hadoop 2.2.0 on redhat linux using maven. Ran some tests on both a secure Hadoop 0.23 cluster and a secure Hadoop 2.2.0 cluster. Verified signatures and md5.
Tom
On Tuesday, December 10, 2013 6:49 PM, Patrick Wendell <pw...@gmail.com> wrote:
Please vote on releasing the following candidate as Apache Spark
(incubating) version 0.8.1.
The tag to be voted on is v0.8.1-incubating (commit b87d31d):
https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
The release files, including signatures, digests, etc can be found
at:
http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/pwendell.asc
The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-040/
The documentation corresponding to this release can be found at:
http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
For information about the contents of this release see:
https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
Please vote on releasing this package as Apache Spark 0.8.1-incubating!
The vote is open until Saturday, December 14th at 01:00 UTC and
passes if a majority of at least 3 +1 PPMC votes are cast.
[ ] +1 Release this package as Apache Spark 0.8.1-incubating
[ ] -1 Do not release this package because ...
To learn more about Apache Spark, please see
http://spark.incubator.apache.org/
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Taka Shinagawa <ta...@gmail.com>.
That's a good point. Although it's definitely not a blocker for this
release, it would be more user friendly to mention sbt/sbt
assembly/assembly as well as Maven build instructions in the README and
quick-start files (at least until the time-consuming sbt assembly process
on a regular hardware gets resolved).
I've been able to build RC4 (against Hadoop 2.2.0) with Maven on my slower
Mac and Ubuntu 12.0.4 LTS successfully. On my faster Mac, sbt/sbt assembly
has completed in 4min.
Thanks for fixing the Maven build for Hadoop 2.2.0!
On Tue, Dec 10, 2013 at 11:58 PM, Prashant Sharma <sc...@gmail.com>wrote:
> Hi Patrick and Matei,
>
> Was trying out this and followed the quick start guide which says do
> sbt/sbt assembly, like few others I was also stuck for few minutes on
> linux. On the other hand if I use sbt/sbt assembly/assembly it is much
> faster.
>
> Should we change the documentation to reflect this. It will not be great
> for first time users to get stuck there.
>
>
> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <matei.zaharia@gmail.com
> >wrote:
>
> > +1
> >
> > Built and tested it on Mac OS X.
> >
> > Matei
> >
> >
> > On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com> wrote:
> >
> > > Please vote on releasing the following candidate as Apache Spark
> > > (incubating) version 0.8.1.
> > >
> > > The tag to be voted on is v0.8.1-incubating (commit b87d31d):
> > >
> >
> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
> > >
> > > The release files, including signatures, digests, etc can be found at:
> > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
> > >
> > > Release artifacts are signed with the following key:
> > > https://people.apache.org/keys/committer/pwendell.asc
> > >
> > > The staging repository for this release can be found at:
> > > https://repository.apache.org/content/repositories/orgapachespark-040/
> > >
> > > The documentation corresponding to this release can be found at:
> > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
> > >
> > > For information about the contents of this release see:
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
> > >
> > > Please vote on releasing this package as Apache Spark 0.8.1-incubating!
> > >
> > > The vote is open until Saturday, December 14th at 01:00 UTC and
> > > passes if a majority of at least 3 +1 PPMC votes are cast.
> > >
> > > [ ] +1 Release this package as Apache Spark 0.8.1-incubating
> > > [ ] -1 Do not release this package because ...
> > >
> > > To learn more about Apache Spark, please see
> > > http://spark.incubator.apache.org/
> >
> >
>
>
> --
> s
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Tom Graves <tg...@yahoo.com>.
I re-downloaded the source tarball and it works now.
Tom
On Wednesday, December 11, 2013 6:27 PM, Patrick Wendell <pw...@gmail.com> wrote:
I also talked to a few people who got corrupted binaries when
downloading from the people.apache HTTP. In that case the checksum
failed but if they re-downloaded it worked. So maybe just re-download
and try again?
On Wed, Dec 11, 2013 at 3:15 PM, Patrick Wendell <pw...@gmail.com> wrote:
> Hey Tom,
>
> I re-verified the signatures and got someone else to do it. It seemed
> fine. Here is what I did.
>
> gpg --recv-key 9E4FE3AF
> wget http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz.asc
> wget http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz
> gpg --verify spark-0.8.1-incubating.tgz.asc spark-0.8.1-incubating.tgz
> gpg: Signature made Tue 10 Dec 2013 02:53:15 PM PST using RSA key ID 9E4FE3AF
> gpg: Good signature from "Patrick Wendell <pw...@gmail.com>"
>
> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>> I don't know how to make sense of the numbers, but here's what I've got
>> from a very small sample size.
>>
>> For both v0.8.0-incubating and v0.8.1-incubating, building separate
>> assemblies is faster than `./sbt/sbt assembly` and the times for building
>> separate assemblies for 0.8.0 and 0.8.1 are about the same.
>>
>> For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as the
>> sum of the separate assemblies.
>> For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as the
>> sum of the separate assemblies.
>>
>> Weird.
>>
>>
>> On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell <pw...@gmail.com>wrote:
>>
>>> I'll +1 myself also.
>>>
>>> For anyone who has the slow build problem: does this issue happen when
>>> building v0.8.0-incubating also? Trying to figure out whether it's
>>> related to something we added in 0.8.1 or if it's a long standing
>>> issue.
>>>
>>> - Patrick
>>>
>>> On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <ma...@gmail.com>
>>> wrote:
>>> > Woah, weird, but definitely good to know.
>>> >
>>> > If you’re doing Spark development, there’s also a more convenient option
>>> added by Shivaram in the master branch. You can do sbt assemble-deps to
>>> package *just* the dependencies of each project in a special assembly JAR,
>>> and then use sbt compile to update the code. This will use the classes
>>> directly out of the target/scala-2.9.3/classes directories. You have to
>>> redo assemble-deps only if your external dependencies change.
>>> >
>>> > Matei
>>> >
>>> > On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com>
>>> wrote:
>>> >
>>> >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can
>>> help.
>>> >> Again this is not a blocker for the release from my side either.
>>> >>
>>> >>
>>> >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <mark@clearstorydata.com
>>> >wrote:
>>> >>
>>> >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly`
>>> takes
>>> >>> a long, long, looooong time to complete (a MBP, in my case), building
>>> three
>>> >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
>>> >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less
>>> time.
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <
>>> scrapcodes@gmail.com
>>> >>>> wrote:
>>> >>>
>>> >>>> forgot to mention, after running sbt/sbt assembly/assembly running
>>> >>> sbt/sbt
>>> >>>> examples/assembly takes just 37s. Not to mention my hardware is not
>>> >>> really
>>> >>>> great.
>>> >>>>
>>> >>>>
>>> >>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <
>>> scrapcodes@gmail.com
>>> >>>>> wrote:
>>> >>>>
>>> >>>>> Hi Patrick and Matei,
>>> >>>>>
>>> >>>>> Was trying out this and followed the quick start guide which says do
>>> >>>>> sbt/sbt assembly, like few others I was also stuck for few minutes on
>>> >>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is
>>> much
>>> >>>>> faster.
>>> >>>>>
>>> >>>>> Should we change the documentation to reflect this. It will not be
>>> >>> great
>>> >>>>> for first time users to get stuck there.
>>> >>>>>
>>> >>>>>
>>> >>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
>>> >>> matei.zaharia@gmail.com
>>> >>>>> wrote:
>>> >>>>>
>>> >>>>>> +1
>>> >>>>>>
>>> >>>>>> Built and tested it on Mac OS X.
>>> >>>>>>
>>> >>>>>> Matei
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com>
>>> >>>> wrote:
>>> >>>>>>
>>> >>>>>>> Please vote on releasing the following candidate as Apache Spark
>>> >>>>>>> (incubating) version 0.8.1.
>>> >>>>>>>
>>> >>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
>>> >>>>>>>
>>> >>>>>>
>>> >>>>
>>> >>>
>>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
>>> >>>>>>>
>>> >>>>>>> The release files, including signatures, digests, etc can be found
>>> >>> at:
>>> >>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
>>> >>>>>>>
>>> >>>>>>> Release artifacts are signed with the following key:
>>> >>>>>>> https://people.apache.org/keys/committer/pwendell.asc
>>> >>>>>>>
>>> >>>>>>> The staging repository for this release can be found at:
>>> >>>>>>>
>>> >>>>
>>> https://repository.apache.org/content/repositories/orgapachespark-040/
>>> >>>>>>>
>>> >>>>>>> The documentation corresponding to this release can be found at:
>>> >>>>>>>
>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
>>> >>>>>>>
>>> >>>>>>> For information about the contents of this release see:
>>> >>>>>>>
>>> >>>>>>
>>> >>>>
>>> >>>
>>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
>>> >>>>>>>
>>> >>>>>>> Please vote on releasing this package as Apache Spark
>>> >>>> 0.8.1-incubating!
>>> >>>>>>>
>>> >>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
>>> >>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
>>> >>>>>>>
>>> >>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
>>> >>>>>>> [ ] -1 Do not release this package because ...
>>> >>>>>>>
>>> >>>>>>> To learn more about Apache Spark, please see
>>> >>>>>>> http://spark.incubator.apache.org/
>>> >>>>>>
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> s
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> s
>>> >>>>
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> s
>>> >
>>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Prashant Sharma <sc...@gmail.com>.
+1, built and tested on linux.
On Thu, Dec 12, 2013 at 5:57 AM, Patrick Wendell <pw...@gmail.com> wrote:
> I also talked to a few people who got corrupted binaries when
> downloading from the people.apache HTTP. In that case the checksum
> failed but if they re-downloaded it worked. So maybe just re-download
> and try again?
>
> On Wed, Dec 11, 2013 at 3:15 PM, Patrick Wendell <pw...@gmail.com>
> wrote:
> > Hey Tom,
> >
> > I re-verified the signatures and got someone else to do it. It seemed
> > fine. Here is what I did.
> >
> > gpg --recv-key 9E4FE3AF
> > wget
> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz.asc
> > wget
> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz
> > gpg --verify spark-0.8.1-incubating.tgz.asc spark-0.8.1-incubating.tgz
> > gpg: Signature made Tue 10 Dec 2013 02:53:15 PM PST using RSA key ID
> 9E4FE3AF
> > gpg: Good signature from "Patrick Wendell <pw...@gmail.com>"
> >
> > On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <ma...@clearstorydata.com>
> wrote:
> >> I don't know how to make sense of the numbers, but here's what I've got
> >> from a very small sample size.
> >>
> >> For both v0.8.0-incubating and v0.8.1-incubating, building separate
> >> assemblies is faster than `./sbt/sbt assembly` and the times for
> building
> >> separate assemblies for 0.8.0 and 0.8.1 are about the same.
> >>
> >> For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as
> the
> >> sum of the separate assemblies.
> >> For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as
> the
> >> sum of the separate assemblies.
> >>
> >> Weird.
> >>
> >>
> >> On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell <pwendell@gmail.com
> >wrote:
> >>
> >>> I'll +1 myself also.
> >>>
> >>> For anyone who has the slow build problem: does this issue happen when
> >>> building v0.8.0-incubating also? Trying to figure out whether it's
> >>> related to something we added in 0.8.1 or if it's a long standing
> >>> issue.
> >>>
> >>> - Patrick
> >>>
> >>> On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <
> matei.zaharia@gmail.com>
> >>> wrote:
> >>> > Woah, weird, but definitely good to know.
> >>> >
> >>> > If you’re doing Spark development, there’s also a more convenient
> option
> >>> added by Shivaram in the master branch. You can do sbt assemble-deps to
> >>> package *just* the dependencies of each project in a special assembly
> JAR,
> >>> and then use sbt compile to update the code. This will use the classes
> >>> directly out of the target/scala-2.9.3/classes directories. You have to
> >>> redo assemble-deps only if your external dependencies change.
> >>> >
> >>> > Matei
> >>> >
> >>> > On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com>
> >>> wrote:
> >>> >
> >>> >> I hope this PR https://github.com/apache/incubator-spark/pull/252can
> >>> help.
> >>> >> Again this is not a blocker for the release from my side either.
> >>> >>
> >>> >>
> >>> >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <
> mark@clearstorydata.com
> >>> >wrote:
> >>> >>
> >>> >>> Interesting, and confirmed: On my machine where `./sbt/sbt
> assembly`
> >>> takes
> >>> >>> a long, long, looooong time to complete (a MBP, in my case),
> building
> >>> three
> >>> >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
> >>> >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much
> less
> >>> time.
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <
> >>> scrapcodes@gmail.com
> >>> >>>> wrote:
> >>> >>>
> >>> >>>> forgot to mention, after running sbt/sbt assembly/assembly running
> >>> >>> sbt/sbt
> >>> >>>> examples/assembly takes just 37s. Not to mention my hardware is
> not
> >>> >>> really
> >>> >>>> great.
> >>> >>>>
> >>> >>>>
> >>> >>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <
> >>> scrapcodes@gmail.com
> >>> >>>>> wrote:
> >>> >>>>
> >>> >>>>> Hi Patrick and Matei,
> >>> >>>>>
> >>> >>>>> Was trying out this and followed the quick start guide which
> says do
> >>> >>>>> sbt/sbt assembly, like few others I was also stuck for few
> minutes on
> >>> >>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is
> >>> much
> >>> >>>>> faster.
> >>> >>>>>
> >>> >>>>> Should we change the documentation to reflect this. It will not
> be
> >>> >>> great
> >>> >>>>> for first time users to get stuck there.
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
> >>> >>> matei.zaharia@gmail.com
> >>> >>>>> wrote:
> >>> >>>>>
> >>> >>>>>> +1
> >>> >>>>>>
> >>> >>>>>> Built and tested it on Mac OS X.
> >>> >>>>>>
> >>> >>>>>> Matei
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <
> pwendell@gmail.com>
> >>> >>>> wrote:
> >>> >>>>>>
> >>> >>>>>>> Please vote on releasing the following candidate as Apache
> Spark
> >>> >>>>>>> (incubating) version 0.8.1.
> >>> >>>>>>>
> >>> >>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
> >>> >>>>>>>
> >>> >>>>>>
> >>> >>>>
> >>> >>>
> >>>
> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
> >>> >>>>>>>
> >>> >>>>>>> The release files, including signatures, digests, etc can be
> found
> >>> >>> at:
> >>> >>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
> >>> >>>>>>>
> >>> >>>>>>> Release artifacts are signed with the following key:
> >>> >>>>>>> https://people.apache.org/keys/committer/pwendell.asc
> >>> >>>>>>>
> >>> >>>>>>> The staging repository for this release can be found at:
> >>> >>>>>>>
> >>> >>>>
> >>> https://repository.apache.org/content/repositories/orgapachespark-040/
> >>> >>>>>>>
> >>> >>>>>>> The documentation corresponding to this release can be found
> at:
> >>> >>>>>>>
> >>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
> >>> >>>>>>>
> >>> >>>>>>> For information about the contents of this release see:
> >>> >>>>>>>
> >>> >>>>>>
> >>> >>>>
> >>> >>>
> >>>
> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
> >>> >>>>>>>
> >>> >>>>>>> Please vote on releasing this package as Apache Spark
> >>> >>>> 0.8.1-incubating!
> >>> >>>>>>>
> >>> >>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
> >>> >>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
> >>> >>>>>>>
> >>> >>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
> >>> >>>>>>> [ ] -1 Do not release this package because ...
> >>> >>>>>>>
> >>> >>>>>>> To learn more about Apache Spark, please see
> >>> >>>>>>> http://spark.incubator.apache.org/
> >>> >>>>>>
> >>> >>>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> --
> >>> >>>>> s
> >>> >>>>>
> >>> >>>>
> >>> >>>>
> >>> >>>>
> >>> >>>> --
> >>> >>>> s
> >>> >>>>
> >>> >>>
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> s
> >>> >
> >>>
>
--
s
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Patrick Wendell <pw...@gmail.com>.
I also talked to a few people who got corrupted binaries when
downloading from the people.apache HTTP. In that case the checksum
failed but if they re-downloaded it worked. So maybe just re-download
and try again?
On Wed, Dec 11, 2013 at 3:15 PM, Patrick Wendell <pw...@gmail.com> wrote:
> Hey Tom,
>
> I re-verified the signatures and got someone else to do it. It seemed
> fine. Here is what I did.
>
> gpg --recv-key 9E4FE3AF
> wget http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz.asc
> wget http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz
> gpg --verify spark-0.8.1-incubating.tgz.asc spark-0.8.1-incubating.tgz
> gpg: Signature made Tue 10 Dec 2013 02:53:15 PM PST using RSA key ID 9E4FE3AF
> gpg: Good signature from "Patrick Wendell <pw...@gmail.com>"
>
> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <ma...@clearstorydata.com> wrote:
>> I don't know how to make sense of the numbers, but here's what I've got
>> from a very small sample size.
>>
>> For both v0.8.0-incubating and v0.8.1-incubating, building separate
>> assemblies is faster than `./sbt/sbt assembly` and the times for building
>> separate assemblies for 0.8.0 and 0.8.1 are about the same.
>>
>> For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as the
>> sum of the separate assemblies.
>> For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as the
>> sum of the separate assemblies.
>>
>> Weird.
>>
>>
>> On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell <pw...@gmail.com>wrote:
>>
>>> I'll +1 myself also.
>>>
>>> For anyone who has the slow build problem: does this issue happen when
>>> building v0.8.0-incubating also? Trying to figure out whether it's
>>> related to something we added in 0.8.1 or if it's a long standing
>>> issue.
>>>
>>> - Patrick
>>>
>>> On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <ma...@gmail.com>
>>> wrote:
>>> > Woah, weird, but definitely good to know.
>>> >
>>> > If you’re doing Spark development, there’s also a more convenient option
>>> added by Shivaram in the master branch. You can do sbt assemble-deps to
>>> package *just* the dependencies of each project in a special assembly JAR,
>>> and then use sbt compile to update the code. This will use the classes
>>> directly out of the target/scala-2.9.3/classes directories. You have to
>>> redo assemble-deps only if your external dependencies change.
>>> >
>>> > Matei
>>> >
>>> > On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com>
>>> wrote:
>>> >
>>> >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can
>>> help.
>>> >> Again this is not a blocker for the release from my side either.
>>> >>
>>> >>
>>> >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <mark@clearstorydata.com
>>> >wrote:
>>> >>
>>> >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly`
>>> takes
>>> >>> a long, long, looooong time to complete (a MBP, in my case), building
>>> three
>>> >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
>>> >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less
>>> time.
>>> >>>
>>> >>>
>>> >>>
>>> >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <
>>> scrapcodes@gmail.com
>>> >>>> wrote:
>>> >>>
>>> >>>> forgot to mention, after running sbt/sbt assembly/assembly running
>>> >>> sbt/sbt
>>> >>>> examples/assembly takes just 37s. Not to mention my hardware is not
>>> >>> really
>>> >>>> great.
>>> >>>>
>>> >>>>
>>> >>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <
>>> scrapcodes@gmail.com
>>> >>>>> wrote:
>>> >>>>
>>> >>>>> Hi Patrick and Matei,
>>> >>>>>
>>> >>>>> Was trying out this and followed the quick start guide which says do
>>> >>>>> sbt/sbt assembly, like few others I was also stuck for few minutes on
>>> >>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is
>>> much
>>> >>>>> faster.
>>> >>>>>
>>> >>>>> Should we change the documentation to reflect this. It will not be
>>> >>> great
>>> >>>>> for first time users to get stuck there.
>>> >>>>>
>>> >>>>>
>>> >>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
>>> >>> matei.zaharia@gmail.com
>>> >>>>> wrote:
>>> >>>>>
>>> >>>>>> +1
>>> >>>>>>
>>> >>>>>> Built and tested it on Mac OS X.
>>> >>>>>>
>>> >>>>>> Matei
>>> >>>>>>
>>> >>>>>>
>>> >>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com>
>>> >>>> wrote:
>>> >>>>>>
>>> >>>>>>> Please vote on releasing the following candidate as Apache Spark
>>> >>>>>>> (incubating) version 0.8.1.
>>> >>>>>>>
>>> >>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
>>> >>>>>>>
>>> >>>>>>
>>> >>>>
>>> >>>
>>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
>>> >>>>>>>
>>> >>>>>>> The release files, including signatures, digests, etc can be found
>>> >>> at:
>>> >>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
>>> >>>>>>>
>>> >>>>>>> Release artifacts are signed with the following key:
>>> >>>>>>> https://people.apache.org/keys/committer/pwendell.asc
>>> >>>>>>>
>>> >>>>>>> The staging repository for this release can be found at:
>>> >>>>>>>
>>> >>>>
>>> https://repository.apache.org/content/repositories/orgapachespark-040/
>>> >>>>>>>
>>> >>>>>>> The documentation corresponding to this release can be found at:
>>> >>>>>>>
>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
>>> >>>>>>>
>>> >>>>>>> For information about the contents of this release see:
>>> >>>>>>>
>>> >>>>>>
>>> >>>>
>>> >>>
>>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
>>> >>>>>>>
>>> >>>>>>> Please vote on releasing this package as Apache Spark
>>> >>>> 0.8.1-incubating!
>>> >>>>>>>
>>> >>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
>>> >>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
>>> >>>>>>>
>>> >>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
>>> >>>>>>> [ ] -1 Do not release this package because ...
>>> >>>>>>>
>>> >>>>>>> To learn more about Apache Spark, please see
>>> >>>>>>> http://spark.incubator.apache.org/
>>> >>>>>>
>>> >>>>>>
>>> >>>>>
>>> >>>>>
>>> >>>>> --
>>> >>>>> s
>>> >>>>>
>>> >>>>
>>> >>>>
>>> >>>>
>>> >>>> --
>>> >>>> s
>>> >>>>
>>> >>>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> s
>>> >
>>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Patrick Wendell <pw...@gmail.com>.
Hey Tom,
I re-verified the signatures and got someone else to do it. It seemed
fine. Here is what I did.
gpg --recv-key 9E4FE3AF
wget http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz.asc
wget http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz
gpg --verify spark-0.8.1-incubating.tgz.asc spark-0.8.1-incubating.tgz
gpg: Signature made Tue 10 Dec 2013 02:53:15 PM PST using RSA key ID 9E4FE3AF
gpg: Good signature from "Patrick Wendell <pw...@gmail.com>"
On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <ma...@clearstorydata.com> wrote:
> I don't know how to make sense of the numbers, but here's what I've got
> from a very small sample size.
>
> For both v0.8.0-incubating and v0.8.1-incubating, building separate
> assemblies is faster than `./sbt/sbt assembly` and the times for building
> separate assemblies for 0.8.0 and 0.8.1 are about the same.
>
> For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as the
> sum of the separate assemblies.
> For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as the
> sum of the separate assemblies.
>
> Weird.
>
>
> On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell <pw...@gmail.com>wrote:
>
>> I'll +1 myself also.
>>
>> For anyone who has the slow build problem: does this issue happen when
>> building v0.8.0-incubating also? Trying to figure out whether it's
>> related to something we added in 0.8.1 or if it's a long standing
>> issue.
>>
>> - Patrick
>>
>> On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <ma...@gmail.com>
>> wrote:
>> > Woah, weird, but definitely good to know.
>> >
>> > If you’re doing Spark development, there’s also a more convenient option
>> added by Shivaram in the master branch. You can do sbt assemble-deps to
>> package *just* the dependencies of each project in a special assembly JAR,
>> and then use sbt compile to update the code. This will use the classes
>> directly out of the target/scala-2.9.3/classes directories. You have to
>> redo assemble-deps only if your external dependencies change.
>> >
>> > Matei
>> >
>> > On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com>
>> wrote:
>> >
>> >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can
>> help.
>> >> Again this is not a blocker for the release from my side either.
>> >>
>> >>
>> >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <mark@clearstorydata.com
>> >wrote:
>> >>
>> >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly`
>> takes
>> >>> a long, long, looooong time to complete (a MBP, in my case), building
>> three
>> >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
>> >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less
>> time.
>> >>>
>> >>>
>> >>>
>> >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <
>> scrapcodes@gmail.com
>> >>>> wrote:
>> >>>
>> >>>> forgot to mention, after running sbt/sbt assembly/assembly running
>> >>> sbt/sbt
>> >>>> examples/assembly takes just 37s. Not to mention my hardware is not
>> >>> really
>> >>>> great.
>> >>>>
>> >>>>
>> >>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <
>> scrapcodes@gmail.com
>> >>>>> wrote:
>> >>>>
>> >>>>> Hi Patrick and Matei,
>> >>>>>
>> >>>>> Was trying out this and followed the quick start guide which says do
>> >>>>> sbt/sbt assembly, like few others I was also stuck for few minutes on
>> >>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is
>> much
>> >>>>> faster.
>> >>>>>
>> >>>>> Should we change the documentation to reflect this. It will not be
>> >>> great
>> >>>>> for first time users to get stuck there.
>> >>>>>
>> >>>>>
>> >>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
>> >>> matei.zaharia@gmail.com
>> >>>>> wrote:
>> >>>>>
>> >>>>>> +1
>> >>>>>>
>> >>>>>> Built and tested it on Mac OS X.
>> >>>>>>
>> >>>>>> Matei
>> >>>>>>
>> >>>>>>
>> >>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com>
>> >>>> wrote:
>> >>>>>>
>> >>>>>>> Please vote on releasing the following candidate as Apache Spark
>> >>>>>>> (incubating) version 0.8.1.
>> >>>>>>>
>> >>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
>> >>>>>>>
>> >>>>>>
>> >>>>
>> >>>
>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
>> >>>>>>>
>> >>>>>>> The release files, including signatures, digests, etc can be found
>> >>> at:
>> >>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
>> >>>>>>>
>> >>>>>>> Release artifacts are signed with the following key:
>> >>>>>>> https://people.apache.org/keys/committer/pwendell.asc
>> >>>>>>>
>> >>>>>>> The staging repository for this release can be found at:
>> >>>>>>>
>> >>>>
>> https://repository.apache.org/content/repositories/orgapachespark-040/
>> >>>>>>>
>> >>>>>>> The documentation corresponding to this release can be found at:
>> >>>>>>>
>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
>> >>>>>>>
>> >>>>>>> For information about the contents of this release see:
>> >>>>>>>
>> >>>>>>
>> >>>>
>> >>>
>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
>> >>>>>>>
>> >>>>>>> Please vote on releasing this package as Apache Spark
>> >>>> 0.8.1-incubating!
>> >>>>>>>
>> >>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
>> >>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
>> >>>>>>>
>> >>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
>> >>>>>>> [ ] -1 Do not release this package because ...
>> >>>>>>>
>> >>>>>>> To learn more about Apache Spark, please see
>> >>>>>>> http://spark.incubator.apache.org/
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>> s
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>> s
>> >>>>
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> s
>> >
>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Azuryy Yu <az...@gmail.com>.
Hi Matei,
Thanks for your response. I am using 0.8.1, and yes, It was using
protobuf-2.5. Sorry I made a mistake before this email.
I used -Phadoop2-yarn, so I don't find it using pb-2.5, actually, I should
use -Pnew-yarn.
Thank you Matei.
On Mon, Dec 16, 2013 at 4:22 PM, Matei Zaharia <ma...@gmail.com>wrote:
> Are you using 0.8.1? It will build with protobuf 2.5 instead of 2.4 as
> long as you make it depend on Hadoop 2.2. But make sure you build it with
> SPARK_HADOOP_VERSION=2.2.0 or whatever.
>
> Spark 0.8.0 doesn’t support Hadoop 2.2 due to this issue.
>
> Matei
>
> On Dec 15, 2013, at 10:25 PM, Azuryy Yu <az...@gmail.com> wrote:
>
> > Maybe I am not give a clear description. I am runing Spark on yarn.
> instead
> > of Mesos. I just want build Spark with protobuf2.5. I am not care about
> > Mesos.
> >
> > I've changed Spark pom.xml to probobuf2.5 manually.
> >
> >
> >
> >
> > On Mon, Dec 16, 2013 at 2:02 PM, Matei Zaharia <matei.zaharia@gmail.com
> >wrote:
> >
> >> Mesos will almost certainly compile fine with protobuf 2.5. The protobuf
> >> compiler and binary format is forward-compatible across releases, it’s
> just
> >> the Java artifacts that aren’t. You’ll need to ask Mesos to provide a
> >> version with protobuf 2.5, and use that with these versions of Hadoop.
> >>
> >> Matei
> >>
> >> On Dec 15, 2013, at 7:00 PM, Liu, Raymond <ra...@intel.com>
> wrote:
> >>
> >>> That issue is for 0.9's solution.
> >>>
> >>> And if you mean for 0.8.1, when you build against hadoop 2.2 Yarn,
> >> protobuf is already using 2.5.0 instead of 2.4.1. so it will works fine
> >> with hadoop 2.2
> >>> And regarding on 0.8.1 you build against hadoop 2.2 Yarn, while run
> upon
> >> mesos... strange combination, I am not sure, might have problem. If have
> >> problem, you might need to build mesos against 2.5.0, I don't test
> that, if
> >> you got time, mind take a test?
> >>>
> >>> Best Regards,
> >>> Raymond Liu
> >>>
> >>>
> >>> -----Original Message-----
> >>> From: Liu, Raymond [mailto:raymond.liu@intel.com]
> >>> Sent: Monday, December 16, 2013 10:48 AM
> >>> To: dev@spark.incubator.apache.org
> >>> Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
> >>>
> >>> Hi Azuryy
> >>>
> >>> Please Check https://spark-project.atlassian.net/browse/SPARK-995 for
> >> this protobuf version issue
> >>>
> >>> Best Regards,
> >>> Raymond Liu
> >>>
> >>> -----Original Message-----
> >>> From: Azuryy Yu [mailto:azuryyyu@gmail.com]
> >>> Sent: Monday, December 16, 2013 10:30 AM
> >>> To: dev@spark.incubator.apache.org
> >>> Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
> >>>
> >>> Hi here,
> >>> Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some
> >> uncompatable API between these two versions.
> >>> Hadoop-2.x using protobuf-2.5.0
> >>>
> >>>
> >>> but if some guys want to run Spark on mesos, then mesos using
> >>> protobuf-2.4.1 currently. so we may discuss here for a better solution.
> >>>
> >>>
> >>>
> >>> On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu <az...@gmail.com> wrote:
> >>>
> >>>> Thanks Patrick.
> >>>> On 16 Dec 2013 02:43, "Patrick Wendell" <pw...@gmail.com> wrote:
> >>>>
> >>>>> You can checkout the docs mentioned in the vote thread. There is also
> >>>>> a pre-build binary for hadoop2 that is compiled for YARN 2.2
> >>>>>
> >>>>> - Patrick
> >>>>>
> >>>>> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu <az...@gmail.com>
> wrote:
> >>>>>> yarn 2.2, not yarn 0.22, I am so sorry.
> >>>>>>
> >>>>>>
> >>>>>> On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu <az...@gmail.com>
> >> wrote:
> >>>>>>
> >>>>>>> Hi,
> >>>>>>> Spark-0.8.1 supports yarn 0.22 right? where to find the release
> note?
> >>>>>>> Thanks.
> >>>>>>>
> >>>>>>>
> >>>>>>> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra <
> >>>>> henry.saputra@gmail.com>wrote:
> >>>>>>>
> >>>>>>>> Yeah seems like it. He was ok with our prev release.
> >>>>>>>> Let's wait for his reply
> >>>>>>>>
> >>>>>>>> On Saturday, December 14, 2013, Patrick Wendell wrote:
> >>>>>>>>
> >>>>>>>>> Henry - from that thread it looks like sebb's concern was
> >>>>>>>>> something different than this.
> >>>>>>>>>
> >>>>>>>>> On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <
> >>>>>>>> henry.saputra@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>> Hi Patrick,
> >>>>>>>>>>
> >>>>>>>>>> Yeap I agree, but technically ASF VOTE release on source
> >>>>>>>>>> only,
> >>>>> there
> >>>>>>>>>> even debate about it =), so putting it in the vote staging
> >>>>> artifact
> >>>>>>>>>> could confuse people because in our case we do package 3rd
> >>>>>>>>>> party libraries in the binary jars.
> >>>>>>>>>>
> >>>>>>>>>> I have sent email to sebb asking clarification about his
> >>>>>>>>>> concern
> >>>>> in
> >>>>>>>>>> general@ list.
> >>>>>>>>>>
> >>>>>>>>>> - Henry
> >>>>>>>>>>
> >>>>>>>>>> On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <
> >>>>> pwendell@gmail.com
> >>>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>> Hey Henry,
> >>>>>>>>>>>
> >>>>>>>>>>> One thing a lot of people do during the vote is test the
> >>>>> binaries and
> >>>>>>>>>>> make sure they work. This is really valuable. If you'd like
> >>>>>>>>>>> I
> >>>>> could
> >>>>>>>>>>> add a caveat to the vote thread explaining that we are only
> >>>>> voting on
> >>>>>>>>>>> the source.
> >>>>>>>>>>>
> >>>>>>>>>>> - Patrick
> >>>>>>>>>>>
> >>>>>>>>>>> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <
> >>>>>>>>> henry.saputra@gmail.com> wrote:
> >>>>>>>>>>>> Actually we should be fine putting the binaries there as
> >>>>>>>>>>>> long
> >>>>> as the
> >>>>>>>>>>>> VOTE is for the source.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Let's verify with sebb in the general@ list about his
> concern.
> >>>>>>>>>>>>
> >>>>>>>>>>>> - Henry
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <
> >>>>>>>>> henry.saputra@gmail.com> wrote:
> >>>>>>>>>>>>> Hi Patrick, as sebb has mentioned let's move the binaries
> >>>>>>>>>>>>> from
> >>>>> the
> >>>>>>>>>>>>> voting directory in your people.apache.org directory.
> >>>>>>>>>>>>> ASF release voting is for source code and not binaries,
> >>>>>>>>>>>>> and technically we provide binaries for convenience.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> And add link to the KEYS location in the dist[1] to let
> >>>>>>>>>>>>> verify
> >>>>>>>>> signatures.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Sorry for the late response to the VOTE thread, guys.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> - Henry
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> [1]
> >>>>>>>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <
> >>>>>>>> pwendell@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>> The vote is now closed. This vote passes with 5 PPMC +1's
> >>>>>>>>>>>>>> and
> >>>>> no 0
> >>>>>>>>> or -1
> >>>>>>>>>>>>>> votes.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> +1 (5 Total)
> >>>>>>>>>>>>>> Matei Zaharia*
> >>>>>>>>>>>>>> Nick Pentreath*
> >>>>>>>>>>>>>> Patrick Wendell*
> >>>>>>>>>>>>>> Prashant Sharma*
> >>>>>>>>>>>>>> Tom Graves*
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> 0 (0 Total)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> -1 (0 Total)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> * = Binding Vote
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> As per the incubator release guide [1] I'll be sending
> >>>>>>>>>>>>>> this
> >>>>> to the
> >>>>>>>>>>>>>> general incubator list for a final vote from IPMC members.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>
> http://incubator.apache.org/guides/releasemanagement.html#best-practi
> >>>>> ce-incubator-release-
> >>>>>>>>>>>>>> vote
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan
> >>>>>>>>>>>>>> <ev...@ooyala.com>
> >>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I'd be personally fine with a standard workflow of
> >>>>> assemble-deps
> >>>>>>>> +
> >>>>>>>>>>>>>>> packaging just the Spark files as separate packages, if
> >>>>>>>>>>>>>>> it
> >>>>>>>> speeds up
> >>>>>>>>>>>>>>> everyone's development time.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <
> >>>>>>>>> mark@clearstorydata.com
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I don't know how to make sense of the numbers, but
> >>>>>>>>>>>>>>>> here's
> >>>>> what
> >>>>>>>>> I've got
> >>>>>>>>>>>>>>>> from a very small sample size.
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>
> >>
>
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Matei Zaharia <ma...@gmail.com>.
Are you using 0.8.1? It will build with protobuf 2.5 instead of 2.4 as long as you make it depend on Hadoop 2.2. But make sure you build it with SPARK_HADOOP_VERSION=2.2.0 or whatever.
Spark 0.8.0 doesn’t support Hadoop 2.2 due to this issue.
Matei
On Dec 15, 2013, at 10:25 PM, Azuryy Yu <az...@gmail.com> wrote:
> Maybe I am not give a clear description. I am runing Spark on yarn. instead
> of Mesos. I just want build Spark with protobuf2.5. I am not care about
> Mesos.
>
> I've changed Spark pom.xml to probobuf2.5 manually.
>
>
>
>
> On Mon, Dec 16, 2013 at 2:02 PM, Matei Zaharia <ma...@gmail.com>wrote:
>
>> Mesos will almost certainly compile fine with protobuf 2.5. The protobuf
>> compiler and binary format is forward-compatible across releases, it’s just
>> the Java artifacts that aren’t. You’ll need to ask Mesos to provide a
>> version with protobuf 2.5, and use that with these versions of Hadoop.
>>
>> Matei
>>
>> On Dec 15, 2013, at 7:00 PM, Liu, Raymond <ra...@intel.com> wrote:
>>
>>> That issue is for 0.9's solution.
>>>
>>> And if you mean for 0.8.1, when you build against hadoop 2.2 Yarn,
>> protobuf is already using 2.5.0 instead of 2.4.1. so it will works fine
>> with hadoop 2.2
>>> And regarding on 0.8.1 you build against hadoop 2.2 Yarn, while run upon
>> mesos... strange combination, I am not sure, might have problem. If have
>> problem, you might need to build mesos against 2.5.0, I don't test that, if
>> you got time, mind take a test?
>>>
>>> Best Regards,
>>> Raymond Liu
>>>
>>>
>>> -----Original Message-----
>>> From: Liu, Raymond [mailto:raymond.liu@intel.com]
>>> Sent: Monday, December 16, 2013 10:48 AM
>>> To: dev@spark.incubator.apache.org
>>> Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
>>>
>>> Hi Azuryy
>>>
>>> Please Check https://spark-project.atlassian.net/browse/SPARK-995 for
>> this protobuf version issue
>>>
>>> Best Regards,
>>> Raymond Liu
>>>
>>> -----Original Message-----
>>> From: Azuryy Yu [mailto:azuryyyu@gmail.com]
>>> Sent: Monday, December 16, 2013 10:30 AM
>>> To: dev@spark.incubator.apache.org
>>> Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
>>>
>>> Hi here,
>>> Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some
>> uncompatable API between these two versions.
>>> Hadoop-2.x using protobuf-2.5.0
>>>
>>>
>>> but if some guys want to run Spark on mesos, then mesos using
>>> protobuf-2.4.1 currently. so we may discuss here for a better solution.
>>>
>>>
>>>
>>> On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu <az...@gmail.com> wrote:
>>>
>>>> Thanks Patrick.
>>>> On 16 Dec 2013 02:43, "Patrick Wendell" <pw...@gmail.com> wrote:
>>>>
>>>>> You can checkout the docs mentioned in the vote thread. There is also
>>>>> a pre-build binary for hadoop2 that is compiled for YARN 2.2
>>>>>
>>>>> - Patrick
>>>>>
>>>>> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu <az...@gmail.com> wrote:
>>>>>> yarn 2.2, not yarn 0.22, I am so sorry.
>>>>>>
>>>>>>
>>>>>> On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu <az...@gmail.com>
>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> Spark-0.8.1 supports yarn 0.22 right? where to find the release note?
>>>>>>> Thanks.
>>>>>>>
>>>>>>>
>>>>>>> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra <
>>>>> henry.saputra@gmail.com>wrote:
>>>>>>>
>>>>>>>> Yeah seems like it. He was ok with our prev release.
>>>>>>>> Let's wait for his reply
>>>>>>>>
>>>>>>>> On Saturday, December 14, 2013, Patrick Wendell wrote:
>>>>>>>>
>>>>>>>>> Henry - from that thread it looks like sebb's concern was
>>>>>>>>> something different than this.
>>>>>>>>>
>>>>>>>>> On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <
>>>>>>>> henry.saputra@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>> Hi Patrick,
>>>>>>>>>>
>>>>>>>>>> Yeap I agree, but technically ASF VOTE release on source
>>>>>>>>>> only,
>>>>> there
>>>>>>>>>> even debate about it =), so putting it in the vote staging
>>>>> artifact
>>>>>>>>>> could confuse people because in our case we do package 3rd
>>>>>>>>>> party libraries in the binary jars.
>>>>>>>>>>
>>>>>>>>>> I have sent email to sebb asking clarification about his
>>>>>>>>>> concern
>>>>> in
>>>>>>>>>> general@ list.
>>>>>>>>>>
>>>>>>>>>> - Henry
>>>>>>>>>>
>>>>>>>>>> On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <
>>>>> pwendell@gmail.com
>>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>> Hey Henry,
>>>>>>>>>>>
>>>>>>>>>>> One thing a lot of people do during the vote is test the
>>>>> binaries and
>>>>>>>>>>> make sure they work. This is really valuable. If you'd like
>>>>>>>>>>> I
>>>>> could
>>>>>>>>>>> add a caveat to the vote thread explaining that we are only
>>>>> voting on
>>>>>>>>>>> the source.
>>>>>>>>>>>
>>>>>>>>>>> - Patrick
>>>>>>>>>>>
>>>>>>>>>>> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <
>>>>>>>>> henry.saputra@gmail.com> wrote:
>>>>>>>>>>>> Actually we should be fine putting the binaries there as
>>>>>>>>>>>> long
>>>>> as the
>>>>>>>>>>>> VOTE is for the source.
>>>>>>>>>>>>
>>>>>>>>>>>> Let's verify with sebb in the general@ list about his concern.
>>>>>>>>>>>>
>>>>>>>>>>>> - Henry
>>>>>>>>>>>>
>>>>>>>>>>>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <
>>>>>>>>> henry.saputra@gmail.com> wrote:
>>>>>>>>>>>>> Hi Patrick, as sebb has mentioned let's move the binaries
>>>>>>>>>>>>> from
>>>>> the
>>>>>>>>>>>>> voting directory in your people.apache.org directory.
>>>>>>>>>>>>> ASF release voting is for source code and not binaries,
>>>>>>>>>>>>> and technically we provide binaries for convenience.
>>>>>>>>>>>>>
>>>>>>>>>>>>> And add link to the KEYS location in the dist[1] to let
>>>>>>>>>>>>> verify
>>>>>>>>> signatures.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sorry for the late response to the VOTE thread, guys.
>>>>>>>>>>>>>
>>>>>>>>>>>>> - Henry
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1]
>>>>>>>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <
>>>>>>>> pwendell@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>>> The vote is now closed. This vote passes with 5 PPMC +1's
>>>>>>>>>>>>>> and
>>>>> no 0
>>>>>>>>> or -1
>>>>>>>>>>>>>> votes.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> +1 (5 Total)
>>>>>>>>>>>>>> Matei Zaharia*
>>>>>>>>>>>>>> Nick Pentreath*
>>>>>>>>>>>>>> Patrick Wendell*
>>>>>>>>>>>>>> Prashant Sharma*
>>>>>>>>>>>>>> Tom Graves*
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 0 (0 Total)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> -1 (0 Total)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> * = Binding Vote
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As per the incubator release guide [1] I'll be sending
>>>>>>>>>>>>>> this
>>>>> to the
>>>>>>>>>>>>>> general incubator list for a final vote from IPMC members.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>> http://incubator.apache.org/guides/releasemanagement.html#best-practi
>>>>> ce-incubator-release-
>>>>>>>>>>>>>> vote
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan
>>>>>>>>>>>>>> <ev...@ooyala.com>
>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I'd be personally fine with a standard workflow of
>>>>> assemble-deps
>>>>>>>> +
>>>>>>>>>>>>>>> packaging just the Spark files as separate packages, if
>>>>>>>>>>>>>>> it
>>>>>>>> speeds up
>>>>>>>>>>>>>>> everyone's development time.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <
>>>>>>>>> mark@clearstorydata.com
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I don't know how to make sense of the numbers, but
>>>>>>>>>>>>>>>> here's
>>>>> what
>>>>>>>>> I've got
>>>>>>>>>>>>>>>> from a very small sample size.
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>
>>
>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Azuryy Yu <az...@gmail.com>.
Maybe I am not give a clear description. I am runing Spark on yarn. instead
of Mesos. I just want build Spark with protobuf2.5. I am not care about
Mesos.
I've changed Spark pom.xml to probobuf2.5 manually.
On Mon, Dec 16, 2013 at 2:02 PM, Matei Zaharia <ma...@gmail.com>wrote:
> Mesos will almost certainly compile fine with protobuf 2.5. The protobuf
> compiler and binary format is forward-compatible across releases, it’s just
> the Java artifacts that aren’t. You’ll need to ask Mesos to provide a
> version with protobuf 2.5, and use that with these versions of Hadoop.
>
> Matei
>
> On Dec 15, 2013, at 7:00 PM, Liu, Raymond <ra...@intel.com> wrote:
>
> > That issue is for 0.9's solution.
> >
> > And if you mean for 0.8.1, when you build against hadoop 2.2 Yarn,
> protobuf is already using 2.5.0 instead of 2.4.1. so it will works fine
> with hadoop 2.2
> > And regarding on 0.8.1 you build against hadoop 2.2 Yarn, while run upon
> mesos... strange combination, I am not sure, might have problem. If have
> problem, you might need to build mesos against 2.5.0, I don't test that, if
> you got time, mind take a test?
> >
> > Best Regards,
> > Raymond Liu
> >
> >
> > -----Original Message-----
> > From: Liu, Raymond [mailto:raymond.liu@intel.com]
> > Sent: Monday, December 16, 2013 10:48 AM
> > To: dev@spark.incubator.apache.org
> > Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
> >
> > Hi Azuryy
> >
> > Please Check https://spark-project.atlassian.net/browse/SPARK-995 for
> this protobuf version issue
> >
> > Best Regards,
> > Raymond Liu
> >
> > -----Original Message-----
> > From: Azuryy Yu [mailto:azuryyyu@gmail.com]
> > Sent: Monday, December 16, 2013 10:30 AM
> > To: dev@spark.incubator.apache.org
> > Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
> >
> > Hi here,
> > Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some
> uncompatable API between these two versions.
> > Hadoop-2.x using protobuf-2.5.0
> >
> >
> > but if some guys want to run Spark on mesos, then mesos using
> > protobuf-2.4.1 currently. so we may discuss here for a better solution.
> >
> >
> >
> > On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu <az...@gmail.com> wrote:
> >
> >> Thanks Patrick.
> >> On 16 Dec 2013 02:43, "Patrick Wendell" <pw...@gmail.com> wrote:
> >>
> >>> You can checkout the docs mentioned in the vote thread. There is also
> >>> a pre-build binary for hadoop2 that is compiled for YARN 2.2
> >>>
> >>> - Patrick
> >>>
> >>> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu <az...@gmail.com> wrote:
> >>>> yarn 2.2, not yarn 0.22, I am so sorry.
> >>>>
> >>>>
> >>>> On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu <az...@gmail.com>
> wrote:
> >>>>
> >>>>> Hi,
> >>>>> Spark-0.8.1 supports yarn 0.22 right? where to find the release note?
> >>>>> Thanks.
> >>>>>
> >>>>>
> >>>>> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra <
> >>> henry.saputra@gmail.com>wrote:
> >>>>>
> >>>>>> Yeah seems like it. He was ok with our prev release.
> >>>>>> Let's wait for his reply
> >>>>>>
> >>>>>> On Saturday, December 14, 2013, Patrick Wendell wrote:
> >>>>>>
> >>>>>>> Henry - from that thread it looks like sebb's concern was
> >>>>>>> something different than this.
> >>>>>>>
> >>>>>>> On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <
> >>>>>> henry.saputra@gmail.com>
> >>>>>>> wrote:
> >>>>>>>> Hi Patrick,
> >>>>>>>>
> >>>>>>>> Yeap I agree, but technically ASF VOTE release on source
> >>>>>>>> only,
> >>> there
> >>>>>>>> even debate about it =), so putting it in the vote staging
> >>> artifact
> >>>>>>>> could confuse people because in our case we do package 3rd
> >>>>>>>> party libraries in the binary jars.
> >>>>>>>>
> >>>>>>>> I have sent email to sebb asking clarification about his
> >>>>>>>> concern
> >>> in
> >>>>>>>> general@ list.
> >>>>>>>>
> >>>>>>>> - Henry
> >>>>>>>>
> >>>>>>>> On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <
> >>> pwendell@gmail.com
> >>>>>>>
> >>>>>>> wrote:
> >>>>>>>>> Hey Henry,
> >>>>>>>>>
> >>>>>>>>> One thing a lot of people do during the vote is test the
> >>> binaries and
> >>>>>>>>> make sure they work. This is really valuable. If you'd like
> >>>>>>>>> I
> >>> could
> >>>>>>>>> add a caveat to the vote thread explaining that we are only
> >>> voting on
> >>>>>>>>> the source.
> >>>>>>>>>
> >>>>>>>>> - Patrick
> >>>>>>>>>
> >>>>>>>>> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <
> >>>>>>> henry.saputra@gmail.com> wrote:
> >>>>>>>>>> Actually we should be fine putting the binaries there as
> >>>>>>>>>> long
> >>> as the
> >>>>>>>>>> VOTE is for the source.
> >>>>>>>>>>
> >>>>>>>>>> Let's verify with sebb in the general@ list about his concern.
> >>>>>>>>>>
> >>>>>>>>>> - Henry
> >>>>>>>>>>
> >>>>>>>>>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <
> >>>>>>> henry.saputra@gmail.com> wrote:
> >>>>>>>>>>> Hi Patrick, as sebb has mentioned let's move the binaries
> >>>>>>>>>>> from
> >>> the
> >>>>>>>>>>> voting directory in your people.apache.org directory.
> >>>>>>>>>>> ASF release voting is for source code and not binaries,
> >>>>>>>>>>> and technically we provide binaries for convenience.
> >>>>>>>>>>>
> >>>>>>>>>>> And add link to the KEYS location in the dist[1] to let
> >>>>>>>>>>> verify
> >>>>>>> signatures.
> >>>>>>>>>>>
> >>>>>>>>>>> Sorry for the late response to the VOTE thread, guys.
> >>>>>>>>>>>
> >>>>>>>>>>> - Henry
> >>>>>>>>>>>
> >>>>>>>>>>> [1]
> >>>>>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <
> >>>>>> pwendell@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>>>> The vote is now closed. This vote passes with 5 PPMC +1's
> >>>>>>>>>>>> and
> >>> no 0
> >>>>>>> or -1
> >>>>>>>>>>>> votes.
> >>>>>>>>>>>>
> >>>>>>>>>>>> +1 (5 Total)
> >>>>>>>>>>>> Matei Zaharia*
> >>>>>>>>>>>> Nick Pentreath*
> >>>>>>>>>>>> Patrick Wendell*
> >>>>>>>>>>>> Prashant Sharma*
> >>>>>>>>>>>> Tom Graves*
> >>>>>>>>>>>>
> >>>>>>>>>>>> 0 (0 Total)
> >>>>>>>>>>>>
> >>>>>>>>>>>> -1 (0 Total)
> >>>>>>>>>>>>
> >>>>>>>>>>>> * = Binding Vote
> >>>>>>>>>>>>
> >>>>>>>>>>>> As per the incubator release guide [1] I'll be sending
> >>>>>>>>>>>> this
> >>> to the
> >>>>>>>>>>>> general incubator list for a final vote from IPMC members.
> >>>>>>>>>>>>
> >>>>>>>>>>>> [1]
> >>>>>>>>>>>>
> >>>>>>>
> >>>>>>
> >>> http://incubator.apache.org/guides/releasemanagement.html#best-practi
> >>> ce-incubator-release-
> >>>>>>>>>>>> vote
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan
> >>>>>>>>>>>> <ev...@ooyala.com>
> >>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> I'd be personally fine with a standard workflow of
> >>> assemble-deps
> >>>>>> +
> >>>>>>>>>>>>> packaging just the Spark files as separate packages, if
> >>>>>>>>>>>>> it
> >>>>>> speeds up
> >>>>>>>>>>>>> everyone's development time.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <
> >>>>>>> mark@clearstorydata.com
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> I don't know how to make sense of the numbers, but
> >>>>>>>>>>>>>> here's
> >>> what
> >>>>>>> I've got
> >>>>>>>>>>>>>> from a very small sample size.
> >>>>>>
> >>>>>
> >>>>>
> >>>
> >>
>
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Matei Zaharia <ma...@gmail.com>.
Mesos will almost certainly compile fine with protobuf 2.5. The protobuf compiler and binary format is forward-compatible across releases, it’s just the Java artifacts that aren’t. You’ll need to ask Mesos to provide a version with protobuf 2.5, and use that with these versions of Hadoop.
Matei
On Dec 15, 2013, at 7:00 PM, Liu, Raymond <ra...@intel.com> wrote:
> That issue is for 0.9's solution.
>
> And if you mean for 0.8.1, when you build against hadoop 2.2 Yarn, protobuf is already using 2.5.0 instead of 2.4.1. so it will works fine with hadoop 2.2
> And regarding on 0.8.1 you build against hadoop 2.2 Yarn, while run upon mesos... strange combination, I am not sure, might have problem. If have problem, you might need to build mesos against 2.5.0, I don't test that, if you got time, mind take a test?
>
> Best Regards,
> Raymond Liu
>
>
> -----Original Message-----
> From: Liu, Raymond [mailto:raymond.liu@intel.com]
> Sent: Monday, December 16, 2013 10:48 AM
> To: dev@spark.incubator.apache.org
> Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
>
> Hi Azuryy
>
> Please Check https://spark-project.atlassian.net/browse/SPARK-995 for this protobuf version issue
>
> Best Regards,
> Raymond Liu
>
> -----Original Message-----
> From: Azuryy Yu [mailto:azuryyyu@gmail.com]
> Sent: Monday, December 16, 2013 10:30 AM
> To: dev@spark.incubator.apache.org
> Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
>
> Hi here,
> Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some uncompatable API between these two versions.
> Hadoop-2.x using protobuf-2.5.0
>
>
> but if some guys want to run Spark on mesos, then mesos using
> protobuf-2.4.1 currently. so we may discuss here for a better solution.
>
>
>
> On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu <az...@gmail.com> wrote:
>
>> Thanks Patrick.
>> On 16 Dec 2013 02:43, "Patrick Wendell" <pw...@gmail.com> wrote:
>>
>>> You can checkout the docs mentioned in the vote thread. There is also
>>> a pre-build binary for hadoop2 that is compiled for YARN 2.2
>>>
>>> - Patrick
>>>
>>> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu <az...@gmail.com> wrote:
>>>> yarn 2.2, not yarn 0.22, I am so sorry.
>>>>
>>>>
>>>> On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu <az...@gmail.com> wrote:
>>>>
>>>>> Hi,
>>>>> Spark-0.8.1 supports yarn 0.22 right? where to find the release note?
>>>>> Thanks.
>>>>>
>>>>>
>>>>> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra <
>>> henry.saputra@gmail.com>wrote:
>>>>>
>>>>>> Yeah seems like it. He was ok with our prev release.
>>>>>> Let's wait for his reply
>>>>>>
>>>>>> On Saturday, December 14, 2013, Patrick Wendell wrote:
>>>>>>
>>>>>>> Henry - from that thread it looks like sebb's concern was
>>>>>>> something different than this.
>>>>>>>
>>>>>>> On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <
>>>>>> henry.saputra@gmail.com>
>>>>>>> wrote:
>>>>>>>> Hi Patrick,
>>>>>>>>
>>>>>>>> Yeap I agree, but technically ASF VOTE release on source
>>>>>>>> only,
>>> there
>>>>>>>> even debate about it =), so putting it in the vote staging
>>> artifact
>>>>>>>> could confuse people because in our case we do package 3rd
>>>>>>>> party libraries in the binary jars.
>>>>>>>>
>>>>>>>> I have sent email to sebb asking clarification about his
>>>>>>>> concern
>>> in
>>>>>>>> general@ list.
>>>>>>>>
>>>>>>>> - Henry
>>>>>>>>
>>>>>>>> On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <
>>> pwendell@gmail.com
>>>>>>>
>>>>>>> wrote:
>>>>>>>>> Hey Henry,
>>>>>>>>>
>>>>>>>>> One thing a lot of people do during the vote is test the
>>> binaries and
>>>>>>>>> make sure they work. This is really valuable. If you'd like
>>>>>>>>> I
>>> could
>>>>>>>>> add a caveat to the vote thread explaining that we are only
>>> voting on
>>>>>>>>> the source.
>>>>>>>>>
>>>>>>>>> - Patrick
>>>>>>>>>
>>>>>>>>> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <
>>>>>>> henry.saputra@gmail.com> wrote:
>>>>>>>>>> Actually we should be fine putting the binaries there as
>>>>>>>>>> long
>>> as the
>>>>>>>>>> VOTE is for the source.
>>>>>>>>>>
>>>>>>>>>> Let's verify with sebb in the general@ list about his concern.
>>>>>>>>>>
>>>>>>>>>> - Henry
>>>>>>>>>>
>>>>>>>>>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <
>>>>>>> henry.saputra@gmail.com> wrote:
>>>>>>>>>>> Hi Patrick, as sebb has mentioned let's move the binaries
>>>>>>>>>>> from
>>> the
>>>>>>>>>>> voting directory in your people.apache.org directory.
>>>>>>>>>>> ASF release voting is for source code and not binaries,
>>>>>>>>>>> and technically we provide binaries for convenience.
>>>>>>>>>>>
>>>>>>>>>>> And add link to the KEYS location in the dist[1] to let
>>>>>>>>>>> verify
>>>>>>> signatures.
>>>>>>>>>>>
>>>>>>>>>>> Sorry for the late response to the VOTE thread, guys.
>>>>>>>>>>>
>>>>>>>>>>> - Henry
>>>>>>>>>>>
>>>>>>>>>>> [1]
>>>>>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <
>>>>>> pwendell@gmail.com>
>>>>>>> wrote:
>>>>>>>>>>>> The vote is now closed. This vote passes with 5 PPMC +1's
>>>>>>>>>>>> and
>>> no 0
>>>>>>> or -1
>>>>>>>>>>>> votes.
>>>>>>>>>>>>
>>>>>>>>>>>> +1 (5 Total)
>>>>>>>>>>>> Matei Zaharia*
>>>>>>>>>>>> Nick Pentreath*
>>>>>>>>>>>> Patrick Wendell*
>>>>>>>>>>>> Prashant Sharma*
>>>>>>>>>>>> Tom Graves*
>>>>>>>>>>>>
>>>>>>>>>>>> 0 (0 Total)
>>>>>>>>>>>>
>>>>>>>>>>>> -1 (0 Total)
>>>>>>>>>>>>
>>>>>>>>>>>> * = Binding Vote
>>>>>>>>>>>>
>>>>>>>>>>>> As per the incubator release guide [1] I'll be sending
>>>>>>>>>>>> this
>>> to the
>>>>>>>>>>>> general incubator list for a final vote from IPMC members.
>>>>>>>>>>>>
>>>>>>>>>>>> [1]
>>>>>>>>>>>>
>>>>>>>
>>>>>>
>>> http://incubator.apache.org/guides/releasemanagement.html#best-practi
>>> ce-incubator-release-
>>>>>>>>>>>> vote
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan
>>>>>>>>>>>> <ev...@ooyala.com>
>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I'd be personally fine with a standard workflow of
>>> assemble-deps
>>>>>> +
>>>>>>>>>>>>> packaging just the Spark files as separate packages, if
>>>>>>>>>>>>> it
>>>>>> speeds up
>>>>>>>>>>>>> everyone's development time.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <
>>>>>>> mark@clearstorydata.com
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I don't know how to make sense of the numbers, but
>>>>>>>>>>>>>> here's
>>> what
>>>>>>> I've got
>>>>>>>>>>>>>> from a very small sample size.
>>>>>>
>>>>>
>>>>>
>>>
>>
RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by "Liu, Raymond" <ra...@intel.com>.
That issue is for 0.9's solution.
And if you mean for 0.8.1, when you build against hadoop 2.2 Yarn, protobuf is already using 2.5.0 instead of 2.4.1. so it will works fine with hadoop 2.2
And regarding on 0.8.1 you build against hadoop 2.2 Yarn, while run upon mesos... strange combination, I am not sure, might have problem. If have problem, you might need to build mesos against 2.5.0, I don't test that, if you got time, mind take a test?
Best Regards,
Raymond Liu
-----Original Message-----
From: Liu, Raymond [mailto:raymond.liu@intel.com]
Sent: Monday, December 16, 2013 10:48 AM
To: dev@spark.incubator.apache.org
Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Hi Azuryy
Please Check https://spark-project.atlassian.net/browse/SPARK-995 for this protobuf version issue
Best Regards,
Raymond Liu
-----Original Message-----
From: Azuryy Yu [mailto:azuryyyu@gmail.com]
Sent: Monday, December 16, 2013 10:30 AM
To: dev@spark.incubator.apache.org
Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Hi here,
Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some uncompatable API between these two versions.
Hadoop-2.x using protobuf-2.5.0
but if some guys want to run Spark on mesos, then mesos using
protobuf-2.4.1 currently. so we may discuss here for a better solution.
On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu <az...@gmail.com> wrote:
> Thanks Patrick.
> On 16 Dec 2013 02:43, "Patrick Wendell" <pw...@gmail.com> wrote:
>
>> You can checkout the docs mentioned in the vote thread. There is also
>> a pre-build binary for hadoop2 that is compiled for YARN 2.2
>>
>> - Patrick
>>
>> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu <az...@gmail.com> wrote:
>> > yarn 2.2, not yarn 0.22, I am so sorry.
>> >
>> >
>> > On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu <az...@gmail.com> wrote:
>> >
>> >> Hi,
>> >> Spark-0.8.1 supports yarn 0.22 right? where to find the release note?
>> >> Thanks.
>> >>
>> >>
>> >> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra <
>> henry.saputra@gmail.com>wrote:
>> >>
>> >>> Yeah seems like it. He was ok with our prev release.
>> >>> Let's wait for his reply
>> >>>
>> >>> On Saturday, December 14, 2013, Patrick Wendell wrote:
>> >>>
>> >>> > Henry - from that thread it looks like sebb's concern was
>> >>> > something different than this.
>> >>> >
>> >>> > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <
>> >>> henry.saputra@gmail.com>
>> >>> > wrote:
>> >>> > > Hi Patrick,
>> >>> > >
>> >>> > > Yeap I agree, but technically ASF VOTE release on source
>> >>> > > only,
>> there
>> >>> > > even debate about it =), so putting it in the vote staging
>> artifact
>> >>> > > could confuse people because in our case we do package 3rd
>> >>> > > party libraries in the binary jars.
>> >>> > >
>> >>> > > I have sent email to sebb asking clarification about his
>> >>> > > concern
>> in
>> >>> > > general@ list.
>> >>> > >
>> >>> > > - Henry
>> >>> > >
>> >>> > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <
>> pwendell@gmail.com
>> >>> >
>> >>> > wrote:
>> >>> > >> Hey Henry,
>> >>> > >>
>> >>> > >> One thing a lot of people do during the vote is test the
>> binaries and
>> >>> > >> make sure they work. This is really valuable. If you'd like
>> >>> > >> I
>> could
>> >>> > >> add a caveat to the vote thread explaining that we are only
>> voting on
>> >>> > >> the source.
>> >>> > >>
>> >>> > >> - Patrick
>> >>> > >>
>> >>> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <
>> >>> > henry.saputra@gmail.com> wrote:
>> >>> > >>> Actually we should be fine putting the binaries there as
>> >>> > >>> long
>> as the
>> >>> > >>> VOTE is for the source.
>> >>> > >>>
>> >>> > >>> Let's verify with sebb in the general@ list about his concern.
>> >>> > >>>
>> >>> > >>> - Henry
>> >>> > >>>
>> >>> > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <
>> >>> > henry.saputra@gmail.com> wrote:
>> >>> > >>>> Hi Patrick, as sebb has mentioned let's move the binaries
>> >>> > >>>> from
>> the
>> >>> > >>>> voting directory in your people.apache.org directory.
>> >>> > >>>> ASF release voting is for source code and not binaries,
>> >>> > >>>> and technically we provide binaries for convenience.
>> >>> > >>>>
>> >>> > >>>> And add link to the KEYS location in the dist[1] to let
>> >>> > >>>> verify
>> >>> > signatures.
>> >>> > >>>>
>> >>> > >>>> Sorry for the late response to the VOTE thread, guys.
>> >>> > >>>>
>> >>> > >>>> - Henry
>> >>> > >>>>
>> >>> > >>>> [1]
>> >>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
>> >>> > >>>>
>> >>> > >>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <
>> >>> pwendell@gmail.com>
>> >>> > wrote:
>> >>> > >>>>> The vote is now closed. This vote passes with 5 PPMC +1's
>> >>> > >>>>> and
>> no 0
>> >>> > or -1
>> >>> > >>>>> votes.
>> >>> > >>>>>
>> >>> > >>>>> +1 (5 Total)
>> >>> > >>>>> Matei Zaharia*
>> >>> > >>>>> Nick Pentreath*
>> >>> > >>>>> Patrick Wendell*
>> >>> > >>>>> Prashant Sharma*
>> >>> > >>>>> Tom Graves*
>> >>> > >>>>>
>> >>> > >>>>> 0 (0 Total)
>> >>> > >>>>>
>> >>> > >>>>> -1 (0 Total)
>> >>> > >>>>>
>> >>> > >>>>> * = Binding Vote
>> >>> > >>>>>
>> >>> > >>>>> As per the incubator release guide [1] I'll be sending
>> >>> > >>>>> this
>> to the
>> >>> > >>>>> general incubator list for a final vote from IPMC members.
>> >>> > >>>>>
>> >>> > >>>>> [1]
>> >>> > >>>>>
>> >>> >
>> >>>
>> http://incubator.apache.org/guides/releasemanagement.html#best-practi
>> ce-incubator-release-
>> >>> > >>>>> vote
>> >>> > >>>>>
>> >>> > >>>>>
>> >>> > >>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan
>> >>> > >>>>> <ev...@ooyala.com>
>> wrote:
>> >>> > >>>>>
>> >>> > >>>>>> I'd be personally fine with a standard workflow of
>> assemble-deps
>> >>> +
>> >>> > >>>>>> packaging just the Spark files as separate packages, if
>> >>> > >>>>>> it
>> >>> speeds up
>> >>> > >>>>>> everyone's development time.
>> >>> > >>>>>>
>> >>> > >>>>>>
>> >>> > >>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <
>> >>> > mark@clearstorydata.com
>> >>> > >>>>>> >wrote:
>> >>> > >>>>>>
>> >>> > >>>>>> > I don't know how to make sense of the numbers, but
>> >>> > >>>>>> > here's
>> what
>> >>> > I've got
>> >>> > >>>>>> > from a very small sample size.
>> >>>
>> >>
>> >>
>>
>
RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by "Liu, Raymond" <ra...@intel.com>.
Hi Azuryy
Please Check https://spark-project.atlassian.net/browse/SPARK-995 for this protobuf version issue
Best Regards,
Raymond Liu
-----Original Message-----
From: Azuryy Yu [mailto:azuryyyu@gmail.com]
Sent: Monday, December 16, 2013 10:30 AM
To: dev@spark.incubator.apache.org
Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Hi here,
Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some uncompatable API between these two versions.
Hadoop-2.x using protobuf-2.5.0
but if some guys want to run Spark on mesos, then mesos using
protobuf-2.4.1 currently. so we may discuss here for a better solution.
On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu <az...@gmail.com> wrote:
> Thanks Patrick.
> On 16 Dec 2013 02:43, "Patrick Wendell" <pw...@gmail.com> wrote:
>
>> You can checkout the docs mentioned in the vote thread. There is also
>> a pre-build binary for hadoop2 that is compiled for YARN 2.2
>>
>> - Patrick
>>
>> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu <az...@gmail.com> wrote:
>> > yarn 2.2, not yarn 0.22, I am so sorry.
>> >
>> >
>> > On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu <az...@gmail.com> wrote:
>> >
>> >> Hi,
>> >> Spark-0.8.1 supports yarn 0.22 right? where to find the release note?
>> >> Thanks.
>> >>
>> >>
>> >> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra <
>> henry.saputra@gmail.com>wrote:
>> >>
>> >>> Yeah seems like it. He was ok with our prev release.
>> >>> Let's wait for his reply
>> >>>
>> >>> On Saturday, December 14, 2013, Patrick Wendell wrote:
>> >>>
>> >>> > Henry - from that thread it looks like sebb's concern was
>> >>> > something different than this.
>> >>> >
>> >>> > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <
>> >>> henry.saputra@gmail.com>
>> >>> > wrote:
>> >>> > > Hi Patrick,
>> >>> > >
>> >>> > > Yeap I agree, but technically ASF VOTE release on source
>> >>> > > only,
>> there
>> >>> > > even debate about it =), so putting it in the vote staging
>> artifact
>> >>> > > could confuse people because in our case we do package 3rd
>> >>> > > party libraries in the binary jars.
>> >>> > >
>> >>> > > I have sent email to sebb asking clarification about his
>> >>> > > concern
>> in
>> >>> > > general@ list.
>> >>> > >
>> >>> > > - Henry
>> >>> > >
>> >>> > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <
>> pwendell@gmail.com
>> >>> >
>> >>> > wrote:
>> >>> > >> Hey Henry,
>> >>> > >>
>> >>> > >> One thing a lot of people do during the vote is test the
>> binaries and
>> >>> > >> make sure they work. This is really valuable. If you'd like
>> >>> > >> I
>> could
>> >>> > >> add a caveat to the vote thread explaining that we are only
>> voting on
>> >>> > >> the source.
>> >>> > >>
>> >>> > >> - Patrick
>> >>> > >>
>> >>> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <
>> >>> > henry.saputra@gmail.com> wrote:
>> >>> > >>> Actually we should be fine putting the binaries there as
>> >>> > >>> long
>> as the
>> >>> > >>> VOTE is for the source.
>> >>> > >>>
>> >>> > >>> Let's verify with sebb in the general@ list about his concern.
>> >>> > >>>
>> >>> > >>> - Henry
>> >>> > >>>
>> >>> > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <
>> >>> > henry.saputra@gmail.com> wrote:
>> >>> > >>>> Hi Patrick, as sebb has mentioned let's move the binaries
>> >>> > >>>> from
>> the
>> >>> > >>>> voting directory in your people.apache.org directory.
>> >>> > >>>> ASF release voting is for source code and not binaries,
>> >>> > >>>> and technically we provide binaries for convenience.
>> >>> > >>>>
>> >>> > >>>> And add link to the KEYS location in the dist[1] to let
>> >>> > >>>> verify
>> >>> > signatures.
>> >>> > >>>>
>> >>> > >>>> Sorry for the late response to the VOTE thread, guys.
>> >>> > >>>>
>> >>> > >>>> - Henry
>> >>> > >>>>
>> >>> > >>>> [1]
>> >>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
>> >>> > >>>>
>> >>> > >>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <
>> >>> pwendell@gmail.com>
>> >>> > wrote:
>> >>> > >>>>> The vote is now closed. This vote passes with 5 PPMC +1's
>> >>> > >>>>> and
>> no 0
>> >>> > or -1
>> >>> > >>>>> votes.
>> >>> > >>>>>
>> >>> > >>>>> +1 (5 Total)
>> >>> > >>>>> Matei Zaharia*
>> >>> > >>>>> Nick Pentreath*
>> >>> > >>>>> Patrick Wendell*
>> >>> > >>>>> Prashant Sharma*
>> >>> > >>>>> Tom Graves*
>> >>> > >>>>>
>> >>> > >>>>> 0 (0 Total)
>> >>> > >>>>>
>> >>> > >>>>> -1 (0 Total)
>> >>> > >>>>>
>> >>> > >>>>> * = Binding Vote
>> >>> > >>>>>
>> >>> > >>>>> As per the incubator release guide [1] I'll be sending
>> >>> > >>>>> this
>> to the
>> >>> > >>>>> general incubator list for a final vote from IPMC members.
>> >>> > >>>>>
>> >>> > >>>>> [1]
>> >>> > >>>>>
>> >>> >
>> >>>
>> http://incubator.apache.org/guides/releasemanagement.html#best-practi
>> ce-incubator-release-
>> >>> > >>>>> vote
>> >>> > >>>>>
>> >>> > >>>>>
>> >>> > >>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan
>> >>> > >>>>> <ev...@ooyala.com>
>> wrote:
>> >>> > >>>>>
>> >>> > >>>>>> I'd be personally fine with a standard workflow of
>> assemble-deps
>> >>> +
>> >>> > >>>>>> packaging just the Spark files as separate packages, if
>> >>> > >>>>>> it
>> >>> speeds up
>> >>> > >>>>>> everyone's development time.
>> >>> > >>>>>>
>> >>> > >>>>>>
>> >>> > >>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <
>> >>> > mark@clearstorydata.com
>> >>> > >>>>>> >wrote:
>> >>> > >>>>>>
>> >>> > >>>>>> > I don't know how to make sense of the numbers, but
>> >>> > >>>>>> > here's
>> what
>> >>> > I've got
>> >>> > >>>>>> > from a very small sample size.
>> >>>
>> >>
>> >>
>>
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Azuryy Yu <az...@gmail.com>.
Hi here,
Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some
uncompatable API between these two versions.
Hadoop-2.x using protobuf-2.5.0
but if some guys want to run Spark on mesos, then mesos using
protobuf-2.4.1 currently. so we may discuss here for a better solution.
On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu <az...@gmail.com> wrote:
> Thanks Patrick.
> On 16 Dec 2013 02:43, "Patrick Wendell" <pw...@gmail.com> wrote:
>
>> You can checkout the docs mentioned in the vote thread. There is also
>> a pre-build binary for hadoop2 that is compiled for YARN 2.2
>>
>> - Patrick
>>
>> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu <az...@gmail.com> wrote:
>> > yarn 2.2, not yarn 0.22, I am so sorry.
>> >
>> >
>> > On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu <az...@gmail.com> wrote:
>> >
>> >> Hi,
>> >> Spark-0.8.1 supports yarn 0.22 right? where to find the release note?
>> >> Thanks.
>> >>
>> >>
>> >> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra <
>> henry.saputra@gmail.com>wrote:
>> >>
>> >>> Yeah seems like it. He was ok with our prev release.
>> >>> Let's wait for his reply
>> >>>
>> >>> On Saturday, December 14, 2013, Patrick Wendell wrote:
>> >>>
>> >>> > Henry - from that thread it looks like sebb's concern was something
>> >>> > different than this.
>> >>> >
>> >>> > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <
>> >>> henry.saputra@gmail.com>
>> >>> > wrote:
>> >>> > > Hi Patrick,
>> >>> > >
>> >>> > > Yeap I agree, but technically ASF VOTE release on source only,
>> there
>> >>> > > even debate about it =), so putting it in the vote staging
>> artifact
>> >>> > > could confuse people because in our case we do package 3rd party
>> >>> > > libraries in the binary jars.
>> >>> > >
>> >>> > > I have sent email to sebb asking clarification about his concern
>> in
>> >>> > > general@ list.
>> >>> > >
>> >>> > > - Henry
>> >>> > >
>> >>> > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <
>> pwendell@gmail.com
>> >>> >
>> >>> > wrote:
>> >>> > >> Hey Henry,
>> >>> > >>
>> >>> > >> One thing a lot of people do during the vote is test the
>> binaries and
>> >>> > >> make sure they work. This is really valuable. If you'd like I
>> could
>> >>> > >> add a caveat to the vote thread explaining that we are only
>> voting on
>> >>> > >> the source.
>> >>> > >>
>> >>> > >> - Patrick
>> >>> > >>
>> >>> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <
>> >>> > henry.saputra@gmail.com> wrote:
>> >>> > >>> Actually we should be fine putting the binaries there as long
>> as the
>> >>> > >>> VOTE is for the source.
>> >>> > >>>
>> >>> > >>> Let's verify with sebb in the general@ list about his concern.
>> >>> > >>>
>> >>> > >>> - Henry
>> >>> > >>>
>> >>> > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <
>> >>> > henry.saputra@gmail.com> wrote:
>> >>> > >>>> Hi Patrick, as sebb has mentioned let's move the binaries from
>> the
>> >>> > >>>> voting directory in your people.apache.org directory.
>> >>> > >>>> ASF release voting is for source code and not binaries, and
>> >>> > >>>> technically we provide binaries for convenience.
>> >>> > >>>>
>> >>> > >>>> And add link to the KEYS location in the dist[1] to let verify
>> >>> > signatures.
>> >>> > >>>>
>> >>> > >>>> Sorry for the late response to the VOTE thread, guys.
>> >>> > >>>>
>> >>> > >>>> - Henry
>> >>> > >>>>
>> >>> > >>>> [1]
>> >>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
>> >>> > >>>>
>> >>> > >>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <
>> >>> pwendell@gmail.com>
>> >>> > wrote:
>> >>> > >>>>> The vote is now closed. This vote passes with 5 PPMC +1's and
>> no 0
>> >>> > or -1
>> >>> > >>>>> votes.
>> >>> > >>>>>
>> >>> > >>>>> +1 (5 Total)
>> >>> > >>>>> Matei Zaharia*
>> >>> > >>>>> Nick Pentreath*
>> >>> > >>>>> Patrick Wendell*
>> >>> > >>>>> Prashant Sharma*
>> >>> > >>>>> Tom Graves*
>> >>> > >>>>>
>> >>> > >>>>> 0 (0 Total)
>> >>> > >>>>>
>> >>> > >>>>> -1 (0 Total)
>> >>> > >>>>>
>> >>> > >>>>> * = Binding Vote
>> >>> > >>>>>
>> >>> > >>>>> As per the incubator release guide [1] I'll be sending this
>> to the
>> >>> > >>>>> general incubator list for a final vote from IPMC members.
>> >>> > >>>>>
>> >>> > >>>>> [1]
>> >>> > >>>>>
>> >>> >
>> >>>
>> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-
>> >>> > >>>>> vote
>> >>> > >>>>>
>> >>> > >>>>>
>> >>> > >>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan <ev...@ooyala.com>
>> wrote:
>> >>> > >>>>>
>> >>> > >>>>>> I'd be personally fine with a standard workflow of
>> assemble-deps
>> >>> +
>> >>> > >>>>>> packaging just the Spark files as separate packages, if it
>> >>> speeds up
>> >>> > >>>>>> everyone's development time.
>> >>> > >>>>>>
>> >>> > >>>>>>
>> >>> > >>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <
>> >>> > mark@clearstorydata.com
>> >>> > >>>>>> >wrote:
>> >>> > >>>>>>
>> >>> > >>>>>> > I don't know how to make sense of the numbers, but here's
>> what
>> >>> > I've got
>> >>> > >>>>>> > from a very small sample size.
>> >>>
>> >>
>> >>
>>
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Azuryy Yu <az...@gmail.com>.
Thanks Patrick.
On 16 Dec 2013 02:43, "Patrick Wendell" <pw...@gmail.com> wrote:
> You can checkout the docs mentioned in the vote thread. There is also
> a pre-build binary for hadoop2 that is compiled for YARN 2.2
>
> - Patrick
>
> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu <az...@gmail.com> wrote:
> > yarn 2.2, not yarn 0.22, I am so sorry.
> >
> >
> > On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu <az...@gmail.com> wrote:
> >
> >> Hi,
> >> Spark-0.8.1 supports yarn 0.22 right? where to find the release note?
> >> Thanks.
> >>
> >>
> >> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra <henry.saputra@gmail.com
> >wrote:
> >>
> >>> Yeah seems like it. He was ok with our prev release.
> >>> Let's wait for his reply
> >>>
> >>> On Saturday, December 14, 2013, Patrick Wendell wrote:
> >>>
> >>> > Henry - from that thread it looks like sebb's concern was something
> >>> > different than this.
> >>> >
> >>> > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <
> >>> henry.saputra@gmail.com>
> >>> > wrote:
> >>> > > Hi Patrick,
> >>> > >
> >>> > > Yeap I agree, but technically ASF VOTE release on source only,
> there
> >>> > > even debate about it =), so putting it in the vote staging artifact
> >>> > > could confuse people because in our case we do package 3rd party
> >>> > > libraries in the binary jars.
> >>> > >
> >>> > > I have sent email to sebb asking clarification about his concern in
> >>> > > general@ list.
> >>> > >
> >>> > > - Henry
> >>> > >
> >>> > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <
> pwendell@gmail.com
> >>> >
> >>> > wrote:
> >>> > >> Hey Henry,
> >>> > >>
> >>> > >> One thing a lot of people do during the vote is test the binaries
> and
> >>> > >> make sure they work. This is really valuable. If you'd like I
> could
> >>> > >> add a caveat to the vote thread explaining that we are only
> voting on
> >>> > >> the source.
> >>> > >>
> >>> > >> - Patrick
> >>> > >>
> >>> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <
> >>> > henry.saputra@gmail.com> wrote:
> >>> > >>> Actually we should be fine putting the binaries there as long as
> the
> >>> > >>> VOTE is for the source.
> >>> > >>>
> >>> > >>> Let's verify with sebb in the general@ list about his concern.
> >>> > >>>
> >>> > >>> - Henry
> >>> > >>>
> >>> > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <
> >>> > henry.saputra@gmail.com> wrote:
> >>> > >>>> Hi Patrick, as sebb has mentioned let's move the binaries from
> the
> >>> > >>>> voting directory in your people.apache.org directory.
> >>> > >>>> ASF release voting is for source code and not binaries, and
> >>> > >>>> technically we provide binaries for convenience.
> >>> > >>>>
> >>> > >>>> And add link to the KEYS location in the dist[1] to let verify
> >>> > signatures.
> >>> > >>>>
> >>> > >>>> Sorry for the late response to the VOTE thread, guys.
> >>> > >>>>
> >>> > >>>> - Henry
> >>> > >>>>
> >>> > >>>> [1]
> >>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
> >>> > >>>>
> >>> > >>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <
> >>> pwendell@gmail.com>
> >>> > wrote:
> >>> > >>>>> The vote is now closed. This vote passes with 5 PPMC +1's and
> no 0
> >>> > or -1
> >>> > >>>>> votes.
> >>> > >>>>>
> >>> > >>>>> +1 (5 Total)
> >>> > >>>>> Matei Zaharia*
> >>> > >>>>> Nick Pentreath*
> >>> > >>>>> Patrick Wendell*
> >>> > >>>>> Prashant Sharma*
> >>> > >>>>> Tom Graves*
> >>> > >>>>>
> >>> > >>>>> 0 (0 Total)
> >>> > >>>>>
> >>> > >>>>> -1 (0 Total)
> >>> > >>>>>
> >>> > >>>>> * = Binding Vote
> >>> > >>>>>
> >>> > >>>>> As per the incubator release guide [1] I'll be sending this to
> the
> >>> > >>>>> general incubator list for a final vote from IPMC members.
> >>> > >>>>>
> >>> > >>>>> [1]
> >>> > >>>>>
> >>> >
> >>>
> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-
> >>> > >>>>> vote
> >>> > >>>>>
> >>> > >>>>>
> >>> > >>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan <ev...@ooyala.com>
> wrote:
> >>> > >>>>>
> >>> > >>>>>> I'd be personally fine with a standard workflow of
> assemble-deps
> >>> +
> >>> > >>>>>> packaging just the Spark files as separate packages, if it
> >>> speeds up
> >>> > >>>>>> everyone's development time.
> >>> > >>>>>>
> >>> > >>>>>>
> >>> > >>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <
> >>> > mark@clearstorydata.com
> >>> > >>>>>> >wrote:
> >>> > >>>>>>
> >>> > >>>>>> > I don't know how to make sense of the numbers, but here's
> what
> >>> > I've got
> >>> > >>>>>> > from a very small sample size.
> >>>
> >>
> >>
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Patrick Wendell <pw...@gmail.com>.
You can checkout the docs mentioned in the vote thread. There is also
a pre-build binary for hadoop2 that is compiled for YARN 2.2
- Patrick
On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu <az...@gmail.com> wrote:
> yarn 2.2, not yarn 0.22, I am so sorry.
>
>
> On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu <az...@gmail.com> wrote:
>
>> Hi,
>> Spark-0.8.1 supports yarn 0.22 right? where to find the release note?
>> Thanks.
>>
>>
>> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra <he...@gmail.com>wrote:
>>
>>> Yeah seems like it. He was ok with our prev release.
>>> Let's wait for his reply
>>>
>>> On Saturday, December 14, 2013, Patrick Wendell wrote:
>>>
>>> > Henry - from that thread it looks like sebb's concern was something
>>> > different than this.
>>> >
>>> > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <
>>> henry.saputra@gmail.com>
>>> > wrote:
>>> > > Hi Patrick,
>>> > >
>>> > > Yeap I agree, but technically ASF VOTE release on source only, there
>>> > > even debate about it =), so putting it in the vote staging artifact
>>> > > could confuse people because in our case we do package 3rd party
>>> > > libraries in the binary jars.
>>> > >
>>> > > I have sent email to sebb asking clarification about his concern in
>>> > > general@ list.
>>> > >
>>> > > - Henry
>>> > >
>>> > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <pwendell@gmail.com
>>> >
>>> > wrote:
>>> > >> Hey Henry,
>>> > >>
>>> > >> One thing a lot of people do during the vote is test the binaries and
>>> > >> make sure they work. This is really valuable. If you'd like I could
>>> > >> add a caveat to the vote thread explaining that we are only voting on
>>> > >> the source.
>>> > >>
>>> > >> - Patrick
>>> > >>
>>> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <
>>> > henry.saputra@gmail.com> wrote:
>>> > >>> Actually we should be fine putting the binaries there as long as the
>>> > >>> VOTE is for the source.
>>> > >>>
>>> > >>> Let's verify with sebb in the general@ list about his concern.
>>> > >>>
>>> > >>> - Henry
>>> > >>>
>>> > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <
>>> > henry.saputra@gmail.com> wrote:
>>> > >>>> Hi Patrick, as sebb has mentioned let's move the binaries from the
>>> > >>>> voting directory in your people.apache.org directory.
>>> > >>>> ASF release voting is for source code and not binaries, and
>>> > >>>> technically we provide binaries for convenience.
>>> > >>>>
>>> > >>>> And add link to the KEYS location in the dist[1] to let verify
>>> > signatures.
>>> > >>>>
>>> > >>>> Sorry for the late response to the VOTE thread, guys.
>>> > >>>>
>>> > >>>> - Henry
>>> > >>>>
>>> > >>>> [1]
>>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
>>> > >>>>
>>> > >>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <
>>> pwendell@gmail.com>
>>> > wrote:
>>> > >>>>> The vote is now closed. This vote passes with 5 PPMC +1's and no 0
>>> > or -1
>>> > >>>>> votes.
>>> > >>>>>
>>> > >>>>> +1 (5 Total)
>>> > >>>>> Matei Zaharia*
>>> > >>>>> Nick Pentreath*
>>> > >>>>> Patrick Wendell*
>>> > >>>>> Prashant Sharma*
>>> > >>>>> Tom Graves*
>>> > >>>>>
>>> > >>>>> 0 (0 Total)
>>> > >>>>>
>>> > >>>>> -1 (0 Total)
>>> > >>>>>
>>> > >>>>> * = Binding Vote
>>> > >>>>>
>>> > >>>>> As per the incubator release guide [1] I'll be sending this to the
>>> > >>>>> general incubator list for a final vote from IPMC members.
>>> > >>>>>
>>> > >>>>> [1]
>>> > >>>>>
>>> >
>>> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-
>>> > >>>>> vote
>>> > >>>>>
>>> > >>>>>
>>> > >>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan <ev...@ooyala.com> wrote:
>>> > >>>>>
>>> > >>>>>> I'd be personally fine with a standard workflow of assemble-deps
>>> +
>>> > >>>>>> packaging just the Spark files as separate packages, if it
>>> speeds up
>>> > >>>>>> everyone's development time.
>>> > >>>>>>
>>> > >>>>>>
>>> > >>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <
>>> > mark@clearstorydata.com
>>> > >>>>>> >wrote:
>>> > >>>>>>
>>> > >>>>>> > I don't know how to make sense of the numbers, but here's what
>>> > I've got
>>> > >>>>>> > from a very small sample size.
>>>
>>
>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Azuryy Yu <az...@gmail.com>.
yarn 2.2, not yarn 0.22, I am so sorry.
On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu <az...@gmail.com> wrote:
> Hi,
> Spark-0.8.1 supports yarn 0.22 right? where to find the release note?
> Thanks.
>
>
> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra <he...@gmail.com>wrote:
>
>> Yeah seems like it. He was ok with our prev release.
>> Let's wait for his reply
>>
>> On Saturday, December 14, 2013, Patrick Wendell wrote:
>>
>> > Henry - from that thread it looks like sebb's concern was something
>> > different than this.
>> >
>> > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <
>> henry.saputra@gmail.com>
>> > wrote:
>> > > Hi Patrick,
>> > >
>> > > Yeap I agree, but technically ASF VOTE release on source only, there
>> > > even debate about it =), so putting it in the vote staging artifact
>> > > could confuse people because in our case we do package 3rd party
>> > > libraries in the binary jars.
>> > >
>> > > I have sent email to sebb asking clarification about his concern in
>> > > general@ list.
>> > >
>> > > - Henry
>> > >
>> > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <pwendell@gmail.com
>> >
>> > wrote:
>> > >> Hey Henry,
>> > >>
>> > >> One thing a lot of people do during the vote is test the binaries and
>> > >> make sure they work. This is really valuable. If you'd like I could
>> > >> add a caveat to the vote thread explaining that we are only voting on
>> > >> the source.
>> > >>
>> > >> - Patrick
>> > >>
>> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <
>> > henry.saputra@gmail.com> wrote:
>> > >>> Actually we should be fine putting the binaries there as long as the
>> > >>> VOTE is for the source.
>> > >>>
>> > >>> Let's verify with sebb in the general@ list about his concern.
>> > >>>
>> > >>> - Henry
>> > >>>
>> > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <
>> > henry.saputra@gmail.com> wrote:
>> > >>>> Hi Patrick, as sebb has mentioned let's move the binaries from the
>> > >>>> voting directory in your people.apache.org directory.
>> > >>>> ASF release voting is for source code and not binaries, and
>> > >>>> technically we provide binaries for convenience.
>> > >>>>
>> > >>>> And add link to the KEYS location in the dist[1] to let verify
>> > signatures.
>> > >>>>
>> > >>>> Sorry for the late response to the VOTE thread, guys.
>> > >>>>
>> > >>>> - Henry
>> > >>>>
>> > >>>> [1]
>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
>> > >>>>
>> > >>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <
>> pwendell@gmail.com>
>> > wrote:
>> > >>>>> The vote is now closed. This vote passes with 5 PPMC +1's and no 0
>> > or -1
>> > >>>>> votes.
>> > >>>>>
>> > >>>>> +1 (5 Total)
>> > >>>>> Matei Zaharia*
>> > >>>>> Nick Pentreath*
>> > >>>>> Patrick Wendell*
>> > >>>>> Prashant Sharma*
>> > >>>>> Tom Graves*
>> > >>>>>
>> > >>>>> 0 (0 Total)
>> > >>>>>
>> > >>>>> -1 (0 Total)
>> > >>>>>
>> > >>>>> * = Binding Vote
>> > >>>>>
>> > >>>>> As per the incubator release guide [1] I'll be sending this to the
>> > >>>>> general incubator list for a final vote from IPMC members.
>> > >>>>>
>> > >>>>> [1]
>> > >>>>>
>> >
>> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-
>> > >>>>> vote
>> > >>>>>
>> > >>>>>
>> > >>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan <ev...@ooyala.com> wrote:
>> > >>>>>
>> > >>>>>> I'd be personally fine with a standard workflow of assemble-deps
>> +
>> > >>>>>> packaging just the Spark files as separate packages, if it
>> speeds up
>> > >>>>>> everyone's development time.
>> > >>>>>>
>> > >>>>>>
>> > >>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <
>> > mark@clearstorydata.com
>> > >>>>>> >wrote:
>> > >>>>>>
>> > >>>>>> > I don't know how to make sense of the numbers, but here's what
>> > I've got
>> > >>>>>> > from a very small sample size.
>>
>
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Azuryy Yu <az...@gmail.com>.
Hi,
Spark-0.8.1 supports yarn 0.22 right? where to find the release note?
Thanks.
On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra <he...@gmail.com>wrote:
> Yeah seems like it. He was ok with our prev release.
> Let's wait for his reply
>
> On Saturday, December 14, 2013, Patrick Wendell wrote:
>
> > Henry - from that thread it looks like sebb's concern was something
> > different than this.
> >
> > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <henry.saputra@gmail.com
> >
> > wrote:
> > > Hi Patrick,
> > >
> > > Yeap I agree, but technically ASF VOTE release on source only, there
> > > even debate about it =), so putting it in the vote staging artifact
> > > could confuse people because in our case we do package 3rd party
> > > libraries in the binary jars.
> > >
> > > I have sent email to sebb asking clarification about his concern in
> > > general@ list.
> > >
> > > - Henry
> > >
> > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <pw...@gmail.com>
> > wrote:
> > >> Hey Henry,
> > >>
> > >> One thing a lot of people do during the vote is test the binaries and
> > >> make sure they work. This is really valuable. If you'd like I could
> > >> add a caveat to the vote thread explaining that we are only voting on
> > >> the source.
> > >>
> > >> - Patrick
> > >>
> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <
> > henry.saputra@gmail.com> wrote:
> > >>> Actually we should be fine putting the binaries there as long as the
> > >>> VOTE is for the source.
> > >>>
> > >>> Let's verify with sebb in the general@ list about his concern.
> > >>>
> > >>> - Henry
> > >>>
> > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <
> > henry.saputra@gmail.com> wrote:
> > >>>> Hi Patrick, as sebb has mentioned let's move the binaries from the
> > >>>> voting directory in your people.apache.org directory.
> > >>>> ASF release voting is for source code and not binaries, and
> > >>>> technically we provide binaries for convenience.
> > >>>>
> > >>>> And add link to the KEYS location in the dist[1] to let verify
> > signatures.
> > >>>>
> > >>>> Sorry for the late response to the VOTE thread, guys.
> > >>>>
> > >>>> - Henry
> > >>>>
> > >>>> [1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
> > >>>>
> > >>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <
> pwendell@gmail.com>
> > wrote:
> > >>>>> The vote is now closed. This vote passes with 5 PPMC +1's and no 0
> > or -1
> > >>>>> votes.
> > >>>>>
> > >>>>> +1 (5 Total)
> > >>>>> Matei Zaharia*
> > >>>>> Nick Pentreath*
> > >>>>> Patrick Wendell*
> > >>>>> Prashant Sharma*
> > >>>>> Tom Graves*
> > >>>>>
> > >>>>> 0 (0 Total)
> > >>>>>
> > >>>>> -1 (0 Total)
> > >>>>>
> > >>>>> * = Binding Vote
> > >>>>>
> > >>>>> As per the incubator release guide [1] I'll be sending this to the
> > >>>>> general incubator list for a final vote from IPMC members.
> > >>>>>
> > >>>>> [1]
> > >>>>>
> >
> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-
> > >>>>> vote
> > >>>>>
> > >>>>>
> > >>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan <ev...@ooyala.com> wrote:
> > >>>>>
> > >>>>>> I'd be personally fine with a standard workflow of assemble-deps +
> > >>>>>> packaging just the Spark files as separate packages, if it speeds
> up
> > >>>>>> everyone's development time.
> > >>>>>>
> > >>>>>>
> > >>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <
> > mark@clearstorydata.com
> > >>>>>> >wrote:
> > >>>>>>
> > >>>>>> > I don't know how to make sense of the numbers, but here's what
> > I've got
> > >>>>>> > from a very small sample size.
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Henry Saputra <he...@gmail.com>.
Yeah seems like it. He was ok with our prev release.
Let's wait for his reply
On Saturday, December 14, 2013, Patrick Wendell wrote:
> Henry - from that thread it looks like sebb's concern was something
> different than this.
>
> On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <he...@gmail.com>
> wrote:
> > Hi Patrick,
> >
> > Yeap I agree, but technically ASF VOTE release on source only, there
> > even debate about it =), so putting it in the vote staging artifact
> > could confuse people because in our case we do package 3rd party
> > libraries in the binary jars.
> >
> > I have sent email to sebb asking clarification about his concern in
> > general@ list.
> >
> > - Henry
> >
> > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <pw...@gmail.com>
> wrote:
> >> Hey Henry,
> >>
> >> One thing a lot of people do during the vote is test the binaries and
> >> make sure they work. This is really valuable. If you'd like I could
> >> add a caveat to the vote thread explaining that we are only voting on
> >> the source.
> >>
> >> - Patrick
> >>
> >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <
> henry.saputra@gmail.com> wrote:
> >>> Actually we should be fine putting the binaries there as long as the
> >>> VOTE is for the source.
> >>>
> >>> Let's verify with sebb in the general@ list about his concern.
> >>>
> >>> - Henry
> >>>
> >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <
> henry.saputra@gmail.com> wrote:
> >>>> Hi Patrick, as sebb has mentioned let's move the binaries from the
> >>>> voting directory in your people.apache.org directory.
> >>>> ASF release voting is for source code and not binaries, and
> >>>> technically we provide binaries for convenience.
> >>>>
> >>>> And add link to the KEYS location in the dist[1] to let verify
> signatures.
> >>>>
> >>>> Sorry for the late response to the VOTE thread, guys.
> >>>>
> >>>> - Henry
> >>>>
> >>>> [1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
> >>>>
> >>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <pw...@gmail.com>
> wrote:
> >>>>> The vote is now closed. This vote passes with 5 PPMC +1's and no 0
> or -1
> >>>>> votes.
> >>>>>
> >>>>> +1 (5 Total)
> >>>>> Matei Zaharia*
> >>>>> Nick Pentreath*
> >>>>> Patrick Wendell*
> >>>>> Prashant Sharma*
> >>>>> Tom Graves*
> >>>>>
> >>>>> 0 (0 Total)
> >>>>>
> >>>>> -1 (0 Total)
> >>>>>
> >>>>> * = Binding Vote
> >>>>>
> >>>>> As per the incubator release guide [1] I'll be sending this to the
> >>>>> general incubator list for a final vote from IPMC members.
> >>>>>
> >>>>> [1]
> >>>>>
> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-
> >>>>> vote
> >>>>>
> >>>>>
> >>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan <ev...@ooyala.com> wrote:
> >>>>>
> >>>>>> I'd be personally fine with a standard workflow of assemble-deps +
> >>>>>> packaging just the Spark files as separate packages, if it speeds up
> >>>>>> everyone's development time.
> >>>>>>
> >>>>>>
> >>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <
> mark@clearstorydata.com
> >>>>>> >wrote:
> >>>>>>
> >>>>>> > I don't know how to make sense of the numbers, but here's what
> I've got
> >>>>>> > from a very small sample size.
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Patrick Wendell <pw...@gmail.com>.
Henry - from that thread it looks like sebb's concern was something
different than this.
On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra <he...@gmail.com> wrote:
> Hi Patrick,
>
> Yeap I agree, but technically ASF VOTE release on source only, there
> even debate about it =), so putting it in the vote staging artifact
> could confuse people because in our case we do package 3rd party
> libraries in the binary jars.
>
> I have sent email to sebb asking clarification about his concern in
> general@ list.
>
> - Henry
>
> On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <pw...@gmail.com> wrote:
>> Hey Henry,
>>
>> One thing a lot of people do during the vote is test the binaries and
>> make sure they work. This is really valuable. If you'd like I could
>> add a caveat to the vote thread explaining that we are only voting on
>> the source.
>>
>> - Patrick
>>
>> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <he...@gmail.com> wrote:
>>> Actually we should be fine putting the binaries there as long as the
>>> VOTE is for the source.
>>>
>>> Let's verify with sebb in the general@ list about his concern.
>>>
>>> - Henry
>>>
>>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <he...@gmail.com> wrote:
>>>> Hi Patrick, as sebb has mentioned let's move the binaries from the
>>>> voting directory in your people.apache.org directory.
>>>> ASF release voting is for source code and not binaries, and
>>>> technically we provide binaries for convenience.
>>>>
>>>> And add link to the KEYS location in the dist[1] to let verify signatures.
>>>>
>>>> Sorry for the late response to the VOTE thread, guys.
>>>>
>>>> - Henry
>>>>
>>>> [1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
>>>>
>>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <pw...@gmail.com> wrote:
>>>>> The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1
>>>>> votes.
>>>>>
>>>>> +1 (5 Total)
>>>>> Matei Zaharia*
>>>>> Nick Pentreath*
>>>>> Patrick Wendell*
>>>>> Prashant Sharma*
>>>>> Tom Graves*
>>>>>
>>>>> 0 (0 Total)
>>>>>
>>>>> -1 (0 Total)
>>>>>
>>>>> * = Binding Vote
>>>>>
>>>>> As per the incubator release guide [1] I'll be sending this to the
>>>>> general incubator list for a final vote from IPMC members.
>>>>>
>>>>> [1]
>>>>> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-
>>>>> vote
>>>>>
>>>>>
>>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan <ev...@ooyala.com> wrote:
>>>>>
>>>>>> I'd be personally fine with a standard workflow of assemble-deps +
>>>>>> packaging just the Spark files as separate packages, if it speeds up
>>>>>> everyone's development time.
>>>>>>
>>>>>>
>>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <mark@clearstorydata.com
>>>>>> >wrote:
>>>>>>
>>>>>> > I don't know how to make sense of the numbers, but here's what I've got
>>>>>> > from a very small sample size.
>>>>>> >
>>>>>> > For both v0.8.0-incubating and v0.8.1-incubating, building separate
>>>>>> > assemblies is faster than `./sbt/sbt assembly` and the times for building
>>>>>> > separate assemblies for 0.8.0 and 0.8.1 are about the same.
>>>>>> >
>>>>>> > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as
>>>>>> the
>>>>>> > sum of the separate assemblies.
>>>>>> > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as
>>>>>> the
>>>>>> > sum of the separate assemblies.
>>>>>> >
>>>>>> > Weird.
>>>>>> >
>>>>>> >
>>>>>> > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell <pwendell@gmail.com
>>>>>> > >wrote:
>>>>>> >
>>>>>> > > I'll +1 myself also.
>>>>>> > >
>>>>>> > > For anyone who has the slow build problem: does this issue happen when
>>>>>> > > building v0.8.0-incubating also? Trying to figure out whether it's
>>>>>> > > related to something we added in 0.8.1 or if it's a long standing
>>>>>> > > issue.
>>>>>> > >
>>>>>> > > - Patrick
>>>>>> > >
>>>>>> > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <
>>>>>> matei.zaharia@gmail.com
>>>>>> > >
>>>>>> > > wrote:
>>>>>> > > > Woah, weird, but definitely good to know.
>>>>>> > > >
>>>>>> > > > If you’re doing Spark development, there’s also a more convenient
>>>>>> > option
>>>>>> > > added by Shivaram in the master branch. You can do sbt assemble-deps to
>>>>>> > > package *just* the dependencies of each project in a special assembly
>>>>>> > JAR,
>>>>>> > > and then use sbt compile to update the code. This will use the classes
>>>>>> > > directly out of the target/scala-2.9.3/classes directories. You have to
>>>>>> > > redo assemble-deps only if your external dependencies change.
>>>>>> > > >
>>>>>> > > > Matei
>>>>>> > > >
>>>>>> > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com>
>>>>>> > > wrote:
>>>>>> > > >
>>>>>> > > >> I hope this PR https://github.com/apache/incubator-spark/pull/252can
>>>>>> > > help.
>>>>>> > > >> Again this is not a blocker for the release from my side either.
>>>>>> > > >>
>>>>>> > > >>
>>>>>> > > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <
>>>>>> > mark@clearstorydata.com
>>>>>> > > >wrote:
>>>>>> > > >>
>>>>>> > > >>> Interesting, and confirmed: On my machine where `./sbt/sbt
>>>>>> assembly`
>>>>>> > > takes
>>>>>> > > >>> a long, long, looooong time to complete (a MBP, in my case),
>>>>>> building
>>>>>> > > three
>>>>>> > > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
>>>>>> > > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much
>>>>>> less
>>>>>> > > time.
>>>>>> > > >>>
>>>>>> > > >>>
>>>>>> > > >>>
>>>>>> > > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <
>>>>>> > > scrapcodes@gmail.com
>>>>>> > > >>>> wrote:
>>>>>> > > >>>
>>>>>> > > >>>> forgot to mention, after running sbt/sbt assembly/assembly running
>>>>>> > > >>> sbt/sbt
>>>>>> > > >>>> examples/assembly takes just 37s. Not to mention my hardware is
>>>>>> not
>>>>>> > > >>> really
>>>>>> > > >>>> great.
>>>>>> > > >>>>
>>>>>> > > >>>>
>>>>>> > > >>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <
>>>>>> > > scrapcodes@gmail.com
>>>>>> > > >>>>> wrote:
>>>>>> > > >>>>
>>>>>> > > >>>>> Hi Patrick and Matei,
>>>>>> > > >>>>>
>>>>>> > > >>>>> Was trying out this and followed the quick start guide which says
>>>>>> > do
>>>>>> > > >>>>> sbt/sbt assembly, like few others I was also stuck for few
>>>>>> minutes
>>>>>> > on
>>>>>> > > >>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is
>>>>>> > > much
>>>>>> > > >>>>> faster.
>>>>>> > > >>>>>
>>>>>> > > >>>>> Should we change the documentation to reflect this. It will not
>>>>>> be
>>>>>> > > >>> great
>>>>>> > > >>>>> for first time users to get stuck there.
>>>>>> > > >>>>>
>>>>>> > > >>>>>
>>>>>> > > >>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
>>>>>> > > >>> matei.zaharia@gmail.com
>>>>>> > > >>>>> wrote:
>>>>>> > > >>>>>
>>>>>> > > >>>>>> +1
>>>>>> > > >>>>>>
>>>>>> > > >>>>>> Built and tested it on Mac OS X.
>>>>>> > > >>>>>>
>>>>>> > > >>>>>> Matei
>>>>>> > > >>>>>>
>>>>>> > > >>>>>>
>>>>>> > > >>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <
>>>>>> pwendell@gmail.com>
>>>>>> > > >>>> wrote:
>>>>>> > > >>>>>>
>>>>>> > > >>>>>>> Please vote on releasing the following candidate as Apache
>>>>>> Spark
>>>>>> > > >>>>>>> (incubating) version 0.8.1.
>>>>>> > > >>>>>>>
>>>>>> > > >>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
>>>>>> > > >>>>>>>
>>>>>> > > >>>>>>
>>>>>> > > >>>>
>>>>>> > > >>>
>>>>>> > >
>>>>>> >
>>>>>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
>>>>>> > > >>>>>>>
>>>>>> > > >>>>>>> The release files, including signatures, digests, etc can be
>>>>>> > found
>>>>>> > > >>> at:
>>>>>> > > >>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
>>>>>> > > >>>>>>>
>>>>>> > > >>>>>>> Release artifacts are signed with the following key:
>>>>>> > > >>>>>>> https://people.apache.org/keys/committer/pwendell.asc
>>>>>> > > >>>>>>>
>>>>>> > > >>>>>>> The staging repository for this release can be found at:
>>>>>> > > >>>>>>>
>>>>>> > > >>>>
>>>>>> > > https://repository.apache.org/content/repositories/orgapachespark-040/
>>>>>> > > >>>>>>>
>>>>>> > > >>>>>>> The documentation corresponding to this release can be found
>>>>>> at:
>>>>>> > > >>>>>>>
>>>>>> > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
>>>>>> > > >>>>>>>
>>>>>> > > >>>>>>> For information about the contents of this release see:
>>>>>> > > >>>>>>>
>>>>>> > > >>>>>>
>>>>>> > > >>>>
>>>>>> > > >>>
>>>>>> > >
>>>>>> >
>>>>>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
>>>>>> > > >>>>>>>
>>>>>> > > >>>>>>> Please vote on releasing this package as Apache Spark
>>>>>> > > >>>> 0.8.1-incubating!
>>>>>> > > >>>>>>>
>>>>>> > > >>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
>>>>>> > > >>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
>>>>>> > > >>>>>>>
>>>>>> > > >>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
>>>>>> > > >>>>>>> [ ] -1 Do not release this package because ...
>>>>>> > > >>>>>>>
>>>>>> > > >>>>>>> To learn more about Apache Spark, please see
>>>>>> > > >>>>>>> http://spark.incubator.apache.org/
>>>>>> > > >>>>>>
>>>>>> > > >>>>>>
>>>>>> > > >>>>>
>>>>>> > > >>>>>
>>>>>> > > >>>>> --
>>>>>> > > >>>>> s
>>>>>> > > >>>>>
>>>>>> > > >>>>
>>>>>> > > >>>>
>>>>>> > > >>>>
>>>>>> > > >>>> --
>>>>>> > > >>>> s
>>>>>> > > >>>>
>>>>>> > > >>>
>>>>>> > > >>
>>>>>> > > >>
>>>>>> > > >>
>>>>>> > > >> --
>>>>>> > > >> s
>>>>>> > > >
>>>>>> > >
>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> --
>>>>>> Evan Chan
>>>>>> Staff Engineer
>>>>>> ev@ooyala.com |
>>>>>>
>>>>>> <http://www.ooyala.com/>
>>>>>> <http://www.facebook.com/ooyala><http://www.linkedin.com/company/ooyala><
>>>>>> http://www.twitter.com/ooyala>
>>>>>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Henry Saputra <he...@gmail.com>.
Hi Patrick,
Yeap I agree, but technically ASF VOTE release on source only, there
even debate about it =), so putting it in the vote staging artifact
could confuse people because in our case we do package 3rd party
libraries in the binary jars.
I have sent email to sebb asking clarification about his concern in
general@ list.
- Henry
On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell <pw...@gmail.com> wrote:
> Hey Henry,
>
> One thing a lot of people do during the vote is test the binaries and
> make sure they work. This is really valuable. If you'd like I could
> add a caveat to the vote thread explaining that we are only voting on
> the source.
>
> - Patrick
>
> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <he...@gmail.com> wrote:
>> Actually we should be fine putting the binaries there as long as the
>> VOTE is for the source.
>>
>> Let's verify with sebb in the general@ list about his concern.
>>
>> - Henry
>>
>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <he...@gmail.com> wrote:
>>> Hi Patrick, as sebb has mentioned let's move the binaries from the
>>> voting directory in your people.apache.org directory.
>>> ASF release voting is for source code and not binaries, and
>>> technically we provide binaries for convenience.
>>>
>>> And add link to the KEYS location in the dist[1] to let verify signatures.
>>>
>>> Sorry for the late response to the VOTE thread, guys.
>>>
>>> - Henry
>>>
>>> [1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
>>>
>>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <pw...@gmail.com> wrote:
>>>> The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1
>>>> votes.
>>>>
>>>> +1 (5 Total)
>>>> Matei Zaharia*
>>>> Nick Pentreath*
>>>> Patrick Wendell*
>>>> Prashant Sharma*
>>>> Tom Graves*
>>>>
>>>> 0 (0 Total)
>>>>
>>>> -1 (0 Total)
>>>>
>>>> * = Binding Vote
>>>>
>>>> As per the incubator release guide [1] I'll be sending this to the
>>>> general incubator list for a final vote from IPMC members.
>>>>
>>>> [1]
>>>> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-
>>>> vote
>>>>
>>>>
>>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan <ev...@ooyala.com> wrote:
>>>>
>>>>> I'd be personally fine with a standard workflow of assemble-deps +
>>>>> packaging just the Spark files as separate packages, if it speeds up
>>>>> everyone's development time.
>>>>>
>>>>>
>>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <mark@clearstorydata.com
>>>>> >wrote:
>>>>>
>>>>> > I don't know how to make sense of the numbers, but here's what I've got
>>>>> > from a very small sample size.
>>>>> >
>>>>> > For both v0.8.0-incubating and v0.8.1-incubating, building separate
>>>>> > assemblies is faster than `./sbt/sbt assembly` and the times for building
>>>>> > separate assemblies for 0.8.0 and 0.8.1 are about the same.
>>>>> >
>>>>> > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as
>>>>> the
>>>>> > sum of the separate assemblies.
>>>>> > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as
>>>>> the
>>>>> > sum of the separate assemblies.
>>>>> >
>>>>> > Weird.
>>>>> >
>>>>> >
>>>>> > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell <pwendell@gmail.com
>>>>> > >wrote:
>>>>> >
>>>>> > > I'll +1 myself also.
>>>>> > >
>>>>> > > For anyone who has the slow build problem: does this issue happen when
>>>>> > > building v0.8.0-incubating also? Trying to figure out whether it's
>>>>> > > related to something we added in 0.8.1 or if it's a long standing
>>>>> > > issue.
>>>>> > >
>>>>> > > - Patrick
>>>>> > >
>>>>> > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <
>>>>> matei.zaharia@gmail.com
>>>>> > >
>>>>> > > wrote:
>>>>> > > > Woah, weird, but definitely good to know.
>>>>> > > >
>>>>> > > > If you’re doing Spark development, there’s also a more convenient
>>>>> > option
>>>>> > > added by Shivaram in the master branch. You can do sbt assemble-deps to
>>>>> > > package *just* the dependencies of each project in a special assembly
>>>>> > JAR,
>>>>> > > and then use sbt compile to update the code. This will use the classes
>>>>> > > directly out of the target/scala-2.9.3/classes directories. You have to
>>>>> > > redo assemble-deps only if your external dependencies change.
>>>>> > > >
>>>>> > > > Matei
>>>>> > > >
>>>>> > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com>
>>>>> > > wrote:
>>>>> > > >
>>>>> > > >> I hope this PR https://github.com/apache/incubator-spark/pull/252can
>>>>> > > help.
>>>>> > > >> Again this is not a blocker for the release from my side either.
>>>>> > > >>
>>>>> > > >>
>>>>> > > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <
>>>>> > mark@clearstorydata.com
>>>>> > > >wrote:
>>>>> > > >>
>>>>> > > >>> Interesting, and confirmed: On my machine where `./sbt/sbt
>>>>> assembly`
>>>>> > > takes
>>>>> > > >>> a long, long, looooong time to complete (a MBP, in my case),
>>>>> building
>>>>> > > three
>>>>> > > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
>>>>> > > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much
>>>>> less
>>>>> > > time.
>>>>> > > >>>
>>>>> > > >>>
>>>>> > > >>>
>>>>> > > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <
>>>>> > > scrapcodes@gmail.com
>>>>> > > >>>> wrote:
>>>>> > > >>>
>>>>> > > >>>> forgot to mention, after running sbt/sbt assembly/assembly running
>>>>> > > >>> sbt/sbt
>>>>> > > >>>> examples/assembly takes just 37s. Not to mention my hardware is
>>>>> not
>>>>> > > >>> really
>>>>> > > >>>> great.
>>>>> > > >>>>
>>>>> > > >>>>
>>>>> > > >>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <
>>>>> > > scrapcodes@gmail.com
>>>>> > > >>>>> wrote:
>>>>> > > >>>>
>>>>> > > >>>>> Hi Patrick and Matei,
>>>>> > > >>>>>
>>>>> > > >>>>> Was trying out this and followed the quick start guide which says
>>>>> > do
>>>>> > > >>>>> sbt/sbt assembly, like few others I was also stuck for few
>>>>> minutes
>>>>> > on
>>>>> > > >>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is
>>>>> > > much
>>>>> > > >>>>> faster.
>>>>> > > >>>>>
>>>>> > > >>>>> Should we change the documentation to reflect this. It will not
>>>>> be
>>>>> > > >>> great
>>>>> > > >>>>> for first time users to get stuck there.
>>>>> > > >>>>>
>>>>> > > >>>>>
>>>>> > > >>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
>>>>> > > >>> matei.zaharia@gmail.com
>>>>> > > >>>>> wrote:
>>>>> > > >>>>>
>>>>> > > >>>>>> +1
>>>>> > > >>>>>>
>>>>> > > >>>>>> Built and tested it on Mac OS X.
>>>>> > > >>>>>>
>>>>> > > >>>>>> Matei
>>>>> > > >>>>>>
>>>>> > > >>>>>>
>>>>> > > >>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <
>>>>> pwendell@gmail.com>
>>>>> > > >>>> wrote:
>>>>> > > >>>>>>
>>>>> > > >>>>>>> Please vote on releasing the following candidate as Apache
>>>>> Spark
>>>>> > > >>>>>>> (incubating) version 0.8.1.
>>>>> > > >>>>>>>
>>>>> > > >>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
>>>>> > > >>>>>>>
>>>>> > > >>>>>>
>>>>> > > >>>>
>>>>> > > >>>
>>>>> > >
>>>>> >
>>>>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
>>>>> > > >>>>>>>
>>>>> > > >>>>>>> The release files, including signatures, digests, etc can be
>>>>> > found
>>>>> > > >>> at:
>>>>> > > >>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
>>>>> > > >>>>>>>
>>>>> > > >>>>>>> Release artifacts are signed with the following key:
>>>>> > > >>>>>>> https://people.apache.org/keys/committer/pwendell.asc
>>>>> > > >>>>>>>
>>>>> > > >>>>>>> The staging repository for this release can be found at:
>>>>> > > >>>>>>>
>>>>> > > >>>>
>>>>> > > https://repository.apache.org/content/repositories/orgapachespark-040/
>>>>> > > >>>>>>>
>>>>> > > >>>>>>> The documentation corresponding to this release can be found
>>>>> at:
>>>>> > > >>>>>>>
>>>>> > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
>>>>> > > >>>>>>>
>>>>> > > >>>>>>> For information about the contents of this release see:
>>>>> > > >>>>>>>
>>>>> > > >>>>>>
>>>>> > > >>>>
>>>>> > > >>>
>>>>> > >
>>>>> >
>>>>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
>>>>> > > >>>>>>>
>>>>> > > >>>>>>> Please vote on releasing this package as Apache Spark
>>>>> > > >>>> 0.8.1-incubating!
>>>>> > > >>>>>>>
>>>>> > > >>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
>>>>> > > >>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
>>>>> > > >>>>>>>
>>>>> > > >>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
>>>>> > > >>>>>>> [ ] -1 Do not release this package because ...
>>>>> > > >>>>>>>
>>>>> > > >>>>>>> To learn more about Apache Spark, please see
>>>>> > > >>>>>>> http://spark.incubator.apache.org/
>>>>> > > >>>>>>
>>>>> > > >>>>>>
>>>>> > > >>>>>
>>>>> > > >>>>>
>>>>> > > >>>>> --
>>>>> > > >>>>> s
>>>>> > > >>>>>
>>>>> > > >>>>
>>>>> > > >>>>
>>>>> > > >>>>
>>>>> > > >>>> --
>>>>> > > >>>> s
>>>>> > > >>>>
>>>>> > > >>>
>>>>> > > >>
>>>>> > > >>
>>>>> > > >>
>>>>> > > >> --
>>>>> > > >> s
>>>>> > > >
>>>>> > >
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --
>>>>> Evan Chan
>>>>> Staff Engineer
>>>>> ev@ooyala.com |
>>>>>
>>>>> <http://www.ooyala.com/>
>>>>> <http://www.facebook.com/ooyala><http://www.linkedin.com/company/ooyala><
>>>>> http://www.twitter.com/ooyala>
>>>>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Patrick Wendell <pw...@gmail.com>.
Hey Henry,
One thing a lot of people do during the vote is test the binaries and
make sure they work. This is really valuable. If you'd like I could
add a caveat to the vote thread explaining that we are only voting on
the source.
- Patrick
On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra <he...@gmail.com> wrote:
> Actually we should be fine putting the binaries there as long as the
> VOTE is for the source.
>
> Let's verify with sebb in the general@ list about his concern.
>
> - Henry
>
> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <he...@gmail.com> wrote:
>> Hi Patrick, as sebb has mentioned let's move the binaries from the
>> voting directory in your people.apache.org directory.
>> ASF release voting is for source code and not binaries, and
>> technically we provide binaries for convenience.
>>
>> And add link to the KEYS location in the dist[1] to let verify signatures.
>>
>> Sorry for the late response to the VOTE thread, guys.
>>
>> - Henry
>>
>> [1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
>>
>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <pw...@gmail.com> wrote:
>>> The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1
>>> votes.
>>>
>>> +1 (5 Total)
>>> Matei Zaharia*
>>> Nick Pentreath*
>>> Patrick Wendell*
>>> Prashant Sharma*
>>> Tom Graves*
>>>
>>> 0 (0 Total)
>>>
>>> -1 (0 Total)
>>>
>>> * = Binding Vote
>>>
>>> As per the incubator release guide [1] I'll be sending this to the
>>> general incubator list for a final vote from IPMC members.
>>>
>>> [1]
>>> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-
>>> vote
>>>
>>>
>>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan <ev...@ooyala.com> wrote:
>>>
>>>> I'd be personally fine with a standard workflow of assemble-deps +
>>>> packaging just the Spark files as separate packages, if it speeds up
>>>> everyone's development time.
>>>>
>>>>
>>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <mark@clearstorydata.com
>>>> >wrote:
>>>>
>>>> > I don't know how to make sense of the numbers, but here's what I've got
>>>> > from a very small sample size.
>>>> >
>>>> > For both v0.8.0-incubating and v0.8.1-incubating, building separate
>>>> > assemblies is faster than `./sbt/sbt assembly` and the times for building
>>>> > separate assemblies for 0.8.0 and 0.8.1 are about the same.
>>>> >
>>>> > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as
>>>> the
>>>> > sum of the separate assemblies.
>>>> > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as
>>>> the
>>>> > sum of the separate assemblies.
>>>> >
>>>> > Weird.
>>>> >
>>>> >
>>>> > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell <pwendell@gmail.com
>>>> > >wrote:
>>>> >
>>>> > > I'll +1 myself also.
>>>> > >
>>>> > > For anyone who has the slow build problem: does this issue happen when
>>>> > > building v0.8.0-incubating also? Trying to figure out whether it's
>>>> > > related to something we added in 0.8.1 or if it's a long standing
>>>> > > issue.
>>>> > >
>>>> > > - Patrick
>>>> > >
>>>> > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <
>>>> matei.zaharia@gmail.com
>>>> > >
>>>> > > wrote:
>>>> > > > Woah, weird, but definitely good to know.
>>>> > > >
>>>> > > > If you’re doing Spark development, there’s also a more convenient
>>>> > option
>>>> > > added by Shivaram in the master branch. You can do sbt assemble-deps to
>>>> > > package *just* the dependencies of each project in a special assembly
>>>> > JAR,
>>>> > > and then use sbt compile to update the code. This will use the classes
>>>> > > directly out of the target/scala-2.9.3/classes directories. You have to
>>>> > > redo assemble-deps only if your external dependencies change.
>>>> > > >
>>>> > > > Matei
>>>> > > >
>>>> > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com>
>>>> > > wrote:
>>>> > > >
>>>> > > >> I hope this PR https://github.com/apache/incubator-spark/pull/252can
>>>> > > help.
>>>> > > >> Again this is not a blocker for the release from my side either.
>>>> > > >>
>>>> > > >>
>>>> > > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <
>>>> > mark@clearstorydata.com
>>>> > > >wrote:
>>>> > > >>
>>>> > > >>> Interesting, and confirmed: On my machine where `./sbt/sbt
>>>> assembly`
>>>> > > takes
>>>> > > >>> a long, long, looooong time to complete (a MBP, in my case),
>>>> building
>>>> > > three
>>>> > > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
>>>> > > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much
>>>> less
>>>> > > time.
>>>> > > >>>
>>>> > > >>>
>>>> > > >>>
>>>> > > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <
>>>> > > scrapcodes@gmail.com
>>>> > > >>>> wrote:
>>>> > > >>>
>>>> > > >>>> forgot to mention, after running sbt/sbt assembly/assembly running
>>>> > > >>> sbt/sbt
>>>> > > >>>> examples/assembly takes just 37s. Not to mention my hardware is
>>>> not
>>>> > > >>> really
>>>> > > >>>> great.
>>>> > > >>>>
>>>> > > >>>>
>>>> > > >>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <
>>>> > > scrapcodes@gmail.com
>>>> > > >>>>> wrote:
>>>> > > >>>>
>>>> > > >>>>> Hi Patrick and Matei,
>>>> > > >>>>>
>>>> > > >>>>> Was trying out this and followed the quick start guide which says
>>>> > do
>>>> > > >>>>> sbt/sbt assembly, like few others I was also stuck for few
>>>> minutes
>>>> > on
>>>> > > >>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is
>>>> > > much
>>>> > > >>>>> faster.
>>>> > > >>>>>
>>>> > > >>>>> Should we change the documentation to reflect this. It will not
>>>> be
>>>> > > >>> great
>>>> > > >>>>> for first time users to get stuck there.
>>>> > > >>>>>
>>>> > > >>>>>
>>>> > > >>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
>>>> > > >>> matei.zaharia@gmail.com
>>>> > > >>>>> wrote:
>>>> > > >>>>>
>>>> > > >>>>>> +1
>>>> > > >>>>>>
>>>> > > >>>>>> Built and tested it on Mac OS X.
>>>> > > >>>>>>
>>>> > > >>>>>> Matei
>>>> > > >>>>>>
>>>> > > >>>>>>
>>>> > > >>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <
>>>> pwendell@gmail.com>
>>>> > > >>>> wrote:
>>>> > > >>>>>>
>>>> > > >>>>>>> Please vote on releasing the following candidate as Apache
>>>> Spark
>>>> > > >>>>>>> (incubating) version 0.8.1.
>>>> > > >>>>>>>
>>>> > > >>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
>>>> > > >>>>>>>
>>>> > > >>>>>>
>>>> > > >>>>
>>>> > > >>>
>>>> > >
>>>> >
>>>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
>>>> > > >>>>>>>
>>>> > > >>>>>>> The release files, including signatures, digests, etc can be
>>>> > found
>>>> > > >>> at:
>>>> > > >>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
>>>> > > >>>>>>>
>>>> > > >>>>>>> Release artifacts are signed with the following key:
>>>> > > >>>>>>> https://people.apache.org/keys/committer/pwendell.asc
>>>> > > >>>>>>>
>>>> > > >>>>>>> The staging repository for this release can be found at:
>>>> > > >>>>>>>
>>>> > > >>>>
>>>> > > https://repository.apache.org/content/repositories/orgapachespark-040/
>>>> > > >>>>>>>
>>>> > > >>>>>>> The documentation corresponding to this release can be found
>>>> at:
>>>> > > >>>>>>>
>>>> > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
>>>> > > >>>>>>>
>>>> > > >>>>>>> For information about the contents of this release see:
>>>> > > >>>>>>>
>>>> > > >>>>>>
>>>> > > >>>>
>>>> > > >>>
>>>> > >
>>>> >
>>>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
>>>> > > >>>>>>>
>>>> > > >>>>>>> Please vote on releasing this package as Apache Spark
>>>> > > >>>> 0.8.1-incubating!
>>>> > > >>>>>>>
>>>> > > >>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
>>>> > > >>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
>>>> > > >>>>>>>
>>>> > > >>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
>>>> > > >>>>>>> [ ] -1 Do not release this package because ...
>>>> > > >>>>>>>
>>>> > > >>>>>>> To learn more about Apache Spark, please see
>>>> > > >>>>>>> http://spark.incubator.apache.org/
>>>> > > >>>>>>
>>>> > > >>>>>>
>>>> > > >>>>>
>>>> > > >>>>>
>>>> > > >>>>> --
>>>> > > >>>>> s
>>>> > > >>>>>
>>>> > > >>>>
>>>> > > >>>>
>>>> > > >>>>
>>>> > > >>>> --
>>>> > > >>>> s
>>>> > > >>>>
>>>> > > >>>
>>>> > > >>
>>>> > > >>
>>>> > > >>
>>>> > > >> --
>>>> > > >> s
>>>> > > >
>>>> > >
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> --
>>>> Evan Chan
>>>> Staff Engineer
>>>> ev@ooyala.com |
>>>>
>>>> <http://www.ooyala.com/>
>>>> <http://www.facebook.com/ooyala><http://www.linkedin.com/company/ooyala><
>>>> http://www.twitter.com/ooyala>
>>>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Henry Saputra <he...@gmail.com>.
Actually we should be fine putting the binaries there as long as the
VOTE is for the source.
Let's verify with sebb in the general@ list about his concern.
- Henry
On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra <he...@gmail.com> wrote:
> Hi Patrick, as sebb has mentioned let's move the binaries from the
> voting directory in your people.apache.org directory.
> ASF release voting is for source code and not binaries, and
> technically we provide binaries for convenience.
>
> And add link to the KEYS location in the dist[1] to let verify signatures.
>
> Sorry for the late response to the VOTE thread, guys.
>
> - Henry
>
> [1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
>
> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <pw...@gmail.com> wrote:
>> The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1
>> votes.
>>
>> +1 (5 Total)
>> Matei Zaharia*
>> Nick Pentreath*
>> Patrick Wendell*
>> Prashant Sharma*
>> Tom Graves*
>>
>> 0 (0 Total)
>>
>> -1 (0 Total)
>>
>> * = Binding Vote
>>
>> As per the incubator release guide [1] I'll be sending this to the
>> general incubator list for a final vote from IPMC members.
>>
>> [1]
>> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-
>> vote
>>
>>
>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan <ev...@ooyala.com> wrote:
>>
>>> I'd be personally fine with a standard workflow of assemble-deps +
>>> packaging just the Spark files as separate packages, if it speeds up
>>> everyone's development time.
>>>
>>>
>>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <mark@clearstorydata.com
>>> >wrote:
>>>
>>> > I don't know how to make sense of the numbers, but here's what I've got
>>> > from a very small sample size.
>>> >
>>> > For both v0.8.0-incubating and v0.8.1-incubating, building separate
>>> > assemblies is faster than `./sbt/sbt assembly` and the times for building
>>> > separate assemblies for 0.8.0 and 0.8.1 are about the same.
>>> >
>>> > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as
>>> the
>>> > sum of the separate assemblies.
>>> > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as
>>> the
>>> > sum of the separate assemblies.
>>> >
>>> > Weird.
>>> >
>>> >
>>> > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell <pwendell@gmail.com
>>> > >wrote:
>>> >
>>> > > I'll +1 myself also.
>>> > >
>>> > > For anyone who has the slow build problem: does this issue happen when
>>> > > building v0.8.0-incubating also? Trying to figure out whether it's
>>> > > related to something we added in 0.8.1 or if it's a long standing
>>> > > issue.
>>> > >
>>> > > - Patrick
>>> > >
>>> > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <
>>> matei.zaharia@gmail.com
>>> > >
>>> > > wrote:
>>> > > > Woah, weird, but definitely good to know.
>>> > > >
>>> > > > If you’re doing Spark development, there’s also a more convenient
>>> > option
>>> > > added by Shivaram in the master branch. You can do sbt assemble-deps to
>>> > > package *just* the dependencies of each project in a special assembly
>>> > JAR,
>>> > > and then use sbt compile to update the code. This will use the classes
>>> > > directly out of the target/scala-2.9.3/classes directories. You have to
>>> > > redo assemble-deps only if your external dependencies change.
>>> > > >
>>> > > > Matei
>>> > > >
>>> > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com>
>>> > > wrote:
>>> > > >
>>> > > >> I hope this PR https://github.com/apache/incubator-spark/pull/252can
>>> > > help.
>>> > > >> Again this is not a blocker for the release from my side either.
>>> > > >>
>>> > > >>
>>> > > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <
>>> > mark@clearstorydata.com
>>> > > >wrote:
>>> > > >>
>>> > > >>> Interesting, and confirmed: On my machine where `./sbt/sbt
>>> assembly`
>>> > > takes
>>> > > >>> a long, long, looooong time to complete (a MBP, in my case),
>>> building
>>> > > three
>>> > > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
>>> > > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much
>>> less
>>> > > time.
>>> > > >>>
>>> > > >>>
>>> > > >>>
>>> > > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <
>>> > > scrapcodes@gmail.com
>>> > > >>>> wrote:
>>> > > >>>
>>> > > >>>> forgot to mention, after running sbt/sbt assembly/assembly running
>>> > > >>> sbt/sbt
>>> > > >>>> examples/assembly takes just 37s. Not to mention my hardware is
>>> not
>>> > > >>> really
>>> > > >>>> great.
>>> > > >>>>
>>> > > >>>>
>>> > > >>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <
>>> > > scrapcodes@gmail.com
>>> > > >>>>> wrote:
>>> > > >>>>
>>> > > >>>>> Hi Patrick and Matei,
>>> > > >>>>>
>>> > > >>>>> Was trying out this and followed the quick start guide which says
>>> > do
>>> > > >>>>> sbt/sbt assembly, like few others I was also stuck for few
>>> minutes
>>> > on
>>> > > >>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is
>>> > > much
>>> > > >>>>> faster.
>>> > > >>>>>
>>> > > >>>>> Should we change the documentation to reflect this. It will not
>>> be
>>> > > >>> great
>>> > > >>>>> for first time users to get stuck there.
>>> > > >>>>>
>>> > > >>>>>
>>> > > >>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
>>> > > >>> matei.zaharia@gmail.com
>>> > > >>>>> wrote:
>>> > > >>>>>
>>> > > >>>>>> +1
>>> > > >>>>>>
>>> > > >>>>>> Built and tested it on Mac OS X.
>>> > > >>>>>>
>>> > > >>>>>> Matei
>>> > > >>>>>>
>>> > > >>>>>>
>>> > > >>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <
>>> pwendell@gmail.com>
>>> > > >>>> wrote:
>>> > > >>>>>>
>>> > > >>>>>>> Please vote on releasing the following candidate as Apache
>>> Spark
>>> > > >>>>>>> (incubating) version 0.8.1.
>>> > > >>>>>>>
>>> > > >>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
>>> > > >>>>>>>
>>> > > >>>>>>
>>> > > >>>>
>>> > > >>>
>>> > >
>>> >
>>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
>>> > > >>>>>>>
>>> > > >>>>>>> The release files, including signatures, digests, etc can be
>>> > found
>>> > > >>> at:
>>> > > >>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
>>> > > >>>>>>>
>>> > > >>>>>>> Release artifacts are signed with the following key:
>>> > > >>>>>>> https://people.apache.org/keys/committer/pwendell.asc
>>> > > >>>>>>>
>>> > > >>>>>>> The staging repository for this release can be found at:
>>> > > >>>>>>>
>>> > > >>>>
>>> > > https://repository.apache.org/content/repositories/orgapachespark-040/
>>> > > >>>>>>>
>>> > > >>>>>>> The documentation corresponding to this release can be found
>>> at:
>>> > > >>>>>>>
>>> > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
>>> > > >>>>>>>
>>> > > >>>>>>> For information about the contents of this release see:
>>> > > >>>>>>>
>>> > > >>>>>>
>>> > > >>>>
>>> > > >>>
>>> > >
>>> >
>>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
>>> > > >>>>>>>
>>> > > >>>>>>> Please vote on releasing this package as Apache Spark
>>> > > >>>> 0.8.1-incubating!
>>> > > >>>>>>>
>>> > > >>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
>>> > > >>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
>>> > > >>>>>>>
>>> > > >>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
>>> > > >>>>>>> [ ] -1 Do not release this package because ...
>>> > > >>>>>>>
>>> > > >>>>>>> To learn more about Apache Spark, please see
>>> > > >>>>>>> http://spark.incubator.apache.org/
>>> > > >>>>>>
>>> > > >>>>>>
>>> > > >>>>>
>>> > > >>>>>
>>> > > >>>>> --
>>> > > >>>>> s
>>> > > >>>>>
>>> > > >>>>
>>> > > >>>>
>>> > > >>>>
>>> > > >>>> --
>>> > > >>>> s
>>> > > >>>>
>>> > > >>>
>>> > > >>
>>> > > >>
>>> > > >>
>>> > > >> --
>>> > > >> s
>>> > > >
>>> > >
>>> >
>>>
>>>
>>>
>>> --
>>> --
>>> Evan Chan
>>> Staff Engineer
>>> ev@ooyala.com |
>>>
>>> <http://www.ooyala.com/>
>>> <http://www.facebook.com/ooyala><http://www.linkedin.com/company/ooyala><
>>> http://www.twitter.com/ooyala>
>>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Henry Saputra <he...@gmail.com>.
Hi Patrick, as sebb has mentioned let's move the binaries from the
voting directory in your people.apache.org directory.
ASF release voting is for source code and not binaries, and
technically we provide binaries for convenience.
And add link to the KEYS location in the dist[1] to let verify signatures.
Sorry for the late response to the VOTE thread, guys.
- Henry
[1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS
On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell <pw...@gmail.com> wrote:
> The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1
> votes.
>
> +1 (5 Total)
> Matei Zaharia*
> Nick Pentreath*
> Patrick Wendell*
> Prashant Sharma*
> Tom Graves*
>
> 0 (0 Total)
>
> -1 (0 Total)
>
> * = Binding Vote
>
> As per the incubator release guide [1] I'll be sending this to the
> general incubator list for a final vote from IPMC members.
>
> [1]
> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-
> vote
>
>
> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan <ev...@ooyala.com> wrote:
>
>> I'd be personally fine with a standard workflow of assemble-deps +
>> packaging just the Spark files as separate packages, if it speeds up
>> everyone's development time.
>>
>>
>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <mark@clearstorydata.com
>> >wrote:
>>
>> > I don't know how to make sense of the numbers, but here's what I've got
>> > from a very small sample size.
>> >
>> > For both v0.8.0-incubating and v0.8.1-incubating, building separate
>> > assemblies is faster than `./sbt/sbt assembly` and the times for building
>> > separate assemblies for 0.8.0 and 0.8.1 are about the same.
>> >
>> > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as
>> the
>> > sum of the separate assemblies.
>> > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as
>> the
>> > sum of the separate assemblies.
>> >
>> > Weird.
>> >
>> >
>> > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell <pwendell@gmail.com
>> > >wrote:
>> >
>> > > I'll +1 myself also.
>> > >
>> > > For anyone who has the slow build problem: does this issue happen when
>> > > building v0.8.0-incubating also? Trying to figure out whether it's
>> > > related to something we added in 0.8.1 or if it's a long standing
>> > > issue.
>> > >
>> > > - Patrick
>> > >
>> > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <
>> matei.zaharia@gmail.com
>> > >
>> > > wrote:
>> > > > Woah, weird, but definitely good to know.
>> > > >
>> > > > If you’re doing Spark development, there’s also a more convenient
>> > option
>> > > added by Shivaram in the master branch. You can do sbt assemble-deps to
>> > > package *just* the dependencies of each project in a special assembly
>> > JAR,
>> > > and then use sbt compile to update the code. This will use the classes
>> > > directly out of the target/scala-2.9.3/classes directories. You have to
>> > > redo assemble-deps only if your external dependencies change.
>> > > >
>> > > > Matei
>> > > >
>> > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com>
>> > > wrote:
>> > > >
>> > > >> I hope this PR https://github.com/apache/incubator-spark/pull/252can
>> > > help.
>> > > >> Again this is not a blocker for the release from my side either.
>> > > >>
>> > > >>
>> > > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <
>> > mark@clearstorydata.com
>> > > >wrote:
>> > > >>
>> > > >>> Interesting, and confirmed: On my machine where `./sbt/sbt
>> assembly`
>> > > takes
>> > > >>> a long, long, looooong time to complete (a MBP, in my case),
>> building
>> > > three
>> > > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
>> > > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much
>> less
>> > > time.
>> > > >>>
>> > > >>>
>> > > >>>
>> > > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <
>> > > scrapcodes@gmail.com
>> > > >>>> wrote:
>> > > >>>
>> > > >>>> forgot to mention, after running sbt/sbt assembly/assembly running
>> > > >>> sbt/sbt
>> > > >>>> examples/assembly takes just 37s. Not to mention my hardware is
>> not
>> > > >>> really
>> > > >>>> great.
>> > > >>>>
>> > > >>>>
>> > > >>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <
>> > > scrapcodes@gmail.com
>> > > >>>>> wrote:
>> > > >>>>
>> > > >>>>> Hi Patrick and Matei,
>> > > >>>>>
>> > > >>>>> Was trying out this and followed the quick start guide which says
>> > do
>> > > >>>>> sbt/sbt assembly, like few others I was also stuck for few
>> minutes
>> > on
>> > > >>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is
>> > > much
>> > > >>>>> faster.
>> > > >>>>>
>> > > >>>>> Should we change the documentation to reflect this. It will not
>> be
>> > > >>> great
>> > > >>>>> for first time users to get stuck there.
>> > > >>>>>
>> > > >>>>>
>> > > >>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
>> > > >>> matei.zaharia@gmail.com
>> > > >>>>> wrote:
>> > > >>>>>
>> > > >>>>>> +1
>> > > >>>>>>
>> > > >>>>>> Built and tested it on Mac OS X.
>> > > >>>>>>
>> > > >>>>>> Matei
>> > > >>>>>>
>> > > >>>>>>
>> > > >>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <
>> pwendell@gmail.com>
>> > > >>>> wrote:
>> > > >>>>>>
>> > > >>>>>>> Please vote on releasing the following candidate as Apache
>> Spark
>> > > >>>>>>> (incubating) version 0.8.1.
>> > > >>>>>>>
>> > > >>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
>> > > >>>>>>>
>> > > >>>>>>
>> > > >>>>
>> > > >>>
>> > >
>> >
>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
>> > > >>>>>>>
>> > > >>>>>>> The release files, including signatures, digests, etc can be
>> > found
>> > > >>> at:
>> > > >>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
>> > > >>>>>>>
>> > > >>>>>>> Release artifacts are signed with the following key:
>> > > >>>>>>> https://people.apache.org/keys/committer/pwendell.asc
>> > > >>>>>>>
>> > > >>>>>>> The staging repository for this release can be found at:
>> > > >>>>>>>
>> > > >>>>
>> > > https://repository.apache.org/content/repositories/orgapachespark-040/
>> > > >>>>>>>
>> > > >>>>>>> The documentation corresponding to this release can be found
>> at:
>> > > >>>>>>>
>> > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
>> > > >>>>>>>
>> > > >>>>>>> For information about the contents of this release see:
>> > > >>>>>>>
>> > > >>>>>>
>> > > >>>>
>> > > >>>
>> > >
>> >
>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
>> > > >>>>>>>
>> > > >>>>>>> Please vote on releasing this package as Apache Spark
>> > > >>>> 0.8.1-incubating!
>> > > >>>>>>>
>> > > >>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
>> > > >>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
>> > > >>>>>>>
>> > > >>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
>> > > >>>>>>> [ ] -1 Do not release this package because ...
>> > > >>>>>>>
>> > > >>>>>>> To learn more about Apache Spark, please see
>> > > >>>>>>> http://spark.incubator.apache.org/
>> > > >>>>>>
>> > > >>>>>>
>> > > >>>>>
>> > > >>>>>
>> > > >>>>> --
>> > > >>>>> s
>> > > >>>>>
>> > > >>>>
>> > > >>>>
>> > > >>>>
>> > > >>>> --
>> > > >>>> s
>> > > >>>>
>> > > >>>
>> > > >>
>> > > >>
>> > > >>
>> > > >> --
>> > > >> s
>> > > >
>> > >
>> >
>>
>>
>>
>> --
>> --
>> Evan Chan
>> Staff Engineer
>> ev@ooyala.com |
>>
>> <http://www.ooyala.com/>
>> <http://www.facebook.com/ooyala><http://www.linkedin.com/company/ooyala><
>> http://www.twitter.com/ooyala>
>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Patrick Wendell <pw...@gmail.com>.
The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1
votes.
+1 (5 Total)
Matei Zaharia*
Nick Pentreath*
Patrick Wendell*
Prashant Sharma*
Tom Graves*
0 (0 Total)
-1 (0 Total)
* = Binding Vote
As per the incubator release guide [1] I'll be sending this to the
general incubator list for a final vote from IPMC members.
[1]
http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release-
vote
On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan <ev...@ooyala.com> wrote:
> I'd be personally fine with a standard workflow of assemble-deps +
> packaging just the Spark files as separate packages, if it speeds up
> everyone's development time.
>
>
> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <mark@clearstorydata.com
> >wrote:
>
> > I don't know how to make sense of the numbers, but here's what I've got
> > from a very small sample size.
> >
> > For both v0.8.0-incubating and v0.8.1-incubating, building separate
> > assemblies is faster than `./sbt/sbt assembly` and the times for building
> > separate assemblies for 0.8.0 and 0.8.1 are about the same.
> >
> > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as
> the
> > sum of the separate assemblies.
> > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as
> the
> > sum of the separate assemblies.
> >
> > Weird.
> >
> >
> > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell <pwendell@gmail.com
> > >wrote:
> >
> > > I'll +1 myself also.
> > >
> > > For anyone who has the slow build problem: does this issue happen when
> > > building v0.8.0-incubating also? Trying to figure out whether it's
> > > related to something we added in 0.8.1 or if it's a long standing
> > > issue.
> > >
> > > - Patrick
> > >
> > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <
> matei.zaharia@gmail.com
> > >
> > > wrote:
> > > > Woah, weird, but definitely good to know.
> > > >
> > > > If you’re doing Spark development, there’s also a more convenient
> > option
> > > added by Shivaram in the master branch. You can do sbt assemble-deps to
> > > package *just* the dependencies of each project in a special assembly
> > JAR,
> > > and then use sbt compile to update the code. This will use the classes
> > > directly out of the target/scala-2.9.3/classes directories. You have to
> > > redo assemble-deps only if your external dependencies change.
> > > >
> > > > Matei
> > > >
> > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com>
> > > wrote:
> > > >
> > > >> I hope this PR https://github.com/apache/incubator-spark/pull/252can
> > > help.
> > > >> Again this is not a blocker for the release from my side either.
> > > >>
> > > >>
> > > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <
> > mark@clearstorydata.com
> > > >wrote:
> > > >>
> > > >>> Interesting, and confirmed: On my machine where `./sbt/sbt
> assembly`
> > > takes
> > > >>> a long, long, looooong time to complete (a MBP, in my case),
> building
> > > three
> > > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
> > > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much
> less
> > > time.
> > > >>>
> > > >>>
> > > >>>
> > > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <
> > > scrapcodes@gmail.com
> > > >>>> wrote:
> > > >>>
> > > >>>> forgot to mention, after running sbt/sbt assembly/assembly running
> > > >>> sbt/sbt
> > > >>>> examples/assembly takes just 37s. Not to mention my hardware is
> not
> > > >>> really
> > > >>>> great.
> > > >>>>
> > > >>>>
> > > >>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <
> > > scrapcodes@gmail.com
> > > >>>>> wrote:
> > > >>>>
> > > >>>>> Hi Patrick and Matei,
> > > >>>>>
> > > >>>>> Was trying out this and followed the quick start guide which says
> > do
> > > >>>>> sbt/sbt assembly, like few others I was also stuck for few
> minutes
> > on
> > > >>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is
> > > much
> > > >>>>> faster.
> > > >>>>>
> > > >>>>> Should we change the documentation to reflect this. It will not
> be
> > > >>> great
> > > >>>>> for first time users to get stuck there.
> > > >>>>>
> > > >>>>>
> > > >>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
> > > >>> matei.zaharia@gmail.com
> > > >>>>> wrote:
> > > >>>>>
> > > >>>>>> +1
> > > >>>>>>
> > > >>>>>> Built and tested it on Mac OS X.
> > > >>>>>>
> > > >>>>>> Matei
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <
> pwendell@gmail.com>
> > > >>>> wrote:
> > > >>>>>>
> > > >>>>>>> Please vote on releasing the following candidate as Apache
> Spark
> > > >>>>>>> (incubating) version 0.8.1.
> > > >>>>>>>
> > > >>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
> > > >>>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>
> > >
> >
> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
> > > >>>>>>>
> > > >>>>>>> The release files, including signatures, digests, etc can be
> > found
> > > >>> at:
> > > >>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
> > > >>>>>>>
> > > >>>>>>> Release artifacts are signed with the following key:
> > > >>>>>>> https://people.apache.org/keys/committer/pwendell.asc
> > > >>>>>>>
> > > >>>>>>> The staging repository for this release can be found at:
> > > >>>>>>>
> > > >>>>
> > > https://repository.apache.org/content/repositories/orgapachespark-040/
> > > >>>>>>>
> > > >>>>>>> The documentation corresponding to this release can be found
> at:
> > > >>>>>>>
> > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
> > > >>>>>>>
> > > >>>>>>> For information about the contents of this release see:
> > > >>>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
> > > >>>>>>>
> > > >>>>>>> Please vote on releasing this package as Apache Spark
> > > >>>> 0.8.1-incubating!
> > > >>>>>>>
> > > >>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
> > > >>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
> > > >>>>>>>
> > > >>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
> > > >>>>>>> [ ] -1 Do not release this package because ...
> > > >>>>>>>
> > > >>>>>>> To learn more about Apache Spark, please see
> > > >>>>>>> http://spark.incubator.apache.org/
> > > >>>>>>
> > > >>>>>>
> > > >>>>>
> > > >>>>>
> > > >>>>> --
> > > >>>>> s
> > > >>>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> --
> > > >>>> s
> > > >>>>
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> s
> > > >
> > >
> >
>
>
>
> --
> --
> Evan Chan
> Staff Engineer
> ev@ooyala.com |
>
> <http://www.ooyala.com/>
> <http://www.facebook.com/ooyala><http://www.linkedin.com/company/ooyala><
> http://www.twitter.com/ooyala>
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Evan Chan <ev...@ooyala.com>.
I'd be personally fine with a standard workflow of assemble-deps +
packaging just the Spark files as separate packages, if it speeds up
everyone's development time.
On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra <ma...@clearstorydata.com>wrote:
> I don't know how to make sense of the numbers, but here's what I've got
> from a very small sample size.
>
> For both v0.8.0-incubating and v0.8.1-incubating, building separate
> assemblies is faster than `./sbt/sbt assembly` and the times for building
> separate assemblies for 0.8.0 and 0.8.1 are about the same.
>
> For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as the
> sum of the separate assemblies.
> For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as the
> sum of the separate assemblies.
>
> Weird.
>
>
> On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell <pwendell@gmail.com
> >wrote:
>
> > I'll +1 myself also.
> >
> > For anyone who has the slow build problem: does this issue happen when
> > building v0.8.0-incubating also? Trying to figure out whether it's
> > related to something we added in 0.8.1 or if it's a long standing
> > issue.
> >
> > - Patrick
> >
> > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <matei.zaharia@gmail.com
> >
> > wrote:
> > > Woah, weird, but definitely good to know.
> > >
> > > If you’re doing Spark development, there’s also a more convenient
> option
> > added by Shivaram in the master branch. You can do sbt assemble-deps to
> > package *just* the dependencies of each project in a special assembly
> JAR,
> > and then use sbt compile to update the code. This will use the classes
> > directly out of the target/scala-2.9.3/classes directories. You have to
> > redo assemble-deps only if your external dependencies change.
> > >
> > > Matei
> > >
> > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com>
> > wrote:
> > >
> > >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can
> > help.
> > >> Again this is not a blocker for the release from my side either.
> > >>
> > >>
> > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <
> mark@clearstorydata.com
> > >wrote:
> > >>
> > >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly`
> > takes
> > >>> a long, long, looooong time to complete (a MBP, in my case), building
> > three
> > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
> > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less
> > time.
> > >>>
> > >>>
> > >>>
> > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <
> > scrapcodes@gmail.com
> > >>>> wrote:
> > >>>
> > >>>> forgot to mention, after running sbt/sbt assembly/assembly running
> > >>> sbt/sbt
> > >>>> examples/assembly takes just 37s. Not to mention my hardware is not
> > >>> really
> > >>>> great.
> > >>>>
> > >>>>
> > >>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <
> > scrapcodes@gmail.com
> > >>>>> wrote:
> > >>>>
> > >>>>> Hi Patrick and Matei,
> > >>>>>
> > >>>>> Was trying out this and followed the quick start guide which says
> do
> > >>>>> sbt/sbt assembly, like few others I was also stuck for few minutes
> on
> > >>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is
> > much
> > >>>>> faster.
> > >>>>>
> > >>>>> Should we change the documentation to reflect this. It will not be
> > >>> great
> > >>>>> for first time users to get stuck there.
> > >>>>>
> > >>>>>
> > >>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
> > >>> matei.zaharia@gmail.com
> > >>>>> wrote:
> > >>>>>
> > >>>>>> +1
> > >>>>>>
> > >>>>>> Built and tested it on Mac OS X.
> > >>>>>>
> > >>>>>> Matei
> > >>>>>>
> > >>>>>>
> > >>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com>
> > >>>> wrote:
> > >>>>>>
> > >>>>>>> Please vote on releasing the following candidate as Apache Spark
> > >>>>>>> (incubating) version 0.8.1.
> > >>>>>>>
> > >>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
> > >>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> >
> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
> > >>>>>>>
> > >>>>>>> The release files, including signatures, digests, etc can be
> found
> > >>> at:
> > >>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
> > >>>>>>>
> > >>>>>>> Release artifacts are signed with the following key:
> > >>>>>>> https://people.apache.org/keys/committer/pwendell.asc
> > >>>>>>>
> > >>>>>>> The staging repository for this release can be found at:
> > >>>>>>>
> > >>>>
> > https://repository.apache.org/content/repositories/orgapachespark-040/
> > >>>>>>>
> > >>>>>>> The documentation corresponding to this release can be found at:
> > >>>>>>>
> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
> > >>>>>>>
> > >>>>>>> For information about the contents of this release see:
> > >>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> >
> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
> > >>>>>>>
> > >>>>>>> Please vote on releasing this package as Apache Spark
> > >>>> 0.8.1-incubating!
> > >>>>>>>
> > >>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
> > >>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
> > >>>>>>>
> > >>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
> > >>>>>>> [ ] -1 Do not release this package because ...
> > >>>>>>>
> > >>>>>>> To learn more about Apache Spark, please see
> > >>>>>>> http://spark.incubator.apache.org/
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> s
> > >>>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> --
> > >>>> s
> > >>>>
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> s
> > >
> >
>
--
--
Evan Chan
Staff Engineer
ev@ooyala.com |
<http://www.ooyala.com/>
<http://www.facebook.com/ooyala><http://www.linkedin.com/company/ooyala><http://www.twitter.com/ooyala>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Mark Hamstra <ma...@clearstorydata.com>.
I don't know how to make sense of the numbers, but here's what I've got
from a very small sample size.
For both v0.8.0-incubating and v0.8.1-incubating, building separate
assemblies is faster than `./sbt/sbt assembly` and the times for building
separate assemblies for 0.8.0 and 0.8.1 are about the same.
For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as the
sum of the separate assemblies.
For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as the
sum of the separate assemblies.
Weird.
On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell <pw...@gmail.com>wrote:
> I'll +1 myself also.
>
> For anyone who has the slow build problem: does this issue happen when
> building v0.8.0-incubating also? Trying to figure out whether it's
> related to something we added in 0.8.1 or if it's a long standing
> issue.
>
> - Patrick
>
> On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <ma...@gmail.com>
> wrote:
> > Woah, weird, but definitely good to know.
> >
> > If you’re doing Spark development, there’s also a more convenient option
> added by Shivaram in the master branch. You can do sbt assemble-deps to
> package *just* the dependencies of each project in a special assembly JAR,
> and then use sbt compile to update the code. This will use the classes
> directly out of the target/scala-2.9.3/classes directories. You have to
> redo assemble-deps only if your external dependencies change.
> >
> > Matei
> >
> > On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com>
> wrote:
> >
> >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can
> help.
> >> Again this is not a blocker for the release from my side either.
> >>
> >>
> >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <mark@clearstorydata.com
> >wrote:
> >>
> >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly`
> takes
> >>> a long, long, looooong time to complete (a MBP, in my case), building
> three
> >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
> >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less
> time.
> >>>
> >>>
> >>>
> >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <
> scrapcodes@gmail.com
> >>>> wrote:
> >>>
> >>>> forgot to mention, after running sbt/sbt assembly/assembly running
> >>> sbt/sbt
> >>>> examples/assembly takes just 37s. Not to mention my hardware is not
> >>> really
> >>>> great.
> >>>>
> >>>>
> >>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <
> scrapcodes@gmail.com
> >>>>> wrote:
> >>>>
> >>>>> Hi Patrick and Matei,
> >>>>>
> >>>>> Was trying out this and followed the quick start guide which says do
> >>>>> sbt/sbt assembly, like few others I was also stuck for few minutes on
> >>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is
> much
> >>>>> faster.
> >>>>>
> >>>>> Should we change the documentation to reflect this. It will not be
> >>> great
> >>>>> for first time users to get stuck there.
> >>>>>
> >>>>>
> >>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
> >>> matei.zaharia@gmail.com
> >>>>> wrote:
> >>>>>
> >>>>>> +1
> >>>>>>
> >>>>>> Built and tested it on Mac OS X.
> >>>>>>
> >>>>>> Matei
> >>>>>>
> >>>>>>
> >>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Please vote on releasing the following candidate as Apache Spark
> >>>>>>> (incubating) version 0.8.1.
> >>>>>>>
> >>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
> >>>>>>>
> >>>>>>> The release files, including signatures, digests, etc can be found
> >>> at:
> >>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
> >>>>>>>
> >>>>>>> Release artifacts are signed with the following key:
> >>>>>>> https://people.apache.org/keys/committer/pwendell.asc
> >>>>>>>
> >>>>>>> The staging repository for this release can be found at:
> >>>>>>>
> >>>>
> https://repository.apache.org/content/repositories/orgapachespark-040/
> >>>>>>>
> >>>>>>> The documentation corresponding to this release can be found at:
> >>>>>>>
> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
> >>>>>>>
> >>>>>>> For information about the contents of this release see:
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
> >>>>>>>
> >>>>>>> Please vote on releasing this package as Apache Spark
> >>>> 0.8.1-incubating!
> >>>>>>>
> >>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
> >>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
> >>>>>>>
> >>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
> >>>>>>> [ ] -1 Do not release this package because ...
> >>>>>>>
> >>>>>>> To learn more about Apache Spark, please see
> >>>>>>> http://spark.incubator.apache.org/
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> s
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> s
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> s
> >
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Taka Shinagawa <ta...@gmail.com>.
On my Core 2 Duo (slower) Mac, I'm having the same problem with building
v0.8.0-incubating as well.
I'm suspecting the slow build problem is caused by sbt-assembly as reported
here.
https://github.com/sbt/sbt-assembly/issues/68
I tried to upgrade sbt-assembly to version 0.10.1 (and sbt to 0.13.0). But
because of the "sbt-assembly 0.10.1 ==> sbt 0.13 ==> sbt-dependency-graph
0.7.4 ==> Scala 2.10" dependency, I wasn't able to use sbt-assembly 0.10.1
for this release.
According to this thread ( https://github.com/sbt/sbt-assembly/issues/96 ),
with sbt-assembly 0.9.2, setting "assemblyCacheOutput in assembly :=
false" might
fix. I haven't verified, though.
On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell <pw...@gmail.com>wrote:
> I'll +1 myself also.
>
> For anyone who has the slow build problem: does this issue happen when
> building v0.8.0-incubating also? Trying to figure out whether it's
> related to something we added in 0.8.1 or if it's a long standing
> issue.
>
> - Patrick
>
> On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <ma...@gmail.com>
> wrote:
> > Woah, weird, but definitely good to know.
> >
> > If you’re doing Spark development, there’s also a more convenient option
> added by Shivaram in the master branch. You can do sbt assemble-deps to
> package *just* the dependencies of each project in a special assembly JAR,
> and then use sbt compile to update the code. This will use the classes
> directly out of the target/scala-2.9.3/classes directories. You have to
> redo assemble-deps only if your external dependencies change.
> >
> > Matei
> >
> > On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com>
> wrote:
> >
> >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can
> help.
> >> Again this is not a blocker for the release from my side either.
> >>
> >>
> >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <mark@clearstorydata.com
> >wrote:
> >>
> >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly`
> takes
> >>> a long, long, looooong time to complete (a MBP, in my case), building
> three
> >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
> >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less
> time.
> >>>
> >>>
> >>>
> >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <
> scrapcodes@gmail.com
> >>>> wrote:
> >>>
> >>>> forgot to mention, after running sbt/sbt assembly/assembly running
> >>> sbt/sbt
> >>>> examples/assembly takes just 37s. Not to mention my hardware is not
> >>> really
> >>>> great.
> >>>>
> >>>>
> >>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <
> scrapcodes@gmail.com
> >>>>> wrote:
> >>>>
> >>>>> Hi Patrick and Matei,
> >>>>>
> >>>>> Was trying out this and followed the quick start guide which says do
> >>>>> sbt/sbt assembly, like few others I was also stuck for few minutes on
> >>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is
> much
> >>>>> faster.
> >>>>>
> >>>>> Should we change the documentation to reflect this. It will not be
> >>> great
> >>>>> for first time users to get stuck there.
> >>>>>
> >>>>>
> >>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
> >>> matei.zaharia@gmail.com
> >>>>> wrote:
> >>>>>
> >>>>>> +1
> >>>>>>
> >>>>>> Built and tested it on Mac OS X.
> >>>>>>
> >>>>>> Matei
> >>>>>>
> >>>>>>
> >>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Please vote on releasing the following candidate as Apache Spark
> >>>>>>> (incubating) version 0.8.1.
> >>>>>>>
> >>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
> >>>>>>>
> >>>>>>> The release files, including signatures, digests, etc can be found
> >>> at:
> >>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
> >>>>>>>
> >>>>>>> Release artifacts are signed with the following key:
> >>>>>>> https://people.apache.org/keys/committer/pwendell.asc
> >>>>>>>
> >>>>>>> The staging repository for this release can be found at:
> >>>>>>>
> >>>>
> https://repository.apache.org/content/repositories/orgapachespark-040/
> >>>>>>>
> >>>>>>> The documentation corresponding to this release can be found at:
> >>>>>>>
> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
> >>>>>>>
> >>>>>>> For information about the contents of this release see:
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
> >>>>>>>
> >>>>>>> Please vote on releasing this package as Apache Spark
> >>>> 0.8.1-incubating!
> >>>>>>>
> >>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
> >>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
> >>>>>>>
> >>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
> >>>>>>> [ ] -1 Do not release this package because ...
> >>>>>>>
> >>>>>>> To learn more about Apache Spark, please see
> >>>>>>> http://spark.incubator.apache.org/
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> s
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> s
> >>>>
> >>>
> >>
> >>
> >>
> >> --
> >> s
> >
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Tom Graves <tg...@yahoo.com>.
Hey folks,
I'm trying to verify the signature on the rc4 but am getting a BAD signature, is it working for others? Perhaps I messed up the import of the asc file.
$ gpg --verify spark-0.8.1-incubating.tgz.asc spark-0.8.1-incubating.tgz
gpg: Signature made Tue 10 Dec 2013 10:53:15 PM UTC using RSA key ID 9E4FE3AF
gpg: BAD signature from "Patrick Wendell <pw...@gmail.com>"
Tom
On Wednesday, December 11, 2013 1:50 PM, Patrick Wendell <pw...@gmail.com> wrote:
I'll +1 myself also.
For anyone who has the slow build problem: does this issue happen when
building v0.8.0-incubating also? Trying to figure out whether it's
related to something we added in 0.8.1
or if it's a long standing
issue.
- Patrick
On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <ma...@gmail.com> wrote:
> Woah, weird, but definitely good to know.
>
> If you’re doing Spark development, there’s also a more convenient option added by Shivaram in the master branch. You can do sbt assemble-deps to package *just* the dependencies of each project in a special assembly JAR, and then use sbt compile to update the code. This will use the classes directly out of the target/scala-2.9.3/classes directories. You have to redo assemble-deps only if your external dependencies change.
>
> Matei
>
> On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com> wrote:
>
>> I hope this PR https://github.com/apache/incubator-spark/pull/252 can help.
>> Again this is not a blocker for the release from my side either.
>>
>>
>> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <ma...@clearstorydata.com>wrote:
>>
>>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` takes
>>> a long, long, looooong time to complete (a MBP, in
my case), building three
>>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
>>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less time.
>>>
>>>
>>>
>>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <scrapcodes@gmail.com
>>>> wrote:
>>>
>>>> forgot to mention, after running sbt/sbt assembly/assembly running
>>> sbt/sbt
>>>> examples/assembly takes just 37s. Not to mention my hardware is not
>>> really
>>>> great.
>>>>
>>>>
>>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <scrapcodes@gmail.com
>>>>> wrote:
>>>>
>>>>> Hi Patrick and Matei,
>>>>>
>>>>> Was trying out this and followed the quick start guide which says do
>>>>> sbt/sbt assembly, like few others I was also stuck for few minutes on
>>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is much
>>>>> faster.
>>>>>
>>>>> Should we change the documentation to reflect this. It will not be
>>> great
>>>>> for first time
users to get stuck there.
>>>>>
>>>>>
>>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
>>> matei.zaharia@gmail.com
>>>>> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> Built and tested it on Mac OS X.
>>>>>>
>>>>>> Matei
>>>>>>
>>>>>>
>>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com>
>>>> wrote:
>>>>>>
>>>>>>> Please vote on releasing the following candidate as Apache Spark
>>>>>>> (incubating) version 0.8.1.
>>>>>>>
>>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
>>>>>>>
>>>>>>
>>>>
>>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
>>>>>>>
>>>>>>> The
release files, including signatures, digests, etc can be found
>>> at:
>>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
>>>>>>>
>>>>>>> Release artifacts are signed with the following key:
>>>>>>> https://people.apache.org/keys/committer/pwendell.asc
>>>>>>>
>>>>>>> The staging repository for this release can be found at:
>>>>>>>
>>>> https://repository.apache.org/content/repositories/orgapachespark-040/
>>>>>>>
>>>>>>> The documentation corresponding to this release can be found at:
>>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
>>>>>>>
>>>>>>> For information about the contents of this release see:
>>>>>>>
>>>>>>
>>>>
>>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
>>>>>>>
>>>>>>> Please vote on releasing this package as Apache Spark
>>>> 0.8.1-incubating!
>>>>>>>
>>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
>>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
>>>>>>>
>>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
>>>>>>> [ ] -1 Do not release this package because ...
>>>>>>>
>>>>>>> To learn more about
Apache Spark, please see
>>>>>>> http://spark.incubator.apache.org/
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> s
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> s
>>>>
>>>
>>
>>
>>
>> --
>> s
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Patrick Wendell <pw...@gmail.com>.
I'll +1 myself also.
For anyone who has the slow build problem: does this issue happen when
building v0.8.0-incubating also? Trying to figure out whether it's
related to something we added in 0.8.1 or if it's a long standing
issue.
- Patrick
On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia <ma...@gmail.com> wrote:
> Woah, weird, but definitely good to know.
>
> If you’re doing Spark development, there’s also a more convenient option added by Shivaram in the master branch. You can do sbt assemble-deps to package *just* the dependencies of each project in a special assembly JAR, and then use sbt compile to update the code. This will use the classes directly out of the target/scala-2.9.3/classes directories. You have to redo assemble-deps only if your external dependencies change.
>
> Matei
>
> On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com> wrote:
>
>> I hope this PR https://github.com/apache/incubator-spark/pull/252 can help.
>> Again this is not a blocker for the release from my side either.
>>
>>
>> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <ma...@clearstorydata.com>wrote:
>>
>>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` takes
>>> a long, long, looooong time to complete (a MBP, in my case), building three
>>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
>>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less time.
>>>
>>>
>>>
>>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <scrapcodes@gmail.com
>>>> wrote:
>>>
>>>> forgot to mention, after running sbt/sbt assembly/assembly running
>>> sbt/sbt
>>>> examples/assembly takes just 37s. Not to mention my hardware is not
>>> really
>>>> great.
>>>>
>>>>
>>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <scrapcodes@gmail.com
>>>>> wrote:
>>>>
>>>>> Hi Patrick and Matei,
>>>>>
>>>>> Was trying out this and followed the quick start guide which says do
>>>>> sbt/sbt assembly, like few others I was also stuck for few minutes on
>>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is much
>>>>> faster.
>>>>>
>>>>> Should we change the documentation to reflect this. It will not be
>>> great
>>>>> for first time users to get stuck there.
>>>>>
>>>>>
>>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
>>> matei.zaharia@gmail.com
>>>>> wrote:
>>>>>
>>>>>> +1
>>>>>>
>>>>>> Built and tested it on Mac OS X.
>>>>>>
>>>>>> Matei
>>>>>>
>>>>>>
>>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com>
>>>> wrote:
>>>>>>
>>>>>>> Please vote on releasing the following candidate as Apache Spark
>>>>>>> (incubating) version 0.8.1.
>>>>>>>
>>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
>>>>>>>
>>>>>>
>>>>
>>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
>>>>>>>
>>>>>>> The release files, including signatures, digests, etc can be found
>>> at:
>>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
>>>>>>>
>>>>>>> Release artifacts are signed with the following key:
>>>>>>> https://people.apache.org/keys/committer/pwendell.asc
>>>>>>>
>>>>>>> The staging repository for this release can be found at:
>>>>>>>
>>>> https://repository.apache.org/content/repositories/orgapachespark-040/
>>>>>>>
>>>>>>> The documentation corresponding to this release can be found at:
>>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
>>>>>>>
>>>>>>> For information about the contents of this release see:
>>>>>>>
>>>>>>
>>>>
>>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
>>>>>>>
>>>>>>> Please vote on releasing this package as Apache Spark
>>>> 0.8.1-incubating!
>>>>>>>
>>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
>>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
>>>>>>>
>>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
>>>>>>> [ ] -1 Do not release this package because ...
>>>>>>>
>>>>>>> To learn more about Apache Spark, please see
>>>>>>> http://spark.incubator.apache.org/
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> s
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> s
>>>>
>>>
>>
>>
>>
>> --
>> s
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Matei Zaharia <ma...@gmail.com>.
Woah, weird, but definitely good to know.
If you’re doing Spark development, there’s also a more convenient option added by Shivaram in the master branch. You can do sbt assemble-deps to package *just* the dependencies of each project in a special assembly JAR, and then use sbt compile to update the code. This will use the classes directly out of the target/scala-2.9.3/classes directories. You have to redo assemble-deps only if your external dependencies change.
Matei
On Dec 11, 2013, at 1:04 AM, Prashant Sharma <sc...@gmail.com> wrote:
> I hope this PR https://github.com/apache/incubator-spark/pull/252 can help.
> Again this is not a blocker for the release from my side either.
>
>
> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <ma...@clearstorydata.com>wrote:
>
>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` takes
>> a long, long, looooong time to complete (a MBP, in my case), building three
>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less time.
>>
>>
>>
>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <scrapcodes@gmail.com
>>> wrote:
>>
>>> forgot to mention, after running sbt/sbt assembly/assembly running
>> sbt/sbt
>>> examples/assembly takes just 37s. Not to mention my hardware is not
>> really
>>> great.
>>>
>>>
>>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <scrapcodes@gmail.com
>>>> wrote:
>>>
>>>> Hi Patrick and Matei,
>>>>
>>>> Was trying out this and followed the quick start guide which says do
>>>> sbt/sbt assembly, like few others I was also stuck for few minutes on
>>>> linux. On the other hand if I use sbt/sbt assembly/assembly it is much
>>>> faster.
>>>>
>>>> Should we change the documentation to reflect this. It will not be
>> great
>>>> for first time users to get stuck there.
>>>>
>>>>
>>>> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
>> matei.zaharia@gmail.com
>>>> wrote:
>>>>
>>>>> +1
>>>>>
>>>>> Built and tested it on Mac OS X.
>>>>>
>>>>> Matei
>>>>>
>>>>>
>>>>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com>
>>> wrote:
>>>>>
>>>>>> Please vote on releasing the following candidate as Apache Spark
>>>>>> (incubating) version 0.8.1.
>>>>>>
>>>>>> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
>>>>>>
>>>>>
>>>
>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
>>>>>>
>>>>>> The release files, including signatures, digests, etc can be found
>> at:
>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
>>>>>>
>>>>>> Release artifacts are signed with the following key:
>>>>>> https://people.apache.org/keys/committer/pwendell.asc
>>>>>>
>>>>>> The staging repository for this release can be found at:
>>>>>>
>>> https://repository.apache.org/content/repositories/orgapachespark-040/
>>>>>>
>>>>>> The documentation corresponding to this release can be found at:
>>>>>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
>>>>>>
>>>>>> For information about the contents of this release see:
>>>>>>
>>>>>
>>>
>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
>>>>>>
>>>>>> Please vote on releasing this package as Apache Spark
>>> 0.8.1-incubating!
>>>>>>
>>>>>> The vote is open until Saturday, December 14th at 01:00 UTC and
>>>>>> passes if a majority of at least 3 +1 PPMC votes are cast.
>>>>>>
>>>>>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
>>>>>> [ ] -1 Do not release this package because ...
>>>>>>
>>>>>> To learn more about Apache Spark, please see
>>>>>> http://spark.incubator.apache.org/
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> s
>>>>
>>>
>>>
>>>
>>> --
>>> s
>>>
>>
>
>
>
> --
> s
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Nick Pentreath <ni...@gmail.com>.
- Successfully built via sbt/sbt assembly/assembly on Mac OS X, as well
as on a dev Ubuntu EC2 box
- Successfully tested via sbt/sbt test locally
- Successfully built and tested using mvn package locally
- I've tested my own Spark jobs (built against 0.8.0-incubating) on this
RC and all works fine, as well as tested with my job server (also built
against 0.8.0-incubating)
- Ran a few spark examples and the shell and PySpark shell
- For my part, tested the MLlib implicit code I added, and checked docs
I'm +1
On Wed, Dec 11, 2013 at 11:04 AM, Prashant Sharma <sc...@gmail.com>wrote:
> I hope this PR https://github.com/apache/incubator-spark/pull/252 can
> help.
> Again this is not a blocker for the release from my side either.
>
>
> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <mark@clearstorydata.com
> >wrote:
>
> > Interesting, and confirmed: On my machine where `./sbt/sbt assembly`
> takes
> > a long, long, looooong time to complete (a MBP, in my case), building
> three
> > separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
> > examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less
> time.
> >
> >
> >
> > On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <scrapcodes@gmail.com
> > >wrote:
> >
> > > forgot to mention, after running sbt/sbt assembly/assembly running
> > sbt/sbt
> > > examples/assembly takes just 37s. Not to mention my hardware is not
> > really
> > > great.
> > >
> > >
> > > On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <scrapcodes@gmail.com
> > > >wrote:
> > >
> > > > Hi Patrick and Matei,
> > > >
> > > > Was trying out this and followed the quick start guide which says do
> > > > sbt/sbt assembly, like few others I was also stuck for few minutes on
> > > > linux. On the other hand if I use sbt/sbt assembly/assembly it is
> much
> > > > faster.
> > > >
> > > > Should we change the documentation to reflect this. It will not be
> > great
> > > > for first time users to get stuck there.
> > > >
> > > >
> > > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
> > matei.zaharia@gmail.com
> > > >wrote:
> > > >
> > > >> +1
> > > >>
> > > >> Built and tested it on Mac OS X.
> > > >>
> > > >> Matei
> > > >>
> > > >>
> > > >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com>
> > > wrote:
> > > >>
> > > >> > Please vote on releasing the following candidate as Apache Spark
> > > >> > (incubating) version 0.8.1.
> > > >> >
> > > >> > The tag to be voted on is v0.8.1-incubating (commit b87d31d):
> > > >> >
> > > >>
> > >
> >
> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
> > > >> >
> > > >> > The release files, including signatures, digests, etc can be found
> > at:
> > > >> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
> > > >> >
> > > >> > Release artifacts are signed with the following key:
> > > >> > https://people.apache.org/keys/committer/pwendell.asc
> > > >> >
> > > >> > The staging repository for this release can be found at:
> > > >> >
> > > https://repository.apache.org/content/repositories/orgapachespark-040/
> > > >> >
> > > >> > The documentation corresponding to this release can be found at:
> > > >> >
> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
> > > >> >
> > > >> > For information about the contents of this release see:
> > > >> >
> > > >>
> > >
> >
> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
> > > >> >
> > > >> > Please vote on releasing this package as Apache Spark
> > > 0.8.1-incubating!
> > > >> >
> > > >> > The vote is open until Saturday, December 14th at 01:00 UTC and
> > > >> > passes if a majority of at least 3 +1 PPMC votes are cast.
> > > >> >
> > > >> > [ ] +1 Release this package as Apache Spark 0.8.1-incubating
> > > >> > [ ] -1 Do not release this package because ...
> > > >> >
> > > >> > To learn more about Apache Spark, please see
> > > >> > http://spark.incubator.apache.org/
> > > >>
> > > >>
> > > >
> > > >
> > > > --
> > > > s
> > > >
> > >
> > >
> > >
> > > --
> > > s
> > >
> >
>
>
>
> --
> s
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Prashant Sharma <sc...@gmail.com>.
I hope this PR https://github.com/apache/incubator-spark/pull/252 can help.
Again this is not a blocker for the release from my side either.
On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra <ma...@clearstorydata.com>wrote:
> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` takes
> a long, long, looooong time to complete (a MBP, in my case), building three
> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less time.
>
>
>
> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <scrapcodes@gmail.com
> >wrote:
>
> > forgot to mention, after running sbt/sbt assembly/assembly running
> sbt/sbt
> > examples/assembly takes just 37s. Not to mention my hardware is not
> really
> > great.
> >
> >
> > On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <scrapcodes@gmail.com
> > >wrote:
> >
> > > Hi Patrick and Matei,
> > >
> > > Was trying out this and followed the quick start guide which says do
> > > sbt/sbt assembly, like few others I was also stuck for few minutes on
> > > linux. On the other hand if I use sbt/sbt assembly/assembly it is much
> > > faster.
> > >
> > > Should we change the documentation to reflect this. It will not be
> great
> > > for first time users to get stuck there.
> > >
> > >
> > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <
> matei.zaharia@gmail.com
> > >wrote:
> > >
> > >> +1
> > >>
> > >> Built and tested it on Mac OS X.
> > >>
> > >> Matei
> > >>
> > >>
> > >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com>
> > wrote:
> > >>
> > >> > Please vote on releasing the following candidate as Apache Spark
> > >> > (incubating) version 0.8.1.
> > >> >
> > >> > The tag to be voted on is v0.8.1-incubating (commit b87d31d):
> > >> >
> > >>
> >
> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
> > >> >
> > >> > The release files, including signatures, digests, etc can be found
> at:
> > >> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
> > >> >
> > >> > Release artifacts are signed with the following key:
> > >> > https://people.apache.org/keys/committer/pwendell.asc
> > >> >
> > >> > The staging repository for this release can be found at:
> > >> >
> > https://repository.apache.org/content/repositories/orgapachespark-040/
> > >> >
> > >> > The documentation corresponding to this release can be found at:
> > >> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
> > >> >
> > >> > For information about the contents of this release see:
> > >> >
> > >>
> >
> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
> > >> >
> > >> > Please vote on releasing this package as Apache Spark
> > 0.8.1-incubating!
> > >> >
> > >> > The vote is open until Saturday, December 14th at 01:00 UTC and
> > >> > passes if a majority of at least 3 +1 PPMC votes are cast.
> > >> >
> > >> > [ ] +1 Release this package as Apache Spark 0.8.1-incubating
> > >> > [ ] -1 Do not release this package because ...
> > >> >
> > >> > To learn more about Apache Spark, please see
> > >> > http://spark.incubator.apache.org/
> > >>
> > >>
> > >
> > >
> > > --
> > > s
> > >
> >
> >
> >
> > --
> > s
> >
>
--
s
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Mark Hamstra <ma...@clearstorydata.com>.
Interesting, and confirmed: On my machine where `./sbt/sbt assembly` takes
a long, long, looooong time to complete (a MBP, in my case), building three
separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt
examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less time.
On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma <sc...@gmail.com>wrote:
> forgot to mention, after running sbt/sbt assembly/assembly running sbt/sbt
> examples/assembly takes just 37s. Not to mention my hardware is not really
> great.
>
>
> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <scrapcodes@gmail.com
> >wrote:
>
> > Hi Patrick and Matei,
> >
> > Was trying out this and followed the quick start guide which says do
> > sbt/sbt assembly, like few others I was also stuck for few minutes on
> > linux. On the other hand if I use sbt/sbt assembly/assembly it is much
> > faster.
> >
> > Should we change the documentation to reflect this. It will not be great
> > for first time users to get stuck there.
> >
> >
> > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <matei.zaharia@gmail.com
> >wrote:
> >
> >> +1
> >>
> >> Built and tested it on Mac OS X.
> >>
> >> Matei
> >>
> >>
> >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com>
> wrote:
> >>
> >> > Please vote on releasing the following candidate as Apache Spark
> >> > (incubating) version 0.8.1.
> >> >
> >> > The tag to be voted on is v0.8.1-incubating (commit b87d31d):
> >> >
> >>
> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
> >> >
> >> > The release files, including signatures, digests, etc can be found at:
> >> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
> >> >
> >> > Release artifacts are signed with the following key:
> >> > https://people.apache.org/keys/committer/pwendell.asc
> >> >
> >> > The staging repository for this release can be found at:
> >> >
> https://repository.apache.org/content/repositories/orgapachespark-040/
> >> >
> >> > The documentation corresponding to this release can be found at:
> >> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
> >> >
> >> > For information about the contents of this release see:
> >> >
> >>
> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
> >> >
> >> > Please vote on releasing this package as Apache Spark
> 0.8.1-incubating!
> >> >
> >> > The vote is open until Saturday, December 14th at 01:00 UTC and
> >> > passes if a majority of at least 3 +1 PPMC votes are cast.
> >> >
> >> > [ ] +1 Release this package as Apache Spark 0.8.1-incubating
> >> > [ ] -1 Do not release this package because ...
> >> >
> >> > To learn more about Apache Spark, please see
> >> > http://spark.incubator.apache.org/
> >>
> >>
> >
> >
> > --
> > s
> >
>
>
>
> --
> s
>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Prashant Sharma <sc...@gmail.com>.
forgot to mention, after running sbt/sbt assembly/assembly running sbt/sbt
examples/assembly takes just 37s. Not to mention my hardware is not really
great.
On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma <sc...@gmail.com>wrote:
> Hi Patrick and Matei,
>
> Was trying out this and followed the quick start guide which says do
> sbt/sbt assembly, like few others I was also stuck for few minutes on
> linux. On the other hand if I use sbt/sbt assembly/assembly it is much
> faster.
>
> Should we change the documentation to reflect this. It will not be great
> for first time users to get stuck there.
>
>
> On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <ma...@gmail.com>wrote:
>
>> +1
>>
>> Built and tested it on Mac OS X.
>>
>> Matei
>>
>>
>> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com> wrote:
>>
>> > Please vote on releasing the following candidate as Apache Spark
>> > (incubating) version 0.8.1.
>> >
>> > The tag to be voted on is v0.8.1-incubating (commit b87d31d):
>> >
>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
>> >
>> > The release files, including signatures, digests, etc can be found at:
>> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
>> >
>> > Release artifacts are signed with the following key:
>> > https://people.apache.org/keys/committer/pwendell.asc
>> >
>> > The staging repository for this release can be found at:
>> > https://repository.apache.org/content/repositories/orgapachespark-040/
>> >
>> > The documentation corresponding to this release can be found at:
>> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
>> >
>> > For information about the contents of this release see:
>> >
>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
>> >
>> > Please vote on releasing this package as Apache Spark 0.8.1-incubating!
>> >
>> > The vote is open until Saturday, December 14th at 01:00 UTC and
>> > passes if a majority of at least 3 +1 PPMC votes are cast.
>> >
>> > [ ] +1 Release this package as Apache Spark 0.8.1-incubating
>> > [ ] -1 Do not release this package because ...
>> >
>> > To learn more about Apache Spark, please see
>> > http://spark.incubator.apache.org/
>>
>>
>
>
> --
> s
>
--
s
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Prashant Sharma <sc...@gmail.com>.
Hi Patrick and Matei,
Was trying out this and followed the quick start guide which says do
sbt/sbt assembly, like few others I was also stuck for few minutes on
linux. On the other hand if I use sbt/sbt assembly/assembly it is much
faster.
Should we change the documentation to reflect this. It will not be great
for first time users to get stuck there.
On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia <ma...@gmail.com>wrote:
> +1
>
> Built and tested it on Mac OS X.
>
> Matei
>
>
> On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com> wrote:
>
> > Please vote on releasing the following candidate as Apache Spark
> > (incubating) version 0.8.1.
> >
> > The tag to be voted on is v0.8.1-incubating (commit b87d31d):
> >
> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
> >
> > The release files, including signatures, digests, etc can be found at:
> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/pwendell.asc
> >
> > The staging repository for this release can be found at:
> > https://repository.apache.org/content/repositories/orgapachespark-040/
> >
> > The documentation corresponding to this release can be found at:
> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
> >
> > For information about the contents of this release see:
> >
> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
> >
> > Please vote on releasing this package as Apache Spark 0.8.1-incubating!
> >
> > The vote is open until Saturday, December 14th at 01:00 UTC and
> > passes if a majority of at least 3 +1 PPMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Spark 0.8.1-incubating
> > [ ] -1 Do not release this package because ...
> >
> > To learn more about Apache Spark, please see
> > http://spark.incubator.apache.org/
>
>
--
s
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Posted by Matei Zaharia <ma...@gmail.com>.
+1
Built and tested it on Mac OS X.
Matei
On Dec 10, 2013, at 4:49 PM, Patrick Wendell <pw...@gmail.com> wrote:
> Please vote on releasing the following candidate as Apache Spark
> (incubating) version 0.8.1.
>
> The tag to be voted on is v0.8.1-incubating (commit b87d31d):
> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e
>
> The release files, including signatures, digests, etc can be found at:
> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-040/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/
>
> For information about the contents of this release see:
> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8
>
> Please vote on releasing this package as Apache Spark 0.8.1-incubating!
>
> The vote is open until Saturday, December 14th at 01:00 UTC and
> passes if a majority of at least 3 +1 PPMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 0.8.1-incubating
> [ ] -1 Do not release this package because ...
>
> To learn more about Apache Spark, please see
> http://spark.incubator.apache.org/