You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Reynold Xin <rx...@databricks.com> on 2015/09/24 09:27:25 UTC
[VOTE] Release Apache Spark 1.5.1 (RC1)
Please vote on releasing the following candidate as Apache Spark version
1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if
a majority of at least 3 +1 PMC votes are cast.
[ ] +1 Release this package as Apache Spark 1.5.1
[ ] -1 Do not release this package because ...
The release fixes 81 known issues in Spark 1.5.0, listed here:
http://s.apache.org/spark-1.5.1
The tag to be voted on is v1.5.1-rc1:
https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
The release files, including signatures, digests, etc. can be found at:
http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/pwendell.asc
The staging repository for this release (1.5.1) can be found at:
*https://repository.apache.org/content/repositories/orgapachespark-1148/
<https://repository.apache.org/content/repositories/orgapachespark-1148/>*
The documentation corresponding to this release can be found at:
http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
=======================================
How can I help test this release?
=======================================
If you are a Spark user, you can help us test this release by taking an
existing Spark workload and running on this release candidate, then
reporting any regressions.
================================================
What justifies a -1 vote for this release?
================================================
-1 vote should occur for regressions from Spark 1.5.0. Bugs already present
in 1.5.0 will not block this release.
===============================================================
What should happen to JIRA tickets still targeting 1.5.1?
===============================================================
Please target 1.5.2 or 1.6.0.
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Sean Owen <so...@cloudera.com>.
+1 non-binding. This is the first time I've seen all tests pass the
first time with Java 8 + Ubuntu + "-Pyarn -Phadoop-2.6 -Phive
-Phive-thriftserver". Clearly the test improvement efforts are paying
off.
As usual the license, sigs, etc are OK.
On Thu, Sep 24, 2015 at 8:27 AM, Reynold Xin <rx...@databricks.com> wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if a
> majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.5.1
> [ ] -1 Do not release this package because ...
>
>
> The release fixes 81 known issues in Spark 1.5.0, listed here:
> http://s.apache.org/spark-1.5.1
>
> The tag to be voted on is v1.5.1-rc1:
> https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release (1.5.1) can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1148/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>
>
> =======================================
> How can I help test this release?
> =======================================
> If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> ================================================
> What justifies a -1 vote for this release?
> ================================================
> -1 vote should occur for regressions from Spark 1.5.0. Bugs already present
> in 1.5.0 will not block this release.
>
> ===============================================================
> What should happen to JIRA tickets still targeting 1.5.1?
> ===============================================================
> Please target 1.5.2 or 1.6.0.
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Richard Hillegas <rh...@us.ibm.com>.
Hi Sean and Wendell,
I share your concerns about how difficult and important it is to get this
right. I think that the Spark community has compiled a very readable and
well organized NOTICE file. A lot of careful thought went into gathering
together 3rd party projects which share the same license text.
All I can offer is my own experience of having served as a release manager
for a sister Apache project (Derby) over the past ten years. The Derby
NOTICE file recites 3rd party licenses verbatim. This is also the approach
taken by the THIRDPARTYLICENSEREADME.txt in the JDK. I am not a lawyer.
However, I have great respect for the experience and legal sensitivities of
the people who compile that JDK license file.
Under your guidance, I would be happy to help compile a NOTICE file which
follows the pattern used by Derby and the JDK. This effort might proceed in
parallel with vetting 1.5.1 and could be targeted at a later release
vehicle. I don't think that the ASF's exposure is greatly increased by one
more release which follows the old pattern.
Another comment inline...
Patrick Wendell <pw...@gmail.com> wrote on 09/24/2015 10:24:25 AM:
> From: Patrick Wendell <pw...@gmail.com>
> To: Sean Owen <so...@cloudera.com>
> Cc: Richard Hillegas/San Francisco/IBM@IBMUS, "dev@spark.apache.org"
> <de...@spark.apache.org>
> Date: 09/24/2015 10:24 AM
> Subject: Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
>
> Hey Richard,
>
> My assessment (just looked before I saw Sean's email) is the same as
> his. The NOTICE file embeds other projects' licenses.
This may be where our perspectives diverge. I did not find those licenses
embedded in the NOTICE file. As I see it, the licenses are cited but not
included.
Thanks,
-Rick
> If those
> licenses themselves have pointers to other files or dependencies, we
> don't embed them. I think this is standard practice.
>
> - Patrick
>
> On Thu, Sep 24, 2015 at 10:00 AM, Sean Owen <so...@cloudera.com> wrote:
> > Hi Richard, those are messages reproduced from other projects' NOTICE
> > files, not created by Spark. They need to be reproduced in Spark's
> > NOTICE file to comply with the license, but their text may or may not
> > apply to Spark's distribution. The intent is that users would track
> > this back to the source project if interested to investigate what the
> > upstream notice is about.
> >
> > Requirements vary by license, but I do not believe there is additional
> > requirement to reproduce these other files. Their license information
> > is already indicated in accordance with the license terms.
> >
> > What licenses are you looking for in LICENSE that you believe
> should be there?
> >
> > Getting all this right is both difficult and important. I've made some
> > efforts over time to strictly comply with the Apache take on
> > licensing, which is at http://www.apache.org/legal/resolved.html It's
> > entirely possible there's still a mistake somewhere in here (possibly
> > a new dependency, etc). Please point it out if you see such a thing.
> >
> > But so far what you describe is "working as intended", as far as I
> > know, according to Apache.
> >
> >
> > On Thu, Sep 24, 2015 at 5:52 PM, Richard Hillegas
> <rh...@us.ibm.com> wrote:
> >> -1 (non-binding)
> >>
> >> I was able to build Spark cleanly from the source distribution using
the
> >> command in README.md:
> >>
> >> build/mvn -DskipTests clean package
> >>
> >> However, while I was waiting for the build to complete, I started
going
> >> through the NOTICE file. I was confused about where to find
> licenses for 3rd
> >> party software bundled with Spark. About halfway through the NOTICE
file,
> >> starting with Java Collections Framework, there is a list of
> licenses of the
> >> form
> >>
> >> license/*.txt
> >>
> >> But there is no license subdirectory in the source distro. I couldn't
find
> >> the *.txt license files for Java Collections Framework, Base64
Encoder, or
> >> JZlib anywhere in the source distro. I couldn't find those files in
license
> >> subdirectories at the indicated home pages for those projects. (I did
find
> >> the license for JZLIB somewhere else, however:
> >> http://www.jcraft.com/jzlib/LICENSE.txt.)
> >>
> >> In addition, I couldn't find licenses for those projects in the master
> >> LICENSE file.
> >>
> >> Are users supposed to get licenses from the indicated 3rd party web
sites?
> >> Those online licenses could change. I would feel more comfortableif
the ASF
> >> were protected by our bundling the licenses inside our source distros.
> >>
> >> After looking for those three licenses, I stopped reading the NOTICE
file.
> >> Maybe I'm confused about how to read the NOTICE file. Where should
users
> >> expect to find the 3rd party licenses?
> >>
> >> Thanks,
> >> -Rick
> >>
> >> Reynold Xin <rx...@databricks.com> wrote on 09/24/2015 12:27:25 AM:
> >>
> >>> From: Reynold Xin <rx...@databricks.com>
> >>> To: "dev@spark.apache.org" <de...@spark.apache.org>
> >>> Date: 09/24/2015 12:28 AM
> >>> Subject: [VOTE] Release Apache Spark 1.5.1 (RC1)
> >>
> >>
> >>>
> >>> Please vote on releasing the following candidate as Apache Spark
> >>> version 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC
> >>> and passes if a majority of at least 3 +1 PMC votes are cast.
> >>>
> >>> [ ] +1 Release this package as Apache Spark 1.5.1
> >>> [ ] -1 Do not release this package because ...
> >>>
> >>> The release fixes 81 known issues in Spark 1.5.0, listed here:
> >>> http://s.apache.org/spark-1.5.1
> >>>
> >>> The tag to be voted on is v1.5.1-rc1:
> >>> https://github.com/apache/spark/commit/
> >>> 4df97937dbf68a9868de58408b9be0bf87dbbb94
> >>>
> >>> The release files, including signatures, digests, etc. can be found
at:
> >>>
http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
> >>>
> >>> Release artifacts are signed with the following key:
> >>> https://people.apache.org/keys/committer/pwendell.asc
> >>>
> >>> The staging repository for this release (1.5.1) can be found at:
> >>>
https://repository.apache.org/content/repositories/orgapachespark-1148/
> >>>
> >>> The documentation corresponding to this release can be found at:
> >>>
http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
> >>>
> >>> =======================================
> >>> How can I help test this release?
> >>> =======================================
> >>> If you are a Spark user, you can help us test this release by taking
> >>> an existing Spark workload and running on this release candidate,
> >>> then reporting any regressions.
> >>>
> >>> ================================================
> >>> What justifies a -1 vote for this release?
> >>> ================================================
> >>> -1 vote should occur for regressions from Spark 1.5.0. Bugs already
> >>> present in 1.5.0 will not block this release.
> >>>
> >>> ===============================================================
> >>> What should happen to JIRA tickets still targeting 1.5.1?
> >>> ===============================================================
> >>> Please target 1.5.2 or 1.6.0.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> > For additional commands, e-mail: dev-help@spark.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Patrick Wendell <pw...@gmail.com>.
Hey Richard,
My assessment (just looked before I saw Sean's email) is the same as
his. The NOTICE file embeds other projects' licenses. If those
licenses themselves have pointers to other files or dependencies, we
don't embed them. I think this is standard practice.
- Patrick
On Thu, Sep 24, 2015 at 10:00 AM, Sean Owen <so...@cloudera.com> wrote:
> Hi Richard, those are messages reproduced from other projects' NOTICE
> files, not created by Spark. They need to be reproduced in Spark's
> NOTICE file to comply with the license, but their text may or may not
> apply to Spark's distribution. The intent is that users would track
> this back to the source project if interested to investigate what the
> upstream notice is about.
>
> Requirements vary by license, but I do not believe there is additional
> requirement to reproduce these other files. Their license information
> is already indicated in accordance with the license terms.
>
> What licenses are you looking for in LICENSE that you believe should be there?
>
> Getting all this right is both difficult and important. I've made some
> efforts over time to strictly comply with the Apache take on
> licensing, which is at http://www.apache.org/legal/resolved.html It's
> entirely possible there's still a mistake somewhere in here (possibly
> a new dependency, etc). Please point it out if you see such a thing.
>
> But so far what you describe is "working as intended", as far as I
> know, according to Apache.
>
>
> On Thu, Sep 24, 2015 at 5:52 PM, Richard Hillegas <rh...@us.ibm.com> wrote:
>> -1 (non-binding)
>>
>> I was able to build Spark cleanly from the source distribution using the
>> command in README.md:
>>
>> build/mvn -DskipTests clean package
>>
>> However, while I was waiting for the build to complete, I started going
>> through the NOTICE file. I was confused about where to find licenses for 3rd
>> party software bundled with Spark. About halfway through the NOTICE file,
>> starting with Java Collections Framework, there is a list of licenses of the
>> form
>>
>> license/*.txt
>>
>> But there is no license subdirectory in the source distro. I couldn't find
>> the *.txt license files for Java Collections Framework, Base64 Encoder, or
>> JZlib anywhere in the source distro. I couldn't find those files in license
>> subdirectories at the indicated home pages for those projects. (I did find
>> the license for JZLIB somewhere else, however:
>> http://www.jcraft.com/jzlib/LICENSE.txt.)
>>
>> In addition, I couldn't find licenses for those projects in the master
>> LICENSE file.
>>
>> Are users supposed to get licenses from the indicated 3rd party web sites?
>> Those online licenses could change. I would feel more comfortable if the ASF
>> were protected by our bundling the licenses inside our source distros.
>>
>> After looking for those three licenses, I stopped reading the NOTICE file.
>> Maybe I'm confused about how to read the NOTICE file. Where should users
>> expect to find the 3rd party licenses?
>>
>> Thanks,
>> -Rick
>>
>> Reynold Xin <rx...@databricks.com> wrote on 09/24/2015 12:27:25 AM:
>>
>>> From: Reynold Xin <rx...@databricks.com>
>>> To: "dev@spark.apache.org" <de...@spark.apache.org>
>>> Date: 09/24/2015 12:28 AM
>>> Subject: [VOTE] Release Apache Spark 1.5.1 (RC1)
>>
>>
>>>
>>> Please vote on releasing the following candidate as Apache Spark
>>> version 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC
>>> and passes if a majority of at least 3 +1 PMC votes are cast.
>>>
>>> [ ] +1 Release this package as Apache Spark 1.5.1
>>> [ ] -1 Do not release this package because ...
>>>
>>> The release fixes 81 known issues in Spark 1.5.0, listed here:
>>> http://s.apache.org/spark-1.5.1
>>>
>>> The tag to be voted on is v1.5.1-rc1:
>>> https://github.com/apache/spark/commit/
>>> 4df97937dbf68a9868de58408b9be0bf87dbbb94
>>>
>>> The release files, including signatures, digests, etc. can be found at:
>>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>>>
>>> Release artifacts are signed with the following key:
>>> https://people.apache.org/keys/committer/pwendell.asc
>>>
>>> The staging repository for this release (1.5.1) can be found at:
>>> https://repository.apache.org/content/repositories/orgapachespark-1148/
>>>
>>> The documentation corresponding to this release can be found at:
>>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>>>
>>> =======================================
>>> How can I help test this release?
>>> =======================================
>>> If you are a Spark user, you can help us test this release by taking
>>> an existing Spark workload and running on this release candidate,
>>> then reporting any regressions.
>>>
>>> ================================================
>>> What justifies a -1 vote for this release?
>>> ================================================
>>> -1 vote should occur for regressions from Spark 1.5.0. Bugs already
>>> present in 1.5.0 will not block this release.
>>>
>>> ===============================================================
>>> What should happen to JIRA tickets still targeting 1.5.1?
>>> ===============================================================
>>> Please target 1.5.2 or 1.6.0.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Sean Owen <so...@cloudera.com>.
Hi Richard, those are messages reproduced from other projects' NOTICE
files, not created by Spark. They need to be reproduced in Spark's
NOTICE file to comply with the license, but their text may or may not
apply to Spark's distribution. The intent is that users would track
this back to the source project if interested to investigate what the
upstream notice is about.
Requirements vary by license, but I do not believe there is additional
requirement to reproduce these other files. Their license information
is already indicated in accordance with the license terms.
What licenses are you looking for in LICENSE that you believe should be there?
Getting all this right is both difficult and important. I've made some
efforts over time to strictly comply with the Apache take on
licensing, which is at http://www.apache.org/legal/resolved.html It's
entirely possible there's still a mistake somewhere in here (possibly
a new dependency, etc). Please point it out if you see such a thing.
But so far what you describe is "working as intended", as far as I
know, according to Apache.
On Thu, Sep 24, 2015 at 5:52 PM, Richard Hillegas <rh...@us.ibm.com> wrote:
> -1 (non-binding)
>
> I was able to build Spark cleanly from the source distribution using the
> command in README.md:
>
> build/mvn -DskipTests clean package
>
> However, while I was waiting for the build to complete, I started going
> through the NOTICE file. I was confused about where to find licenses for 3rd
> party software bundled with Spark. About halfway through the NOTICE file,
> starting with Java Collections Framework, there is a list of licenses of the
> form
>
> license/*.txt
>
> But there is no license subdirectory in the source distro. I couldn't find
> the *.txt license files for Java Collections Framework, Base64 Encoder, or
> JZlib anywhere in the source distro. I couldn't find those files in license
> subdirectories at the indicated home pages for those projects. (I did find
> the license for JZLIB somewhere else, however:
> http://www.jcraft.com/jzlib/LICENSE.txt.)
>
> In addition, I couldn't find licenses for those projects in the master
> LICENSE file.
>
> Are users supposed to get licenses from the indicated 3rd party web sites?
> Those online licenses could change. I would feel more comfortable if the ASF
> were protected by our bundling the licenses inside our source distros.
>
> After looking for those three licenses, I stopped reading the NOTICE file.
> Maybe I'm confused about how to read the NOTICE file. Where should users
> expect to find the 3rd party licenses?
>
> Thanks,
> -Rick
>
> Reynold Xin <rx...@databricks.com> wrote on 09/24/2015 12:27:25 AM:
>
>> From: Reynold Xin <rx...@databricks.com>
>> To: "dev@spark.apache.org" <de...@spark.apache.org>
>> Date: 09/24/2015 12:28 AM
>> Subject: [VOTE] Release Apache Spark 1.5.1 (RC1)
>
>
>>
>> Please vote on releasing the following candidate as Apache Spark
>> version 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC
>> and passes if a majority of at least 3 +1 PMC votes are cast.
>>
>> [ ] +1 Release this package as Apache Spark 1.5.1
>> [ ] -1 Do not release this package because ...
>>
>> The release fixes 81 known issues in Spark 1.5.0, listed here:
>> http://s.apache.org/spark-1.5.1
>>
>> The tag to be voted on is v1.5.1-rc1:
>> https://github.com/apache/spark/commit/
>> 4df97937dbf68a9868de58408b9be0bf87dbbb94
>>
>> The release files, including signatures, digests, etc. can be found at:
>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>>
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/pwendell.asc
>>
>> The staging repository for this release (1.5.1) can be found at:
>> https://repository.apache.org/content/repositories/orgapachespark-1148/
>>
>> The documentation corresponding to this release can be found at:
>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>>
>> =======================================
>> How can I help test this release?
>> =======================================
>> If you are a Spark user, you can help us test this release by taking
>> an existing Spark workload and running on this release candidate,
>> then reporting any regressions.
>>
>> ================================================
>> What justifies a -1 vote for this release?
>> ================================================
>> -1 vote should occur for regressions from Spark 1.5.0. Bugs already
>> present in 1.5.0 will not block this release.
>>
>> ===============================================================
>> What should happen to JIRA tickets still targeting 1.5.1?
>> ===============================================================
>> Please target 1.5.2 or 1.6.0.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Richard Hillegas <rh...@us.ibm.com>.
-1 (non-binding)
I was able to build Spark cleanly from the source distribution using the
command in README.md:
build/mvn -DskipTests clean package
However, while I was waiting for the build to complete, I started going
through the NOTICE file. I was confused about where to find licenses for
3rd party software bundled with Spark. About halfway through the NOTICE
file, starting with Java Collections Framework, there is a list of licenses
of the form
license/*.txt
But there is no license subdirectory in the source distro. I couldn't find
the *.txt license files for Java Collections Framework, Base64 Encoder, or
JZlib anywhere in the source distro. I couldn't find those files in license
subdirectories at the indicated home pages for those projects. (I did find
the license for JZLIB somewhere else, however:
http://www.jcraft.com/jzlib/LICENSE.txt.)
In addition, I couldn't find licenses for those projects in the master
LICENSE file.
Are users supposed to get licenses from the indicated 3rd party web sites?
Those online licenses could change. I would feel more comfortable if the
ASF were protected by our bundling the licenses inside our source distros.
After looking for those three licenses, I stopped reading the NOTICE file.
Maybe I'm confused about how to read the NOTICE file. Where should users
expect to find the 3rd party licenses?
Thanks,
-Rick
Reynold Xin <rx...@databricks.com> wrote on 09/24/2015 12:27:25 AM:
> From: Reynold Xin <rx...@databricks.com>
> To: "dev@spark.apache.org" <de...@spark.apache.org>
> Date: 09/24/2015 12:28 AM
> Subject: [VOTE] Release Apache Spark 1.5.1 (RC1)
>
> Please vote on releasing the following candidate as Apache Spark
> version 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC
> and passes if a majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.5.1
> [ ] -1 Do not release this package because ...
>
> The release fixes 81 known issues in Spark 1.5.0, listed here:
> http://s.apache.org/spark-1.5.1
>
> The tag to be voted on is v1.5.1-rc1:
> https://github.com/apache/spark/commit/
> 4df97937dbf68a9868de58408b9be0bf87dbbb94
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release (1.5.1) can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1148/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>
> =======================================
> How can I help test this release?
> =======================================
> If you are a Spark user, you can help us test this release by taking
> an existing Spark workload and running on this release candidate,
> then reporting any regressions.
>
> ================================================
> What justifies a -1 vote for this release?
> ================================================
> -1 vote should occur for regressions from Spark 1.5.0. Bugs already
> present in 1.5.0 will not block this release.
>
> ===============================================================
> What should happen to JIRA tickets still targeting 1.5.1?
> ===============================================================
> Please target 1.5.2 or 1.6.0.
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Sean McNamara <Se...@Webtrends.com>.
Ran tests + built/ran an internal spark streaming app /w 1.5.1 artifacts.
+1
Cheers,
Sean
On Sep 24, 2015, at 1:28 AM, Reynold Xin <rx...@databricks.com>> wrote:
Please vote on releasing the following candidate as Apache Spark version 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast.
[ ] +1 Release this package as Apache Spark 1.5.1
[ ] -1 Do not release this package because ...
The release fixes 81 known issues in Spark 1.5.0, listed here:
http://s.apache.org/spark-1.5.1
The tag to be voted on is v1.5.1-rc1:
https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
The release files, including signatures, digests, etc. can be found at:
http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/pwendell.asc
The staging repository for this release (1.5.1) can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1148/
The documentation corresponding to this release can be found at:
http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
=======================================
How can I help test this release?
=======================================
If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.
================================================
What justifies a -1 vote for this release?
================================================
-1 vote should occur for regressions from Spark 1.5.0. Bugs already present in 1.5.0 will not block this release.
===============================================================
What should happen to JIRA tickets still targeting 1.5.1?
===============================================================
Please target 1.5.2 or 1.6.0.
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Sean Owen <so...@cloudera.com>.
It's on Maven Central already. These various updates have to happen in
some order, and you'll probably see an inconsistent state for a day or
so while things get slowly updated. Consider it released when there's
an announcement, I suppose.
On Mon, Sep 28, 2015 at 11:07 PM, Jerry Lam <ch...@gmail.com> wrote:
> Hi Spark Developers,
>
> The Spark 1.5.1 documentation is already publicly accessible
> (https://spark.apache.org/docs/latest/index.html) but the release is not. Is
> it intentional?
>
> Best Regards,
>
> Jerry
>
> On Mon, Sep 28, 2015 at 9:21 AM, james <yi...@gmail.com> wrote:
>>
>> +1
>>
>> 1) Build binary instruction: ./make-distribution.sh --tgz --skip-java-test
>> -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver
>> -DskipTests
>> 2) Run Spark SQL with YARN client mode
>>
>> This 1.5.1 RC1 package have better test results than previous 1.5.0 except
>> for Spark-10484,Spark-4266 open issue.
>>
>>
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-5-1-RC1-tp14310p14388.html
>> Sent from the Apache Spark Developers List mailing list archive at
>> Nabble.com.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> For additional commands, e-mail: dev-help@spark.apache.org
>>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Jerry Lam <ch...@gmail.com>.
Hi Spark Developers,
The Spark 1.5.1 documentation is already publicly accessible (
https://spark.apache.org/docs/latest/index.html) but the release is not. Is
it intentional?
Best Regards,
Jerry
On Mon, Sep 28, 2015 at 9:21 AM, james <yi...@gmail.com> wrote:
> +1
>
> 1) Build binary instruction: ./make-distribution.sh --tgz --skip-java-test
> -Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver
> -DskipTests
> 2) Run Spark SQL with YARN client mode
>
> This 1.5.1 RC1 package have better test results than previous 1.5.0 except
> for Spark-10484,Spark-4266 open issue.
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-5-1-RC1-tp14310p14388.html
> Sent from the Apache Spark Developers List mailing list archive at
> Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by james <yi...@gmail.com>.
+1
1) Build binary instruction: ./make-distribution.sh --tgz --skip-java-test
-Pyarn -Phadoop-2.6 -Dhadoop.version=2.6.0 -Phive -Phive-thriftserver
-DskipTests
2) Run Spark SQL with YARN client mode
This 1.5.1 RC1 package have better test results than previous 1.5.0 except
for Spark-10484,Spark-4266 open issue.
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-5-1-RC1-tp14310p14388.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Marcelo Vanzin <va...@cloudera.com>.
Ignoring my previous question, +1. Tested several different jobs on
YARN and standalone with dynamic allocation on.
On Fri, Sep 25, 2015 at 11:32 AM, Marcelo Vanzin <va...@cloudera.com> wrote:
> Mostly for my education (I hope), but I was testing
> "spark-1.5.1-bin-without-hadoop.tgz" assuming it would contain
> everything (including HiveContext support), just without the Hadoop
> common jars in the assembly. But HiveContext is not there.
>
> Is this expected?
>
> On Thu, Sep 24, 2015 at 12:27 AM, Reynold Xin <rx...@databricks.com> wrote:
>> Please vote on releasing the following candidate as Apache Spark version
>> 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if a
>> majority of at least 3 +1 PMC votes are cast.
>>
>> [ ] +1 Release this package as Apache Spark 1.5.1
>> [ ] -1 Do not release this package because ...
>>
>>
>> The release fixes 81 known issues in Spark 1.5.0, listed here:
>> http://s.apache.org/spark-1.5.1
>>
>> The tag to be voted on is v1.5.1-rc1:
>> https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
>>
>> The release files, including signatures, digests, etc. can be found at:
>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>>
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/pwendell.asc
>>
>> The staging repository for this release (1.5.1) can be found at:
>> https://repository.apache.org/content/repositories/orgapachespark-1148/
>>
>> The documentation corresponding to this release can be found at:
>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>>
>>
>> =======================================
>> How can I help test this release?
>> =======================================
>> If you are a Spark user, you can help us test this release by taking an
>> existing Spark workload and running on this release candidate, then
>> reporting any regressions.
>>
>> ================================================
>> What justifies a -1 vote for this release?
>> ================================================
>> -1 vote should occur for regressions from Spark 1.5.0. Bugs already present
>> in 1.5.0 will not block this release.
>>
>> ===============================================================
>> What should happen to JIRA tickets still targeting 1.5.1?
>> ===============================================================
>> Please target 1.5.2 or 1.6.0.
>>
>>
>>
>
>
>
> --
> Marcelo
--
Marcelo
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Marcelo Vanzin <va...@cloudera.com>.
Mostly for my education (I hope), but I was testing
"spark-1.5.1-bin-without-hadoop.tgz" assuming it would contain
everything (including HiveContext support), just without the Hadoop
common jars in the assembly. But HiveContext is not there.
Is this expected?
On Thu, Sep 24, 2015 at 12:27 AM, Reynold Xin <rx...@databricks.com> wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if a
> majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.5.1
> [ ] -1 Do not release this package because ...
>
>
> The release fixes 81 known issues in Spark 1.5.0, listed here:
> http://s.apache.org/spark-1.5.1
>
> The tag to be voted on is v1.5.1-rc1:
> https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release (1.5.1) can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1148/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>
>
> =======================================
> How can I help test this release?
> =======================================
> If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> ================================================
> What justifies a -1 vote for this release?
> ================================================
> -1 vote should occur for regressions from Spark 1.5.0. Bugs already present
> in 1.5.0 will not block this release.
>
> ===============================================================
> What should happen to JIRA tickets still targeting 1.5.1?
> ===============================================================
> Please target 1.5.2 or 1.6.0.
>
>
>
--
Marcelo
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Michael Armbrust <mi...@databricks.com>.
+1 - Ran TPCDS and some other micro benchmarks
On Fri, Sep 25, 2015 at 11:09 AM, Tom Graves <tg...@yahoo.com.invalid>
wrote:
> +1. Tested Spark on Yarn on Hadoop 2.6 and 2.7.
>
> Tom
>
>
>
> On Thursday, September 24, 2015 2:34 AM, Reynold Xin <rx...@databricks.com>
> wrote:
>
>
> Please vote on releasing the following candidate as Apache Spark version
> 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if
> a majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.5.1
> [ ] -1 Do not release this package because ...
>
>
> The release fixes 81 known issues in Spark 1.5.0, listed here:
> http://s.apache.org/spark-1.5.1
>
> The tag to be voted on is v1.5.1-rc1:
>
> https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release (1.5.1) can be found at:
> *https://repository.apache.org/content/repositories/orgapachespark-1148/
> <https://repository.apache.org/content/repositories/orgapachespark-1148/>*
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>
>
> =======================================
> How can I help test this release?
> =======================================
> If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> ================================================
> What justifies a -1 vote for this release?
> ================================================
> -1 vote should occur for regressions from Spark 1.5.0. Bugs already
> present in 1.5.0 will not block this release.
>
> ===============================================================
> What should happen to JIRA tickets still targeting 1.5.1?
> ===============================================================
> Please target 1.5.2 or 1.6.0.
>
>
>
>
>
>
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Tom Graves <tg...@yahoo.com.INVALID>.
+1. Tested Spark on Yarn on Hadoop 2.6 and 2.7.
Tom
On Thursday, September 24, 2015 2:34 AM, Reynold Xin <rx...@databricks.com> wrote:
Please vote on releasing the following candidate as Apache Spark version 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast.
[ ] +1 Release this package as Apache Spark 1.5.1[ ] -1 Do not release this package because ...
The release fixes 81 known issues in Spark 1.5.0, listed here:http://s.apache.org/spark-1.5.1
The tag to be voted on is v1.5.1-rc1:https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
The release files, including signatures, digests, etc. can be found at:http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
Release artifacts are signed with the following key:https://people.apache.org/keys/committer/pwendell.asc
The staging repository for this release (1.5.1) can be found at:https://repository.apache.org/content/repositories/orgapachespark-1148/
The documentation corresponding to this release can be found at:http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
=======================================How can I help test this release?=======================================If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.
================================================What justifies a -1 vote for this release?================================================-1 vote should occur for regressions from Spark 1.5.0. Bugs already present in 1.5.0 will not block this release.
===============================================================What should happen to JIRA tickets still targeting 1.5.1?===============================================================Please target 1.5.2 or 1.6.0.
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Joseph Bradley <jo...@databricks.com>.
+1 Tested MLlib on Mac OS X
On Thu, Sep 24, 2015 at 6:14 PM, Reynold Xin <rx...@databricks.com> wrote:
> Krishna,
>
> Thanks for testing every release!
>
>
> On Thu, Sep 24, 2015 at 6:08 PM, Krishna Sankar <ks...@gmail.com>
> wrote:
>
>> +1 (non-binding, of course)
>>
>> 1. Compiled OSX 10.10 (Yosemite) OK Total time: 26:48 min
>> mvn clean package -Pyarn -Phadoop-2.6 -DskipTests
>> 2. Tested pyspark, mllib (iPython 4.0, FYI, notebook install is separate
>> “conda install python” and then “conda install jupyter”)
>> 2.1. statistics (min,max,mean,Pearson,Spearman) OK
>> 2.2. Linear/Ridge/Laso Regression OK
>> 2.3. Decision Tree, Naive Bayes OK
>> 2.4. KMeans OK
>> Center And Scale OK
>> 2.5. RDD operations OK
>> State of the Union Texts - MapReduce, Filter,sortByKey (word count)
>> 2.6. Recommendation (Movielens medium dataset ~1 M ratings) OK
>> Model evaluation/optimization (rank, numIter, lambda) with
>> itertools OK
>> 3. Scala - MLlib
>> 3.1. statistics (min,max,mean,Pearson,Spearman) OK
>> 3.2. LinearRegressionWithSGD OK
>> 3.3. Decision Tree OK
>> 3.4. KMeans OK
>> 3.5. Recommendation (Movielens medium dataset ~1 M ratings) OK
>> 3.6. saveAsParquetFile OK
>> 3.7. Read and verify the 4.3 save(above) - sqlContext.parquetFile,
>> registerTempTable, sql OK
>> 3.8. result = sqlContext.sql("SELECT
>> OrderDetails.OrderID,ShipCountry,UnitPrice,Qty,Discount FROM Orders INNER
>> JOIN OrderDetails ON Orders.OrderID = OrderDetails.OrderID") OK
>> 4.0. Spark SQL from Python OK
>> 4.1. result = sqlContext.sql("SELECT * from people WHERE State = 'WA'") OK
>> 5.0. Packages
>> 5.1. com.databricks.spark.csv - read/write OK (--packages
>> com.databricks:spark-csv_2.10:1.2.0)
>> 6.0. DataFrames
>> 6.1. cast,dtypes OK
>> 6.2. groupBy,avg,crosstab,corr,isNull,na.drop OK
>> 6.3. All joins,sql,set operations,udf OK
>> *Notes:*
>> 1. Speed improvement in DataFrame functions groupBy, avg,sum et al. *Good
>> work*. I am working on a project to reduce processing time from ~24 hrs
>> to ... Let us see what Spark does. The speedups would help a lot.
>> 2. FYI, UDFs getM and getY work now (Thanks). Slower; saturates the CPU.
>> A non-scientific snapshot below. I know that this really has to be done
>> more rigorously, on a bigger machine, with more cores et al..
>> [image: Inline image 1] [image: Inline image 2]
>>
>> On Thu, Sep 24, 2015 at 12:27 AM, Reynold Xin <rx...@databricks.com>
>> wrote:
>>
>>> Please vote on releasing the following candidate as Apache Spark version
>>> 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if
>>> a majority of at least 3 +1 PMC votes are cast.
>>>
>>> [ ] +1 Release this package as Apache Spark 1.5.1
>>> [ ] -1 Do not release this package because ...
>>>
>>>
>>> The release fixes 81 known issues in Spark 1.5.0, listed here:
>>> http://s.apache.org/spark-1.5.1
>>>
>>> The tag to be voted on is v1.5.1-rc1:
>>>
>>> https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
>>>
>>> The release files, including signatures, digests, etc. can be found at:
>>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>>>
>>> Release artifacts are signed with the following key:
>>> https://people.apache.org/keys/committer/pwendell.asc
>>>
>>> The staging repository for this release (1.5.1) can be found at:
>>> *https://repository.apache.org/content/repositories/orgapachespark-1148/
>>> <https://repository.apache.org/content/repositories/orgapachespark-1148/>*
>>>
>>> The documentation corresponding to this release can be found at:
>>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>>>
>>>
>>> =======================================
>>> How can I help test this release?
>>> =======================================
>>> If you are a Spark user, you can help us test this release by taking an
>>> existing Spark workload and running on this release candidate, then
>>> reporting any regressions.
>>>
>>> ================================================
>>> What justifies a -1 vote for this release?
>>> ================================================
>>> -1 vote should occur for regressions from Spark 1.5.0. Bugs already
>>> present in 1.5.0 will not block this release.
>>>
>>> ===============================================================
>>> What should happen to JIRA tickets still targeting 1.5.1?
>>> ===============================================================
>>> Please target 1.5.2 or 1.6.0.
>>>
>>>
>>>
>>>
>>
>
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Reynold Xin <rx...@databricks.com>.
Krishna,
Thanks for testing every release!
On Thu, Sep 24, 2015 at 6:08 PM, Krishna Sankar <ks...@gmail.com> wrote:
> +1 (non-binding, of course)
>
> 1. Compiled OSX 10.10 (Yosemite) OK Total time: 26:48 min
> mvn clean package -Pyarn -Phadoop-2.6 -DskipTests
> 2. Tested pyspark, mllib (iPython 4.0, FYI, notebook install is separate
> “conda install python” and then “conda install jupyter”)
> 2.1. statistics (min,max,mean,Pearson,Spearman) OK
> 2.2. Linear/Ridge/Laso Regression OK
> 2.3. Decision Tree, Naive Bayes OK
> 2.4. KMeans OK
> Center And Scale OK
> 2.5. RDD operations OK
> State of the Union Texts - MapReduce, Filter,sortByKey (word count)
> 2.6. Recommendation (Movielens medium dataset ~1 M ratings) OK
> Model evaluation/optimization (rank, numIter, lambda) with
> itertools OK
> 3. Scala - MLlib
> 3.1. statistics (min,max,mean,Pearson,Spearman) OK
> 3.2. LinearRegressionWithSGD OK
> 3.3. Decision Tree OK
> 3.4. KMeans OK
> 3.5. Recommendation (Movielens medium dataset ~1 M ratings) OK
> 3.6. saveAsParquetFile OK
> 3.7. Read and verify the 4.3 save(above) - sqlContext.parquetFile,
> registerTempTable, sql OK
> 3.8. result = sqlContext.sql("SELECT
> OrderDetails.OrderID,ShipCountry,UnitPrice,Qty,Discount FROM Orders INNER
> JOIN OrderDetails ON Orders.OrderID = OrderDetails.OrderID") OK
> 4.0. Spark SQL from Python OK
> 4.1. result = sqlContext.sql("SELECT * from people WHERE State = 'WA'") OK
> 5.0. Packages
> 5.1. com.databricks.spark.csv - read/write OK (--packages
> com.databricks:spark-csv_2.10:1.2.0)
> 6.0. DataFrames
> 6.1. cast,dtypes OK
> 6.2. groupBy,avg,crosstab,corr,isNull,na.drop OK
> 6.3. All joins,sql,set operations,udf OK
> *Notes:*
> 1. Speed improvement in DataFrame functions groupBy, avg,sum et al. *Good
> work*. I am working on a project to reduce processing time from ~24 hrs
> to ... Let us see what Spark does. The speedups would help a lot.
> 2. FYI, UDFs getM and getY work now (Thanks). Slower; saturates the CPU. A
> non-scientific snapshot below. I know that this really has to be done more
> rigorously, on a bigger machine, with more cores et al..
> [image: Inline image 1] [image: Inline image 2]
>
> On Thu, Sep 24, 2015 at 12:27 AM, Reynold Xin <rx...@databricks.com> wrote:
>
>> Please vote on releasing the following candidate as Apache Spark version
>> 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if
>> a majority of at least 3 +1 PMC votes are cast.
>>
>> [ ] +1 Release this package as Apache Spark 1.5.1
>> [ ] -1 Do not release this package because ...
>>
>>
>> The release fixes 81 known issues in Spark 1.5.0, listed here:
>> http://s.apache.org/spark-1.5.1
>>
>> The tag to be voted on is v1.5.1-rc1:
>>
>> https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
>>
>> The release files, including signatures, digests, etc. can be found at:
>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>>
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/pwendell.asc
>>
>> The staging repository for this release (1.5.1) can be found at:
>> *https://repository.apache.org/content/repositories/orgapachespark-1148/
>> <https://repository.apache.org/content/repositories/orgapachespark-1148/>*
>>
>> The documentation corresponding to this release can be found at:
>> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>>
>>
>> =======================================
>> How can I help test this release?
>> =======================================
>> If you are a Spark user, you can help us test this release by taking an
>> existing Spark workload and running on this release candidate, then
>> reporting any regressions.
>>
>> ================================================
>> What justifies a -1 vote for this release?
>> ================================================
>> -1 vote should occur for regressions from Spark 1.5.0. Bugs already
>> present in 1.5.0 will not block this release.
>>
>> ===============================================================
>> What should happen to JIRA tickets still targeting 1.5.1?
>> ===============================================================
>> Please target 1.5.2 or 1.6.0.
>>
>>
>>
>>
>
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Krishna Sankar <ks...@gmail.com>.
+1 (non-binding, of course)
1. Compiled OSX 10.10 (Yosemite) OK Total time: 26:48 min
mvn clean package -Pyarn -Phadoop-2.6 -DskipTests
2. Tested pyspark, mllib (iPython 4.0, FYI, notebook install is separate
“conda install python” and then “conda install jupyter”)
2.1. statistics (min,max,mean,Pearson,Spearman) OK
2.2. Linear/Ridge/Laso Regression OK
2.3. Decision Tree, Naive Bayes OK
2.4. KMeans OK
Center And Scale OK
2.5. RDD operations OK
State of the Union Texts - MapReduce, Filter,sortByKey (word count)
2.6. Recommendation (Movielens medium dataset ~1 M ratings) OK
Model evaluation/optimization (rank, numIter, lambda) with itertools
OK
3. Scala - MLlib
3.1. statistics (min,max,mean,Pearson,Spearman) OK
3.2. LinearRegressionWithSGD OK
3.3. Decision Tree OK
3.4. KMeans OK
3.5. Recommendation (Movielens medium dataset ~1 M ratings) OK
3.6. saveAsParquetFile OK
3.7. Read and verify the 4.3 save(above) - sqlContext.parquetFile,
registerTempTable, sql OK
3.8. result = sqlContext.sql("SELECT
OrderDetails.OrderID,ShipCountry,UnitPrice,Qty,Discount FROM Orders INNER
JOIN OrderDetails ON Orders.OrderID = OrderDetails.OrderID") OK
4.0. Spark SQL from Python OK
4.1. result = sqlContext.sql("SELECT * from people WHERE State = 'WA'") OK
5.0. Packages
5.1. com.databricks.spark.csv - read/write OK (--packages
com.databricks:spark-csv_2.10:1.2.0)
6.0. DataFrames
6.1. cast,dtypes OK
6.2. groupBy,avg,crosstab,corr,isNull,na.drop OK
6.3. All joins,sql,set operations,udf OK
*Notes:*
1. Speed improvement in DataFrame functions groupBy, avg,sum et al. *Good
work*. I am working on a project to reduce processing time from ~24 hrs to
... Let us see what Spark does. The speedups would help a lot.
2. FYI, UDFs getM and getY work now (Thanks). Slower; saturates the CPU. A
non-scientific snapshot below. I know that this really has to be done more
rigorously, on a bigger machine, with more cores et al..
[image: Inline image 1] [image: Inline image 2]
On Thu, Sep 24, 2015 at 12:27 AM, Reynold Xin <rx...@databricks.com> wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if
> a majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.5.1
> [ ] -1 Do not release this package because ...
>
>
> The release fixes 81 known issues in Spark 1.5.0, listed here:
> http://s.apache.org/spark-1.5.1
>
> The tag to be voted on is v1.5.1-rc1:
>
> https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release (1.5.1) can be found at:
> *https://repository.apache.org/content/repositories/orgapachespark-1148/
> <https://repository.apache.org/content/repositories/orgapachespark-1148/>*
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>
>
> =======================================
> How can I help test this release?
> =======================================
> If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> ================================================
> What justifies a -1 vote for this release?
> ================================================
> -1 vote should occur for regressions from Spark 1.5.0. Bugs already
> present in 1.5.0 will not block this release.
>
> ===============================================================
> What should happen to JIRA tickets still targeting 1.5.1?
> ===============================================================
> Please target 1.5.2 or 1.6.0.
>
>
>
>
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Suresh Thalamati <su...@gmail.com>.
+1 (non-binding.)
Tested jdbc data source, and some of the tpc-ds queries.
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Yin Huai <yh...@databricks.com>.
+1
Tested 1.5.1 SQL blockers.
On Sat, Sep 26, 2015 at 1:36 PM, robineast <ro...@xense.co.uk> wrote:
> +1
>
>
> build/mvn clean package -DskipTests -Pyarn -Phadoop-2.6
> OK
> Basic graph tests
> Load graph using edgeListFile...SUCCESS
> Run PageRank...SUCCESS
> Minimum Spanning Tree Algorithm
> Run basic Minimum Spanning Tree algorithm...SUCCESS
> Run Minimum Spanning Tree taxonomy creation...SUCCESS
>
>
>
> --
> View this message in context:
> http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-5-1-RC1-tp14310p14380.html
> Sent from the Apache Spark Developers List mailing list archive at
> Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by robineast <ro...@xense.co.uk>.
+1
build/mvn clean package -DskipTests -Pyarn -Phadoop-2.6
OK
Basic graph tests
Load graph using edgeListFile...SUCCESS
Run PageRank...SUCCESS
Minimum Spanning Tree Algorithm
Run basic Minimum Spanning Tree algorithm...SUCCESS
Run Minimum Spanning Tree taxonomy creation...SUCCESS
--
View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/VOTE-Release-Apache-Spark-1-5-1-RC1-tp14310p14380.html
Sent from the Apache Spark Developers List mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Reynold Xin <rx...@databricks.com>.
Thanks everybody for voting. I'm going to close the vote now. The vote
passes with 17 +1 votes and 1 -1 vote. I will work on packaging this asap.
+1:
Reynold Xin*
Sean Owen
Hossein Falaki
Xiangrui Meng*
Krishna Sankar
Joseph Bradley
Sean McNamara*
Luciano Resende
Doug Balog
Eugene Zhulenev
Vaquar Khan
Tom Graves*
Michael Armbrust*
Marcelo Vanzin
Robin East
Yin Huai
Suresh Thalamati
0:
-1:
Richard Hillegas [see note]
Note: Richard Hillegas did say in a separate thread the issue he brought up
should not block the release. However, he did not explicitly amend his vote
so I'm including it as a -1 here.
On Thu, Sep 24, 2015 at 12:27 AM, Reynold Xin <rx...@databricks.com> wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if
> a majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.5.1
> [ ] -1 Do not release this package because ...
>
>
> The release fixes 81 known issues in Spark 1.5.0, listed here:
> http://s.apache.org/spark-1.5.1
>
> The tag to be voted on is v1.5.1-rc1:
>
> https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release (1.5.1) can be found at:
> *https://repository.apache.org/content/repositories/orgapachespark-1148/
> <https://repository.apache.org/content/repositories/orgapachespark-1148/>*
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>
>
> =======================================
> How can I help test this release?
> =======================================
> If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> ================================================
> What justifies a -1 vote for this release?
> ================================================
> -1 vote should occur for regressions from Spark 1.5.0. Bugs already
> present in 1.5.0 will not block this release.
>
> ===============================================================
> What should happen to JIRA tickets still targeting 1.5.1?
> ===============================================================
> Please target 1.5.2 or 1.6.0.
>
>
>
>
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Luciano Resende <lu...@gmail.com>.
+1 (non-binding)
Compiled in Mac OS with :
build/mvn -Pyarn,sparkr,hive,hive-thriftserver
-Phadoop-2.6 -Dhadoop.version=2.6.0 -DskipTests clean package
Checked around R
Looked into legal files
All looks good.
On Thu, Sep 24, 2015 at 12:27 AM, Reynold Xin <rx...@databricks.com> wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if
> a majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.5.1
> [ ] -1 Do not release this package because ...
>
>
> The release fixes 81 known issues in Spark 1.5.0, listed here:
> http://s.apache.org/spark-1.5.1
>
> The tag to be voted on is v1.5.1-rc1:
>
> https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release (1.5.1) can be found at:
> *https://repository.apache.org/content/repositories/orgapachespark-1148/
> <https://repository.apache.org/content/repositories/orgapachespark-1148/>*
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>
>
> =======================================
> How can I help test this release?
> =======================================
> If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions.
>
> ================================================
> What justifies a -1 vote for this release?
> ================================================
> -1 vote should occur for regressions from Spark 1.5.0. Bugs already
> present in 1.5.0 will not block this release.
>
> ===============================================================
> What should happen to JIRA tickets still targeting 1.5.1?
> ===============================================================
> Please target 1.5.2 or 1.6.0.
>
>
>
>
--
Luciano Resende
http://people.apache.org/~lresende
http://twitter.com/lresende1975
http://lresende.blogspot.com/
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by vaquar khan <va...@gmail.com>.
+1 (non-binding)
Regards,
Vaquar khan
On 25 Sep 2015 18:28, "Eugene Zhulenev" <eu...@gmail.com> wrote:
> +1
>
> Running latest build from 1.5 branch, SO much more stable than 1.5.0
> release.
>
> On Fri, Sep 25, 2015 at 8:55 AM, Doug Balog <do...@dugos.com>
> wrote:
>
>> +1 (non-binding)
>>
>> Tested on secure YARN cluster with HIVE.
>>
>> Notes: SPARK-10422, SPARK-10737 were causing us problems with 1.5.0. We
>> see 1.5.1 as a big improvement.
>>
>> Cheers,
>>
>> Doug
>>
>>
>> > On Sep 24, 2015, at 3:27 AM, Reynold Xin <rx...@databricks.com> wrote:
>> >
>> > Please vote on releasing the following candidate as Apache Spark
>> version 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and
>> passes if a majority of at least 3 +1 PMC votes are cast.
>> >
>> > [ ] +1 Release this package as Apache Spark 1.5.1
>> > [ ] -1 Do not release this package because ...
>> >
>> >
>> > The release fixes 81 known issues in Spark 1.5.0, listed here:
>> > http://s.apache.org/spark-1.5.1
>> >
>> > The tag to be voted on is v1.5.1-rc1:
>> >
>> https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
>> >
>> > The release files, including signatures, digests, etc. can be found at:
>> > http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>> >
>> > Release artifacts are signed with the following key:
>> > https://people.apache.org/keys/committer/pwendell.asc
>> >
>> > The staging repository for this release (1.5.1) can be found at:
>> > https://repository.apache.org/content/repositories/orgapachespark-1148/
>> >
>> > The documentation corresponding to this release can be found at:
>> > http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>> >
>> >
>> > =======================================
>> > How can I help test this release?
>> > =======================================
>> > If you are a Spark user, you can help us test this release by taking an
>> existing Spark workload and running on this release candidate, then
>> reporting any regressions.
>> >
>> > ================================================
>> > What justifies a -1 vote for this release?
>> > ================================================
>> > -1 vote should occur for regressions from Spark 1.5.0. Bugs already
>> present in 1.5.0 will not block this release.
>> >
>> > ===============================================================
>> > What should happen to JIRA tickets still targeting 1.5.1?
>> > ===============================================================
>> > Please target 1.5.2 or 1.6.0.
>> >
>> >
>> >
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> For additional commands, e-mail: dev-help@spark.apache.org
>>
>>
>
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Eugene Zhulenev <eu...@gmail.com>.
+1
Running latest build from 1.5 branch, SO much more stable than 1.5.0
release.
On Fri, Sep 25, 2015 at 8:55 AM, Doug Balog <do...@dugos.com> wrote:
> +1 (non-binding)
>
> Tested on secure YARN cluster with HIVE.
>
> Notes: SPARK-10422, SPARK-10737 were causing us problems with 1.5.0. We
> see 1.5.1 as a big improvement.
>
> Cheers,
>
> Doug
>
>
> > On Sep 24, 2015, at 3:27 AM, Reynold Xin <rx...@databricks.com> wrote:
> >
> > Please vote on releasing the following candidate as Apache Spark version
> 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if
> a majority of at least 3 +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Spark 1.5.1
> > [ ] -1 Do not release this package because ...
> >
> >
> > The release fixes 81 known issues in Spark 1.5.0, listed here:
> > http://s.apache.org/spark-1.5.1
> >
> > The tag to be voted on is v1.5.1-rc1:
> >
> https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
> >
> > The release files, including signatures, digests, etc. can be found at:
> > http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
> >
> > Release artifacts are signed with the following key:
> > https://people.apache.org/keys/committer/pwendell.asc
> >
> > The staging repository for this release (1.5.1) can be found at:
> > https://repository.apache.org/content/repositories/orgapachespark-1148/
> >
> > The documentation corresponding to this release can be found at:
> > http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
> >
> >
> > =======================================
> > How can I help test this release?
> > =======================================
> > If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions.
> >
> > ================================================
> > What justifies a -1 vote for this release?
> > ================================================
> > -1 vote should occur for regressions from Spark 1.5.0. Bugs already
> present in 1.5.0 will not block this release.
> >
> > ===============================================================
> > What should happen to JIRA tickets still targeting 1.5.1?
> > ===============================================================
> > Please target 1.5.2 or 1.6.0.
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>
Re: [VOTE] Release Apache Spark 1.5.1 (RC1)
Posted by Doug Balog <do...@dugos.com>.
+1 (non-binding)
Tested on secure YARN cluster with HIVE.
Notes: SPARK-10422, SPARK-10737 were causing us problems with 1.5.0. We see 1.5.1 as a big improvement.
Cheers,
Doug
> On Sep 24, 2015, at 3:27 AM, Reynold Xin <rx...@databricks.com> wrote:
>
> Please vote on releasing the following candidate as Apache Spark version 1.5.1. The vote is open until Sun, Sep 27, 2015 at 10:00 UTC and passes if a majority of at least 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.5.1
> [ ] -1 Do not release this package because ...
>
>
> The release fixes 81 known issues in Spark 1.5.0, listed here:
> http://s.apache.org/spark-1.5.1
>
> The tag to be voted on is v1.5.1-rc1:
> https://github.com/apache/spark/commit/4df97937dbf68a9868de58408b9be0bf87dbbb94
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-bin/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release (1.5.1) can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1148/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.5.1-rc1-docs/
>
>
> =======================================
> How can I help test this release?
> =======================================
> If you are a Spark user, you can help us test this release by taking an existing Spark workload and running on this release candidate, then reporting any regressions.
>
> ================================================
> What justifies a -1 vote for this release?
> ================================================
> -1 vote should occur for regressions from Spark 1.5.0. Bugs already present in 1.5.0 will not block this release.
>
> ===============================================================
> What should happen to JIRA tickets still targeting 1.5.1?
> ===============================================================
> Please target 1.5.2 or 1.6.0.
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org