You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Reynold Xin <rx...@databricks.com> on 2016/06/20 04:24:53 UTC
[VOTE] Release Apache Spark 1.6.2 (RC2)
Please vote on releasing the following candidate as Apache Spark version
1.6.2. The vote is open until Wednesday, June 22, 2016 at 22:00 PDT and
passes if a majority of at least 3+1 PMC votes are cast.
[ ] +1 Release this package as Apache Spark 1.6.2
[ ] -1 Do not release this package because ...
The tag to be voted on is v1.6.2-rc2
(54b1121f351f056d6b67d2bb4efe0d553c0f7482)
The release files, including signatures, digests, etc. can be found at:
http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-bin/
Release artifacts are signed with the following key:
https://people.apache.org/keys/committer/pwendell.asc
The staging repository for this release can be found at:
https://repository.apache.org/content/repositories/orgapachespark-1186/
The documentation corresponding to this release can be found at:
http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-docs/
=======================================
== How can I help test this release? ==
=======================================
If you are a Spark user, you can help us test this release by taking an
existing Spark workload and running on this release candidate, then
reporting any regressions from 1.6.1.
================================================
== What justifies a -1 vote for this release? ==
================================================
This is a maintenance release in the 1.6.x series. Bugs already present in
1.6.1, missing features, or bugs related to new features will not
necessarily block this release.
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Sameer Agarwal <sa...@databricks.com>.
+1
On Wed, Jun 22, 2016 at 1:07 PM, Kousuke Saruta <sa...@oss.nttdata.co.jp>
wrote:
> +1 (non-binding)
>
> On 2016/06/23 4:53, Reynold Xin wrote:
>
> +1 myself
>
>
> On Wed, Jun 22, 2016 at 12:19 PM, Sean McNamara <
> Sean.McNamara@webtrends.com> wrote:
>
>> +1
>>
>> On Jun 22, 2016, at 1:14 PM, Michael Armbrust <mi...@databricks.com>
>> wrote:
>>
>> +1
>>
>> On Wed, Jun 22, 2016 at 11:33 AM, Jonathan Kelly <jo...@gmail.com>
>> wrote:
>>
>>> +1
>>>
>>> On Wed, Jun 22, 2016 at 10:41 AM Tim Hunter < <ti...@databricks.com>
>>> timhunter@databricks.com> wrote:
>>>
>>>> +1 This release passes all tests on the graphframes and tensorframes
>>>> packages.
>>>>
>>>> On Wed, Jun 22, 2016 at 7:19 AM, Cody Koeninger < <co...@koeninger.org>
>>>> cody@koeninger.org> wrote:
>>>>
>>>>> If we're considering backporting changes for the 0.8 kafka
>>>>> integration, I am sure there are people who would like to get
>>>>>
>>>>> https://issues.apache.org/jira/browse/SPARK-10963
>>>>>
>>>>> into 1.6.x as well
>>>>>
>>>>> On Wed, Jun 22, 2016 at 7:41 AM, Sean Owen < <so...@cloudera.com>
>>>>> sowen@cloudera.com> wrote:
>>>>> > Good call, probably worth back-porting, I'll try to do that. I don't
>>>>> > think it blocks a release, but would be good to get into a next RC if
>>>>> > any.
>>>>> >
>>>>> > On Wed, Jun 22, 2016 at 11:38 AM, Pete Robbins <
>>>>> <ro...@gmail.com> wrote:
>>>>> >> This has failed on our 1.6 stream builds regularly.
>>>>> >> ( <https://issues.apache.org/jira/browse/SPARK-6005>
>>>>> https://issues.apache.org/jira/browse/SPARK-6005) looks fixed in 2.0?
>>>>> >>
>>>>> >> On Wed, 22 Jun 2016 at 11:15 Sean Owen < <so...@cloudera.com>
>>>>> sowen@cloudera.com> wrote:
>>>>> >>>
>>>>> >>> Oops, one more in the "does anybody else see this" department:
>>>>> >>>
>>>>> >>> - offset recovery *** FAILED ***
>>>>> >>> recoveredOffsetRanges.forall(((or:
>>>>> (org.apache.spark.streaming.Time,
>>>>> >>> Array[org.apache.spark.streaming.kafka.OffsetRange])) =>
>>>>> >>>
>>>>> >>>
>>>>> earlierOffsetRangesAsSets.contains(scala.Tuple2.apply[org.apache.spark.streaming.Time,
>>>>> >>>
>>>>> >>>
>>>>> scala.collection.immutable.Set[org.apache.spark.streaming.kafka.OffsetRange]](or._1,
>>>>> >>>
>>>>> >>>
>>>>> scala.this.Predef.refArrayOps[org.apache.spark.streaming.kafka.OffsetRange](or._2).toSet[org.apache.spark.streaming.kafka.OffsetRange]))))
>>>>> >>> was false Recovered ranges are not the same as the ones generated
>>>>> >>> (DirectKafkaStreamSuite.scala:301)
>>>>> >>>
>>>>> >>> This actually fails consistently for me too in the Kafka
>>>>> integration
>>>>> >>> code. Not timezone related, I think.
>>>>> >
>>>>> > ---------------------------------------------------------------------
>>>>> > To unsubscribe, e-mail: <de...@spark.apache.org>
>>>>> dev-unsubscribe@spark.apache.org
>>>>> > For additional commands, e-mail: <de...@spark.apache.org>
>>>>> dev-help@spark.apache.org
>>>>> >
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: <de...@spark.apache.org>
>>>>> dev-unsubscribe@spark.apache.org
>>>>> For additional commands, e-mail: <de...@spark.apache.org>
>>>>> dev-help@spark.apache.org
>>>>>
>>>>>
>>>>
>>
>>
>
>
--
Sameer Agarwal
Software Engineer | Databricks Inc.
http://cs.berkeley.edu/~sameerag
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Kousuke Saruta <sa...@oss.nttdata.co.jp>.
+1 (non-binding)
On 2016/06/23 4:53, Reynold Xin wrote:
> +1 myself
>
>
> On Wed, Jun 22, 2016 at 12:19 PM, Sean McNamara
> <Sean.McNamara@webtrends.com <ma...@webtrends.com>> wrote:
>
> +1
>
>> On Jun 22, 2016, at 1:14 PM, Michael Armbrust
>> <michael@databricks.com <ma...@databricks.com>> wrote:
>>
>> +1
>>
>> On Wed, Jun 22, 2016 at 11:33 AM, Jonathan Kelly
>> <jonathakamzn@gmail.com <ma...@gmail.com>> wrote:
>>
>> +1
>>
>> On Wed, Jun 22, 2016 at 10:41 AM Tim Hunter
>> <timhunter@databricks.com <ma...@databricks.com>>
>> wrote:
>>
>> +1 This release passes all tests on the graphframes and
>> tensorframes packages.
>>
>> On Wed, Jun 22, 2016 at 7:19 AM, Cody Koeninger
>> <cody@koeninger.org <ma...@koeninger.org>> wrote:
>>
>> If we're considering backporting changes for the 0.8
>> kafka
>> integration, I am sure there are people who would
>> like to get
>>
>> https://issues.apache.org/jira/browse/SPARK-10963
>>
>> into 1.6.x as well
>>
>> On Wed, Jun 22, 2016 at 7:41 AM, Sean Owen
>> <sowen@cloudera.com <ma...@cloudera.com>> wrote:
>> > Good call, probably worth back-porting, I'll try to
>> do that. I don't
>> > think it blocks a release, but would be good to get
>> into a next RC if
>> > any.
>> >
>> > On Wed, Jun 22, 2016 at 11:38 AM, Pete Robbins
>> <robbinspg@gmail.com <ma...@gmail.com>> wrote:
>> >> This has failed on our 1.6 stream builds regularly.
>> >> (https://issues.apache.org/jira/browse/SPARK-6005)
>> looks fixed in 2.0?
>> >>
>> >> On Wed, 22 Jun 2016 at 11:15 Sean Owen
>> <sowen@cloudera.com <ma...@cloudera.com>> wrote:
>> >>>
>> >>> Oops, one more in the "does anybody else see
>> this" department:
>> >>>
>> >>> - offset recovery *** FAILED ***
>> >>> recoveredOffsetRanges.forall(((or:
>> (org.apache.spark.streaming.Time,
>> >>>
>> Array[org.apache.spark.streaming.kafka.OffsetRange])) =>
>> >>>
>> >>>
>> earlierOffsetRangesAsSets.contains(scala.Tuple2.apply[org.apache.spark.streaming.Time,
>> >>>
>> >>>
>> scala.collection.immutable.Set[org.apache.spark.streaming.kafka.OffsetRange]](or._1,
>> >>>
>> >>>
>> scala.this.Predef.refArrayOps[org.apache.spark.streaming.kafka.OffsetRange](or._2).toSet[org.apache.spark.streaming.kafka.OffsetRange]))))
>> >>> was false Recovered ranges are not the same as
>> the ones generated
>> >>> (DirectKafkaStreamSuite.scala:301)
>> >>>
>> >>> This actually fails consistently for me too in
>> the Kafka integration
>> >>> code. Not timezone related, I think.
>> >
>> >
>> ---------------------------------------------------------------------
>> > To unsubscribe, e-mail:
>> dev-unsubscribe@spark.apache.org
>> <ma...@spark.apache.org>
>> > For additional commands, e-mail:
>> dev-help@spark.apache.org
>> <ma...@spark.apache.org>
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> dev-unsubscribe@spark.apache.org
>> <ma...@spark.apache.org>
>> For additional commands, e-mail:
>> dev-help@spark.apache.org
>> <ma...@spark.apache.org>
>>
>>
>>
>
>
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Reynold Xin <rx...@databricks.com>.
+1 myself
On Wed, Jun 22, 2016 at 12:19 PM, Sean McNamara <Sean.McNamara@webtrends.com
> wrote:
> +1
>
> On Jun 22, 2016, at 1:14 PM, Michael Armbrust <mi...@databricks.com>
> wrote:
>
> +1
>
> On Wed, Jun 22, 2016 at 11:33 AM, Jonathan Kelly <jo...@gmail.com>
> wrote:
>
>> +1
>>
>> On Wed, Jun 22, 2016 at 10:41 AM Tim Hunter <ti...@databricks.com>
>> wrote:
>>
>>> +1 This release passes all tests on the graphframes and tensorframes
>>> packages.
>>>
>>> On Wed, Jun 22, 2016 at 7:19 AM, Cody Koeninger <co...@koeninger.org>
>>> wrote:
>>>
>>>> If we're considering backporting changes for the 0.8 kafka
>>>> integration, I am sure there are people who would like to get
>>>>
>>>> https://issues.apache.org/jira/browse/SPARK-10963
>>>>
>>>> into 1.6.x as well
>>>>
>>>> On Wed, Jun 22, 2016 at 7:41 AM, Sean Owen <so...@cloudera.com> wrote:
>>>> > Good call, probably worth back-porting, I'll try to do that. I don't
>>>> > think it blocks a release, but would be good to get into a next RC if
>>>> > any.
>>>> >
>>>> > On Wed, Jun 22, 2016 at 11:38 AM, Pete Robbins <ro...@gmail.com>
>>>> wrote:
>>>> >> This has failed on our 1.6 stream builds regularly.
>>>> >> (https://issues.apache.org/jira/browse/SPARK-6005) looks fixed in
>>>> 2.0?
>>>> >>
>>>> >> On Wed, 22 Jun 2016 at 11:15 Sean Owen <so...@cloudera.com> wrote:
>>>> >>>
>>>> >>> Oops, one more in the "does anybody else see this" department:
>>>> >>>
>>>> >>> - offset recovery *** FAILED ***
>>>> >>> recoveredOffsetRanges.forall(((or:
>>>> (org.apache.spark.streaming.Time,
>>>> >>> Array[org.apache.spark.streaming.kafka.OffsetRange])) =>
>>>> >>>
>>>> >>>
>>>> earlierOffsetRangesAsSets.contains(scala.Tuple2.apply[org.apache.spark.streaming.Time,
>>>> >>>
>>>> >>>
>>>> scala.collection.immutable.Set[org.apache.spark.streaming.kafka.OffsetRange]](or._1,
>>>> >>>
>>>> >>>
>>>> scala.this.Predef.refArrayOps[org.apache.spark.streaming.kafka.OffsetRange](or._2).toSet[org.apache.spark.streaming.kafka.OffsetRange]))))
>>>> >>> was false Recovered ranges are not the same as the ones generated
>>>> >>> (DirectKafkaStreamSuite.scala:301)
>>>> >>>
>>>> >>> This actually fails consistently for me too in the Kafka integration
>>>> >>> code. Not timezone related, I think.
>>>> >
>>>> > ---------------------------------------------------------------------
>>>> > To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>>> > For additional commands, e-mail: dev-help@spark.apache.org
>>>> >
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>>> For additional commands, e-mail: dev-help@spark.apache.org
>>>>
>>>>
>>>
>
>
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Sean McNamara <Se...@Webtrends.com>.
+1
On Jun 22, 2016, at 1:14 PM, Michael Armbrust <mi...@databricks.com>> wrote:
+1
On Wed, Jun 22, 2016 at 11:33 AM, Jonathan Kelly <jo...@gmail.com>> wrote:
+1
On Wed, Jun 22, 2016 at 10:41 AM Tim Hunter <ti...@databricks.com>> wrote:
+1 This release passes all tests on the graphframes and tensorframes packages.
On Wed, Jun 22, 2016 at 7:19 AM, Cody Koeninger <co...@koeninger.org>> wrote:
If we're considering backporting changes for the 0.8 kafka
integration, I am sure there are people who would like to get
https://issues.apache.org/jira/browse/SPARK-10963
into 1.6.x as well
On Wed, Jun 22, 2016 at 7:41 AM, Sean Owen <so...@cloudera.com>> wrote:
> Good call, probably worth back-porting, I'll try to do that. I don't
> think it blocks a release, but would be good to get into a next RC if
> any.
>
> On Wed, Jun 22, 2016 at 11:38 AM, Pete Robbins <ro...@gmail.com>> wrote:
>> This has failed on our 1.6 stream builds regularly.
>> (https://issues.apache.org/jira/browse/SPARK-6005) looks fixed in 2.0?
>>
>> On Wed, 22 Jun 2016 at 11:15 Sean Owen <so...@cloudera.com>> wrote:
>>>
>>> Oops, one more in the "does anybody else see this" department:
>>>
>>> - offset recovery *** FAILED ***
>>> recoveredOffsetRanges.forall(((or: (org.apache.spark.streaming.Time,
>>> Array[org.apache.spark.streaming.kafka.OffsetRange])) =>
>>>
>>> earlierOffsetRangesAsSets.contains(scala.Tuple2.apply[org.apache.spark.streaming.Time,
>>>
>>> scala.collection.immutable.Set[org.apache.spark.streaming.kafka.OffsetRange]](or._1,
>>>
>>> scala.this.Predef.refArrayOps[org.apache.spark.streaming.kafka.OffsetRange](or._2).toSet[org.apache.spark.streaming.kafka.OffsetRange]))))
>>> was false Recovered ranges are not the same as the ones generated
>>> (DirectKafkaStreamSuite.scala:301)
>>>
>>> This actually fails consistently for me too in the Kafka integration
>>> code. Not timezone related, I think.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org<ma...@spark.apache.org>
> For additional commands, e-mail: dev-help@spark.apache.org<ma...@spark.apache.org>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org<ma...@spark.apache.org>
For additional commands, e-mail: dev-help@spark.apache.org<ma...@spark.apache.org>
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Michael Armbrust <mi...@databricks.com>.
+1
On Wed, Jun 22, 2016 at 11:33 AM, Jonathan Kelly <jo...@gmail.com>
wrote:
> +1
>
> On Wed, Jun 22, 2016 at 10:41 AM Tim Hunter <ti...@databricks.com>
> wrote:
>
>> +1 This release passes all tests on the graphframes and tensorframes
>> packages.
>>
>> On Wed, Jun 22, 2016 at 7:19 AM, Cody Koeninger <co...@koeninger.org>
>> wrote:
>>
>>> If we're considering backporting changes for the 0.8 kafka
>>> integration, I am sure there are people who would like to get
>>>
>>> https://issues.apache.org/jira/browse/SPARK-10963
>>>
>>> into 1.6.x as well
>>>
>>> On Wed, Jun 22, 2016 at 7:41 AM, Sean Owen <so...@cloudera.com> wrote:
>>> > Good call, probably worth back-porting, I'll try to do that. I don't
>>> > think it blocks a release, but would be good to get into a next RC if
>>> > any.
>>> >
>>> > On Wed, Jun 22, 2016 at 11:38 AM, Pete Robbins <ro...@gmail.com>
>>> wrote:
>>> >> This has failed on our 1.6 stream builds regularly.
>>> >> (https://issues.apache.org/jira/browse/SPARK-6005) looks fixed in
>>> 2.0?
>>> >>
>>> >> On Wed, 22 Jun 2016 at 11:15 Sean Owen <so...@cloudera.com> wrote:
>>> >>>
>>> >>> Oops, one more in the "does anybody else see this" department:
>>> >>>
>>> >>> - offset recovery *** FAILED ***
>>> >>> recoveredOffsetRanges.forall(((or:
>>> (org.apache.spark.streaming.Time,
>>> >>> Array[org.apache.spark.streaming.kafka.OffsetRange])) =>
>>> >>>
>>> >>>
>>> earlierOffsetRangesAsSets.contains(scala.Tuple2.apply[org.apache.spark.streaming.Time,
>>> >>>
>>> >>>
>>> scala.collection.immutable.Set[org.apache.spark.streaming.kafka.OffsetRange]](or._1,
>>> >>>
>>> >>>
>>> scala.this.Predef.refArrayOps[org.apache.spark.streaming.kafka.OffsetRange](or._2).toSet[org.apache.spark.streaming.kafka.OffsetRange]))))
>>> >>> was false Recovered ranges are not the same as the ones generated
>>> >>> (DirectKafkaStreamSuite.scala:301)
>>> >>>
>>> >>> This actually fails consistently for me too in the Kafka integration
>>> >>> code. Not timezone related, I think.
>>> >
>>> > ---------------------------------------------------------------------
>>> > To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>> > For additional commands, e-mail: dev-help@spark.apache.org
>>> >
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>>> For additional commands, e-mail: dev-help@spark.apache.org
>>>
>>>
>>
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Jonathan Kelly <jo...@gmail.com>.
+1
On Wed, Jun 22, 2016 at 10:41 AM Tim Hunter <ti...@databricks.com>
wrote:
> +1 This release passes all tests on the graphframes and tensorframes
> packages.
>
> On Wed, Jun 22, 2016 at 7:19 AM, Cody Koeninger <co...@koeninger.org>
> wrote:
>
>> If we're considering backporting changes for the 0.8 kafka
>> integration, I am sure there are people who would like to get
>>
>> https://issues.apache.org/jira/browse/SPARK-10963
>>
>> into 1.6.x as well
>>
>> On Wed, Jun 22, 2016 at 7:41 AM, Sean Owen <so...@cloudera.com> wrote:
>> > Good call, probably worth back-porting, I'll try to do that. I don't
>> > think it blocks a release, but would be good to get into a next RC if
>> > any.
>> >
>> > On Wed, Jun 22, 2016 at 11:38 AM, Pete Robbins <ro...@gmail.com>
>> wrote:
>> >> This has failed on our 1.6 stream builds regularly.
>> >> (https://issues.apache.org/jira/browse/SPARK-6005) looks fixed in 2.0?
>> >>
>> >> On Wed, 22 Jun 2016 at 11:15 Sean Owen <so...@cloudera.com> wrote:
>> >>>
>> >>> Oops, one more in the "does anybody else see this" department:
>> >>>
>> >>> - offset recovery *** FAILED ***
>> >>> recoveredOffsetRanges.forall(((or: (org.apache.spark.streaming.Time,
>> >>> Array[org.apache.spark.streaming.kafka.OffsetRange])) =>
>> >>>
>> >>>
>> earlierOffsetRangesAsSets.contains(scala.Tuple2.apply[org.apache.spark.streaming.Time,
>> >>>
>> >>>
>> scala.collection.immutable.Set[org.apache.spark.streaming.kafka.OffsetRange]](or._1,
>> >>>
>> >>>
>> scala.this.Predef.refArrayOps[org.apache.spark.streaming.kafka.OffsetRange](or._2).toSet[org.apache.spark.streaming.kafka.OffsetRange]))))
>> >>> was false Recovered ranges are not the same as the ones generated
>> >>> (DirectKafkaStreamSuite.scala:301)
>> >>>
>> >>> This actually fails consistently for me too in the Kafka integration
>> >>> code. Not timezone related, I think.
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> > For additional commands, e-mail: dev-help@spark.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
>> For additional commands, e-mail: dev-help@spark.apache.org
>>
>>
>
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Tim Hunter <ti...@databricks.com>.
+1 This release passes all tests on the graphframes and tensorframes
packages.
On Wed, Jun 22, 2016 at 7:19 AM, Cody Koeninger <co...@koeninger.org> wrote:
> If we're considering backporting changes for the 0.8 kafka
> integration, I am sure there are people who would like to get
>
> https://issues.apache.org/jira/browse/SPARK-10963
>
> into 1.6.x as well
>
> On Wed, Jun 22, 2016 at 7:41 AM, Sean Owen <so...@cloudera.com> wrote:
> > Good call, probably worth back-porting, I'll try to do that. I don't
> > think it blocks a release, but would be good to get into a next RC if
> > any.
> >
> > On Wed, Jun 22, 2016 at 11:38 AM, Pete Robbins <ro...@gmail.com>
> wrote:
> >> This has failed on our 1.6 stream builds regularly.
> >> (https://issues.apache.org/jira/browse/SPARK-6005) looks fixed in 2.0?
> >>
> >> On Wed, 22 Jun 2016 at 11:15 Sean Owen <so...@cloudera.com> wrote:
> >>>
> >>> Oops, one more in the "does anybody else see this" department:
> >>>
> >>> - offset recovery *** FAILED ***
> >>> recoveredOffsetRanges.forall(((or: (org.apache.spark.streaming.Time,
> >>> Array[org.apache.spark.streaming.kafka.OffsetRange])) =>
> >>>
> >>>
> earlierOffsetRangesAsSets.contains(scala.Tuple2.apply[org.apache.spark.streaming.Time,
> >>>
> >>>
> scala.collection.immutable.Set[org.apache.spark.streaming.kafka.OffsetRange]](or._1,
> >>>
> >>>
> scala.this.Predef.refArrayOps[org.apache.spark.streaming.kafka.OffsetRange](or._2).toSet[org.apache.spark.streaming.kafka.OffsetRange]))))
> >>> was false Recovered ranges are not the same as the ones generated
> >>> (DirectKafkaStreamSuite.scala:301)
> >>>
> >>> This actually fails consistently for me too in the Kafka integration
> >>> code. Not timezone related, I think.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> > For additional commands, e-mail: dev-help@spark.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Cody Koeninger <co...@koeninger.org>.
If we're considering backporting changes for the 0.8 kafka
integration, I am sure there are people who would like to get
https://issues.apache.org/jira/browse/SPARK-10963
into 1.6.x as well
On Wed, Jun 22, 2016 at 7:41 AM, Sean Owen <so...@cloudera.com> wrote:
> Good call, probably worth back-porting, I'll try to do that. I don't
> think it blocks a release, but would be good to get into a next RC if
> any.
>
> On Wed, Jun 22, 2016 at 11:38 AM, Pete Robbins <ro...@gmail.com> wrote:
>> This has failed on our 1.6 stream builds regularly.
>> (https://issues.apache.org/jira/browse/SPARK-6005) looks fixed in 2.0?
>>
>> On Wed, 22 Jun 2016 at 11:15 Sean Owen <so...@cloudera.com> wrote:
>>>
>>> Oops, one more in the "does anybody else see this" department:
>>>
>>> - offset recovery *** FAILED ***
>>> recoveredOffsetRanges.forall(((or: (org.apache.spark.streaming.Time,
>>> Array[org.apache.spark.streaming.kafka.OffsetRange])) =>
>>>
>>> earlierOffsetRangesAsSets.contains(scala.Tuple2.apply[org.apache.spark.streaming.Time,
>>>
>>> scala.collection.immutable.Set[org.apache.spark.streaming.kafka.OffsetRange]](or._1,
>>>
>>> scala.this.Predef.refArrayOps[org.apache.spark.streaming.kafka.OffsetRange](or._2).toSet[org.apache.spark.streaming.kafka.OffsetRange]))))
>>> was false Recovered ranges are not the same as the ones generated
>>> (DirectKafkaStreamSuite.scala:301)
>>>
>>> This actually fails consistently for me too in the Kafka integration
>>> code. Not timezone related, I think.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Sean Owen <so...@cloudera.com>.
Good call, probably worth back-porting, I'll try to do that. I don't
think it blocks a release, but would be good to get into a next RC if
any.
On Wed, Jun 22, 2016 at 11:38 AM, Pete Robbins <ro...@gmail.com> wrote:
> This has failed on our 1.6 stream builds regularly.
> (https://issues.apache.org/jira/browse/SPARK-6005) looks fixed in 2.0?
>
> On Wed, 22 Jun 2016 at 11:15 Sean Owen <so...@cloudera.com> wrote:
>>
>> Oops, one more in the "does anybody else see this" department:
>>
>> - offset recovery *** FAILED ***
>> recoveredOffsetRanges.forall(((or: (org.apache.spark.streaming.Time,
>> Array[org.apache.spark.streaming.kafka.OffsetRange])) =>
>>
>> earlierOffsetRangesAsSets.contains(scala.Tuple2.apply[org.apache.spark.streaming.Time,
>>
>> scala.collection.immutable.Set[org.apache.spark.streaming.kafka.OffsetRange]](or._1,
>>
>> scala.this.Predef.refArrayOps[org.apache.spark.streaming.kafka.OffsetRange](or._2).toSet[org.apache.spark.streaming.kafka.OffsetRange]))))
>> was false Recovered ranges are not the same as the ones generated
>> (DirectKafkaStreamSuite.scala:301)
>>
>> This actually fails consistently for me too in the Kafka integration
>> code. Not timezone related, I think.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Pete Robbins <ro...@gmail.com>.
This has failed on our 1.6 stream builds regularly. (
https://issues.apache.org/jira/browse/SPARK-6005) looks fixed in 2.0?
On Wed, 22 Jun 2016 at 11:15 Sean Owen <so...@cloudera.com> wrote:
> Oops, one more in the "does anybody else see this" department:
>
> - offset recovery *** FAILED ***
> recoveredOffsetRanges.forall(((or: (org.apache.spark.streaming.Time,
> Array[org.apache.spark.streaming.kafka.OffsetRange])) =>
>
> earlierOffsetRangesAsSets.contains(scala.Tuple2.apply[org.apache.spark.streaming.Time,
>
> scala.collection.immutable.Set[org.apache.spark.streaming.kafka.OffsetRange]](or._1,
>
> scala.this.Predef.refArrayOps[org.apache.spark.streaming.kafka.OffsetRange](or._2).toSet[org.apache.spark.streaming.kafka.OffsetRange]))))
> was false Recovered ranges are not the same as the ones generated
> (DirectKafkaStreamSuite.scala:301)
>
> This actually fails consistently for me too in the Kafka integration
> code. Not timezone related, I think.
>
> On Wed, Jun 22, 2016 at 9:02 AM, Sean Owen <so...@cloudera.com> wrote:
> > I'm fairly convinced this error and others that appear timestamp
> > related are an environment problem. This test and method have been
> > present for several Spark versions, without change. I reviewed the
> > logic and it seems sound, explicitly setting the time zone correctly.
> > I am not sure why it behaves differently on this machine.
> >
> > I'd give a +1 to this release if nobody else is seeing errors like
> > this. The sigs, hashes, other tests pass for me.
> >
> > On Tue, Jun 21, 2016 at 6:49 PM, Sean Owen <so...@cloudera.com> wrote:
> >> UIUtilsSuite:
> >> - formatBatchTime *** FAILED ***
> >> "2015/05/14 [14]:04:40" did not equal "2015/05/14 [21]:04:40"
> >> (UIUtilsSuite.scala:73)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Sean Owen <so...@cloudera.com>.
Oops, one more in the "does anybody else see this" department:
- offset recovery *** FAILED ***
recoveredOffsetRanges.forall(((or: (org.apache.spark.streaming.Time,
Array[org.apache.spark.streaming.kafka.OffsetRange])) =>
earlierOffsetRangesAsSets.contains(scala.Tuple2.apply[org.apache.spark.streaming.Time,
scala.collection.immutable.Set[org.apache.spark.streaming.kafka.OffsetRange]](or._1,
scala.this.Predef.refArrayOps[org.apache.spark.streaming.kafka.OffsetRange](or._2).toSet[org.apache.spark.streaming.kafka.OffsetRange]))))
was false Recovered ranges are not the same as the ones generated
(DirectKafkaStreamSuite.scala:301)
This actually fails consistently for me too in the Kafka integration
code. Not timezone related, I think.
On Wed, Jun 22, 2016 at 9:02 AM, Sean Owen <so...@cloudera.com> wrote:
> I'm fairly convinced this error and others that appear timestamp
> related are an environment problem. This test and method have been
> present for several Spark versions, without change. I reviewed the
> logic and it seems sound, explicitly setting the time zone correctly.
> I am not sure why it behaves differently on this machine.
>
> I'd give a +1 to this release if nobody else is seeing errors like
> this. The sigs, hashes, other tests pass for me.
>
> On Tue, Jun 21, 2016 at 6:49 PM, Sean Owen <so...@cloudera.com> wrote:
>> UIUtilsSuite:
>> - formatBatchTime *** FAILED ***
>> "2015/05/14 [14]:04:40" did not equal "2015/05/14 [21]:04:40"
>> (UIUtilsSuite.scala:73)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Sean Owen <so...@cloudera.com>.
I'm fairly convinced this error and others that appear timestamp
related are an environment problem. This test and method have been
present for several Spark versions, without change. I reviewed the
logic and it seems sound, explicitly setting the time zone correctly.
I am not sure why it behaves differently on this machine.
I'd give a +1 to this release if nobody else is seeing errors like
this. The sigs, hashes, other tests pass for me.
On Tue, Jun 21, 2016 at 6:49 PM, Sean Owen <so...@cloudera.com> wrote:
> UIUtilsSuite:
> - formatBatchTime *** FAILED ***
> "2015/05/14 [14]:04:40" did not equal "2015/05/14 [21]:04:40"
> (UIUtilsSuite.scala:73)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Sean Owen <so...@cloudera.com>.
Nice one, yeah indeed I was doing an incremental build. Not a blocker.
I'll have a look into the others, though I suspect they're problems
with tests rather than production code.
On Tue, Jun 21, 2016 at 6:53 PM, Marcelo Vanzin <va...@cloudera.com> wrote:
> On Tue, Jun 21, 2016 at 10:49 AM, Sean Owen <so...@cloudera.com> wrote:
>> I'm getting some errors building on Ubuntu 16 + Java 7. First is one
>> that may just be down to a Scala bug:
>>
>> [ERROR] bad symbolic reference. A signature in WebUI.class refers to
>> term eclipse
>> in package org which is not available.
>
> This is probably https://issues.apache.org/jira/browse/SPARK-13780. It
> should only affect incremental builds ("mvn -rf ..." or "mvn -pl
> ..."), not clean builds. Not sure about the other ones.
>
> --
> Marcelo
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Marcelo Vanzin <va...@cloudera.com>.
On Tue, Jun 21, 2016 at 10:49 AM, Sean Owen <so...@cloudera.com> wrote:
> I'm getting some errors building on Ubuntu 16 + Java 7. First is one
> that may just be down to a Scala bug:
>
> [ERROR] bad symbolic reference. A signature in WebUI.class refers to
> term eclipse
> in package org which is not available.
This is probably https://issues.apache.org/jira/browse/SPARK-13780. It
should only affect incremental builds ("mvn -rf ..." or "mvn -pl
..."), not clean builds. Not sure about the other ones.
--
Marcelo
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Sean Owen <so...@cloudera.com>.
I'm getting some errors building on Ubuntu 16 + Java 7. First is one
that may just be down to a Scala bug:
[ERROR] bad symbolic reference. A signature in WebUI.class refers to
term eclipse
in package org which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when
compiling WebUI.class.
[ERROR] bad symbolic reference. A signature in WebUI.class refers to term jetty
in value org.eclipse which is not available.
It may be completely missing from the current classpath, or the version on
the classpath might be incompatible with the version used when
compiling WebUI.class.
But I'm seeing some consistent timezone-related failures, from core:
UIUtilsSuite:
- formatBatchTime *** FAILED ***
"2015/05/14 [14]:04:40" did not equal "2015/05/14 [21]:04:40"
(UIUtilsSuite.scala:73)
and several from Spark SQL, like:
- udf_unix_timestamp *** FAILED ***
Results do not match for udf_unix_timestamp:
== Parsed Logical Plan ==
'Project [unresolvedalias(2009-03-20
11:30:01),unresolvedalias('unix_timestamp(2009-03-20 11:30:01))]
+- 'UnresolvedRelation `oneline`, None
== Analyzed Logical Plan ==
_c0: string, _c1: bigint
Project [2009-03-20 11:30:01 AS _c0#122914,unixtimestamp(2009-03-20
11:30:01,yyyy-MM-dd HH:mm:ss) AS _c1#122915L]
+- MetastoreRelation default, oneline, None
== Optimized Logical Plan ==
Project [2009-03-20 11:30:01 AS _c0#122914,1237548601 AS _c1#122915L]
+- MetastoreRelation default, oneline, None
== Physical Plan ==
Project [2009-03-20 11:30:01 AS _c0#122914,1237548601 AS _c1#122915L]
+- HiveTableScan MetastoreRelation default, oneline, None
_c0 _c1
!== HIVE - 1 row(s) == == CATALYST - 1 row(s) ==
!2009-03-20 11:30:01 1237573801 2009-03-20 11:30:01 1237548601
(HiveComparisonTest.scala:458)
I'll start looking into them. It could be real, if possibly minor,
bugs because I presume most of the testing happens on machines in a
PDT timezone instead of UTC? that's at least the timezone of the
machine I'm testing on.
On Mon, Jun 20, 2016 at 5:24 AM, Reynold Xin <rx...@databricks.com> wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.6.2. The vote is open until Wednesday, June 22, 2016 at 22:00 PDT and
> passes if a majority of at least 3+1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.6.2
> [ ] -1 Do not release this package because ...
>
>
> The tag to be voted on is v1.6.2-rc2
> (54b1121f351f056d6b67d2bb4efe0d553c0f7482)
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-bin/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1186/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-docs/
>
>
> =======================================
> == How can I help test this release? ==
> =======================================
> If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions from 1.6.1.
>
> ================================================
> == What justifies a -1 vote for this release? ==
> ================================================
> This is a maintenance release in the 1.6.x series. Bugs already present in
> 1.6.1, missing features, or bugs related to new features will not
> necessarily block this release.
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by vaquar khan <va...@gmail.com>.
+1 (non-binding
Regards,
Vaquar khan
On 23 Jun 2016 07:50, "Sean Owen" <so...@cloudera.com> wrote:
> I don't think that qualifies as a blocker; not even clear it's a
> regression. Even non-binding votes here should focus on whether this
> is OK to release as a maintenance update to 1.6.1.
>
> On Thu, Jun 23, 2016 at 1:45 PM, Maciej Bryński <ma...@brynski.pl> wrote:
> > -1
> >
> > I need SPARK-13283 to be solved.
> >
> > Regards,
> > Maciek Bryński
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Sean Owen <so...@cloudera.com>.
I don't think that qualifies as a blocker; not even clear it's a
regression. Even non-binding votes here should focus on whether this
is OK to release as a maintenance update to 1.6.1.
On Thu, Jun 23, 2016 at 1:45 PM, Maciej Bryński <ma...@brynski.pl> wrote:
> -1
>
> I need SPARK-13283 to be solved.
>
> Regards,
> Maciek Bryński
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
For additional commands, e-mail: dev-help@spark.apache.org
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Reynold Xin <rx...@databricks.com>.
Maciej let's fix SPARK-13283. It won't block 1.6.2 though.
On Thu, Jun 23, 2016 at 5:45 AM, Maciej Bryński <ma...@brynski.pl> wrote:
> -1
>
> I need SPARK-13283 to be solved.
>
> Regards,
> Maciek Bryński
>
> 2016-06-23 0:13 GMT+02:00 Krishna Sankar <ks...@gmail.com>:
>
>> +1 (non-binding, of course)
>>
>> 1. Compiled OSX 10.10 (Yosemite) OK Total time: 37:11 min
>> mvn clean package -Pyarn -Phadoop-2.6 -DskipTests
>> 2. Tested pyspark, mllib (iPython 4.0)
>> 2.0 Spark version is 1.6.2
>> 2.1. statistics (min,max,mean,Pearson,Spearman) OK
>> 2.2. Linear/Ridge/Lasso Regression OK
>> 2.3. Decision Tree, Naive Bayes OK
>> 2.4. KMeans OK
>> Center And Scale OK
>> 2.5. RDD operations OK
>> State of the Union Texts - MapReduce, Filter,sortByKey (word count)
>> 2.6. Recommendation (Movielens medium dataset ~1 M ratings) OK
>> Model evaluation/optimization (rank, numIter, lambda) with
>> itertools OK
>> 3. Scala - MLlib
>> 3.1. statistics (min,max,mean,Pearson,Spearman) OK
>> 3.2. LinearRegressionWithSGD OK
>> 3.3. Decision Tree OK
>> 3.4. KMeans OK
>> 3.5. Recommendation (Movielens medium dataset ~1 M ratings) OK
>> 3.6. saveAsParquetFile OK
>> 3.7. Read and verify the 4.3 save(above) - sqlContext.parquetFile,
>> registerTempTable, sql OK
>> 3.8. result = sqlContext.sql("SELECT
>> OrderDetails.OrderID,ShipCountry,UnitPrice,Qty,Discount FROM Orders INNER
>> JOIN OrderDetails ON Orders.OrderID = OrderDetails.OrderID") OK
>> 4.0. Spark SQL from Python OK
>> 4.1. result = sqlContext.sql("SELECT * from people WHERE State = 'WA'") OK
>> 5.0. Packages
>> 5.1. com.databricks.spark.csv - read/write OK (--packages
>> com.databricks:spark-csv_2.10:1.4.0)
>> 6.0. DataFrames
>> 6.1. cast,dtypes OK
>> 6.2. groupBy,avg,crosstab,corr,isNull,na.drop OK
>> 6.3. All joins,sql,set operations,udf OK
>> 7.0. GraphX/Scala
>> 7.1. Create Graph (small and bigger dataset) OK
>> 7.2. Structure APIs - OK
>> 7.3. Social Network/Community APIs - OK
>> 7.4. Algorithms (PageRank of 2 datasets, aggregateMessages() ) OK
>>
>> Cheers & Good Work, Folks
>> <k/>
>>
>> On Sun, Jun 19, 2016 at 9:24 PM, Reynold Xin <rx...@databricks.com> wrote:
>>
>>> Please vote on releasing the following candidate as Apache Spark version
>>> 1.6.2. The vote is open until Wednesday, June 22, 2016 at 22:00 PDT and
>>> passes if a majority of at least 3+1 PMC votes are cast.
>>>
>>> [ ] +1 Release this package as Apache Spark 1.6.2
>>> [ ] -1 Do not release this package because ...
>>>
>>>
>>> The tag to be voted on is v1.6.2-rc2
>>> (54b1121f351f056d6b67d2bb4efe0d553c0f7482)
>>>
>>> The release files, including signatures, digests, etc. can be found at:
>>> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-bin/
>>>
>>> Release artifacts are signed with the following key:
>>> https://people.apache.org/keys/committer/pwendell.asc
>>>
>>> The staging repository for this release can be found at:
>>> https://repository.apache.org/content/repositories/orgapachespark-1186/
>>>
>>> The documentation corresponding to this release can be found at:
>>> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-docs/
>>>
>>>
>>> =======================================
>>> == How can I help test this release? ==
>>> =======================================
>>> If you are a Spark user, you can help us test this release by taking an
>>> existing Spark workload and running on this release candidate, then
>>> reporting any regressions from 1.6.1.
>>>
>>> ================================================
>>> == What justifies a -1 vote for this release? ==
>>> ================================================
>>> This is a maintenance release in the 1.6.x series. Bugs already present
>>> in 1.6.1, missing features, or bugs related to new features will not
>>> necessarily block this release.
>>>
>>>
>>>
>>>
>>
>
>
> --
> Maciek Bryński
>
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Maciej Bryński <ma...@brynski.pl>.
-1
I need SPARK-13283 to be solved.
Regards,
Maciek Bryński
2016-06-23 0:13 GMT+02:00 Krishna Sankar <ks...@gmail.com>:
> +1 (non-binding, of course)
>
> 1. Compiled OSX 10.10 (Yosemite) OK Total time: 37:11 min
> mvn clean package -Pyarn -Phadoop-2.6 -DskipTests
> 2. Tested pyspark, mllib (iPython 4.0)
> 2.0 Spark version is 1.6.2
> 2.1. statistics (min,max,mean,Pearson,Spearman) OK
> 2.2. Linear/Ridge/Lasso Regression OK
> 2.3. Decision Tree, Naive Bayes OK
> 2.4. KMeans OK
> Center And Scale OK
> 2.5. RDD operations OK
> State of the Union Texts - MapReduce, Filter,sortByKey (word count)
> 2.6. Recommendation (Movielens medium dataset ~1 M ratings) OK
> Model evaluation/optimization (rank, numIter, lambda) with
> itertools OK
> 3. Scala - MLlib
> 3.1. statistics (min,max,mean,Pearson,Spearman) OK
> 3.2. LinearRegressionWithSGD OK
> 3.3. Decision Tree OK
> 3.4. KMeans OK
> 3.5. Recommendation (Movielens medium dataset ~1 M ratings) OK
> 3.6. saveAsParquetFile OK
> 3.7. Read and verify the 4.3 save(above) - sqlContext.parquetFile,
> registerTempTable, sql OK
> 3.8. result = sqlContext.sql("SELECT
> OrderDetails.OrderID,ShipCountry,UnitPrice,Qty,Discount FROM Orders INNER
> JOIN OrderDetails ON Orders.OrderID = OrderDetails.OrderID") OK
> 4.0. Spark SQL from Python OK
> 4.1. result = sqlContext.sql("SELECT * from people WHERE State = 'WA'") OK
> 5.0. Packages
> 5.1. com.databricks.spark.csv - read/write OK (--packages
> com.databricks:spark-csv_2.10:1.4.0)
> 6.0. DataFrames
> 6.1. cast,dtypes OK
> 6.2. groupBy,avg,crosstab,corr,isNull,na.drop OK
> 6.3. All joins,sql,set operations,udf OK
> 7.0. GraphX/Scala
> 7.1. Create Graph (small and bigger dataset) OK
> 7.2. Structure APIs - OK
> 7.3. Social Network/Community APIs - OK
> 7.4. Algorithms (PageRank of 2 datasets, aggregateMessages() ) OK
>
> Cheers & Good Work, Folks
> <k/>
>
> On Sun, Jun 19, 2016 at 9:24 PM, Reynold Xin <rx...@databricks.com> wrote:
>
>> Please vote on releasing the following candidate as Apache Spark version
>> 1.6.2. The vote is open until Wednesday, June 22, 2016 at 22:00 PDT and
>> passes if a majority of at least 3+1 PMC votes are cast.
>>
>> [ ] +1 Release this package as Apache Spark 1.6.2
>> [ ] -1 Do not release this package because ...
>>
>>
>> The tag to be voted on is v1.6.2-rc2
>> (54b1121f351f056d6b67d2bb4efe0d553c0f7482)
>>
>> The release files, including signatures, digests, etc. can be found at:
>> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-bin/
>>
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/pwendell.asc
>>
>> The staging repository for this release can be found at:
>> https://repository.apache.org/content/repositories/orgapachespark-1186/
>>
>> The documentation corresponding to this release can be found at:
>> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-docs/
>>
>>
>> =======================================
>> == How can I help test this release? ==
>> =======================================
>> If you are a Spark user, you can help us test this release by taking an
>> existing Spark workload and running on this release candidate, then
>> reporting any regressions from 1.6.1.
>>
>> ================================================
>> == What justifies a -1 vote for this release? ==
>> ================================================
>> This is a maintenance release in the 1.6.x series. Bugs already present
>> in 1.6.1, missing features, or bugs related to new features will not
>> necessarily block this release.
>>
>>
>>
>>
>
--
Maciek Bryński
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Krishna Sankar <ks...@gmail.com>.
+1 (non-binding, of course)
1. Compiled OSX 10.10 (Yosemite) OK Total time: 37:11 min
mvn clean package -Pyarn -Phadoop-2.6 -DskipTests
2. Tested pyspark, mllib (iPython 4.0)
2.0 Spark version is 1.6.2
2.1. statistics (min,max,mean,Pearson,Spearman) OK
2.2. Linear/Ridge/Lasso Regression OK
2.3. Decision Tree, Naive Bayes OK
2.4. KMeans OK
Center And Scale OK
2.5. RDD operations OK
State of the Union Texts - MapReduce, Filter,sortByKey (word count)
2.6. Recommendation (Movielens medium dataset ~1 M ratings) OK
Model evaluation/optimization (rank, numIter, lambda) with itertools
OK
3. Scala - MLlib
3.1. statistics (min,max,mean,Pearson,Spearman) OK
3.2. LinearRegressionWithSGD OK
3.3. Decision Tree OK
3.4. KMeans OK
3.5. Recommendation (Movielens medium dataset ~1 M ratings) OK
3.6. saveAsParquetFile OK
3.7. Read and verify the 4.3 save(above) - sqlContext.parquetFile,
registerTempTable, sql OK
3.8. result = sqlContext.sql("SELECT
OrderDetails.OrderID,ShipCountry,UnitPrice,Qty,Discount FROM Orders INNER
JOIN OrderDetails ON Orders.OrderID = OrderDetails.OrderID") OK
4.0. Spark SQL from Python OK
4.1. result = sqlContext.sql("SELECT * from people WHERE State = 'WA'") OK
5.0. Packages
5.1. com.databricks.spark.csv - read/write OK (--packages
com.databricks:spark-csv_2.10:1.4.0)
6.0. DataFrames
6.1. cast,dtypes OK
6.2. groupBy,avg,crosstab,corr,isNull,na.drop OK
6.3. All joins,sql,set operations,udf OK
7.0. GraphX/Scala
7.1. Create Graph (small and bigger dataset) OK
7.2. Structure APIs - OK
7.3. Social Network/Community APIs - OK
7.4. Algorithms (PageRank of 2 datasets, aggregateMessages() ) OK
Cheers & Good Work, Folks
<k/>
On Sun, Jun 19, 2016 at 9:24 PM, Reynold Xin <rx...@databricks.com> wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.6.2. The vote is open until Wednesday, June 22, 2016 at 22:00 PDT and
> passes if a majority of at least 3+1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.6.2
> [ ] -1 Do not release this package because ...
>
>
> The tag to be voted on is v1.6.2-rc2
> (54b1121f351f056d6b67d2bb4efe0d553c0f7482)
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-bin/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1186/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-docs/
>
>
> =======================================
> == How can I help test this release? ==
> =======================================
> If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions from 1.6.1.
>
> ================================================
> == What justifies a -1 vote for this release? ==
> ================================================
> This is a maintenance release in the 1.6.x series. Bugs already present
> in 1.6.1, missing features, or bugs related to new features will not
> necessarily block this release.
>
>
>
>
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by "Shixiong(Ryan) Zhu" <sh...@databricks.com>.
Hey Pete,
I just pushed your PR to branch 1.6. As it's not a blocker, it may or may
not be in 1.6.2, depending on if there will be another RC.
On Tue, Jun 21, 2016 at 1:36 PM, Pete Robbins <ro...@gmail.com> wrote:
> It breaks Spark running on machines with less than 3 cores/threads, which
> may be rare, and it is maybe an edge case.
>
> Personally, I like to fix known bugs and the fact there are other blocking
> methods in event loops actually makes it worse not to fix ones that you
> know about.
>
> Probably not a blocker to release though but that's your call.
>
> Cheers,
>
> On Tue, Jun 21, 2016 at 6:40 PM Shixiong(Ryan) Zhu <
> shixiong@databricks.com> wrote:
>
>> Hey Pete,
>>
>> I didn't backport it to 1.6 because it just affects tests in most cases.
>> I'm sure we also have other places calling blocking methods in the event
>> loops, so similar issues are still there even after applying this patch.
>> Hence, I don't think it's a blocker for 1.6.2.
>>
>> On Tue, Jun 21, 2016 at 2:57 AM, Pete Robbins <ro...@gmail.com>
>> wrote:
>>
>>> The PR (https://github.com/apache/spark/pull/13055) to fix
>>> https://issues.apache.org/jira/browse/SPARK-15262 was applied to 1.6.2
>>> however this fix caused another issue
>>> https://issues.apache.org/jira/browse/SPARK-15606 the fix for which (
>>> https://github.com/apache/spark/pull/13355) has not been backported to
>>> the 1.6 branch so I'm now seeing the same failure in 1.6.2
>>>
>>> Cheers,
>>>
>>> On Mon, 20 Jun 2016 at 05:25 Reynold Xin <rx...@databricks.com> wrote:
>>>
>>>> Please vote on releasing the following candidate as Apache Spark
>>>> version 1.6.2. The vote is open until Wednesday, June 22, 2016 at 22:00 PDT
>>>> and passes if a majority of at least 3+1 PMC votes are cast.
>>>>
>>>> [ ] +1 Release this package as Apache Spark 1.6.2
>>>> [ ] -1 Do not release this package because ...
>>>>
>>>>
>>>> The tag to be voted on is v1.6.2-rc2
>>>> (54b1121f351f056d6b67d2bb4efe0d553c0f7482)
>>>>
>>>> The release files, including signatures, digests, etc. can be found at:
>>>> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-bin/
>>>>
>>>> Release artifacts are signed with the following key:
>>>> https://people.apache.org/keys/committer/pwendell.asc
>>>>
>>>> The staging repository for this release can be found at:
>>>> https://repository.apache.org/content/repositories/orgapachespark-1186/
>>>>
>>>> The documentation corresponding to this release can be found at:
>>>> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-docs/
>>>>
>>>>
>>>> =======================================
>>>> == How can I help test this release? ==
>>>> =======================================
>>>> If you are a Spark user, you can help us test this release by taking an
>>>> existing Spark workload and running on this release candidate, then
>>>> reporting any regressions from 1.6.1.
>>>>
>>>> ================================================
>>>> == What justifies a -1 vote for this release? ==
>>>> ================================================
>>>> This is a maintenance release in the 1.6.x series. Bugs already
>>>> present in 1.6.1, missing features, or bugs related to new features will
>>>> not necessarily block this release.
>>>>
>>>>
>>>>
>>>>
>>
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Pete Robbins <ro...@gmail.com>.
It breaks Spark running on machines with less than 3 cores/threads, which
may be rare, and it is maybe an edge case.
Personally, I like to fix known bugs and the fact there are other blocking
methods in event loops actually makes it worse not to fix ones that you
know about.
Probably not a blocker to release though but that's your call.
Cheers,
On Tue, Jun 21, 2016 at 6:40 PM Shixiong(Ryan) Zhu <sh...@databricks.com>
wrote:
> Hey Pete,
>
> I didn't backport it to 1.6 because it just affects tests in most cases.
> I'm sure we also have other places calling blocking methods in the event
> loops, so similar issues are still there even after applying this patch.
> Hence, I don't think it's a blocker for 1.6.2.
>
> On Tue, Jun 21, 2016 at 2:57 AM, Pete Robbins <ro...@gmail.com> wrote:
>
>> The PR (https://github.com/apache/spark/pull/13055) to fix
>> https://issues.apache.org/jira/browse/SPARK-15262 was applied to 1.6.2
>> however this fix caused another issue
>> https://issues.apache.org/jira/browse/SPARK-15606 the fix for which (
>> https://github.com/apache/spark/pull/13355) has not been backported to
>> the 1.6 branch so I'm now seeing the same failure in 1.6.2
>>
>> Cheers,
>>
>> On Mon, 20 Jun 2016 at 05:25 Reynold Xin <rx...@databricks.com> wrote:
>>
>>> Please vote on releasing the following candidate as Apache Spark version
>>> 1.6.2. The vote is open until Wednesday, June 22, 2016 at 22:00 PDT and
>>> passes if a majority of at least 3+1 PMC votes are cast.
>>>
>>> [ ] +1 Release this package as Apache Spark 1.6.2
>>> [ ] -1 Do not release this package because ...
>>>
>>>
>>> The tag to be voted on is v1.6.2-rc2
>>> (54b1121f351f056d6b67d2bb4efe0d553c0f7482)
>>>
>>> The release files, including signatures, digests, etc. can be found at:
>>> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-bin/
>>>
>>> Release artifacts are signed with the following key:
>>> https://people.apache.org/keys/committer/pwendell.asc
>>>
>>> The staging repository for this release can be found at:
>>> https://repository.apache.org/content/repositories/orgapachespark-1186/
>>>
>>> The documentation corresponding to this release can be found at:
>>> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-docs/
>>>
>>>
>>> =======================================
>>> == How can I help test this release? ==
>>> =======================================
>>> If you are a Spark user, you can help us test this release by taking an
>>> existing Spark workload and running on this release candidate, then
>>> reporting any regressions from 1.6.1.
>>>
>>> ================================================
>>> == What justifies a -1 vote for this release? ==
>>> ================================================
>>> This is a maintenance release in the 1.6.x series. Bugs already present
>>> in 1.6.1, missing features, or bugs related to new features will not
>>> necessarily block this release.
>>>
>>>
>>>
>>>
>
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by "Shixiong(Ryan) Zhu" <sh...@databricks.com>.
Hey Pete,
I didn't backport it to 1.6 because it just affects tests in most cases.
I'm sure we also have other places calling blocking methods in the event
loops, so similar issues are still there even after applying this patch.
Hence, I don't think it's a blocker for 1.6.2.
On Tue, Jun 21, 2016 at 2:57 AM, Pete Robbins <ro...@gmail.com> wrote:
> The PR (https://github.com/apache/spark/pull/13055) to fix
> https://issues.apache.org/jira/browse/SPARK-15262 was applied to 1.6.2
> however this fix caused another issue
> https://issues.apache.org/jira/browse/SPARK-15606 the fix for which (
> https://github.com/apache/spark/pull/13355) has not been backported to
> the 1.6 branch so I'm now seeing the same failure in 1.6.2
>
> Cheers,
>
> On Mon, 20 Jun 2016 at 05:25 Reynold Xin <rx...@databricks.com> wrote:
>
>> Please vote on releasing the following candidate as Apache Spark version
>> 1.6.2. The vote is open until Wednesday, June 22, 2016 at 22:00 PDT and
>> passes if a majority of at least 3+1 PMC votes are cast.
>>
>> [ ] +1 Release this package as Apache Spark 1.6.2
>> [ ] -1 Do not release this package because ...
>>
>>
>> The tag to be voted on is v1.6.2-rc2
>> (54b1121f351f056d6b67d2bb4efe0d553c0f7482)
>>
>> The release files, including signatures, digests, etc. can be found at:
>> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-bin/
>>
>> Release artifacts are signed with the following key:
>> https://people.apache.org/keys/committer/pwendell.asc
>>
>> The staging repository for this release can be found at:
>> https://repository.apache.org/content/repositories/orgapachespark-1186/
>>
>> The documentation corresponding to this release can be found at:
>> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-docs/
>>
>>
>> =======================================
>> == How can I help test this release? ==
>> =======================================
>> If you are a Spark user, you can help us test this release by taking an
>> existing Spark workload and running on this release candidate, then
>> reporting any regressions from 1.6.1.
>>
>> ================================================
>> == What justifies a -1 vote for this release? ==
>> ================================================
>> This is a maintenance release in the 1.6.x series. Bugs already present
>> in 1.6.1, missing features, or bugs related to new features will not
>> necessarily block this release.
>>
>>
>>
>>
Re: [VOTE] Release Apache Spark 1.6.2 (RC2)
Posted by Pete Robbins <ro...@gmail.com>.
The PR (https://github.com/apache/spark/pull/13055) to fix
https://issues.apache.org/jira/browse/SPARK-15262 was applied to 1.6.2
however this fix caused another issue
https://issues.apache.org/jira/browse/SPARK-15606 the fix for which (
https://github.com/apache/spark/pull/13355) has not been backported to the
1.6 branch so I'm now seeing the same failure in 1.6.2
Cheers,
On Mon, 20 Jun 2016 at 05:25 Reynold Xin <rx...@databricks.com> wrote:
> Please vote on releasing the following candidate as Apache Spark version
> 1.6.2. The vote is open until Wednesday, June 22, 2016 at 22:00 PDT and
> passes if a majority of at least 3+1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Spark 1.6.2
> [ ] -1 Do not release this package because ...
>
>
> The tag to be voted on is v1.6.2-rc2
> (54b1121f351f056d6b67d2bb4efe0d553c0f7482)
>
> The release files, including signatures, digests, etc. can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-bin/
>
> Release artifacts are signed with the following key:
> https://people.apache.org/keys/committer/pwendell.asc
>
> The staging repository for this release can be found at:
> https://repository.apache.org/content/repositories/orgapachespark-1186/
>
> The documentation corresponding to this release can be found at:
> http://people.apache.org/~pwendell/spark-releases/spark-1.6.2-rc2-docs/
>
>
> =======================================
> == How can I help test this release? ==
> =======================================
> If you are a Spark user, you can help us test this release by taking an
> existing Spark workload and running on this release candidate, then
> reporting any regressions from 1.6.1.
>
> ================================================
> == What justifies a -1 vote for this release? ==
> ================================================
> This is a maintenance release in the 1.6.x series. Bugs already present
> in 1.6.1, missing features, or bugs related to new features will not
> necessarily block this release.
>
>
>
>