You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Wenchen Fan <cl...@gmail.com> on 2018/10/02 02:43:22 UTC
Re: On Scala 2.12.7
My major concern is how it will affect end-users if Spark 2.4 is built with
Scala versions prior to 2.12.7. Generally I'm hesitating to upgrade Scala
version when we are very close to a release, and Scala 2.12 build of Spark
2.4 is beta anyway.
On Sat, Sep 29, 2018 at 6:46 AM Sean Owen <sr...@apache.org> wrote:
> I'm forking the discussion about Scala 2.12.7 from the 2.4.0 RC vote
> thread.
>
> 2.12.7 was released yesterday, and, is even labeled as fixing Spark
> 2.4 compatibility! https://www.scala-lang.org/news/2.12.7 We should
> look into it, yes.
>
> Darcy identified, and they fixed, this issue:
> https://github.com/scala/scala/pull/7156 while finishing the work for
> Scala 2.12.
>
> However we already worked around this in Spark, no? at
>
> https://github.com/apache/spark/commit/f29c2b5287563c0d6f55f936bd5a75707d7b2b1f
>
> So we should go ahead and update to use 2.12.7, yes, and undo this
> workaround?
> But this doesn't necessarily block a 2.4.0 release, if it's already
> worked around.
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
Re: On Scala 2.12.7
Posted by Felix Cheung <fe...@hotmail.com>.
Although like you said, spark support for scala 2.12 is beta anyway then shouldn’t we get it to a working state by basing on 2.12.7? There shouldn’t be a stability issue since it is not officially “supported”
________________________________
From: Wenchen Fan <cl...@gmail.com>
Sent: Monday, October 1, 2018 7:43 PM
To: Sean Owen
Cc: sadhen@zoho.com; Spark dev list
Subject: Re: On Scala 2.12.7
My major concern is how it will affect end-users if Spark 2.4 is built with Scala versions prior to 2.12.7. Generally I'm hesitating to upgrade Scala version when we are very close to a release, and Scala 2.12 build of Spark 2.4 is beta anyway.
On Sat, Sep 29, 2018 at 6:46 AM Sean Owen <sr...@apache.org>> wrote:
I'm forking the discussion about Scala 2.12.7 from the 2.4.0 RC vote thread.
2.12.7 was released yesterday, and, is even labeled as fixing Spark
2.4 compatibility! https://www.scala-lang.org/news/2.12.7 We should
look into it, yes.
Darcy identified, and they fixed, this issue:
https://github.com/scala/scala/pull/7156 while finishing the work for
Scala 2.12.
However we already worked around this in Spark, no? at
https://github.com/apache/spark/commit/f29c2b5287563c0d6f55f936bd5a75707d7b2b1f
So we should go ahead and update to use 2.12.7, yes, and undo this workaround?
But this doesn't necessarily block a 2.4.0 release, if it's already
worked around.
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org<ma...@spark.apache.org>
Re: On Scala 2.12.7
Posted by Sean Owen <sr...@apache.org>.
I don't think so. Spark brings the Scala dependency, and I don't think
the installed scalac matters in this respect.
Darcy there was an open question about whether this enables you to
back out the workaround you created for 2.12.6. I tried removing it
and it failed again, so left it in as still needed.
On Mon, Oct 1, 2018 at 10:20 PM Wenchen Fan <cl...@gmail.com> wrote:
>
> SGTM then. Is there anything we need to do to pick up the 2.12.7 upgrade? like updating Jenkins config?
>
> On Tue, Oct 2, 2018 at 10:53 AM Sean Owen <sr...@apache.org> wrote:
>>
>> I tested both ways, and it actually works fine. It calls into question
>> whether there's really a fix we need with 2.12.7, but, I hear two
>> informed opinions (Darcy and the scala release notes) that it was
>> relevant. As we have no prior 2.12 support, I guess my feeling was
>> indeed to get this update in to the first 2.12-supporting release.
>>
>> On Mon, Oct 1, 2018 at 9:43 PM Wenchen Fan <cl...@gmail.com> wrote:
>> >
>> > My major concern is how it will affect end-users if Spark 2.4 is built with Scala versions prior to 2.12.7. Generally I'm hesitating to upgrade Scala version when we are very close to a release, and Scala 2.12 build of Spark 2.4 is beta anyway.
>> >
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
Re: On Scala 2.12.7
Posted by Wenchen Fan <cl...@gmail.com>.
SGTM then. Is there anything we need to do to pick up the 2.12.7 upgrade?
like updating Jenkins config?
On Tue, Oct 2, 2018 at 10:53 AM Sean Owen <sr...@apache.org> wrote:
> I tested both ways, and it actually works fine. It calls into question
> whether there's really a fix we need with 2.12.7, but, I hear two
> informed opinions (Darcy and the scala release notes) that it was
> relevant. As we have no prior 2.12 support, I guess my feeling was
> indeed to get this update in to the first 2.12-supporting release.
>
> On Mon, Oct 1, 2018 at 9:43 PM Wenchen Fan <cl...@gmail.com> wrote:
> >
> > My major concern is how it will affect end-users if Spark 2.4 is built
> with Scala versions prior to 2.12.7. Generally I'm hesitating to upgrade
> Scala version when we are very close to a release, and Scala 2.12 build of
> Spark 2.4 is beta anyway.
> >
>
Re: On Scala 2.12.7
Posted by Sean Owen <sr...@apache.org>.
I tested both ways, and it actually works fine. It calls into question
whether there's really a fix we need with 2.12.7, but, I hear two
informed opinions (Darcy and the scala release notes) that it was
relevant. As we have no prior 2.12 support, I guess my feeling was
indeed to get this update in to the first 2.12-supporting release.
On Mon, Oct 1, 2018 at 9:43 PM Wenchen Fan <cl...@gmail.com> wrote:
>
> My major concern is how it will affect end-users if Spark 2.4 is built with Scala versions prior to 2.12.7. Generally I'm hesitating to upgrade Scala version when we are very close to a release, and Scala 2.12 build of Spark 2.4 is beta anyway.
>
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org