You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Sean Owen <so...@cloudera.com> on 2017/07/08 07:38:32 UTC
Re: RFC: removing Scala 2.10
I'd like to go ahead and merge SPARK-19810 after the Spark 2.2.0 release is
fully done and dusted. Would anyone object at this stage, to removing 2.10
support?
On Tue, Mar 7, 2017 at 3:55 PM Mark Hamstra <ma...@clearstorydata.com> wrote:
> Sorry, for some reason I was thinking that we have branch-2.2 cut already.
> If we're not going to pull Scala 2.10 out of 2.2.0, then we should wait at
> least until that branch is cut before we pull it out of master -- but I'd
> still argue for not long after that so that the 2.12 work can start.
>
> On Tue, Mar 7, 2017 at 7:50 AM, Mark Hamstra <ma...@clearstorydata.com>
> wrote:
>
>> I don't think 2.10 and 2.12 support can coexist, and soon, 2.12 support
>>> will be important.
>>>
>>
>> Which is why I'm thinking that we should pull 2.10 support out of master
>> soon -- either immediately or right after 2.2 goes into RC or full release.
>>
>> On Tue, Mar 7, 2017 at 2:14 AM, Sean Owen <so...@cloudera.com> wrote:
>>
>>> (2.10 was already deprecated for 2.1, so that's done actually.)
>>>
>>> Personally I'm fine with leaving in 2.10 support for 2.2. (FWIW CDH is
>>> Scala 2.11-only for Spark 2.) If there were no voices in support of keeping
>>> it, might be worth moving on right now, but if there's any substantive
>>> argument against, I'd also punt it another release.
>>>
>>> It's not really driven by cleanup, though that's nice, but 2.12 support.
>>> I don't think 2.10 and 2.12 support can coexist, and soon, 2.12 support
>>> will be important.
>>>
>>> How about tagging this for 2.3.0, as well as targeting 2.12 support for
>>> 2.3.0?
>>>
>>> On Tue, Mar 7, 2017 at 2:38 AM Reynold Xin <rx...@databricks.com> wrote:
>>>
>>>> Thanks for sending an email. I was going to +1 but then I figured I
>>>> should be data driven. I took a look at the distribution of Scala versions
>>>> across all the clusters Databricks runs (which is a very high number across
>>>> a variety of tech startups, SMBs, large enterprises, and this is the chart:
>>>>
>>>> [image: scala-version.png]
>>>>
>>>>
>>>>
>>>> Given 30% are still on Scala 2.10, I'd say we should officially
>>>> deprecate Scala 2.10 in Spark 2.2 and remove the support in a future
>>>> release (e.g. 2.3). Note that in the past we only deprecated Java 7 /
>>>> Python 2.6 in 2.0, and didn't do anything with Scala 2.10.
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Mar 6, 2017 at 1:18 AM, Sean Owen <so...@cloudera.com> wrote:
>>>>
>>>> Another call for comments on removal of Scala 2.10 support, if you
>>>> haven't already. See
>>>>
>>>> https://github.com/apache/spark/pull/17150
>>>> http://issues.apache.org/jira/browse/SPARK-19810
>>>>
>>>> I've heard several votes in support and no specific objections at this
>>>> point, but wanted to make another call to check for any doubts before I go
>>>> ahead for Spark 2.2.
>>>>
>>>>
>>>>
>>
>
Re: RFC: removing Scala 2.10
Posted by Reynold Xin <rx...@databricks.com>.
SGTM.
On Sat, Jul 8, 2017 at 12:38 AM Sean Owen <so...@cloudera.com> wrote:
> I'd like to go ahead and merge SPARK-19810 after the Spark 2.2.0 release
> is fully done and dusted. Would anyone object at this stage, to removing
> 2.10 support?
>
> On Tue, Mar 7, 2017 at 3:55 PM Mark Hamstra <ma...@clearstorydata.com>
> wrote:
>
>> Sorry, for some reason I was thinking that we have branch-2.2 cut
>> already. If we're not going to pull Scala 2.10 out of 2.2.0, then we should
>> wait at least until that branch is cut before we pull it out of master --
>> but I'd still argue for not long after that so that the 2.12 work can start.
>>
>> On Tue, Mar 7, 2017 at 7:50 AM, Mark Hamstra <ma...@clearstorydata.com>
>> wrote:
>>
>>> I don't think 2.10 and 2.12 support can coexist, and soon, 2.12 support
>>>> will be important.
>>>>
>>>
>>> Which is why I'm thinking that we should pull 2.10 support out of master
>>> soon -- either immediately or right after 2.2 goes into RC or full release.
>>>
>>> On Tue, Mar 7, 2017 at 2:14 AM, Sean Owen <so...@cloudera.com> wrote:
>>>
>>>> (2.10 was already deprecated for 2.1, so that's done actually.)
>>>>
>>>> Personally I'm fine with leaving in 2.10 support for 2.2. (FWIW CDH is
>>>> Scala 2.11-only for Spark 2.) If there were no voices in support of keeping
>>>> it, might be worth moving on right now, but if there's any substantive
>>>> argument against, I'd also punt it another release.
>>>>
>>>> It's not really driven by cleanup, though that's nice, but 2.12
>>>> support. I don't think 2.10 and 2.12 support can coexist, and soon, 2.12
>>>> support will be important.
>>>>
>>>> How about tagging this for 2.3.0, as well as targeting 2.12 support for
>>>> 2.3.0?
>>>>
>>>> On Tue, Mar 7, 2017 at 2:38 AM Reynold Xin <rx...@databricks.com> wrote:
>>>>
>>>>> Thanks for sending an email. I was going to +1 but then I figured I
>>>>> should be data driven. I took a look at the distribution of Scala versions
>>>>> across all the clusters Databricks runs (which is a very high number across
>>>>> a variety of tech startups, SMBs, large enterprises, and this is the chart:
>>>>>
>>>>> [image: scala-version.png]
>>>>>
>>>>>
>>>>>
>>>>> Given 30% are still on Scala 2.10, I'd say we should officially
>>>>> deprecate Scala 2.10 in Spark 2.2 and remove the support in a future
>>>>> release (e.g. 2.3). Note that in the past we only deprecated Java 7 /
>>>>> Python 2.6 in 2.0, and didn't do anything with Scala 2.10.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Mar 6, 2017 at 1:18 AM, Sean Owen <so...@cloudera.com> wrote:
>>>>>
>>>>> Another call for comments on removal of Scala 2.10 support, if you
>>>>> haven't already. See
>>>>>
>>>>> https://github.com/apache/spark/pull/17150
>>>>> http://issues.apache.org/jira/browse/SPARK-19810
>>>>>
>>>>> I've heard several votes in support and no specific objections at this
>>>>> point, but wanted to make another call to check for any doubts before I go
>>>>> ahead for Spark 2.2.
>>>>>
>>>>>
>>>>>
>>>
>>
Re: RFC: removing Scala 2.10
Posted by Russell Spitzer <ru...@gmail.com>.
Look at the date of the first email with the chart :)
On Sat, Jul 8, 2017 at 4:42 PM Chris Fregly <ch...@fregly.com> wrote:
> @reynold: why does this chart end in february?
>
> we're well into july. what is the recent breakdown?
>
> *Chris Fregly*
> Research Engineer @ *PipelineAI*
> Founder @ *Advanced Spark and TensorFlow Meetup*
> *San Francisco - Chicago - Washington DC - London*
>
> On Jul 8, 2017, 9:01 AM -0700, Sean Owen <so...@cloudera.com>, wrote:
>
> I'd like to go ahead and merge SPARK-19810 after the Spark 2.2.0 release
> is fully done and dusted. Would anyone object at this stage, to removing
> 2.10 support?
>
> On Tue, Mar 7, 2017 at 3:55 PM Mark Hamstra <ma...@clearstorydata.com>
> wrote:
>
>> Sorry, for some reason I was thinking that we have branch-2.2 cut
>> already. If we're not going to pull Scala 2.10 out of 2.2.0, then we should
>> wait at least until that branch is cut before we pull it out of master --
>> but I'd still argue for not long after that so that the 2.12 work can start.
>>
>> On Tue, Mar 7, 2017 at 7:50 AM, Mark Hamstra <ma...@clearstorydata.com>
>> wrote:
>>
>>> I don't think 2.10 and 2.12 support can coexist, and soon, 2.12 support
>>>> will be important.
>>>>
>>>
>>> Which is why I'm thinking that we should pull 2.10 support out of master
>>> soon -- either immediately or right after 2.2 goes into RC or full release.
>>>
>>> On Tue, Mar 7, 2017 at 2:14 AM, Sean Owen <so...@cloudera.com> wrote:
>>>
>>>> (2.10 was already deprecated for 2.1, so that's done actually.)
>>>>
>>>> Personally I'm fine with leaving in 2.10 support for 2.2. (FWIW CDH is
>>>> Scala 2.11-only for Spark 2.) If there were no voices in support of keeping
>>>> it, might be worth moving on right now, but if there's any substantive
>>>> argument against, I'd also punt it another release.
>>>>
>>>> It's not really driven by cleanup, though that's nice, but 2.12
>>>> support. I don't think 2.10 and 2.12 support can coexist, and soon, 2.12
>>>> support will be important.
>>>>
>>>> How about tagging this for 2.3.0, as well as targeting 2.12 support for
>>>> 2.3.0?
>>>>
>>>> On Tue, Mar 7, 2017 at 2:38 AM Reynold Xin <rx...@databricks.com> wrote:
>>>>
>>>>> Thanks for sending an email. I was going to +1 but then I figured I
>>>>> should be data driven. I took a look at the distribution of Scala versions
>>>>> across all the clusters Databricks runs (which is a very high number across
>>>>> a variety of tech startups, SMBs, large enterprises, and this is the chart:
>>>>>
>>>>> [image: scala-version.png]
>>>>>
>>>>>
>>>>>
>>>>> Given 30% are still on Scala 2.10, I'd say we should officially
>>>>> deprecate Scala 2.10 in Spark 2.2 and remove the support in a future
>>>>> release (e.g. 2.3). Note that in the past we only deprecated Java 7 /
>>>>> Python 2.6 in 2.0, and didn't do anything with Scala 2.10.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Mar 6, 2017 at 1:18 AM, Sean Owen <so...@cloudera.com> wrote:
>>>>>
>>>>> Another call for comments on removal of Scala 2.10 support, if you
>>>>> haven't already. See
>>>>>
>>>>> https://github.com/apache/spark/pull/17150
>>>>> http://issues.apache.org/jira/browse/SPARK-19810
>>>>>
>>>>> I've heard several votes in support and no specific objections at this
>>>>> point, but wanted to make another call to check for any doubts before I go
>>>>> ahead for Spark 2.2.
>>>>>
>>>>>
>>>>>
>>>
>>
Re: RFC: removing Scala 2.10
Posted by Chris Fregly <ch...@fregly.com>.
@reynold: why does this chart end in february?
we're well into july. what is the recent breakdown?
Chris Fregly
Research Engineer @ PipelineAI
Founder @ Advanced Spark and TensorFlow Meetup
San Francisco - Chicago - Washington DC - London
On Jul 8, 2017, 9:01 AM -0700, Sean Owen <so...@cloudera.com>, wrote:
> I'd like to go ahead and merge SPARK-19810 after the Spark 2.2.0 release is fully done and dusted. Would anyone object at this stage, to removing 2.10 support?
>
> > On Tue, Mar 7, 2017 at 3:55 PM Mark Hamstra <ma...@clearstorydata.com> wrote:
> > > Sorry, for some reason I was thinking that we have branch-2.2 cut already. If we're not going to pull Scala 2.10 out of 2.2.0, then we should wait at least until that branch is cut before we pull it out of master -- but I'd still argue for not long after that so that the 2.12 work can start.
> > >
> > > > On Tue, Mar 7, 2017 at 7:50 AM, Mark Hamstra <ma...@clearstorydata.com> wrote:
> > > > > > I don't think 2.10 and 2.12 support can coexist, and soon, 2.12 support will be important.
> > > > >
> > > > > Which is why I'm thinking that we should pull 2.10 support out of master soon -- either immediately or right after 2.2 goes into RC or full release.
> > > > >
> > > > > > On Tue, Mar 7, 2017 at 2:14 AM, Sean Owen <so...@cloudera.com> wrote:
> > > > > > > (2.10 was already deprecated for 2.1, so that's done actually.)
> > > > > > >
> > > > > > > Personally I'm fine with leaving in 2.10 support for 2.2. (FWIW CDH is Scala 2.11-only for Spark 2.) If there were no voices in support of keeping it, might be worth moving on right now, but if there's any substantive argument against, I'd also punt it another release.
> > > > > > >
> > > > > > > It's not really driven by cleanup, though that's nice, but 2.12 support. I don't think 2.10 and 2.12 support can coexist, and soon, 2.12 support will be important.
> > > > > > >
> > > > > > > How about tagging this for 2.3.0, as well as targeting 2.12 support for 2.3.0?
> > > > > > >
> > > > > > > > On Tue, Mar 7, 2017 at 2:38 AM Reynold Xin <rx...@databricks.com> wrote:
> > > > > > > > > Thanks for sending an email. I was going to +1 but then I figured I should be data driven. I took a look at the distribution of Scala versions across all the clusters Databricks runs (which is a very high number across a variety of tech startups, SMBs, large enterprises, and this is the chart:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Given 30% are still on Scala 2.10, I'd say we should officially deprecate Scala 2.10 in Spark 2.2 and remove the support in a future release (e.g. 2.3). Note that in the past we only deprecated Java 7 / Python 2.6 in 2.0, and didn't do anything with Scala 2.10.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Mon, Mar 6, 2017 at 1:18 AM, Sean Owen <so...@cloudera.com> wrote:
> > > > > > > > > > Another call for comments on removal of Scala 2.10 support, if you haven't already. See
> > > > > > > > > >
> > > > > > > > > > https://github.com/apache/spark/pull/17150
> > > > > > > > > > http://issues.apache.org/jira/browse/SPARK-19810
> > > > > > > > > >
> > > > > > > > > > I've heard several votes in support and no specific objections at this point, but wanted to make another call to check for any doubts before I go ahead for Spark 2.2.
> > > > > > > > >
> > > > >
> > >