You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Holden Karau <ho...@pigscanfly.ca> on 2019/09/02 02:04:12 UTC
Re: [DISCUSSION]JDK11 for Apache 2.x?
So I'm not a lawyer, but looking at the JDK8 License FAQ (
https://www.oracle.com/technetwork/java/javase/overview/oracle-jdk-faqs.html )
makes me feel like we should support folks who are on the 2.x branch and
want to migrate away from JDK8. Although the Redhat post about OpenJDK8 is
reassuring (
https://developers.redhat.com/blog/2019/05/22/openjdk-8-and-11-still-in-safe-hands/
).
On Tue, Aug 27, 2019 at 2:22 PM Sean Owen <sr...@gmail.com> wrote:
> Spark 3 will not require Java 11; it will work with Java 8 too. I
> think the question is whether someone who _wants_ Java 11 should have
> a 2.x release instead of 3.0.
>
> In practice... how much are people moving off Java 8 right now? It's
> still my default, because most but not all things work with 11. (Here,
> I'm using 11 as shorthand for 9+; it's the LTS release in the 9 to 11
> series).
> I get that the issue is support, but it seems that Oracle is still
> providing public updates through Dec 2020. I think it's that
> commercial support is now for-pay? And OpenJDK 8 still gets updates?
> that I'm unclear on.
>
> Not that those aren't legitimate concerns, I'm just trying to figure
> out how much need vs want there is and why out there.
>
>
> On Tue, Aug 27, 2019 at 4:15 PM Jean Georges Perrin <jg...@jgp.net> wrote:
> >
> > Not a contributor, but a user perspective…
> >
> > As Spark 3.x will be an evolution, I am not completely shocked that it
> would imply a Java 11 requirement as well. Would be great to have both Java
> 8 and Java 11, but one needs to be able to say goodbye. Java 8 is great,
> still using it actively in production, but we know its time is limited, so,
> by the time we evolve to Spark 3, we could combine it with Java 11.
> >
> > On the other hand, not everybody may think this way and it may slow down
> the adoption of Spark 3…
> >
> > However, I concur with Sean, I don’t think another 2.x is needed for
> Java 11.
> >
> > > On Aug 27, 2019, at 3:09 PM, Sean Owen <sr...@gmail.com> wrote:
> > >
> > > I think one of the key problems here are the required dependency
> > > upgrades. It would mean many minor breaking changes and a few bigger
> > > ones, notably around Hive, and forces a scala 2.12-only update. I
> > > think my question is whether that even makes sense as a minor release?
> > > it wouldn't be backwards compatible with 2.4 enough to call it a
> > > low-risk update. It would be a smaller step than moving all the way to
> > > 3.0, sure. I am not super against it, but we have to keep in mind how
> > > much work it would then be to maintain two LTS 2.x releases, 2.4 and
> > > the sort-of-compatible 2.5, while proceeding with 3.x.
> > >
> > > On Tue, Aug 27, 2019 at 2:01 PM DB Tsai <d_...@apple.com.invalid>
> wrote:
> > >>
> > >> Hello everyone,
> > >>
> > >> Thank you all for working on supporting JDK11 in Apache Spark 3.0 as
> a community.
> > >>
> > >> Java 8 is already end of life for commercial users, and many
> companies are moving to Java 11.
> > >> The release date for Apache Spark 3.0 is still not there yet, and
> there are many API
> > >> incompatibility issues when upgrading from Spark 2.x. As a result,
> asking users to move to
> > >> Spark 3.0 to use JDK 11 is not realistic.
> > >>
> > >> Should we backport PRs for JDK11 and cut a release in 2.x to support
> JDK11?
> > >>
> > >> Should we cut a new Apache Spark 2.5 since the patches involve some
> of the dependencies changes
> > >> which is not desired in minor release?
> > >>
> > >> Thanks.
> > >>
> > >> DB Tsai | Siri Open Source Technologies [not a contribution] |
> Apple, Inc
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
> > >>
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
> > >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
> >
>
> ---------------------------------------------------------------------
> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>
>
--
Twitter: https://twitter.com/holdenkarau
Books (Learning Spark, High Performance Spark, etc.):
https://amzn.to/2MaRAG9 <https://amzn.to/2MaRAG9>
YouTube Live Streams: https://www.youtube.com/user/holdenkarau
Re: [DISCUSSION]JDK11 for Apache 2.x?
Posted by Sean Owen <sr...@gmail.com>.
That's just the Oracle 'commercial' JDK though. OpenJDK is still OSS
and 8 is updated there too. Oracle JDK is still free too, just not for
commercial use and commercial support for 8 (IIUC). I think that's the
'audience' here, and I don't know how large or small that is.
That said a key first question is simply, how much breaking change is
necessary to get 2.x to work on JDK 11? If it's large I think that
might resolve the question right there. SPARK-24417 has most but not
necessarily all necessary changes under it.
Heck it could be maintained as a separate community project too, if
there is both interest from some parties to make it happen, but not
enough to commit to maintain all the backports and updated from 3.x.
On Sun, Sep 1, 2019 at 9:04 PM Holden Karau <ho...@pigscanfly.ca> wrote:
>
> So I'm not a lawyer, but looking at the JDK8 License FAQ ( https://www.oracle.com/technetwork/java/javase/overview/oracle-jdk-faqs.html ) makes me feel like we should support folks who are on the 2.x branch and want to migrate away from JDK8. Although the Redhat post about OpenJDK8 is reassuring ( https://developers.redhat.com/blog/2019/05/22/openjdk-8-and-11-still-in-safe-hands/ ).
>
> On Tue, Aug 27, 2019 at 2:22 PM Sean Owen <sr...@gmail.com> wrote:
>>
>> Spark 3 will not require Java 11; it will work with Java 8 too. I
>> think the question is whether someone who _wants_ Java 11 should have
>> a 2.x release instead of 3.0.
>>
>> In practice... how much are people moving off Java 8 right now? It's
>> still my default, because most but not all things work with 11. (Here,
>> I'm using 11 as shorthand for 9+; it's the LTS release in the 9 to 11
>> series).
>> I get that the issue is support, but it seems that Oracle is still
>> providing public updates through Dec 2020. I think it's that
>> commercial support is now for-pay? And OpenJDK 8 still gets updates?
>> that I'm unclear on.
>>
>> Not that those aren't legitimate concerns, I'm just trying to figure
>> out how much need vs want there is and why out there.
>>
>>
>> On Tue, Aug 27, 2019 at 4:15 PM Jean Georges Perrin <jg...@jgp.net> wrote:
>> >
>> > Not a contributor, but a user perspective…
>> >
>> > As Spark 3.x will be an evolution, I am not completely shocked that it would imply a Java 11 requirement as well. Would be great to have both Java 8 and Java 11, but one needs to be able to say goodbye. Java 8 is great, still using it actively in production, but we know its time is limited, so, by the time we evolve to Spark 3, we could combine it with Java 11.
>> >
>> > On the other hand, not everybody may think this way and it may slow down the adoption of Spark 3…
>> >
>> > However, I concur with Sean, I don’t think another 2.x is needed for Java 11.
>> >
>> > > On Aug 27, 2019, at 3:09 PM, Sean Owen <sr...@gmail.com> wrote:
>> > >
>> > > I think one of the key problems here are the required dependency
>> > > upgrades. It would mean many minor breaking changes and a few bigger
>> > > ones, notably around Hive, and forces a scala 2.12-only update. I
>> > > think my question is whether that even makes sense as a minor release?
>> > > it wouldn't be backwards compatible with 2.4 enough to call it a
>> > > low-risk update. It would be a smaller step than moving all the way to
>> > > 3.0, sure. I am not super against it, but we have to keep in mind how
>> > > much work it would then be to maintain two LTS 2.x releases, 2.4 and
>> > > the sort-of-compatible 2.5, while proceeding with 3.x.
>> > >
>> > > On Tue, Aug 27, 2019 at 2:01 PM DB Tsai <d_...@apple.com.invalid> wrote:
>> > >>
>> > >> Hello everyone,
>> > >>
>> > >> Thank you all for working on supporting JDK11 in Apache Spark 3.0 as a community.
>> > >>
>> > >> Java 8 is already end of life for commercial users, and many companies are moving to Java 11.
>> > >> The release date for Apache Spark 3.0 is still not there yet, and there are many API
>> > >> incompatibility issues when upgrading from Spark 2.x. As a result, asking users to move to
>> > >> Spark 3.0 to use JDK 11 is not realistic.
>> > >>
>> > >> Should we backport PRs for JDK11 and cut a release in 2.x to support JDK11?
>> > >>
>> > >> Should we cut a new Apache Spark 2.5 since the patches involve some of the dependencies changes
>> > >> which is not desired in minor release?
>> > >>
>> > >> Thanks.
>> > >>
>> > >> DB Tsai | Siri Open Source Technologies [not a contribution] | Apple, Inc
>> > >>
>> > >>
>> > >> ---------------------------------------------------------------------
>> > >> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>> > >>
>> > >
>> > > ---------------------------------------------------------------------
>> > > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>> > >
>> >
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe e-mail: dev-unsubscribe@spark.apache.org
>>
>
>
> --
> Twitter: https://twitter.com/holdenkarau
> Books (Learning Spark, High Performance Spark, etc.): https://amzn.to/2MaRAG9
> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org