You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spark.apache.org by Sean Owen <sr...@apache.org> on 2020/10/28 20:44:26 UTC
Spark on JDK 14
For kicks, I tried Spark on JDK 14. 11 -> 14 doesn't change much, not as
much as 8 -> 9 (-> 11), and indeed, virtually all tests pass. For the
interested, these two seem to fail:
- ZooKeeperPersistenceEngine *** FAILED ***
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /spark/master_status
- parsing hour with various patterns *** FAILED ***
java.time.format.DateTimeParseException: Text '2009-12-12 12 am' could
not be parsed: Invalid value for HourOfAmPm (valid values 0 - 11): 12
I'd expect that most applications would just work now on Spark 3 + Java 14.
I'd guess the same is true for Java 16 even, but, we're probably focused on
the LTS releases.
Kris Mok pointed out that Project Panama (in Java 17 maybe?) might have
implications as it changes off-heap memory access.
Re: Spark on JDK 14
Posted by Kazuaki Ishizaki <IS...@jp.ibm.com>.
Java 16 will also includes Vector API (incubator), which is a part of
Project Panama, as shown in
https://mail.openjdk.java.net/pipermail/panama-dev/2020-October/011149.html
When the next LTS will be available, we could exploit it in Spark.
Kazuaki Ishizaki
From: Dongjoon Hyun <do...@gmail.com>
To: Sean Owen <sr...@apache.org>
Cc: dev <de...@spark.apache.org>
Date: 2020/10/29 11:34
Subject: Re: Spark on JDK 14
Thank you for the sharing, Sean.
Although Java 14 is already EOL (Sep. 2020), that is important information
because we are tracking the Java upstream.
Bests,
Dongjoon.
On Wed, Oct 28, 2020 at 1:44 PM Sean Owen <sr...@apache.org> wrote:
For kicks, I tried Spark on JDK 14. 11 -> 14 doesn't change much, not as
much as 8 -> 9 (-> 11), and indeed, virtually all tests pass. For the
interested, these two seem to fail:
- ZooKeeperPersistenceEngine *** FAILED ***
org.apache.zookeeper.KeeperException$ConnectionLossException:
KeeperErrorCode = ConnectionLoss for /spark/master_status
- parsing hour with various patterns *** FAILED ***
java.time.format.DateTimeParseException: Text '2009-12-12 12 am' could
not be parsed: Invalid value for HourOfAmPm (valid values 0 - 11): 12
I'd expect that most applications would just work now on Spark 3 + Java
14. I'd guess the same is true for Java 16 even, but, we're probably
focused on the LTS releases.
Kris Mok pointed out that Project Panama (in Java 17 maybe?) might have
implications as it changes off-heap memory access.
Re: Spark on JDK 14
Posted by Dongjoon Hyun <do...@gmail.com>.
Thank you for the sharing, Sean.
Although Java 14 is already EOL (Sep. 2020), that is important information
because we are tracking the Java upstream.
Bests,
Dongjoon.
On Wed, Oct 28, 2020 at 1:44 PM Sean Owen <sr...@apache.org> wrote:
> For kicks, I tried Spark on JDK 14. 11 -> 14 doesn't change much, not as
> much as 8 -> 9 (-> 11), and indeed, virtually all tests pass. For the
> interested, these two seem to fail:
>
> - ZooKeeperPersistenceEngine *** FAILED ***
> org.apache.zookeeper.KeeperException$ConnectionLossException:
> KeeperErrorCode = ConnectionLoss for /spark/master_status
>
> - parsing hour with various patterns *** FAILED ***
> java.time.format.DateTimeParseException: Text '2009-12-12 12 am' could
> not be parsed: Invalid value for HourOfAmPm (valid values 0 - 11): 12
>
> I'd expect that most applications would just work now on Spark 3 + Java
> 14. I'd guess the same is true for Java 16 even, but, we're probably
> focused on the LTS releases.
>
> Kris Mok pointed out that Project Panama (in Java 17 maybe?) might have
> implications as it changes off-heap memory access.
>