You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pulsar.apache.org by Nicolò Boschi <bo...@gmail.com> on 2022/02/14 13:54:14 UTC
JDK 17 runtime compatibility
Dear Pulsar community,
I would like to start a discussion about having a plan to make Pulsar
runtime-compatible with JDK 17.
The goal is to run Pulsar using JDK17 at runtime, benefitting from the
latest JVM performance improvements and features.
The non-goal is to deprecate JDK8 or JDK11 runtime compatibility. The java
compiler *release* flag will be kept to 8 (this change deserves another
discussion thread).
*Build*
At the moment, the build command (mvn clean install -DskipTests) works well
using JDK17.
We fixed a lot of unit tests and other issues found using JDK17 removing
PowerMockito improper usages: https://github.com/apache/pulsar/pull/14098
Other small pulls were merged fixing other kinds of issues (all related to
the tests themself and not the production code).
*Runtime compatibility*
Currently I haven't seen severe issues running Pulsar on JDK17.
For testing purposes, I’ve created a simple image which pulls the Pulsar
master and installs JDK17-zulu instead of JDK11.
I've tried running the docker image in a Kubernetes environment (deploying
it with the Apache Pulsar helm chart), testing a few scenario:
* Produce and consume message (pulsar cli, pulsar perf)
* Functions/Connectors (embedding Nar files)
All the components/pods are working without visible issues. However my
tests weren't under high traffic and a lot of Pulsar features were not
involved in the tests.
The main problems might be caused by the restrictions around reflection
usage introduced in JDK17 JEP 403: Strongly Encapsulate JDK Internals
<https://openjdk.java.net/jeps/403>
We may need to unlock some packages to reflection to keep some
optimizations, for example for BookKeeper Native IO (
https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/util/NativeIO.java#L98
)
It would be great to chase other improvements to be explicitly unlocked for
other low-level libraries (Netty, GRPC...)
*Continuous Integration*
To maintain the support for JDK17 we need to switch all the CI jobs to
JDK17; the idea is to replace JDK11 completely. We could - not sure - keep
a single job for JDK11 compatibility, like we do with JDK8.
*Docker distribution*
After we get the Pulsar master CI stable, we can move the official Docker
images to JDK17 (currently on JDK11).
*Proposal plan*
We can start moving the CI to JDK17 after the branch-2.10 has been cut and
targeting JDK17 runtime support for Pulsar 2.11
WDYT?
BR,
Nicolò Boschi
Re: JDK 17 runtime compatibility
Posted by Heesung Sohn <he...@streamnative.io.INVALID>.
Hi dev,
I am definitely a latecomer to this issue, but I think we could also
support java17 build as well as this java17 runtime support.
I added this java17 build support and created a pip here,
https://github.com/apache/pulsar/issues/15207.
I think I can help with upgrading the build for java17 and adding the
different jdk compatibility tests (if not done yet)
Thank you,
Heesung
On Wed, Apr 20, 2022 at 10:05 AM Dave Fisher <wa...@apache.org> wrote:
> Bumping this thread so it’s current.
>
> > On Feb 14, 2022, at 6:09 AM, Christophe Bornet <bo...@gmail.com>
> wrote:
> >
> > Big +1
> >
> > Le lun. 14 févr. 2022 à 14:54, Nicolò Boschi <bo...@gmail.com> a
> > écrit :
> >
> >> Dear Pulsar community,
> >>
> >>
> >> I would like to start a discussion about having a plan to make Pulsar
> >> runtime-compatible with JDK 17.
> >>
> >>
> >> The goal is to run Pulsar using JDK17 at runtime, benefitting from the
> >> latest JVM performance improvements and features.
> >>
> >>
> >> The non-goal is to deprecate JDK8 or JDK11 runtime compatibility. The
> java
> >> compiler *release* flag will be kept to 8 (this change deserves another
> >> discussion thread).
> >>
> >>
> >> *Build*
> >>
> >>
> >> At the moment, the build command (mvn clean install -DskipTests) works
> well
> >> using JDK17.
> >>
> >>
> >> We fixed a lot of unit tests and other issues found using JDK17 removing
> >> PowerMockito improper usages:
> https://github.com/apache/pulsar/pull/14098
> >>
> >> Other small pulls were merged fixing other kinds of issues (all related
> to
> >> the tests themself and not the production code).
> >>
> >>
> >> *Runtime compatibility*
> >>
> >>
> >> Currently I haven't seen severe issues running Pulsar on JDK17.
> >>
> >> For testing purposes, I’ve created a simple image which pulls the Pulsar
> >> master and installs JDK17-zulu instead of JDK11.
> >>
> >> I've tried running the docker image in a Kubernetes environment
> (deploying
> >> it with the Apache Pulsar helm chart), testing a few scenario:
> >>
> >> * Produce and consume message (pulsar cli, pulsar perf)
> >>
> >> * Functions/Connectors (embedding Nar files)
> >>
> >>
> >> All the components/pods are working without visible issues. However my
> >> tests weren't under high traffic and a lot of Pulsar features were not
> >> involved in the tests.
> >>
> >>
> >> The main problems might be caused by the restrictions around reflection
> >> usage introduced in JDK17 JEP 403: Strongly Encapsulate JDK Internals
> >> <https://openjdk.java.net/jeps/403>
> >>
> >> We may need to unlock some packages to reflection to keep some
> >> optimizations, for example for BookKeeper Native IO (
> >>
> >>
> https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/util/NativeIO.java#L98
> >> )
> >>
> >>
> >> It would be great to chase other improvements to be explicitly unlocked
> for
> >> other low-level libraries (Netty, GRPC...)
> >>
> >>
> >>
> >> *Continuous Integration*
> >>
> >> To maintain the support for JDK17 we need to switch all the CI jobs to
> >> JDK17; the idea is to replace JDK11 completely. We could - not sure -
> keep
> >> a single job for JDK11 compatibility, like we do with JDK8.
> >>
> >>
> >>
> >> *Docker distribution*
> >>
> >> After we get the Pulsar master CI stable, we can move the official
> Docker
> >> images to JDK17 (currently on JDK11).
> >>
> >>
> >>
> >> *Proposal plan*
> >>
> >> We can start moving the CI to JDK17 after the branch-2.10 has been cut
> and
> >> targeting JDK17 runtime support for Pulsar 2.11
> >>
> >>
> >>
> >>
> >> WDYT?
> >>
> >> BR,
> >> Nicolò Boschi
> >>
>
>
Re: JDK 17 runtime compatibility
Posted by Dave Fisher <wa...@apache.org>.
Bumping this thread so it’s current.
> On Feb 14, 2022, at 6:09 AM, Christophe Bornet <bo...@gmail.com> wrote:
>
> Big +1
>
> Le lun. 14 févr. 2022 à 14:54, Nicolò Boschi <bo...@gmail.com> a
> écrit :
>
>> Dear Pulsar community,
>>
>>
>> I would like to start a discussion about having a plan to make Pulsar
>> runtime-compatible with JDK 17.
>>
>>
>> The goal is to run Pulsar using JDK17 at runtime, benefitting from the
>> latest JVM performance improvements and features.
>>
>>
>> The non-goal is to deprecate JDK8 or JDK11 runtime compatibility. The java
>> compiler *release* flag will be kept to 8 (this change deserves another
>> discussion thread).
>>
>>
>> *Build*
>>
>>
>> At the moment, the build command (mvn clean install -DskipTests) works well
>> using JDK17.
>>
>>
>> We fixed a lot of unit tests and other issues found using JDK17 removing
>> PowerMockito improper usages: https://github.com/apache/pulsar/pull/14098
>>
>> Other small pulls were merged fixing other kinds of issues (all related to
>> the tests themself and not the production code).
>>
>>
>> *Runtime compatibility*
>>
>>
>> Currently I haven't seen severe issues running Pulsar on JDK17.
>>
>> For testing purposes, I’ve created a simple image which pulls the Pulsar
>> master and installs JDK17-zulu instead of JDK11.
>>
>> I've tried running the docker image in a Kubernetes environment (deploying
>> it with the Apache Pulsar helm chart), testing a few scenario:
>>
>> * Produce and consume message (pulsar cli, pulsar perf)
>>
>> * Functions/Connectors (embedding Nar files)
>>
>>
>> All the components/pods are working without visible issues. However my
>> tests weren't under high traffic and a lot of Pulsar features were not
>> involved in the tests.
>>
>>
>> The main problems might be caused by the restrictions around reflection
>> usage introduced in JDK17 JEP 403: Strongly Encapsulate JDK Internals
>> <https://openjdk.java.net/jeps/403>
>>
>> We may need to unlock some packages to reflection to keep some
>> optimizations, for example for BookKeeper Native IO (
>>
>> https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/util/NativeIO.java#L98
>> )
>>
>>
>> It would be great to chase other improvements to be explicitly unlocked for
>> other low-level libraries (Netty, GRPC...)
>>
>>
>>
>> *Continuous Integration*
>>
>> To maintain the support for JDK17 we need to switch all the CI jobs to
>> JDK17; the idea is to replace JDK11 completely. We could - not sure - keep
>> a single job for JDK11 compatibility, like we do with JDK8.
>>
>>
>>
>> *Docker distribution*
>>
>> After we get the Pulsar master CI stable, we can move the official Docker
>> images to JDK17 (currently on JDK11).
>>
>>
>>
>> *Proposal plan*
>>
>> We can start moving the CI to JDK17 after the branch-2.10 has been cut and
>> targeting JDK17 runtime support for Pulsar 2.11
>>
>>
>>
>>
>> WDYT?
>>
>> BR,
>> Nicolò Boschi
>>
Re: JDK 17 runtime compatibility
Posted by Christophe Bornet <bo...@gmail.com>.
Big +1
Le lun. 14 févr. 2022 à 14:54, Nicolò Boschi <bo...@gmail.com> a
écrit :
> Dear Pulsar community,
>
>
> I would like to start a discussion about having a plan to make Pulsar
> runtime-compatible with JDK 17.
>
>
> The goal is to run Pulsar using JDK17 at runtime, benefitting from the
> latest JVM performance improvements and features.
>
>
> The non-goal is to deprecate JDK8 or JDK11 runtime compatibility. The java
> compiler *release* flag will be kept to 8 (this change deserves another
> discussion thread).
>
>
> *Build*
>
>
> At the moment, the build command (mvn clean install -DskipTests) works well
> using JDK17.
>
>
> We fixed a lot of unit tests and other issues found using JDK17 removing
> PowerMockito improper usages: https://github.com/apache/pulsar/pull/14098
>
> Other small pulls were merged fixing other kinds of issues (all related to
> the tests themself and not the production code).
>
>
> *Runtime compatibility*
>
>
> Currently I haven't seen severe issues running Pulsar on JDK17.
>
> For testing purposes, I’ve created a simple image which pulls the Pulsar
> master and installs JDK17-zulu instead of JDK11.
>
> I've tried running the docker image in a Kubernetes environment (deploying
> it with the Apache Pulsar helm chart), testing a few scenario:
>
> * Produce and consume message (pulsar cli, pulsar perf)
>
> * Functions/Connectors (embedding Nar files)
>
>
> All the components/pods are working without visible issues. However my
> tests weren't under high traffic and a lot of Pulsar features were not
> involved in the tests.
>
>
> The main problems might be caused by the restrictions around reflection
> usage introduced in JDK17 JEP 403: Strongly Encapsulate JDK Internals
> <https://openjdk.java.net/jeps/403>
>
> We may need to unlock some packages to reflection to keep some
> optimizations, for example for BookKeeper Native IO (
>
> https://github.com/apache/bookkeeper/blob/master/bookkeeper-server/src/main/java/org/apache/bookkeeper/util/NativeIO.java#L98
> )
>
>
> It would be great to chase other improvements to be explicitly unlocked for
> other low-level libraries (Netty, GRPC...)
>
>
>
> *Continuous Integration*
>
> To maintain the support for JDK17 we need to switch all the CI jobs to
> JDK17; the idea is to replace JDK11 completely. We could - not sure - keep
> a single job for JDK11 compatibility, like we do with JDK8.
>
>
>
> *Docker distribution*
>
> After we get the Pulsar master CI stable, we can move the official Docker
> images to JDK17 (currently on JDK11).
>
>
>
> *Proposal plan*
>
> We can start moving the CI to JDK17 after the branch-2.10 has been cut and
> targeting JDK17 runtime support for Pulsar 2.11
>
>
>
>
> WDYT?
>
> BR,
> Nicolò Boschi
>