[DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Christopher <ct...@apache.org>]

Hi ZK Devs,

With recent advancements in Java (since Java 9), it is now generally
no longer necessary to require that software be developed on an older
JDK in order to have confidence that it will run on the older version
of Java. This is because, as of Java 9, all JDK releases have better
support for cross-compilation to older Java versions.

What this means is that developers can confidently make the build
requirements for a project higher than the Java version that will
actually be supported at runtime.

In fact, ZooKeeper already supports the necessary flags in its Maven
build configuration to ensure that it uses JDK 8 compliance when
building on a newer JDK (I added this way back in ZOOKEEPER-3739 /
https://github.com/apache/zookeeper/pull/1269)

So, I propose that we make JDK 11 the new minimum version to *build*
ZooKeeper with. This would not change the runtime requirement, which
would remain at JDK 8.

The only necessary change to make this happen would be to add the
minimum Java version to the maven-enforcer-plugin (like
https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164)

This would allow ZooKeeper to to streamline its development process a
little bit by reducing the amount of CI testing that is done as part
of the build. In other words, we can drop the CI builds for JDK 8,
which saves on build resources and time. The return on investment is
so low for the JDK 8 builds anyway, because of the improved
cross-compilation in newer JDKs. So, there's not much value in
building on JDK 8 anyway.

Of course, I am only recommending this for *new* release lines,
starting with ZooKeeper 3.7.0/master branch, because I would not want
to change expectations for users who will build their own 3.5 and 3.6
versions as they continue to have patch versions released.

What do you think?

Kind Regards,
Christopher

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Patrick Hunt <ph...@apache.org>]

On Wed, Oct 21, 2020 at 8:04 AM Brent <br...@gmail.com> wrote:

> As a slightly different consideration, if you look at the Long-Term Support
> (LTS) roadmaps for Java, currently Java 8 is set to have full support until
> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
>
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> https://en.wikipedia.org/wiki/Java_version_history
>
> My guess is that a number of companies are still heavily invested at the
> Java 8 level (I know a few) and with that kind of time horizon, they have
> no real motivation to upgrade for quite a while.  If the recent Python 2
> deprecation is anything to go by, they won't do it until they have to.
>
> Not saying Java 8 isn't *very* old (2014 release it seems like?)  and I'm
> not invested heavily either way, but this might suggest that ZK may end up
> with a lot of users potentially locking themselves to 3.6.x for a while as
> Enrico mentioned.
>
> (Not a major contributor, but wanted to chime in since I just had this
> conversation with a bunch of people professionally recently)
>
>
I had similar thoughts as to what Brent has mentioned. Also there are
reports such as this one from inforworld in May of this year:
https://www.infoworld.com/article/3532358/oracle-extends-extended-support-for-java-8.html

"64 percent of Java users polled are using Java SE (Standard Edition) 8 for
their main application in production"

Given Oracle themselves have seen fit to continue supporting java8 I don't
see how we should drop it. In many cases there may be a corporate mandate
in place which keeps folks from utilizing a newer version - iow not a
technical decision.

Regards,

Patrick



> Brent
>
> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <an...@apache.org> wrote:
>
> > Thanks for the summary.
> >
> > I still vote for option 1). Move 3.7.0 to JDK 11 fully. Other projects
> > will upgrade once they’re JDK11 compliant, otherwise they will stay on
> 3.5
> > or 3.6. Both version are quite recent in ZooKeeper-terms, we already
> > planned big changes for 3.7.0 and JDK 11 could be one of them.
> >
> > Don’t put extra burden on the ZK community to help others staying on
> > ancient Java versions.
> >
> > Andor
> >
> >
> >
> > > On 2020. Oct 21., at 10:57, Enrico Olivelli <eo...@gmail.com>
> wrote:
> > >
> > > Let me recap
> > > - Christopher is proposing to move to JDK11
> > > - ZooKeeper client and server are bundled and coded and tested together
> > in
> > > zookeeper-server
> > > - Enrico is concerned about the need of testing ZooKeeper client on
> JDK8
> > > (not a problem to move the server to JDK11)
> > >
> > > ZooKeeper client is used by tons of users and unfortunately many
> projects
> > > are still on JDK8, if we move ZooKeeper to JDK11 the risk is to block
> > users
> > > from the adoption,
> > > that is that we will see the world to stay on 3.6.x and we will have
> > again
> > > a long lived release line, like 3.4.
> > >
> > > Testing the client on JDK8 would be possible if we create some kind of
> > > additional module with system tests, then we can start the server on
> > docker
> > > on JDK11+ and start a client on JDK8
> > > with Maven toolchain it should possible to run surefire tests using a
> > > separate JVM.
> > >
> > > So in my vision 2 options:
> > > 1) fully JDK11 - drop JDK8 at all
> > > 2) build with JDK11 - server only on JDK11 - add system tests with
> docker
> > > and toolchains that ensure the ZooKeeper client (and all dependencies)
> > > still work on JDK8
> > >
> > > From my point of view about the ZooKeeper ecosystem option 2) will be
> far
> > > better, but we need resources to work on a new test suite.
> > >
> > > Enrico
> > >
> > >
> > > Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> andor@apache.org>
> > ha
> > > scritto:
> > >
> > >> Tamas, Enrico,
> > >>
> > >> Sorry I don’t follow. Why do we have to test the client with JDK 8 in
> > >> version 3.7.0?
> > >>
> > >> Andor
> > >>
> > >>
> > >>
> > >>
> > >>> On 2020. Oct 20., at 22:29, Tamas Penzes <tamaas@cloudera.com.INVALID
> >
> > >> wrote:
> > >>>
> > >>> Hi Enrico,
> > >>>
> > >>> Separating ZooKeeper client and server is a huge work, but we might
> not
> > >>> need it.
> > >>> As you mentioned we have to test ZK client with Java 8, what about
> > >>> separating only the test cases which we need to run with Java8 too?
> > >>> In Curator we have the ZK compatibility tests where we run a limited
> > >> amount
> > >>> of Curator's jUnit tests with a different ZK version.
> > >>> We might be able to do the same here, tag tests which are testing ZK
> > >> client
> > >>> and run them separately with Java 8. The only limitation is that
> these
> > >>> tests must stay JDK8 compatible.
> > >>> But from the tags we will see which ones are those.
> > >>>
> > >>> What do you think?
> > >>>
> > >>> Regards, Tamaas
> > >>>
> > >>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <eolivelli@gmail.com
> >
> > >> wrote:
> > >>>
> > >>>> Christopher
> > >>>> I appreciate your idea and I also moved lots of my projects to work
> > the
> > >> way
> > >>>> you are suggesting.
> > >>>>
> > >>>>
> > >>>> We must run tests using real jdk8 to test the Zookeeper client. We
> > must
> > >>>> ensure that Zookeeper works well, especially while dealing with
> > security
> > >>>> stuff.
> > >>>> Currently the client is in the same module of the server and it will
> > >> take a
> > >>>> good (huge) amount of work to separate them
> > >>>>
> > >>>>
> > >>>> Enrico
> > >>>>
> > >>>> Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org> ha
> > scritto:
> > >>>>
> > >>>>> Hi ZK Devs,
> > >>>>>
> > >>>>> With recent advancements in Java (since Java 9), it is now
> generally
> > >>>>> no longer necessary to require that software be developed on an
> older
> > >>>>> JDK in order to have confidence that it will run on the older
> version
> > >>>>> of Java. This is because, as of Java 9, all JDK releases have
> better
> > >>>>> support for cross-compilation to older Java versions.
> > >>>>>
> > >>>>> What this means is that developers can confidently make the build
> > >>>>> requirements for a project higher than the Java version that will
> > >>>>> actually be supported at runtime.
> > >>>>>
> > >>>>> In fact, ZooKeeper already supports the necessary flags in its
> Maven
> > >>>>> build configuration to ensure that it uses JDK 8 compliance when
> > >>>>> building on a newer JDK (I added this way back in ZOOKEEPER-3739 /
> > >>>>> https://github.com/apache/zookeeper/pull/1269)
> > >>>>>
> > >>>>> So, I propose that we make JDK 11 the new minimum version to
> *build*
> > >>>>> ZooKeeper with. This would not change the runtime requirement,
> which
> > >>>>> would remain at JDK 8.
> > >>>>>
> > >>>>> The only necessary change to make this happen would be to add the
> > >>>>> minimum Java version to the maven-enforcer-plugin (like
> > >>>>>
> > >>>>>
> > >>>>
> > >>
> >
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> > >>>>> )
> > >>>>>
> > >>>>> This would allow ZooKeeper to to streamline its development
> process a
> > >>>>> little bit by reducing the amount of CI testing that is done as
> part
> > >>>>> of the build. In other words, we can drop the CI builds for JDK 8,
> > >>>>> which saves on build resources and time. The return on investment
> is
> > >>>>> so low for the JDK 8 builds anyway, because of the improved
> > >>>>> cross-compilation in newer JDKs. So, there's not much value in
> > >>>>> building on JDK 8 anyway.
> > >>>>>
> > >>>>> Of course, I am only recommending this for *new* release lines,
> > >>>>> starting with ZooKeeper 3.7.0/master branch, because I would not
> want
> > >>>>> to change expectations for users who will build their own 3.5 and
> 3.6
> > >>>>> versions as they continue to have patch versions released.
> > >>>>>
> > >>>>> What do you think?
> > >>>>>
> > >>>>> Kind Regards,
> > >>>>> Christopher
> > >>>>>
> > >>>>
> > >>
> > >>
> >
> >
>

R: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Alessandro Luccaroni - Diennea <al...@diennea.com.INVALID>]

I'm not talking on behalf of the Hadoop project but from what Temas reported few days ago it seems that Hadoop is indeed in good shape to support Java 11 but it's not in the position to drop JDK8 any time soon since they just dropped JDK7 in Hadoop 3 (and the majority of the user base is still on Hadoop 2).
If ZK drops JDK8 support I expect that Hadoop will be stuck on an older version of ZK for some time.

I don't have hard numbers but, from what I can see around me, I expect that the Hadoop userbase is a good chunk of the ZK users in the wild: if the Zookeeper team is OK in maintaining ZK 3.6 for as long as Hadoop 3 is mainteined I don't see any issue.

Alessandro

-----Messaggio originale-----
Da: Andor Molnar <an...@apache.org>
Inviato: lunedì 26 ottobre 2020 14:09
A: DevZooKeeper <de...@zookeeper.apache.org>
Oggetto: Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

I can only echo what Chris said in his latest e-mail:

3.7 will be a future release of ZooKeeper and we should look forward what’s coming with Java versions at that time (there probably will be at least 3 stable versions to support). Though I like the idea of “passive support”: I don’t see any harm in fixing Java 8 related bugs, but I would be also happy to make Java 11 features available for contributors.

From what you said even Hadoop is in good shape for Java 11 readiness. What’s (will be) the problem here?

Andor



> On 2020. Oct 22., at 16:17, Alessandro Luccaroni - Diennea <al...@diennea.com.INVALID> wrote:
>
> I know of projects that embed Zookeeper so I don't think it is (only) a matter of run/build time.
> I like what Mate said about announcing this switch today for a later major release (such as 4.0 that could come after 3.7) to give time to downstream applications to plan for this change.
>
> Regards,
> Alessandro Luccaroni
>
> -----Messaggio originale-----
> Da: Ted Dunning <te...@gmail.com>
> Inviato: giovedì 22 ottobre 2020 15:58
> A: dev@zookeeper.apache.org
> Oggetto: Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7
>
> Allesandro,
>
> Would your answer be different if you considered a strategy that
> allows
> JRE8 to run ZK but requires JDK11 to build it?
>
>
>
> On Thu, Oct 22, 2020 at 6:36 AM Szalay-Bekő Máté
> <sz...@gmail.com>
> wrote:
>
>> Hello All,
>>
>> A few reflections:
>>
>> - I don't think that backporting fixes from a JDK 11 only version to
>> a JDK
>> 8 compatible version would be necessarily a harder thing than any
>> regular backport. It kind of depends on us (whether we use many JDK
>> 11 only features or not until we drop JDK 8 from all supported versions).
>> Also if it gets more painful, then we can decide to limit the number
>> of backported commits (e.g. only strictly to security fixes / CVEs)
>> - But even if the above point is true (?), I would not do this cut (e.g.
>> moving entirely to JDK 11 only) in a minor release. I think this
>> should be a major change in 4.0. (maybe together with other more
>> "risky" changes, like the separation of the client and server
>> artifacts/code; or some incompatible changes in the leader election
>> protocol. Although these are separate discussions)
>> - So all-in-all I like the JDK 11 only option the most. But I
>> wouldn't do it in 3.7 (which might happen very soon), but rather in
>> 4.0. The question for me is wether to do any in-between step in 3.x.
>> (like the options 1 or 2 above in Christopher's mail). I think it
>> mainly should depend on the timing of 4.0.
>>
>> Kind regards,
>> Mate
>>
>> On Thu, Oct 22, 2020 at 3:05 PM Flavio Junqueira <fp...@apache.org> wrote:
>>
>>> There are three points that stand out for me in this thread:
>>>
>>> - How do we determine how such a change affects our user base?
>>> - How much effort do the different options induce with respect to
>>> maintenance?
>>> - What's the right timeline for changes and how do we communicate
>>> them so that our users have enough time to prepare?
>>>
>>> Someone mentioned a PMC vote, and I don't think this should be a
>>> closed vote, independent of how the conversation goes.
>>>
>>> -Flavio
>>>
>>>> On 22 Oct 2020, at 08:39, Alessandro Luccaroni - Diennea
>>> <al...@diennea.com.INVALID> wrote:
>>>>
>>>> Hi all,
>>>> If I might chime in as a zookeeper user (in multiple products) and
>>>> a
>>> follower of the project I think the drop of Java8 support (official
>> and/or
>>> unofficial) could be a big mistake.
>>>>
>>>> From my own company point of view we already support Java11 in all
>>>> our
>>> applications so we are not directly impacted (and we have upgrade
>>> path
>> for
>>> older versions to provide to our customers).
>>>> My worries resides in the (high) probability of a userbase
>>> fragmentation: in the recent past Zookeeper development picked up
>>> speed thanks to a bunch of new committers and PMCs after a period of
>>> mostly maintenance focused works, but the number of active
>>> committers and PMCs
>> is
>>> still very low for a project like this.
>>>>
>>>> I foresee the risk of spreading thin the resources of the project
>>>> if we
>>> force the userbase to stick to an older version and, in turn, we are
>> forced
>>> to backport many issue to the 3.6 branch.
>>>>
>>>> Alessandro Luccaroni
>>>> Platform Manager @ Diennea - MagNews
>>>> Tel.: (+39) 0546 066100 Int. 924 - Mob.: (+39) 393 7273519 Viale
>>>> G.Marconi 30/14 - 48018 Faenza (RA) - Italy
>>>>
>>>> -----Messaggio originale-----
>>>> Da: Christopher <ct...@apache.org>
>>>> Inviato: giovedì 22 ottobre 2020 05:21
>>>> A: dev@zookeeper.apache.org
>>>> Oggetto: Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7
>>>>
>>>> I'm happy that this discussion has been so lively! I just want to
>>> emphasize a few things:
>>>>
>>>> I really do understand the desire to continue to support Java 8...
>>>> I
>> get
>>> it. But all the conversations around this seem based on what people
>>> are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
>>>> *future* release... so it should be based more on reasonable
>>> expectations for users in the future, and less based on what is
>>> happening today. I suspect *most* people today are still using 3.4
>>> anyway (it was just so stable for so long...), but that shouldn't
>>> mean the developers should hold back development on 3.5 and 3.6, any
>>> more than today's users
>> of
>>> 3.5/3.6 should hold back 3.7.
>>>>
>>>> Some of the opinions expressed in this discussion seem to propose a
>>> scenario where users are going to be updating to "bleeding edge"
>>>> versions of ZooKeeper, but are going to insist on using Java 8.
>>>> Personally, I find this to be implausible. In my experience, people
>>> either upgrade everything as soon as they are able to, or they
>>> upgrade
>> each
>>> thing individually, only when they are forced to. The first group
>>> will be happy to move to Java 11 and ZK 3.7. The second group will
>>> probably avoid
>>> 3.7 anyway, and are fine sticking with 3.6, but if they had to
>>> update to 3.7, they'd also be fine updating to Java 11 if they had
>>> to in order to
>> use
>>> 3.7. I can't imagine the scenario where people are eagerly choosing
>>> to upgrade to ZK 3.7, but miserly insisting on using Java 8. Perhaps
>>> that scenario exists, but it's hard for me to imagine. Even so, my
>>> proposal would still support even that group of people.
>>>>
>>>> I think there are now effectively three proposals being discussed
>>>> in
>>> this thread:
>>>>
>>>> 1. (Christopher's original proposal) passively support Java 8 at
>> runtime
>>> by making JDK 11 the minimum requirement to build and test.
>>>> This scenario involves continuing to fix bugs, as they are
>>>> discovered
>>> and reported, that affect JDK 8, but passively, rather than proactively.
>>> This proposal does *not* drop Java 8 support, but merely
>>> de-emphasizes it in development of what will be 3.7 in the future,
>>> and drops the
>> requirement
>>> to do dedicated testing with Java 8. I think this is low risk,
>>> because it is very unlikely that the ZK devs would introduce a bug
>>> that would affect only Java 8 and the compiler wouldn't catch it...
>>> because the cross-compilation features of newer JDKs are really good.
>>>>
>>>> 2. (Enrico's alternate proposal) this variation of my proposal
>>>> would
>>> involve continuing to proactively support Java 8 by creating a
>>> dedicated testing suite to test client code on Java 8. I think this
>>> is a good
>> option,
>>> but since it involves a significantly higher amount of work than
>>> option
>> 1,
>>> I think the cost-benefit analysis would show this to be not worth
>>> the effort. Also, if it were implemented, it would need to be done
>>> carefully
>> to
>>> avoid requiring developers to have concurrently installed both Java
>>> 8 and Java 11 in order to perform a build, because requiring Java 8
>>> at build
>> time
>>> while developing would be worse than we have today.
>>>>
>>>> 3. (Andor's preference) move to JDK 11 fully. This would provide no
>>> support, passive or active, for Java 8 in ZK 3.7. To be honest, this
>>> is
>> my
>>> personal favorite, and is the simplest to implement and communicate
>> clearly
>>> to end users in release notes. The only reason I proposed a passive
>> support
>>> of Java 8 instead of this is because I was trying to seek a
>>> compromise
>> from
>>> the start. But, I think by far, this is the best option for the next
>>> *future* release of ZK. If you wanted to make the change even more
>> visible
>>> to users, the version could even be bumped to 4.0.
>>>>
>>>> If this were to come to a VOTE by the PMC, in order to make a final
>>> decision, I would recommend they vote on option 3, and then if that
>> fails,
>>> vote on option 1, and if that fails, keep things the way they are
>> (because
>>> option 2 is more work).
>>>>
>>>> Christopher
>>>>
>>>> P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK
>>>> 11
>>> and it works just fine there (as long as you add the missing runtime
>>> jar for javax.activation:javax.activation-api:1.2.0 to its class
>>> path, but
>> that
>>> was fixed in Hadoop 3.3).
>>>>
>>>> On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes
>> <ta...@cloudera.com.invalid>
>>> wrote:
>>>>>
>>>>> Hi All,
>>>>>
>>>>> I've just talked with a Hadoop/HDFS developer who told me what I
>>> guessed.
>>>>> With Hadoop3 they have just dropped JDK7 support, dropping JDK8
>>>>> would mean a release of Hadoop4.
>>>>> Since HADOOP-15338 is finished, they test with JDK8 and JDK11
>>>>> both. As of today most of the Hadoop users are still on Hadoop2,
>>>>> he doesn't expect
>>>>> Hadoop4 soon.
>>>>> As many Apache components depend on Hadoop and ZooKeeper they
>>>>> won't hurry to JDK11 until they have to (they will probably go
>>>>> one-by-one very slowly), which means if Hadoop stays on JDK8, they
>>>>> would use the last ZK version which works on JDK8.
>>>>> Do we want a ZK 3.4 again?
>>>>>
>>>>> Regards, Tamaas
>>>>>
>>>>>
>>>>> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes
>>>>> <ta...@cloudera.com>
>>> wrote:
>>>>>
>>>>>> Hi All,
>>>>>>
>>>>>> Just to add my two cents.
>>>>>>
>>>>>> Upgrading to JDK11 looks inevitable sooner or later and I would
>>>>>> definitely not wait until 2030 or 2026 when the extended support
>>>>>> of
>>> JDK8 ends.
>>>>>> But on the other side I have to agree with Enrico and Patrick
>>>>>> that far too many users are tied to JDK8 yet (not because they
>>>>>> want to use JDK8, but because they have to), some of them are
>>>>>> components of the Hadoop ecosystem, which would be a loss to tie
>>>>>> them to 3.6.x for
>>> years.
>>>>>> Do we know the state of Hadoop? It builds with JDK8 at the
>>>>>> moment, but do we know what are their plans to go to JDK11?
>>>>>> When they move, we should move too, but I don't think it would be
>>>>>> wise to do it earlier.
>>>>>>
>>>>>> Christopher's option looks like the golden path, but it needs
>>>>>> some investment on the testing side as Enrico pointed it out.
>>>>>> Could we agree on it as a compromise?
>>>>>>
>>>>>> Regards, Tamaas
>>>>>>
>>>>>> On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com>
>>> wrote:
>>>>>>
>>>>>>> I think I was reacting to Enrico's earlier comment of:
>>>>>>>
>>>>>>> " ZooKeeper client is used by tons of users and unfortunately
>>>>>>> many projects are still on JDK8, if we move ZooKeeper to JDK11
>>>>>>> the risk is to block users from the adoption, that is that we
>>>>>>> will see the world to stay on 3.6.x and we will have again a
>>>>>>> long lived release line, like 3.4."
>>>>>>>
>>>>>>> It's a matter of whether or not a long-lived release line is
>>>>>>> desirable/undesirable.  If everyone is OK keeping 3.6.x
>>>>>>> up-to-date security/patch-wise (if not feature-wise) for the
>>>>>>> next N years, then that's a potentially valid approach.  I
>>>>>>> interpreted that comment as "a long-lived release line is
>>>>>>> undesirable", but no one explicitly said that, I just read it that way.
>>>>>>>
>>>>>>> ~Brent
>>>>>>>
>>>>>>> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org>
>>> wrote:
>>>>>>>
>>>>>>>> On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org>
>>> wrote:
>>>>>>>>
>>>>>>>>> As far as I know Hbase, Solr and Kafka are already Java 11 ready.
>>>>>>>>>
>>>>>>>>> IMHO contributors of those projects should also put efforts
>>>>>>>>> into
>>>>>>> moving
>>>>>>>>> forward.
>>>>>>>>>
>>>>>>>>> We’re not saying that you _have_ to move to Java 11.
>>>>>>>>> Staying on Java 8? No problem, 3.6 is for you.
>>>>>>>>> Want the fancy new features of 3.7? Work on it on your side too.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> ppl want things like security fixes. I believe the highlighted
>>>>>>>> downside
>>>>>>> is
>>>>>>>> that we would need to continue to maintain 3.6.x rather than
>>>>>>>> allowing users, and ourselves, to focus on trunk for production
>> -"64
>>> percent"
>>>>>>> would
>>>>>>>> be blocked.
>>>>>>>>
>>>>>>>> Patrick
>>>>>>>>
>>>>>>>>
>>>>>>>>> Andor
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On 2020. Oct 21., at 17:52, Enrico Olivelli
>>>>>>>>>> <eo...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
>>>>>>>> andor@apache.org>
>>>>>>>>> ha
>>>>>>>>>> scritto:
>>>>>>>>>>
>>>>>>>>>>> Correct me if I'm wrong, but Oracle gets paid for extended
>>> support.
>>>>>>>>>>> Java 8 support until 2030 is not free of charge.
>>>>>>>>>>>
>>>>>>>>>>> "ZK may end up with a lot of users potentially locking
>>>>>>>>>>> themselves
>>>>>>> to
>>>>>>>>>>> 3.6.x for a while as Enrico mentioned."
>>>>>>>>>>>
>>>>>>>>>>> That's true. What's the downside of that which we should
>>>>>>>>>>> invest in
>>>>>>> to
>>>>>>>>>>> avoid?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I see ZooKeeper used in many many projects, all of the
>>>>>>>>>> HBase/Pulsar/Kafka/Solr ecosystem...
>>>>>>>>>> they will have to move to JDK11 in order to move to the new
>>>>>>>>>> ZK
>>>>>>> version
>>>>>>>>>> so probably they will stay on ZK 3.6
>>>>>>>>>>
>>>>>>>>>> Probably with Java 17 LTS released the cards will change on
>>>>>>>>>> the
>>>>>>> table
>>>>>>>>>>
>>>>>>>>>> Enrico
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Andor
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
>>>>>>>>>>>> As a slightly different consideration, if you look at the
>>>>>>> Long-Term
>>>>>>>>>>>> Support
>>>>>>>>>>>> (LTS) roadmaps for Java, currently Java 8 is set to have
>>>>>>>>>>>> full
>>>>>>> support
>>>>>>>>>>>> until
>>>>>>>>>>>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>> https://www.oracle.com/java/technologies/java-se-support-roadmap.
>>>>>>>> html
>>>>>>>>>>>> https://en.wikipedia.org/wiki/Java_version_history
>>>>>>>>>>>>
>>>>>>>>>>>> My guess is that a number of companies are still heavily
>>>>>>>>>>>> invested
>>>>>>> at
>>>>>>>>>>>> the
>>>>>>>>>>>> Java 8 level (I know a few) and with that kind of time
>>>>>>>>>>>> horizon,
>>>>>>> they
>>>>>>>>>>>> have
>>>>>>>>>>>> no real motivation to upgrade for quite a while.  If the
>>>>>>>>>>>> recent Python 2 deprecation is anything to go by, they
>>>>>>>>>>>> won't do it until they have to.
>>>>>>>>>>>>
>>>>>>>>>>>> Not saying Java 8 isn't *very* old (2014 release it seems
>>>>>>>>>>>> like?)
>>>>>>> and
>>>>>>>>>>>> I'm
>>>>>>>>>>>> not invested heavily either way, but this might suggest
>>>>>>>>>>>> that ZK
>>>>>>> may
>>>>>>>>>>>> end up
>>>>>>>>>>>> with a lot of users potentially locking themselves to 3.6.x
>>>>>>>>>>>> for a while as Enrico mentioned.
>>>>>>>>>>>>
>>>>>>>>>>>> (Not a major contributor, but wanted to chime in since I
>>>>>>>>>>>> just had this conversation with a bunch of people
>>>>>>>>>>>> professionally recently)
>>>>>>>>>>>>
>>>>>>>>>>>> Brent
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar
>>>>>>>>>>>> <an...@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for the summary.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully.
>>>>>>>>>>>>> Other projects will upgrade once they’re JDK11 compliant,
>>>>>>>>>>>>> otherwise they will
>>>>>>> stay
>>>>>>>>>>>>> on 3.5
>>>>>>>>>>>>> or 3.6. Both version are quite recent in ZooKeeper-terms,
>>>>>>>>>>>>> we already planned big changes for 3.7.0 and JDK 11 could
>>>>>>>>>>>>> be one of them.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Don’t put extra burden on the ZK community to help others
>>>>>>>>>>>>> staying on ancient Java versions.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Andor
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
>>>>>>> eolivelli@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Let me recap
>>>>>>>>>>>>>> - Christopher is proposing to move to JDK11
>>>>>>>>>>>>>> - ZooKeeper client and server are bundled and coded and
>>>>>>>>>>>>>> tested together
>>>>>>>>>>>>> in
>>>>>>>>>>>>>> zookeeper-server
>>>>>>>>>>>>>> - Enrico is concerned about the need of testing ZooKeeper
>>>>>>>>>>>>>> client on JDK8 (not a problem to move the server to
>>>>>>>>>>>>>> JDK11)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ZooKeeper client is used by tons of users and
>>>>>>>>>>>>>> unfortunately many projects are still on JDK8, if we move
>>>>>>>>>>>>>> ZooKeeper to JDK11 the risk is to block
>>>>>>>>>>>>> users
>>>>>>>>>>>>>> from the adoption,
>>>>>>>>>>>>>> that is that we will see the world to stay on 3.6.x and
>>>>>>>>>>>>>> we will have
>>>>>>>>>>>>> again
>>>>>>>>>>>>>> a long lived release line, like 3.4.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Testing the client on JDK8 would be possible if we create
>>>>>>>>>>>>>> some kind of additional module with system tests, then we
>>>>>>>>>>>>>> can start the
>>>>>>> server
>>>>>>>>>>>>>> on
>>>>>>>>>>>>> docker
>>>>>>>>>>>>>> on JDK11+ and start a client on JDK8 with Maven toolchain
>>>>>>>>>>>>>> it should possible to run surefire tests using a separate
>>>>>>>>>>>>>> JVM.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So in my vision 2 options:
>>>>>>>>>>>>>> 1) fully JDK11 - drop JDK8 at all
>>>>>>>>>>>>>> 2) build with JDK11 - server only on JDK11 - add system
>>>>>>>>>>>>>> tests with docker and toolchains that ensure the
>>>>>>>>>>>>>> ZooKeeper client (and all
>>>>>>>>>>>>>> dependencies)
>>>>>>>>>>>>>> still work on JDK8
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> From my point of view about the ZooKeeper ecosystem
>>>>>>>>>>>>>> option 2) will be far better, but we need resources to
>>>>>>>>>>>>>> work on a new test suite.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Enrico
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
>>>>>>>>>>>>>> andor@apache.org>
>>>>>>>>>>>>> ha
>>>>>>>>>>>>>> scritto:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Tamas, Enrico,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sorry I don’t follow. Why do we have to test the client
>>>>>>>>>>>>>>> with JDK 8 in version 3.7.0?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Andor
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
>>>>>>>>>>>>>>>> tamaas@cloudera.com.INVALID>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> Hi Enrico,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Separating ZooKeeper client and server is a huge work,
>>>>>>>>>>>>>>>> but we might not need it.
>>>>>>>>>>>>>>>> As you mentioned we have to test ZK client with Java 8,
>>>>>>>>>>>>>>>> what about separating only the test cases which we need
>>>>>>>>>>>>>>>> to run with
>>>>>>>>>>>>>>>> Java8 too?
>>>>>>>>>>>>>>>> In Curator we have the ZK compatibility tests where we
>>>>>>>>>>>>>>>> run a limited
>>>>>>>>>>>>>>> amount
>>>>>>>>>>>>>>>> of Curator's jUnit tests with a different ZK version.
>>>>>>>>>>>>>>>> We might be able to do the same here, tag tests which
>>>>>>>>>>>>>>>> are testing ZK
>>>>>>>>>>>>>>> client
>>>>>>>>>>>>>>>> and run them separately with Java 8. The only
>>>>>>>>>>>>>>>> limitation is that these tests must stay JDK8
>>>>>>>>>>>>>>>> compatible.
>>>>>>>>>>>>>>>> But from the tags we will see which ones are those.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards, Tamaas
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
>>>>>>>>>>>>>>>> eolivelli@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> Christopher
>>>>>>>>>>>>>>>>> I appreciate your idea and I also moved lots of my
>>>>>>>>>>>>>>>>> projects to work
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>> you are suggesting.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> We must run tests using real jdk8 to test the
>>>>>>>>>>>>>>>>> Zookeeper client. We
>>>>>>>>>>>>> must
>>>>>>>>>>>>>>>>> ensure that Zookeeper works well, especially while
>>>>>>>>>>>>>>>>> dealing with
>>>>>>>>>>>>> security
>>>>>>>>>>>>>>>>> stuff.
>>>>>>>>>>>>>>>>> Currently the client is in the same module of the
>>>>>>>>>>>>>>>>> server and it will
>>>>>>>>>>>>>>> take a
>>>>>>>>>>>>>>>>> good (huge) amount of work to separate them
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Enrico
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher
>>>>>>>>>>>>>>>>> <ct...@apache.org> ha
>>>>>>>>>>>>> scritto:
>>>>>>>>>>>>>>>>>> Hi ZK Devs,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> With recent advancements in Java (since Java 9), it
>>>>>>>>>>>>>>>>>> is now generally no longer necessary to require that
>>>>>>>>>>>>>>>>>> software be developed on an older JDK in order to
>>>>>>>>>>>>>>>>>> have confidence that it will run on the older version
>>>>>>>>>>>>>>>>>> of Java. This is because, as of Java 9, all JDK
>>>>>>>>>>>>>>>>>> releases have better support for cross-compilation to
>>>>>>>>>>>>>>>>>> older Java versions.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> What this means is that developers can confidently
>>>>>>>>>>>>>>>>>> make the build requirements for a project higher than
>>>>>>>>>>>>>>>>>> the Java version that will actually be supported at
>>>>>>>>>>>>>>>>>> runtime.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> In fact, ZooKeeper already supports the necessary
>>>>>>>>>>>>>>>>>> flags in its Maven build configuration to ensure that
>>>>>>>>>>>>>>>>>> it uses JDK 8 compliance when building on a newer JDK
>>>>>>>>>>>>>>>>>> (I added this way back in
>>>>>>>>>>>>>>>>>> ZOOKEEPER-3739 /
>>>>>>>>>>>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> So, I propose that we make JDK 11 the new minimum
>>>>>>>>>>>>>>>>>> version to *build* ZooKeeper with. This would not
>>>>>>>>>>>>>>>>>> change the runtime requirement, which would remain at
>>>>>>>>>>>>>>>>>> JDK 8.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> The only necessary change to make this happen would
>>>>>>>>>>>>>>>>>> be to add the minimum Java version to the
>>>>>>>>>>>>>>>>>> maven-enforcer-plugin (like
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7e
>>>>>>> cdd1
>>>>>>> 1d5dedb146519/pom.xml#L1162-L1164
>>>>>>>>>>>>>>>>>> )
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> This would allow ZooKeeper to to streamline its
>>>>>>>>>>>>>>>>>> development process a little bit by reducing the
>>>>>>>>>>>>>>>>>> amount of CI testing that is done as part of the
>>>>>>>>>>>>>>>>>> build. In other words, we can drop the CI builds for
>>>>>>>>>>>>>>>>>> JDK 8, which saves on build resources and time. The
>>>>>>>>>>>>>>>>>> return on investment is so low for the JDK 8 builds
>>>>>>>>>>>>>>>>>> anyway, because of the improved cross-compilation in
>>>>>>>>>>>>>>>>>> newer JDKs. So, there's not much value in building on
>>>>>>>>>>>>>>>>>> JDK 8 anyway.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Of course, I am only recommending this for *new*
>>>>>>>>>>>>>>>>>> release lines, starting with ZooKeeper 3.7.0/master
>>>>>>>>>>>>>>>>>> branch, because I would not want to change
>>>>>>>>>>>>>>>>>> expectations for users who will build their own
>>>>>>>>>>>>>>>>>> 3.5 and 3.6
>>>>>>>>>>>>>>>>>> versions as they continue to have patch versions
>>>>>>>>>>>>>>>>>> released.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Kind Regards,
>>>>>>>>>>>>>>>>>> Christopher
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>> ________________________________
>>>>
>>>> CONFIDENTIALITY & PRIVACY NOTICE
>>>> This e-mail (including any attachments) is strictly confidential
>>>> and
>> may
>>> also contain privileged information. If you are not the intended
>> recipient
>>> you are not authorised to read, print, save, process or disclose
>>> this message. If you have received this message by mistake, please
>>> inform the sender immediately and destroy this e-mail, its
>>> attachments and any
>> copies.
>>> Any use, distribution, reproduction or disclosure by any person
>>> other
>> than
>>> the intended recipient is strictly prohibited and the person
>>> responsible may incur in penalties.
>>>> The use of this e-mail is only for professional purposes; there is
>>>> no
>>> guarantee that the correspondence towards this e-mail will be read
>>> only
>> by
>>> the recipient, because, under certain circumstances, there may be a
>>> need
>> to
>>> access this email by third subjects belonging to the Company.
>>>
>>>
>>
>
> ________________________________
>
> CONFIDENTIALITY & PRIVACY NOTICE
> This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties.
> The use of this e-mail is only for professional purposes; there is no guarantee that the correspondence towards this e-mail will be read only by the recipient, because, under certain circumstances, there may be a need to access this email by third subjects belonging to the Company.


________________________________

CONFIDENTIALITY & PRIVACY NOTICE
This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties.
The use of this e-mail is only for professional purposes; there is no guarantee that the correspondence towards this e-mail will be read only by the recipient, because, under certain circumstances, there may be a need to access this email by third subjects belonging to the Company.

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Andor Molnar <an...@apache.org>]

I can only echo what Chris said in his latest e-mail:

3.7 will be a future release of ZooKeeper and we should look forward what’s coming with Java versions at that time (there probably will be at least 3 stable versions to support). Though I like the idea of “passive support”: I don’t see any harm in fixing Java 8 related bugs, but I would be also happy to make Java 11 features available for contributors.

From what you said even Hadoop is in good shape for Java 11 readiness. What’s (will be) the problem here?

Andor



> On 2020. Oct 22., at 16:17, Alessandro Luccaroni - Diennea <al...@diennea.com.INVALID> wrote:
> 
> I know of projects that embed Zookeeper so I don't think it is (only) a matter of run/build time.
> I like what Mate said about announcing this switch today for a later major release (such as 4.0 that could come after 3.7) to give time to downstream applications to plan for this change.
> 
> Regards,
> Alessandro Luccaroni
> 
> -----Messaggio originale-----
> Da: Ted Dunning <te...@gmail.com>
> Inviato: giovedì 22 ottobre 2020 15:58
> A: dev@zookeeper.apache.org
> Oggetto: Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7
> 
> Allesandro,
> 
> Would your answer be different if you considered a strategy that allows
> JRE8 to run ZK but requires JDK11 to build it?
> 
> 
> 
> On Thu, Oct 22, 2020 at 6:36 AM Szalay-Bekő Máté <sz...@gmail.com>
> wrote:
> 
>> Hello All,
>> 
>> A few reflections:
>> 
>> - I don't think that backporting fixes from a JDK 11 only version to a
>> JDK
>> 8 compatible version would be necessarily a harder thing than any
>> regular backport. It kind of depends on us (whether we use many JDK 11
>> only features or not until we drop JDK 8 from all supported versions).
>> Also if it gets more painful, then we can decide to limit the number
>> of backported commits (e.g. only strictly to security fixes / CVEs)
>> - But even if the above point is true (?), I would not do this cut (e.g.
>> moving entirely to JDK 11 only) in a minor release. I think this
>> should be a major change in 4.0. (maybe together with other more
>> "risky" changes, like the separation of the client and server
>> artifacts/code; or some incompatible changes in the leader election
>> protocol. Although these are separate discussions)
>> - So all-in-all I like the JDK 11 only option the most. But I wouldn't
>> do it in 3.7 (which might happen very soon), but rather in 4.0. The
>> question for me is wether to do any in-between step in 3.x. (like the
>> options 1 or 2 above in Christopher's mail). I think it mainly should
>> depend on the timing of 4.0.
>> 
>> Kind regards,
>> Mate
>> 
>> On Thu, Oct 22, 2020 at 3:05 PM Flavio Junqueira <fp...@apache.org> wrote:
>> 
>>> There are three points that stand out for me in this thread:
>>> 
>>> - How do we determine how such a change affects our user base?
>>> - How much effort do the different options induce with respect to
>>> maintenance?
>>> - What's the right timeline for changes and how do we communicate
>>> them so that our users have enough time to prepare?
>>> 
>>> Someone mentioned a PMC vote, and I don't think this should be a
>>> closed vote, independent of how the conversation goes.
>>> 
>>> -Flavio
>>> 
>>>> On 22 Oct 2020, at 08:39, Alessandro Luccaroni - Diennea
>>> <al...@diennea.com.INVALID> wrote:
>>>> 
>>>> Hi all,
>>>> If I might chime in as a zookeeper user (in multiple products) and
>>>> a
>>> follower of the project I think the drop of Java8 support (official
>> and/or
>>> unofficial) could be a big mistake.
>>>> 
>>>> From my own company point of view we already support Java11 in all
>>>> our
>>> applications so we are not directly impacted (and we have upgrade
>>> path
>> for
>>> older versions to provide to our customers).
>>>> My worries resides in the (high) probability of a userbase
>>> fragmentation: in the recent past Zookeeper development picked up
>>> speed thanks to a bunch of new committers and PMCs after a period of
>>> mostly maintenance focused works, but the number of active
>>> committers and PMCs
>> is
>>> still very low for a project like this.
>>>> 
>>>> I foresee the risk of spreading thin the resources of the project
>>>> if we
>>> force the userbase to stick to an older version and, in turn, we are
>> forced
>>> to backport many issue to the 3.6 branch.
>>>> 
>>>> Alessandro Luccaroni
>>>> Platform Manager @ Diennea - MagNews
>>>> Tel.: (+39) 0546 066100 Int. 924 - Mob.: (+39) 393 7273519 Viale
>>>> G.Marconi 30/14 - 48018 Faenza (RA) - Italy
>>>> 
>>>> -----Messaggio originale-----
>>>> Da: Christopher <ct...@apache.org>
>>>> Inviato: giovedì 22 ottobre 2020 05:21
>>>> A: dev@zookeeper.apache.org
>>>> Oggetto: Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7
>>>> 
>>>> I'm happy that this discussion has been so lively! I just want to
>>> emphasize a few things:
>>>> 
>>>> I really do understand the desire to continue to support Java 8...
>>>> I
>> get
>>> it. But all the conversations around this seem based on what people
>>> are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
>>>> *future* release... so it should be based more on reasonable
>>> expectations for users in the future, and less based on what is
>>> happening today. I suspect *most* people today are still using 3.4
>>> anyway (it was just so stable for so long...), but that shouldn't
>>> mean the developers should hold back development on 3.5 and 3.6, any
>>> more than today's users
>> of
>>> 3.5/3.6 should hold back 3.7.
>>>> 
>>>> Some of the opinions expressed in this discussion seem to propose
>>>> a
>>> scenario where users are going to be updating to "bleeding edge"
>>>> versions of ZooKeeper, but are going to insist on using Java 8.
>>>> Personally, I find this to be implausible. In my experience,
>>>> people
>>> either upgrade everything as soon as they are able to, or they
>>> upgrade
>> each
>>> thing individually, only when they are forced to. The first group
>>> will be happy to move to Java 11 and ZK 3.7. The second group will
>>> probably avoid
>>> 3.7 anyway, and are fine sticking with 3.6, but if they had to
>>> update to 3.7, they'd also be fine updating to Java 11 if they had
>>> to in order to
>> use
>>> 3.7. I can't imagine the scenario where people are eagerly choosing
>>> to upgrade to ZK 3.7, but miserly insisting on using Java 8. Perhaps
>>> that scenario exists, but it's hard for me to imagine. Even so, my
>>> proposal would still support even that group of people.
>>>> 
>>>> I think there are now effectively three proposals being discussed
>>>> in
>>> this thread:
>>>> 
>>>> 1. (Christopher's original proposal) passively support Java 8 at
>> runtime
>>> by making JDK 11 the minimum requirement to build and test.
>>>> This scenario involves continuing to fix bugs, as they are
>>>> discovered
>>> and reported, that affect JDK 8, but passively, rather than proactively.
>>> This proposal does *not* drop Java 8 support, but merely
>>> de-emphasizes it in development of what will be 3.7 in the future,
>>> and drops the
>> requirement
>>> to do dedicated testing with Java 8. I think this is low risk,
>>> because it is very unlikely that the ZK devs would introduce a bug
>>> that would affect only Java 8 and the compiler wouldn't catch it...
>>> because the cross-compilation features of newer JDKs are really good.
>>>> 
>>>> 2. (Enrico's alternate proposal) this variation of my proposal
>>>> would
>>> involve continuing to proactively support Java 8 by creating a
>>> dedicated testing suite to test client code on Java 8. I think this
>>> is a good
>> option,
>>> but since it involves a significantly higher amount of work than
>>> option
>> 1,
>>> I think the cost-benefit analysis would show this to be not worth
>>> the effort. Also, if it were implemented, it would need to be done
>>> carefully
>> to
>>> avoid requiring developers to have concurrently installed both Java
>>> 8 and Java 11 in order to perform a build, because requiring Java 8
>>> at build
>> time
>>> while developing would be worse than we have today.
>>>> 
>>>> 3. (Andor's preference) move to JDK 11 fully. This would provide
>>>> no
>>> support, passive or active, for Java 8 in ZK 3.7. To be honest, this
>>> is
>> my
>>> personal favorite, and is the simplest to implement and communicate
>> clearly
>>> to end users in release notes. The only reason I proposed a passive
>> support
>>> of Java 8 instead of this is because I was trying to seek a
>>> compromise
>> from
>>> the start. But, I think by far, this is the best option for the next
>>> *future* release of ZK. If you wanted to make the change even more
>> visible
>>> to users, the version could even be bumped to 4.0.
>>>> 
>>>> If this were to come to a VOTE by the PMC, in order to make a
>>>> final
>>> decision, I would recommend they vote on option 3, and then if that
>> fails,
>>> vote on option 1, and if that fails, keep things the way they are
>> (because
>>> option 2 is more work).
>>>> 
>>>> Christopher
>>>> 
>>>> P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK
>>>> 11
>>> and it works just fine there (as long as you add the missing runtime
>>> jar for javax.activation:javax.activation-api:1.2.0 to its class
>>> path, but
>> that
>>> was fixed in Hadoop 3.3).
>>>> 
>>>> On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes
>> <ta...@cloudera.com.invalid>
>>> wrote:
>>>>> 
>>>>> Hi All,
>>>>> 
>>>>> I've just talked with a Hadoop/HDFS developer who told me what I
>>> guessed.
>>>>> With Hadoop3 they have just dropped JDK7 support, dropping JDK8
>>>>> would mean a release of Hadoop4.
>>>>> Since HADOOP-15338 is finished, they test with JDK8 and JDK11
>>>>> both. As of today most of the Hadoop users are still on Hadoop2,
>>>>> he doesn't expect
>>>>> Hadoop4 soon.
>>>>> As many Apache components depend on Hadoop and ZooKeeper they
>>>>> won't hurry to JDK11 until they have to (they will probably go
>>>>> one-by-one very slowly), which means if Hadoop stays on JDK8,
>>>>> they would use the last ZK version which works on JDK8.
>>>>> Do we want a ZK 3.4 again?
>>>>> 
>>>>> Regards, Tamaas
>>>>> 
>>>>> 
>>>>> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes
>>>>> <ta...@cloudera.com>
>>> wrote:
>>>>> 
>>>>>> Hi All,
>>>>>> 
>>>>>> Just to add my two cents.
>>>>>> 
>>>>>> Upgrading to JDK11 looks inevitable sooner or later and I would
>>>>>> definitely not wait until 2030 or 2026 when the extended support
>>>>>> of
>>> JDK8 ends.
>>>>>> But on the other side I have to agree with Enrico and Patrick
>>>>>> that far too many users are tied to JDK8 yet (not because they
>>>>>> want to use JDK8, but because they have to), some of them are
>>>>>> components of the Hadoop ecosystem, which would be a loss to tie
>>>>>> them to 3.6.x for
>>> years.
>>>>>> Do we know the state of Hadoop? It builds with JDK8 at the
>>>>>> moment, but do we know what are their plans to go to JDK11?
>>>>>> When they move, we should move too, but I don't think it would
>>>>>> be wise to do it earlier.
>>>>>> 
>>>>>> Christopher's option looks like the golden path, but it needs
>>>>>> some investment on the testing side as Enrico pointed it out.
>>>>>> Could we agree on it as a compromise?
>>>>>> 
>>>>>> Regards, Tamaas
>>>>>> 
>>>>>> On Wed, Oct 21, 2020 at 7:11 PM Brent
>>>>>> <br...@gmail.com>
>>> wrote:
>>>>>> 
>>>>>>> I think I was reacting to Enrico's earlier comment of:
>>>>>>> 
>>>>>>> " ZooKeeper client is used by tons of users and unfortunately
>>>>>>> many projects are still on JDK8, if we move ZooKeeper to JDK11
>>>>>>> the risk is to block users from the adoption, that is that we
>>>>>>> will see the world to stay on 3.6.x and we will have again a
>>>>>>> long lived release line, like 3.4."
>>>>>>> 
>>>>>>> It's a matter of whether or not a long-lived release line is
>>>>>>> desirable/undesirable.  If everyone is OK keeping 3.6.x
>>>>>>> up-to-date security/patch-wise (if not feature-wise) for the
>>>>>>> next N years, then that's a potentially valid approach.  I
>>>>>>> interpreted that comment as "a long-lived release line is
>>>>>>> undesirable", but no one explicitly said that, I just read it that way.
>>>>>>> 
>>>>>>> ~Brent
>>>>>>> 
>>>>>>> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org>
>>> wrote:
>>>>>>> 
>>>>>>>> On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar
>>>>>>>> <an...@apache.org>
>>> wrote:
>>>>>>>> 
>>>>>>>>> As far as I know Hbase, Solr and Kafka are already Java 11 ready.
>>>>>>>>> 
>>>>>>>>> IMHO contributors of those projects should also put efforts
>>>>>>>>> into
>>>>>>> moving
>>>>>>>>> forward.
>>>>>>>>> 
>>>>>>>>> We’re not saying that you _have_ to move to Java 11.
>>>>>>>>> Staying on Java 8? No problem, 3.6 is for you.
>>>>>>>>> Want the fancy new features of 3.7? Work on it on your side too.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> ppl want things like security fixes. I believe the highlighted
>>>>>>>> downside
>>>>>>> is
>>>>>>>> that we would need to continue to maintain 3.6.x rather than
>>>>>>>> allowing users, and ourselves, to focus on trunk for
>>>>>>>> production
>> -"64
>>> percent"
>>>>>>> would
>>>>>>>> be blocked.
>>>>>>>> 
>>>>>>>> Patrick
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> Andor
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On 2020. Oct 21., at 17:52, Enrico Olivelli
>>>>>>>>>> <eo...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
>>>>>>>> andor@apache.org>
>>>>>>>>> ha
>>>>>>>>>> scritto:
>>>>>>>>>> 
>>>>>>>>>>> Correct me if I'm wrong, but Oracle gets paid for extended
>>> support.
>>>>>>>>>>> Java 8 support until 2030 is not free of charge.
>>>>>>>>>>> 
>>>>>>>>>>> "ZK may end up with a lot of users potentially locking
>>>>>>>>>>> themselves
>>>>>>> to
>>>>>>>>>>> 3.6.x for a while as Enrico mentioned."
>>>>>>>>>>> 
>>>>>>>>>>> That's true. What's the downside of that which we should
>>>>>>>>>>> invest in
>>>>>>> to
>>>>>>>>>>> avoid?
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> I see ZooKeeper used in many many projects, all of the
>>>>>>>>>> HBase/Pulsar/Kafka/Solr ecosystem...
>>>>>>>>>> they will have to move to JDK11 in order to move to the new
>>>>>>>>>> ZK
>>>>>>> version
>>>>>>>>>> so probably they will stay on ZK 3.6
>>>>>>>>>> 
>>>>>>>>>> Probably with Java 17 LTS released the cards will change on
>>>>>>>>>> the
>>>>>>> table
>>>>>>>>>> 
>>>>>>>>>> Enrico
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Andor
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
>>>>>>>>>>>> As a slightly different consideration, if you look at the
>>>>>>> Long-Term
>>>>>>>>>>>> Support
>>>>>>>>>>>> (LTS) roadmaps for Java, currently Java 8 is set to have
>>>>>>>>>>>> full
>>>>>>> support
>>>>>>>>>>>> until
>>>>>>>>>>>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>> https://www.oracle.com/java/technologies/java-se-support-roadmap.
>>>>>>>> html
>>>>>>>>>>>> https://en.wikipedia.org/wiki/Java_version_history
>>>>>>>>>>>> 
>>>>>>>>>>>> My guess is that a number of companies are still heavily
>>>>>>>>>>>> invested
>>>>>>> at
>>>>>>>>>>>> the
>>>>>>>>>>>> Java 8 level (I know a few) and with that kind of time
>>>>>>>>>>>> horizon,
>>>>>>> they
>>>>>>>>>>>> have
>>>>>>>>>>>> no real motivation to upgrade for quite a while.  If the
>>>>>>>>>>>> recent Python 2 deprecation is anything to go by, they
>>>>>>>>>>>> won't do it until they have to.
>>>>>>>>>>>> 
>>>>>>>>>>>> Not saying Java 8 isn't *very* old (2014 release it seems
>>>>>>>>>>>> like?)
>>>>>>> and
>>>>>>>>>>>> I'm
>>>>>>>>>>>> not invested heavily either way, but this might suggest
>>>>>>>>>>>> that ZK
>>>>>>> may
>>>>>>>>>>>> end up
>>>>>>>>>>>> with a lot of users potentially locking themselves to
>>>>>>>>>>>> 3.6.x for a while as Enrico mentioned.
>>>>>>>>>>>> 
>>>>>>>>>>>> (Not a major contributor, but wanted to chime in since I
>>>>>>>>>>>> just had this conversation with a bunch of people
>>>>>>>>>>>> professionally recently)
>>>>>>>>>>>> 
>>>>>>>>>>>> Brent
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar
>>>>>>>>>>>> <an...@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks for the summary.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully.
>>>>>>>>>>>>> Other projects will upgrade once they’re JDK11 compliant,
>>>>>>>>>>>>> otherwise they will
>>>>>>> stay
>>>>>>>>>>>>> on 3.5
>>>>>>>>>>>>> or 3.6. Both version are quite recent in ZooKeeper-terms,
>>>>>>>>>>>>> we already planned big changes for 3.7.0 and JDK 11 could
>>>>>>>>>>>>> be one of them.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Don’t put extra burden on the ZK community to help others
>>>>>>>>>>>>> staying on ancient Java versions.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Andor
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
>>>>>>> eolivelli@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Let me recap
>>>>>>>>>>>>>> - Christopher is proposing to move to JDK11
>>>>>>>>>>>>>> - ZooKeeper client and server are bundled and coded and
>>>>>>>>>>>>>> tested together
>>>>>>>>>>>>> in
>>>>>>>>>>>>>> zookeeper-server
>>>>>>>>>>>>>> - Enrico is concerned about the need of testing
>>>>>>>>>>>>>> ZooKeeper client on JDK8 (not a problem to move the
>>>>>>>>>>>>>> server to
>>>>>>>>>>>>>> JDK11)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> ZooKeeper client is used by tons of users and
>>>>>>>>>>>>>> unfortunately many projects are still on JDK8, if we
>>>>>>>>>>>>>> move ZooKeeper to JDK11 the risk is to block
>>>>>>>>>>>>> users
>>>>>>>>>>>>>> from the adoption,
>>>>>>>>>>>>>> that is that we will see the world to stay on 3.6.x and
>>>>>>>>>>>>>> we will have
>>>>>>>>>>>>> again
>>>>>>>>>>>>>> a long lived release line, like 3.4.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Testing the client on JDK8 would be possible if we
>>>>>>>>>>>>>> create some kind of additional module with system tests,
>>>>>>>>>>>>>> then we can start the
>>>>>>> server
>>>>>>>>>>>>>> on
>>>>>>>>>>>>> docker
>>>>>>>>>>>>>> on JDK11+ and start a client on JDK8 with Maven
>>>>>>>>>>>>>> toolchain it should possible to run surefire tests using
>>>>>>>>>>>>>> a separate JVM.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> So in my vision 2 options:
>>>>>>>>>>>>>> 1) fully JDK11 - drop JDK8 at all
>>>>>>>>>>>>>> 2) build with JDK11 - server only on JDK11 - add system
>>>>>>>>>>>>>> tests with docker and toolchains that ensure the
>>>>>>>>>>>>>> ZooKeeper client (and all
>>>>>>>>>>>>>> dependencies)
>>>>>>>>>>>>>> still work on JDK8
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> From my point of view about the ZooKeeper ecosystem
>>>>>>>>>>>>>> option 2) will be far better, but we need resources to
>>>>>>>>>>>>>> work on a new test suite.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Enrico
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
>>>>>>>>>>>>>> andor@apache.org>
>>>>>>>>>>>>> ha
>>>>>>>>>>>>>> scritto:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Tamas, Enrico,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Sorry I don’t follow. Why do we have to test the client
>>>>>>>>>>>>>>> with JDK 8 in version 3.7.0?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Andor
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
>>>>>>>>>>>>>>>> tamaas@cloudera.com.INVALID>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> Hi Enrico,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Separating ZooKeeper client and server is a huge work,
>>>>>>>>>>>>>>>> but we might not need it.
>>>>>>>>>>>>>>>> As you mentioned we have to test ZK client with Java
>>>>>>>>>>>>>>>> 8, what about separating only the test cases which we
>>>>>>>>>>>>>>>> need to run with
>>>>>>>>>>>>>>>> Java8 too?
>>>>>>>>>>>>>>>> In Curator we have the ZK compatibility tests where we
>>>>>>>>>>>>>>>> run a limited
>>>>>>>>>>>>>>> amount
>>>>>>>>>>>>>>>> of Curator's jUnit tests with a different ZK version.
>>>>>>>>>>>>>>>> We might be able to do the same here, tag tests which
>>>>>>>>>>>>>>>> are testing ZK
>>>>>>>>>>>>>>> client
>>>>>>>>>>>>>>>> and run them separately with Java 8. The only
>>>>>>>>>>>>>>>> limitation is that these tests must stay JDK8
>>>>>>>>>>>>>>>> compatible.
>>>>>>>>>>>>>>>> But from the tags we will see which ones are those.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Regards, Tamaas
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
>>>>>>>>>>>>>>>> eolivelli@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> Christopher
>>>>>>>>>>>>>>>>> I appreciate your idea and I also moved lots of my
>>>>>>>>>>>>>>>>> projects to work
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>> you are suggesting.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> We must run tests using real jdk8 to test the
>>>>>>>>>>>>>>>>> Zookeeper client. We
>>>>>>>>>>>>> must
>>>>>>>>>>>>>>>>> ensure that Zookeeper works well, especially while
>>>>>>>>>>>>>>>>> dealing with
>>>>>>>>>>>>> security
>>>>>>>>>>>>>>>>> stuff.
>>>>>>>>>>>>>>>>> Currently the client is in the same module of the
>>>>>>>>>>>>>>>>> server and it will
>>>>>>>>>>>>>>> take a
>>>>>>>>>>>>>>>>> good (huge) amount of work to separate them
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Enrico
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher
>>>>>>>>>>>>>>>>> <ct...@apache.org> ha
>>>>>>>>>>>>> scritto:
>>>>>>>>>>>>>>>>>> Hi ZK Devs,
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> With recent advancements in Java (since Java 9), it
>>>>>>>>>>>>>>>>>> is now generally no longer necessary to require that
>>>>>>>>>>>>>>>>>> software be developed on an older JDK in order to
>>>>>>>>>>>>>>>>>> have confidence that it will run on the older
>>>>>>>>>>>>>>>>>> version of Java. This is because, as of Java 9, all
>>>>>>>>>>>>>>>>>> JDK releases have better support for
>>>>>>>>>>>>>>>>>> cross-compilation to older Java versions.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> What this means is that developers can confidently
>>>>>>>>>>>>>>>>>> make the build requirements for a project higher
>>>>>>>>>>>>>>>>>> than the Java version that will actually be
>>>>>>>>>>>>>>>>>> supported at runtime.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> In fact, ZooKeeper already supports the necessary
>>>>>>>>>>>>>>>>>> flags in its Maven build configuration to ensure
>>>>>>>>>>>>>>>>>> that it uses JDK 8 compliance when building on a
>>>>>>>>>>>>>>>>>> newer JDK (I added this way back in
>>>>>>>>>>>>>>>>>> ZOOKEEPER-3739 /
>>>>>>>>>>>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> So, I propose that we make JDK 11 the new minimum
>>>>>>>>>>>>>>>>>> version to *build* ZooKeeper with. This would not
>>>>>>>>>>>>>>>>>> change the runtime requirement, which would remain
>>>>>>>>>>>>>>>>>> at JDK 8.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> The only necessary change to make this happen would
>>>>>>>>>>>>>>>>>> be to add the minimum Java version to the
>>>>>>>>>>>>>>>>>> maven-enforcer-plugin (like
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7e
>>>>>>> cdd1
>>>>>>> 1d5dedb146519/pom.xml#L1162-L1164
>>>>>>>>>>>>>>>>>> )
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> This would allow ZooKeeper to to streamline its
>>>>>>>>>>>>>>>>>> development process a little bit by reducing the
>>>>>>>>>>>>>>>>>> amount of CI testing that is done as part of the
>>>>>>>>>>>>>>>>>> build. In other words, we can drop the CI builds for
>>>>>>>>>>>>>>>>>> JDK 8, which saves on build resources and time. The
>>>>>>>>>>>>>>>>>> return on investment is so low for the JDK 8 builds
>>>>>>>>>>>>>>>>>> anyway, because of the improved cross-compilation in
>>>>>>>>>>>>>>>>>> newer JDKs. So, there's not much value in building
>>>>>>>>>>>>>>>>>> on JDK 8 anyway.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Of course, I am only recommending this for *new*
>>>>>>>>>>>>>>>>>> release lines, starting with ZooKeeper 3.7.0/master
>>>>>>>>>>>>>>>>>> branch, because I would not want to change
>>>>>>>>>>>>>>>>>> expectations for users who will build their own
>>>>>>>>>>>>>>>>>> 3.5 and 3.6
>>>>>>>>>>>>>>>>>> versions as they continue to have patch versions
>>>>>>>>>>>>>>>>>> released.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Kind Regards,
>>>>>>>>>>>>>>>>>> Christopher
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>>>> ________________________________
>>>> 
>>>> CONFIDENTIALITY & PRIVACY NOTICE
>>>> This e-mail (including any attachments) is strictly confidential
>>>> and
>> may
>>> also contain privileged information. If you are not the intended
>> recipient
>>> you are not authorised to read, print, save, process or disclose
>>> this message. If you have received this message by mistake, please
>>> inform the sender immediately and destroy this e-mail, its
>>> attachments and any
>> copies.
>>> Any use, distribution, reproduction or disclosure by any person
>>> other
>> than
>>> the intended recipient is strictly prohibited and the person
>>> responsible may incur in penalties.
>>>> The use of this e-mail is only for professional purposes; there is
>>>> no
>>> guarantee that the correspondence towards this e-mail will be read
>>> only
>> by
>>> the recipient, because, under certain circumstances, there may be a
>>> need
>> to
>>> access this email by third subjects belonging to the Company.
>>> 
>>> 
>> 
> 
> ________________________________
> 
> CONFIDENTIALITY & PRIVACY NOTICE
> This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties.
> The use of this e-mail is only for professional purposes; there is no guarantee that the correspondence towards this e-mail will be read only by the recipient, because, under certain circumstances, there may be a need to access this email by third subjects belonging to the Company.

R: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Alessandro Luccaroni - Diennea <al...@diennea.com.INVALID>]

I know of projects that embed Zookeeper so I don't think it is (only) a matter of run/build time.
I like what Mate said about announcing this switch today for a later major release (such as 4.0 that could come after 3.7) to give time to downstream applications to plan for this change.

Regards,
Alessandro Luccaroni

-----Messaggio originale-----
Da: Ted Dunning <te...@gmail.com>
Inviato: giovedì 22 ottobre 2020 15:58
A: dev@zookeeper.apache.org
Oggetto: Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

Allesandro,

Would your answer be different if you considered a strategy that allows
JRE8 to run ZK but requires JDK11 to build it?



On Thu, Oct 22, 2020 at 6:36 AM Szalay-Bekő Máté <sz...@gmail.com>
wrote:

> Hello All,
>
> A few reflections:
>
> - I don't think that backporting fixes from a JDK 11 only version to a
> JDK
> 8 compatible version would be necessarily a harder thing than any
> regular backport. It kind of depends on us (whether we use many JDK 11
> only features or not until we drop JDK 8 from all supported versions).
> Also if it gets more painful, then we can decide to limit the number
> of backported commits (e.g. only strictly to security fixes / CVEs)
> - But even if the above point is true (?), I would not do this cut (e.g.
> moving entirely to JDK 11 only) in a minor release. I think this
> should be a major change in 4.0. (maybe together with other more
> "risky" changes, like the separation of the client and server
> artifacts/code; or some incompatible changes in the leader election
> protocol. Although these are separate discussions)
> - So all-in-all I like the JDK 11 only option the most. But I wouldn't
> do it in 3.7 (which might happen very soon), but rather in 4.0. The
> question for me is wether to do any in-between step in 3.x. (like the
> options 1 or 2 above in Christopher's mail). I think it mainly should
> depend on the timing of 4.0.
>
> Kind regards,
> Mate
>
> On Thu, Oct 22, 2020 at 3:05 PM Flavio Junqueira <fp...@apache.org> wrote:
>
> > There are three points that stand out for me in this thread:
> >
> > - How do we determine how such a change affects our user base?
> > - How much effort do the different options induce with respect to
> > maintenance?
> > - What's the right timeline for changes and how do we communicate
> > them so that our users have enough time to prepare?
> >
> > Someone mentioned a PMC vote, and I don't think this should be a
> > closed vote, independent of how the conversation goes.
> >
> > -Flavio
> >
> > > On 22 Oct 2020, at 08:39, Alessandro Luccaroni - Diennea
> > <al...@diennea.com.INVALID> wrote:
> > >
> > > Hi all,
> > > If I might chime in as a zookeeper user (in multiple products) and
> > > a
> > follower of the project I think the drop of Java8 support (official
> and/or
> > unofficial) could be a big mistake.
> > >
> > > From my own company point of view we already support Java11 in all
> > > our
> > applications so we are not directly impacted (and we have upgrade
> > path
> for
> > older versions to provide to our customers).
> > > My worries resides in the (high) probability of a userbase
> > fragmentation: in the recent past Zookeeper development picked up
> > speed thanks to a bunch of new committers and PMCs after a period of
> > mostly maintenance focused works, but the number of active
> > committers and PMCs
> is
> > still very low for a project like this.
> > >
> > > I foresee the risk of spreading thin the resources of the project
> > > if we
> > force the userbase to stick to an older version and, in turn, we are
> forced
> > to backport many issue to the 3.6 branch.
> > >
> > > Alessandro Luccaroni
> > > Platform Manager @ Diennea - MagNews
> > > Tel.: (+39) 0546 066100 Int. 924 - Mob.: (+39) 393 7273519 Viale
> > > G.Marconi 30/14 - 48018 Faenza (RA) - Italy
> > >
> > > -----Messaggio originale-----
> > > Da: Christopher <ct...@apache.org>
> > > Inviato: giovedì 22 ottobre 2020 05:21
> > > A: dev@zookeeper.apache.org
> > > Oggetto: Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7
> > >
> > > I'm happy that this discussion has been so lively! I just want to
> > emphasize a few things:
> > >
> > > I really do understand the desire to continue to support Java 8...
> > > I
> get
> > it. But all the conversations around this seem based on what people
> > are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
> > > *future* release... so it should be based more on reasonable
> > expectations for users in the future, and less based on what is
> > happening today. I suspect *most* people today are still using 3.4
> > anyway (it was just so stable for so long...), but that shouldn't
> > mean the developers should hold back development on 3.5 and 3.6, any
> > more than today's users
> of
> > 3.5/3.6 should hold back 3.7.
> > >
> > > Some of the opinions expressed in this discussion seem to propose
> > > a
> > scenario where users are going to be updating to "bleeding edge"
> > > versions of ZooKeeper, but are going to insist on using Java 8.
> > > Personally, I find this to be implausible. In my experience,
> > > people
> > either upgrade everything as soon as they are able to, or they
> > upgrade
> each
> > thing individually, only when they are forced to. The first group
> > will be happy to move to Java 11 and ZK 3.7. The second group will
> > probably avoid
> > 3.7 anyway, and are fine sticking with 3.6, but if they had to
> > update to 3.7, they'd also be fine updating to Java 11 if they had
> > to in order to
> use
> > 3.7. I can't imagine the scenario where people are eagerly choosing
> > to upgrade to ZK 3.7, but miserly insisting on using Java 8. Perhaps
> > that scenario exists, but it's hard for me to imagine. Even so, my
> > proposal would still support even that group of people.
> > >
> > > I think there are now effectively three proposals being discussed
> > > in
> > this thread:
> > >
> > > 1. (Christopher's original proposal) passively support Java 8 at
> runtime
> > by making JDK 11 the minimum requirement to build and test.
> > > This scenario involves continuing to fix bugs, as they are
> > > discovered
> > and reported, that affect JDK 8, but passively, rather than proactively.
> > This proposal does *not* drop Java 8 support, but merely
> > de-emphasizes it in development of what will be 3.7 in the future,
> > and drops the
> requirement
> > to do dedicated testing with Java 8. I think this is low risk,
> > because it is very unlikely that the ZK devs would introduce a bug
> > that would affect only Java 8 and the compiler wouldn't catch it...
> > because the cross-compilation features of newer JDKs are really good.
> > >
> > > 2. (Enrico's alternate proposal) this variation of my proposal
> > > would
> > involve continuing to proactively support Java 8 by creating a
> > dedicated testing suite to test client code on Java 8. I think this
> > is a good
> option,
> > but since it involves a significantly higher amount of work than
> > option
> 1,
> > I think the cost-benefit analysis would show this to be not worth
> > the effort. Also, if it were implemented, it would need to be done
> > carefully
> to
> > avoid requiring developers to have concurrently installed both Java
> > 8 and Java 11 in order to perform a build, because requiring Java 8
> > at build
> time
> > while developing would be worse than we have today.
> > >
> > > 3. (Andor's preference) move to JDK 11 fully. This would provide
> > > no
> > support, passive or active, for Java 8 in ZK 3.7. To be honest, this
> > is
> my
> > personal favorite, and is the simplest to implement and communicate
> clearly
> > to end users in release notes. The only reason I proposed a passive
> support
> > of Java 8 instead of this is because I was trying to seek a
> > compromise
> from
> > the start. But, I think by far, this is the best option for the next
> > *future* release of ZK. If you wanted to make the change even more
> visible
> > to users, the version could even be bumped to 4.0.
> > >
> > > If this were to come to a VOTE by the PMC, in order to make a
> > > final
> > decision, I would recommend they vote on option 3, and then if that
> fails,
> > vote on option 1, and if that fails, keep things the way they are
> (because
> > option 2 is more work).
> > >
> > > Christopher
> > >
> > > P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK
> > > 11
> > and it works just fine there (as long as you add the missing runtime
> > jar for javax.activation:javax.activation-api:1.2.0 to its class
> > path, but
> that
> > was fixed in Hadoop 3.3).
> > >
> > > On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes
> <ta...@cloudera.com.invalid>
> > wrote:
> > >>
> > >> Hi All,
> > >>
> > >> I've just talked with a Hadoop/HDFS developer who told me what I
> > guessed.
> > >> With Hadoop3 they have just dropped JDK7 support, dropping JDK8
> > >> would mean a release of Hadoop4.
> > >> Since HADOOP-15338 is finished, they test with JDK8 and JDK11
> > >> both. As of today most of the Hadoop users are still on Hadoop2,
> > >> he doesn't expect
> > >> Hadoop4 soon.
> > >> As many Apache components depend on Hadoop and ZooKeeper they
> > >> won't hurry to JDK11 until they have to (they will probably go
> > >> one-by-one very slowly), which means if Hadoop stays on JDK8,
> > >> they would use the last ZK version which works on JDK8.
> > >> Do we want a ZK 3.4 again?
> > >>
> > >> Regards, Tamaas
> > >>
> > >>
> > >> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes
> > >> <ta...@cloudera.com>
> > wrote:
> > >>
> > >>> Hi All,
> > >>>
> > >>> Just to add my two cents.
> > >>>
> > >>> Upgrading to JDK11 looks inevitable sooner or later and I would
> > >>> definitely not wait until 2030 or 2026 when the extended support
> > >>> of
> > JDK8 ends.
> > >>> But on the other side I have to agree with Enrico and Patrick
> > >>> that far too many users are tied to JDK8 yet (not because they
> > >>> want to use JDK8, but because they have to), some of them are
> > >>> components of the Hadoop ecosystem, which would be a loss to tie
> > >>> them to 3.6.x for
> > years.
> > >>> Do we know the state of Hadoop? It builds with JDK8 at the
> > >>> moment, but do we know what are their plans to go to JDK11?
> > >>> When they move, we should move too, but I don't think it would
> > >>> be wise to do it earlier.
> > >>>
> > >>> Christopher's option looks like the golden path, but it needs
> > >>> some investment on the testing side as Enrico pointed it out.
> > >>> Could we agree on it as a compromise?
> > >>>
> > >>> Regards, Tamaas
> > >>>
> > >>> On Wed, Oct 21, 2020 at 7:11 PM Brent
> > >>> <br...@gmail.com>
> > wrote:
> > >>>
> > >>>> I think I was reacting to Enrico's earlier comment of:
> > >>>>
> > >>>> " ZooKeeper client is used by tons of users and unfortunately
> > >>>> many projects are still on JDK8, if we move ZooKeeper to JDK11
> > >>>> the risk is to block users from the adoption, that is that we
> > >>>> will see the world to stay on 3.6.x and we will have again a
> > >>>> long lived release line, like 3.4."
> > >>>>
> > >>>> It's a matter of whether or not a long-lived release line is
> > >>>> desirable/undesirable.  If everyone is OK keeping 3.6.x
> > >>>> up-to-date security/patch-wise (if not feature-wise) for the
> > >>>> next N years, then that's a potentially valid approach.  I
> > >>>> interpreted that comment as "a long-lived release line is
> > >>>> undesirable", but no one explicitly said that, I just read it that way.
> > >>>>
> > >>>> ~Brent
> > >>>>
> > >>>> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org>
> > wrote:
> > >>>>
> > >>>>> On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar
> > >>>>> <an...@apache.org>
> > wrote:
> > >>>>>
> > >>>>>> As far as I know Hbase, Solr and Kafka are already Java 11 ready.
> > >>>>>>
> > >>>>>> IMHO contributors of those projects should also put efforts
> > >>>>>> into
> > >>>> moving
> > >>>>>> forward.
> > >>>>>>
> > >>>>>> We’re not saying that you _have_ to move to Java 11.
> > >>>>>> Staying on Java 8? No problem, 3.6 is for you.
> > >>>>>> Want the fancy new features of 3.7? Work on it on your side too.
> > >>>>>>
> > >>>>>>
> > >>>>> ppl want things like security fixes. I believe the highlighted
> > >>>>> downside
> > >>>> is
> > >>>>> that we would need to continue to maintain 3.6.x rather than
> > >>>>> allowing users, and ourselves, to focus on trunk for
> > >>>>> production
> -"64
> > percent"
> > >>>> would
> > >>>>> be blocked.
> > >>>>>
> > >>>>> Patrick
> > >>>>>
> > >>>>>
> > >>>>>> Andor
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>> On 2020. Oct 21., at 17:52, Enrico Olivelli
> > >>>>>>> <eo...@gmail.com>
> > >>>>> wrote:
> > >>>>>>>
> > >>>>>>> Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
> > >>>>> andor@apache.org>
> > >>>>>> ha
> > >>>>>>> scritto:
> > >>>>>>>
> > >>>>>>>> Correct me if I'm wrong, but Oracle gets paid for extended
> > support.
> > >>>>>>>> Java 8 support until 2030 is not free of charge.
> > >>>>>>>>
> > >>>>>>>> "ZK may end up with a lot of users potentially locking
> > >>>>>>>> themselves
> > >>>> to
> > >>>>>>>> 3.6.x for a while as Enrico mentioned."
> > >>>>>>>>
> > >>>>>>>> That's true. What's the downside of that which we should
> > >>>>>>>> invest in
> > >>>> to
> > >>>>>>>> avoid?
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>> I see ZooKeeper used in many many projects, all of the
> > >>>>>>> HBase/Pulsar/Kafka/Solr ecosystem...
> > >>>>>>> they will have to move to JDK11 in order to move to the new
> > >>>>>>> ZK
> > >>>> version
> > >>>>>>> so probably they will stay on ZK 3.6
> > >>>>>>>
> > >>>>>>> Probably with Java 17 LTS released the cards will change on
> > >>>>>>> the
> > >>>> table
> > >>>>>>>
> > >>>>>>> Enrico
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> Andor
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> > >>>>>>>>> As a slightly different consideration, if you look at the
> > >>>> Long-Term
> > >>>>>>>>> Support
> > >>>>>>>>> (LTS) roadmaps for Java, currently Java 8 is set to have
> > >>>>>>>>> full
> > >>>> support
> > >>>>>>>>> until
> > >>>>>>>>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>> https://www.oracle.com/java/technologies/java-se-support-roadmap.
> > >>>>> html
> > >>>>>>>>> https://en.wikipedia.org/wiki/Java_version_history
> > >>>>>>>>>
> > >>>>>>>>> My guess is that a number of companies are still heavily
> > >>>>>>>>> invested
> > >>>> at
> > >>>>>>>>> the
> > >>>>>>>>> Java 8 level (I know a few) and with that kind of time
> > >>>>>>>>> horizon,
> > >>>> they
> > >>>>>>>>> have
> > >>>>>>>>> no real motivation to upgrade for quite a while.  If the
> > >>>>>>>>> recent Python 2 deprecation is anything to go by, they
> > >>>>>>>>> won't do it until they have to.
> > >>>>>>>>>
> > >>>>>>>>> Not saying Java 8 isn't *very* old (2014 release it seems
> > >>>>>>>>> like?)
> > >>>> and
> > >>>>>>>>> I'm
> > >>>>>>>>> not invested heavily either way, but this might suggest
> > >>>>>>>>> that ZK
> > >>>> may
> > >>>>>>>>> end up
> > >>>>>>>>> with a lot of users potentially locking themselves to
> > >>>>>>>>> 3.6.x for a while as Enrico mentioned.
> > >>>>>>>>>
> > >>>>>>>>> (Not a major contributor, but wanted to chime in since I
> > >>>>>>>>> just had this conversation with a bunch of people
> > >>>>>>>>> professionally recently)
> > >>>>>>>>>
> > >>>>>>>>> Brent
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar
> > >>>>>>>>> <an...@apache.org>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Thanks for the summary.
> > >>>>>>>>>>
> > >>>>>>>>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully.
> > >>>>>>>>>> Other projects will upgrade once they’re JDK11 compliant,
> > >>>>>>>>>> otherwise they will
> > >>>> stay
> > >>>>>>>>>> on 3.5
> > >>>>>>>>>> or 3.6. Both version are quite recent in ZooKeeper-terms,
> > >>>>>>>>>> we already planned big changes for 3.7.0 and JDK 11 could
> > >>>>>>>>>> be one of them.
> > >>>>>>>>>>
> > >>>>>>>>>> Don’t put extra burden on the ZK community to help others
> > >>>>>>>>>> staying on ancient Java versions.
> > >>>>>>>>>>
> > >>>>>>>>>> Andor
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
> > >>>> eolivelli@gmail.com>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Let me recap
> > >>>>>>>>>>> - Christopher is proposing to move to JDK11
> > >>>>>>>>>>> - ZooKeeper client and server are bundled and coded and
> > >>>>>>>>>>> tested together
> > >>>>>>>>>> in
> > >>>>>>>>>>> zookeeper-server
> > >>>>>>>>>>> - Enrico is concerned about the need of testing
> > >>>>>>>>>>> ZooKeeper client on JDK8 (not a problem to move the
> > >>>>>>>>>>> server to
> > >>>>>>>>>>> JDK11)
> > >>>>>>>>>>>
> > >>>>>>>>>>> ZooKeeper client is used by tons of users and
> > >>>>>>>>>>> unfortunately many projects are still on JDK8, if we
> > >>>>>>>>>>> move ZooKeeper to JDK11 the risk is to block
> > >>>>>>>>>> users
> > >>>>>>>>>>> from the adoption,
> > >>>>>>>>>>> that is that we will see the world to stay on 3.6.x and
> > >>>>>>>>>>> we will have
> > >>>>>>>>>> again
> > >>>>>>>>>>> a long lived release line, like 3.4.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Testing the client on JDK8 would be possible if we
> > >>>>>>>>>>> create some kind of additional module with system tests,
> > >>>>>>>>>>> then we can start the
> > >>>> server
> > >>>>>>>>>>> on
> > >>>>>>>>>> docker
> > >>>>>>>>>>> on JDK11+ and start a client on JDK8 with Maven
> > >>>>>>>>>>> toolchain it should possible to run surefire tests using
> > >>>>>>>>>>> a separate JVM.
> > >>>>>>>>>>>
> > >>>>>>>>>>> So in my vision 2 options:
> > >>>>>>>>>>> 1) fully JDK11 - drop JDK8 at all
> > >>>>>>>>>>> 2) build with JDK11 - server only on JDK11 - add system
> > >>>>>>>>>>> tests with docker and toolchains that ensure the
> > >>>>>>>>>>> ZooKeeper client (and all
> > >>>>>>>>>>> dependencies)
> > >>>>>>>>>>> still work on JDK8
> > >>>>>>>>>>>
> > >>>>>>>>>>> From my point of view about the ZooKeeper ecosystem
> > >>>>>>>>>>> option 2) will be far better, but we need resources to
> > >>>>>>>>>>> work on a new test suite.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Enrico
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> > >>>>>>>>>>> andor@apache.org>
> > >>>>>>>>>> ha
> > >>>>>>>>>>> scritto:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Tamas, Enrico,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Sorry I don’t follow. Why do we have to test the client
> > >>>>>>>>>>>> with JDK 8 in version 3.7.0?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Andor
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
> > >>>>>>>>>>>>> tamaas@cloudera.com.INVALID>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>> Hi Enrico,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Separating ZooKeeper client and server is a huge work,
> > >>>>>>>>>>>>> but we might not need it.
> > >>>>>>>>>>>>> As you mentioned we have to test ZK client with Java
> > >>>>>>>>>>>>> 8, what about separating only the test cases which we
> > >>>>>>>>>>>>> need to run with
> > >>>>>>>>>>>>> Java8 too?
> > >>>>>>>>>>>>> In Curator we have the ZK compatibility tests where we
> > >>>>>>>>>>>>> run a limited
> > >>>>>>>>>>>> amount
> > >>>>>>>>>>>>> of Curator's jUnit tests with a different ZK version.
> > >>>>>>>>>>>>> We might be able to do the same here, tag tests which
> > >>>>>>>>>>>>> are testing ZK
> > >>>>>>>>>>>> client
> > >>>>>>>>>>>>> and run them separately with Java 8. The only
> > >>>>>>>>>>>>> limitation is that these tests must stay JDK8
> > >>>>>>>>>>>>> compatible.
> > >>>>>>>>>>>>> But from the tags we will see which ones are those.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> What do you think?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Regards, Tamaas
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> > >>>>>>>>>>>>> eolivelli@gmail.com>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>> Christopher
> > >>>>>>>>>>>>>> I appreciate your idea and I also moved lots of my
> > >>>>>>>>>>>>>> projects to work
> > >>>>>>>>>> the
> > >>>>>>>>>>>> way
> > >>>>>>>>>>>>>> you are suggesting.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> We must run tests using real jdk8 to test the
> > >>>>>>>>>>>>>> Zookeeper client. We
> > >>>>>>>>>> must
> > >>>>>>>>>>>>>> ensure that Zookeeper works well, especially while
> > >>>>>>>>>>>>>> dealing with
> > >>>>>>>>>> security
> > >>>>>>>>>>>>>> stuff.
> > >>>>>>>>>>>>>> Currently the client is in the same module of the
> > >>>>>>>>>>>>>> server and it will
> > >>>>>>>>>>>> take a
> > >>>>>>>>>>>>>> good (huge) amount of work to separate them
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Enrico
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher
> > >>>>>>>>>>>>>> <ct...@apache.org> ha
> > >>>>>>>>>> scritto:
> > >>>>>>>>>>>>>>> Hi ZK Devs,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> With recent advancements in Java (since Java 9), it
> > >>>>>>>>>>>>>>> is now generally no longer necessary to require that
> > >>>>>>>>>>>>>>> software be developed on an older JDK in order to
> > >>>>>>>>>>>>>>> have confidence that it will run on the older
> > >>>>>>>>>>>>>>> version of Java. This is because, as of Java 9, all
> > >>>>>>>>>>>>>>> JDK releases have better support for
> > >>>>>>>>>>>>>>> cross-compilation to older Java versions.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> What this means is that developers can confidently
> > >>>>>>>>>>>>>>> make the build requirements for a project higher
> > >>>>>>>>>>>>>>> than the Java version that will actually be
> > >>>>>>>>>>>>>>> supported at runtime.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> In fact, ZooKeeper already supports the necessary
> > >>>>>>>>>>>>>>> flags in its Maven build configuration to ensure
> > >>>>>>>>>>>>>>> that it uses JDK 8 compliance when building on a
> > >>>>>>>>>>>>>>> newer JDK (I added this way back in
> > >>>>>>>>>>>>>>> ZOOKEEPER-3739 /
> > >>>>>>>>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> So, I propose that we make JDK 11 the new minimum
> > >>>>>>>>>>>>>>> version to *build* ZooKeeper with. This would not
> > >>>>>>>>>>>>>>> change the runtime requirement, which would remain
> > >>>>>>>>>>>>>>> at JDK 8.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> The only necessary change to make this happen would
> > >>>>>>>>>>>>>>> be to add the minimum Java version to the
> > >>>>>>>>>>>>>>> maven-enforcer-plugin (like
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7e
> > >>>> cdd1
> > >>>> 1d5dedb146519/pom.xml#L1162-L1164
> > >>>>>>>>>>>>>>> )
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> This would allow ZooKeeper to to streamline its
> > >>>>>>>>>>>>>>> development process a little bit by reducing the
> > >>>>>>>>>>>>>>> amount of CI testing that is done as part of the
> > >>>>>>>>>>>>>>> build. In other words, we can drop the CI builds for
> > >>>>>>>>>>>>>>> JDK 8, which saves on build resources and time. The
> > >>>>>>>>>>>>>>> return on investment is so low for the JDK 8 builds
> > >>>>>>>>>>>>>>> anyway, because of the improved cross-compilation in
> > >>>>>>>>>>>>>>> newer JDKs. So, there's not much value in building
> > >>>>>>>>>>>>>>> on JDK 8 anyway.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Of course, I am only recommending this for *new*
> > >>>>>>>>>>>>>>> release lines, starting with ZooKeeper 3.7.0/master
> > >>>>>>>>>>>>>>> branch, because I would not want to change
> > >>>>>>>>>>>>>>> expectations for users who will build their own
> > >>>>>>>>>>>>>>> 3.5 and 3.6
> > >>>>>>>>>>>>>>> versions as they continue to have patch versions
> > >>>>>>>>>>>>>>> released.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> What do you think?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Kind Regards,
> > >>>>>>>>>>>>>>> Christopher
> > >>>>>>>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >
> > > ________________________________
> > >
> > > CONFIDENTIALITY & PRIVACY NOTICE
> > > This e-mail (including any attachments) is strictly confidential
> > > and
> may
> > also contain privileged information. If you are not the intended
> recipient
> > you are not authorised to read, print, save, process or disclose
> > this message. If you have received this message by mistake, please
> > inform the sender immediately and destroy this e-mail, its
> > attachments and any
> copies.
> > Any use, distribution, reproduction or disclosure by any person
> > other
> than
> > the intended recipient is strictly prohibited and the person
> > responsible may incur in penalties.
> > > The use of this e-mail is only for professional purposes; there is
> > > no
> > guarantee that the correspondence towards this e-mail will be read
> > only
> by
> > the recipient, because, under certain circumstances, there may be a
> > need
> to
> > access this email by third subjects belonging to the Company.
> >
> >
>

________________________________

CONFIDENTIALITY & PRIVACY NOTICE
This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties.
The use of this e-mail is only for professional purposes; there is no guarantee that the correspondence towards this e-mail will be read only by the recipient, because, under certain circumstances, there may be a need to access this email by third subjects belonging to the Company.

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Ted Dunning <te...@gmail.com>]

Allesandro,

Would your answer be different if you considered a strategy that allows
JRE8 to run ZK but requires JDK11 to build it?



On Thu, Oct 22, 2020 at 6:36 AM Szalay-Bekő Máté <sz...@gmail.com>
wrote:

> Hello All,
>
> A few reflections:
>
> - I don't think that backporting fixes from a JDK 11 only version to a JDK
> 8 compatible version would be necessarily a harder thing than any regular
> backport. It kind of depends on us (whether we use many JDK 11 only
> features or not until we drop JDK 8 from all supported versions). Also if
> it gets more painful, then we can decide to limit the number of backported
> commits (e.g. only strictly to security fixes / CVEs)
> - But even if the above point is true (?), I would not do this cut (e.g.
> moving entirely to JDK 11 only) in a minor release. I think this should be
> a major change in 4.0. (maybe together with other more "risky" changes,
> like the separation of the client and server artifacts/code; or some
> incompatible changes in the leader election protocol. Although these are
> separate discussions)
> - So all-in-all I like the JDK 11 only option the most. But I wouldn't do
> it in 3.7 (which might happen very soon), but rather in 4.0. The question
> for me is wether to do any in-between step in 3.x. (like the options 1 or 2
> above in Christopher's mail). I think it mainly should depend on the timing
> of 4.0.
>
> Kind regards,
> Mate
>
> On Thu, Oct 22, 2020 at 3:05 PM Flavio Junqueira <fp...@apache.org> wrote:
>
> > There are three points that stand out for me in this thread:
> >
> > - How do we determine how such a change affects our user base?
> > - How much effort do the different options induce with respect to
> > maintenance?
> > - What's the right timeline for changes and how do we communicate them so
> > that our users have enough time to prepare?
> >
> > Someone mentioned a PMC vote, and I don't think this should be a closed
> > vote, independent of how the conversation goes.
> >
> > -Flavio
> >
> > > On 22 Oct 2020, at 08:39, Alessandro Luccaroni - Diennea
> > <al...@diennea.com.INVALID> wrote:
> > >
> > > Hi all,
> > > If I might chime in as a zookeeper user (in multiple products) and a
> > follower of the project I think the drop of Java8 support (official
> and/or
> > unofficial) could be a big mistake.
> > >
> > > From my own company point of view we already support Java11 in all our
> > applications so we are not directly impacted (and we have upgrade path
> for
> > older versions to provide to our customers).
> > > My worries resides in the (high) probability of a userbase
> > fragmentation: in the recent past Zookeeper development picked up speed
> > thanks to a bunch of new committers and PMCs after a period of mostly
> > maintenance focused works, but the number of active committers and PMCs
> is
> > still very low for a project like this.
> > >
> > > I foresee the risk of spreading thin the resources of the project if we
> > force the userbase to stick to an older version and, in turn, we are
> forced
> > to backport many issue to the 3.6 branch.
> > >
> > > Alessandro Luccaroni
> > > Platform Manager @ Diennea - MagNews
> > > Tel.: (+39) 0546 066100 Int. 924 - Mob.: (+39) 393 7273519
> > > Viale G.Marconi 30/14 - 48018 Faenza (RA) - Italy
> > >
> > > -----Messaggio originale-----
> > > Da: Christopher <ct...@apache.org>
> > > Inviato: giovedì 22 ottobre 2020 05:21
> > > A: dev@zookeeper.apache.org
> > > Oggetto: Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7
> > >
> > > I'm happy that this discussion has been so lively! I just want to
> > emphasize a few things:
> > >
> > > I really do understand the desire to continue to support Java 8... I
> get
> > it. But all the conversations around this seem based on what people are
> > doing *today*. But, ZK 3.7 is *tomorrow's* version... a
> > > *future* release... so it should be based more on reasonable
> > expectations for users in the future, and less based on what is happening
> > today. I suspect *most* people today are still using 3.4 anyway (it was
> > just so stable for so long...), but that shouldn't mean the developers
> > should hold back development on 3.5 and 3.6, any more than today's users
> of
> > 3.5/3.6 should hold back 3.7.
> > >
> > > Some of the opinions expressed in this discussion seem to propose a
> > scenario where users are going to be updating to "bleeding edge"
> > > versions of ZooKeeper, but are going to insist on using Java 8.
> > > Personally, I find this to be implausible. In my experience, people
> > either upgrade everything as soon as they are able to, or they upgrade
> each
> > thing individually, only when they are forced to. The first group will be
> > happy to move to Java 11 and ZK 3.7. The second group will probably avoid
> > 3.7 anyway, and are fine sticking with 3.6, but if they had to update to
> > 3.7, they'd also be fine updating to Java 11 if they had to in order to
> use
> > 3.7. I can't imagine the scenario where people are eagerly choosing to
> > upgrade to ZK 3.7, but miserly insisting on using Java 8. Perhaps that
> > scenario exists, but it's hard for me to imagine. Even so, my proposal
> > would still support even that group of people.
> > >
> > > I think there are now effectively three proposals being discussed in
> > this thread:
> > >
> > > 1. (Christopher's original proposal) passively support Java 8 at
> runtime
> > by making JDK 11 the minimum requirement to build and test.
> > > This scenario involves continuing to fix bugs, as they are discovered
> > and reported, that affect JDK 8, but passively, rather than proactively.
> > This proposal does *not* drop Java 8 support, but merely de-emphasizes it
> > in development of what will be 3.7 in the future, and drops the
> requirement
> > to do dedicated testing with Java 8. I think this is low risk, because it
> > is very unlikely that the ZK devs would introduce a bug that would affect
> > only Java 8 and the compiler wouldn't catch it... because the
> > cross-compilation features of newer JDKs are really good.
> > >
> > > 2. (Enrico's alternate proposal) this variation of my proposal would
> > involve continuing to proactively support Java 8 by creating a dedicated
> > testing suite to test client code on Java 8. I think this is a good
> option,
> > but since it involves a significantly higher amount of work than option
> 1,
> > I think the cost-benefit analysis would show this to be not worth the
> > effort. Also, if it were implemented, it would need to be done carefully
> to
> > avoid requiring developers to have concurrently installed both Java 8 and
> > Java 11 in order to perform a build, because requiring Java 8 at build
> time
> > while developing would be worse than we have today.
> > >
> > > 3. (Andor's preference) move to JDK 11 fully. This would provide no
> > support, passive or active, for Java 8 in ZK 3.7. To be honest, this is
> my
> > personal favorite, and is the simplest to implement and communicate
> clearly
> > to end users in release notes. The only reason I proposed a passive
> support
> > of Java 8 instead of this is because I was trying to seek a compromise
> from
> > the start. But, I think by far, this is the best option for the next
> > *future* release of ZK. If you wanted to make the change even more
> visible
> > to users, the version could even be bumped to 4.0.
> > >
> > > If this were to come to a VOTE by the PMC, in order to make a final
> > decision, I would recommend they vote on option 3, and then if that
> fails,
> > vote on option 1, and if that fails, keep things the way they are
> (because
> > option 2 is more work).
> > >
> > > Christopher
> > >
> > > P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK 11
> > and it works just fine there (as long as you add the missing runtime jar
> > for javax.activation:javax.activation-api:1.2.0 to its class path, but
> that
> > was fixed in Hadoop 3.3).
> > >
> > > On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes
> <ta...@cloudera.com.invalid>
> > wrote:
> > >>
> > >> Hi All,
> > >>
> > >> I've just talked with a Hadoop/HDFS developer who told me what I
> > guessed.
> > >> With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would
> > >> mean a release of Hadoop4.
> > >> Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both. As
> > >> of today most of the Hadoop users are still on Hadoop2, he doesn't
> > >> expect
> > >> Hadoop4 soon.
> > >> As many Apache components depend on Hadoop and ZooKeeper they won't
> > >> hurry to JDK11 until they have to (they will probably go one-by-one
> > >> very slowly), which means if Hadoop stays on JDK8, they would use the
> > >> last ZK version which works on JDK8.
> > >> Do we want a ZK 3.4 again?
> > >>
> > >> Regards, Tamaas
> > >>
> > >>
> > >> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <ta...@cloudera.com>
> > wrote:
> > >>
> > >>> Hi All,
> > >>>
> > >>> Just to add my two cents.
> > >>>
> > >>> Upgrading to JDK11 looks inevitable sooner or later and I would
> > >>> definitely not wait until 2030 or 2026 when the extended support of
> > JDK8 ends.
> > >>> But on the other side I have to agree with Enrico and Patrick that
> > >>> far too many users are tied to JDK8 yet (not because they want to
> > >>> use JDK8, but because they have to), some of them are components of
> > >>> the Hadoop ecosystem, which would be a loss to tie them to 3.6.x for
> > years.
> > >>> Do we know the state of Hadoop? It builds with JDK8 at the moment,
> > >>> but do we know what are their plans to go to JDK11?
> > >>> When they move, we should move too, but I don't think it would be
> > >>> wise to do it earlier.
> > >>>
> > >>> Christopher's option looks like the golden path, but it needs some
> > >>> investment on the testing side as Enrico pointed it out.
> > >>> Could we agree on it as a compromise?
> > >>>
> > >>> Regards, Tamaas
> > >>>
> > >>> On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com>
> > wrote:
> > >>>
> > >>>> I think I was reacting to Enrico's earlier comment of:
> > >>>>
> > >>>> " ZooKeeper client is used by tons of users and unfortunately many
> > >>>> projects are still on JDK8, if we move ZooKeeper to JDK11 the risk
> > >>>> is to block users from the adoption, that is that we will see the
> > >>>> world to stay on 3.6.x and we will have again a long lived release
> > >>>> line, like 3.4."
> > >>>>
> > >>>> It's a matter of whether or not a long-lived release line is
> > >>>> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
> > >>>> security/patch-wise (if not feature-wise) for the next N years,
> > >>>> then that's a potentially valid approach.  I interpreted that
> > >>>> comment as "a long-lived release line is undesirable", but no one
> > >>>> explicitly said that, I just read it that way.
> > >>>>
> > >>>> ~Brent
> > >>>>
> > >>>> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org>
> > wrote:
> > >>>>
> > >>>>> On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org>
> > wrote:
> > >>>>>
> > >>>>>> As far as I know Hbase, Solr and Kafka are already Java 11 ready.
> > >>>>>>
> > >>>>>> IMHO contributors of those projects should also put efforts
> > >>>>>> into
> > >>>> moving
> > >>>>>> forward.
> > >>>>>>
> > >>>>>> We’re not saying that you _have_ to move to Java 11.
> > >>>>>> Staying on Java 8? No problem, 3.6 is for you.
> > >>>>>> Want the fancy new features of 3.7? Work on it on your side too.
> > >>>>>>
> > >>>>>>
> > >>>>> ppl want things like security fixes. I believe the highlighted
> > >>>>> downside
> > >>>> is
> > >>>>> that we would need to continue to maintain 3.6.x rather than
> > >>>>> allowing users, and ourselves, to focus on trunk for production
> -"64
> > percent"
> > >>>> would
> > >>>>> be blocked.
> > >>>>>
> > >>>>> Patrick
> > >>>>>
> > >>>>>
> > >>>>>> Andor
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>> On 2020. Oct 21., at 17:52, Enrico Olivelli
> > >>>>>>> <eo...@gmail.com>
> > >>>>> wrote:
> > >>>>>>>
> > >>>>>>> Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
> > >>>>> andor@apache.org>
> > >>>>>> ha
> > >>>>>>> scritto:
> > >>>>>>>
> > >>>>>>>> Correct me if I'm wrong, but Oracle gets paid for extended
> > support.
> > >>>>>>>> Java 8 support until 2030 is not free of charge.
> > >>>>>>>>
> > >>>>>>>> "ZK may end up with a lot of users potentially locking
> > >>>>>>>> themselves
> > >>>> to
> > >>>>>>>> 3.6.x for a while as Enrico mentioned."
> > >>>>>>>>
> > >>>>>>>> That's true. What's the downside of that which we should
> > >>>>>>>> invest in
> > >>>> to
> > >>>>>>>> avoid?
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>> I see ZooKeeper used in many many projects, all of the
> > >>>>>>> HBase/Pulsar/Kafka/Solr ecosystem...
> > >>>>>>> they will have to move to JDK11 in order to move to the new
> > >>>>>>> ZK
> > >>>> version
> > >>>>>>> so probably they will stay on ZK 3.6
> > >>>>>>>
> > >>>>>>> Probably with Java 17 LTS released the cards will change on
> > >>>>>>> the
> > >>>> table
> > >>>>>>>
> > >>>>>>> Enrico
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> Andor
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> > >>>>>>>>> As a slightly different consideration, if you look at the
> > >>>> Long-Term
> > >>>>>>>>> Support
> > >>>>>>>>> (LTS) roadmaps for Java, currently Java 8 is set to have
> > >>>>>>>>> full
> > >>>> support
> > >>>>>>>>> until
> > >>>>>>>>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>> https://www.oracle.com/java/technologies/java-se-support-roadmap.
> > >>>>> html
> > >>>>>>>>> https://en.wikipedia.org/wiki/Java_version_history
> > >>>>>>>>>
> > >>>>>>>>> My guess is that a number of companies are still heavily
> > >>>>>>>>> invested
> > >>>> at
> > >>>>>>>>> the
> > >>>>>>>>> Java 8 level (I know a few) and with that kind of time
> > >>>>>>>>> horizon,
> > >>>> they
> > >>>>>>>>> have
> > >>>>>>>>> no real motivation to upgrade for quite a while.  If the
> > >>>>>>>>> recent Python 2 deprecation is anything to go by, they
> > >>>>>>>>> won't do it until they have to.
> > >>>>>>>>>
> > >>>>>>>>> Not saying Java 8 isn't *very* old (2014 release it seems
> > >>>>>>>>> like?)
> > >>>> and
> > >>>>>>>>> I'm
> > >>>>>>>>> not invested heavily either way, but this might suggest
> > >>>>>>>>> that ZK
> > >>>> may
> > >>>>>>>>> end up
> > >>>>>>>>> with a lot of users potentially locking themselves to 3.6.x
> > >>>>>>>>> for a while as Enrico mentioned.
> > >>>>>>>>>
> > >>>>>>>>> (Not a major contributor, but wanted to chime in since I
> > >>>>>>>>> just had this conversation with a bunch of people
> > >>>>>>>>> professionally recently)
> > >>>>>>>>>
> > >>>>>>>>> Brent
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar
> > >>>>>>>>> <an...@apache.org>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Thanks for the summary.
> > >>>>>>>>>>
> > >>>>>>>>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully.
> > >>>>>>>>>> Other projects will upgrade once they’re JDK11 compliant,
> > >>>>>>>>>> otherwise they will
> > >>>> stay
> > >>>>>>>>>> on 3.5
> > >>>>>>>>>> or 3.6. Both version are quite recent in ZooKeeper-terms,
> > >>>>>>>>>> we already planned big changes for 3.7.0 and JDK 11 could
> > >>>>>>>>>> be one of them.
> > >>>>>>>>>>
> > >>>>>>>>>> Don’t put extra burden on the ZK community to help others
> > >>>>>>>>>> staying on ancient Java versions.
> > >>>>>>>>>>
> > >>>>>>>>>> Andor
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
> > >>>> eolivelli@gmail.com>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Let me recap
> > >>>>>>>>>>> - Christopher is proposing to move to JDK11
> > >>>>>>>>>>> - ZooKeeper client and server are bundled and coded and
> > >>>>>>>>>>> tested together
> > >>>>>>>>>> in
> > >>>>>>>>>>> zookeeper-server
> > >>>>>>>>>>> - Enrico is concerned about the need of testing ZooKeeper
> > >>>>>>>>>>> client on JDK8 (not a problem to move the server to
> > >>>>>>>>>>> JDK11)
> > >>>>>>>>>>>
> > >>>>>>>>>>> ZooKeeper client is used by tons of users and
> > >>>>>>>>>>> unfortunately many projects are still on JDK8, if we move
> > >>>>>>>>>>> ZooKeeper to JDK11 the risk is to block
> > >>>>>>>>>> users
> > >>>>>>>>>>> from the adoption,
> > >>>>>>>>>>> that is that we will see the world to stay on 3.6.x and
> > >>>>>>>>>>> we will have
> > >>>>>>>>>> again
> > >>>>>>>>>>> a long lived release line, like 3.4.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Testing the client on JDK8 would be possible if we create
> > >>>>>>>>>>> some kind of additional module with system tests, then we
> > >>>>>>>>>>> can start the
> > >>>> server
> > >>>>>>>>>>> on
> > >>>>>>>>>> docker
> > >>>>>>>>>>> on JDK11+ and start a client on JDK8 with Maven toolchain
> > >>>>>>>>>>> it should possible to run surefire tests using a separate
> > >>>>>>>>>>> JVM.
> > >>>>>>>>>>>
> > >>>>>>>>>>> So in my vision 2 options:
> > >>>>>>>>>>> 1) fully JDK11 - drop JDK8 at all
> > >>>>>>>>>>> 2) build with JDK11 - server only on JDK11 - add system
> > >>>>>>>>>>> tests with docker and toolchains that ensure the
> > >>>>>>>>>>> ZooKeeper client (and all
> > >>>>>>>>>>> dependencies)
> > >>>>>>>>>>> still work on JDK8
> > >>>>>>>>>>>
> > >>>>>>>>>>> From my point of view about the ZooKeeper ecosystem
> > >>>>>>>>>>> option 2) will be far better, but we need resources to
> > >>>>>>>>>>> work on a new test suite.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Enrico
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> > >>>>>>>>>>> andor@apache.org>
> > >>>>>>>>>> ha
> > >>>>>>>>>>> scritto:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Tamas, Enrico,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Sorry I don’t follow. Why do we have to test the client
> > >>>>>>>>>>>> with JDK 8 in version 3.7.0?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Andor
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
> > >>>>>>>>>>>>> tamaas@cloudera.com.INVALID>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>> Hi Enrico,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Separating ZooKeeper client and server is a huge work,
> > >>>>>>>>>>>>> but we might not need it.
> > >>>>>>>>>>>>> As you mentioned we have to test ZK client with Java 8,
> > >>>>>>>>>>>>> what about separating only the test cases which we need
> > >>>>>>>>>>>>> to run with
> > >>>>>>>>>>>>> Java8 too?
> > >>>>>>>>>>>>> In Curator we have the ZK compatibility tests where we
> > >>>>>>>>>>>>> run a limited
> > >>>>>>>>>>>> amount
> > >>>>>>>>>>>>> of Curator's jUnit tests with a different ZK version.
> > >>>>>>>>>>>>> We might be able to do the same here, tag tests which
> > >>>>>>>>>>>>> are testing ZK
> > >>>>>>>>>>>> client
> > >>>>>>>>>>>>> and run them separately with Java 8. The only
> > >>>>>>>>>>>>> limitation is that these tests must stay JDK8
> > >>>>>>>>>>>>> compatible.
> > >>>>>>>>>>>>> But from the tags we will see which ones are those.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> What do you think?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Regards, Tamaas
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> > >>>>>>>>>>>>> eolivelli@gmail.com>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>> Christopher
> > >>>>>>>>>>>>>> I appreciate your idea and I also moved lots of my
> > >>>>>>>>>>>>>> projects to work
> > >>>>>>>>>> the
> > >>>>>>>>>>>> way
> > >>>>>>>>>>>>>> you are suggesting.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> We must run tests using real jdk8 to test the
> > >>>>>>>>>>>>>> Zookeeper client. We
> > >>>>>>>>>> must
> > >>>>>>>>>>>>>> ensure that Zookeeper works well, especially while
> > >>>>>>>>>>>>>> dealing with
> > >>>>>>>>>> security
> > >>>>>>>>>>>>>> stuff.
> > >>>>>>>>>>>>>> Currently the client is in the same module of the
> > >>>>>>>>>>>>>> server and it will
> > >>>>>>>>>>>> take a
> > >>>>>>>>>>>>>> good (huge) amount of work to separate them
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Enrico
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher
> > >>>>>>>>>>>>>> <ct...@apache.org> ha
> > >>>>>>>>>> scritto:
> > >>>>>>>>>>>>>>> Hi ZK Devs,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> With recent advancements in Java (since Java 9), it
> > >>>>>>>>>>>>>>> is now generally no longer necessary to require that
> > >>>>>>>>>>>>>>> software be developed on an older JDK in order to
> > >>>>>>>>>>>>>>> have confidence that it will run on the older version
> > >>>>>>>>>>>>>>> of Java. This is because, as of Java 9, all JDK
> > >>>>>>>>>>>>>>> releases have better support for cross-compilation to
> > >>>>>>>>>>>>>>> older Java versions.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> What this means is that developers can confidently
> > >>>>>>>>>>>>>>> make the build requirements for a project higher than
> > >>>>>>>>>>>>>>> the Java version that will actually be supported at
> > >>>>>>>>>>>>>>> runtime.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> In fact, ZooKeeper already supports the necessary
> > >>>>>>>>>>>>>>> flags in its Maven build configuration to ensure that
> > >>>>>>>>>>>>>>> it uses JDK 8 compliance when building on a newer JDK
> > >>>>>>>>>>>>>>> (I added this way back in
> > >>>>>>>>>>>>>>> ZOOKEEPER-3739 /
> > >>>>>>>>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> So, I propose that we make JDK 11 the new minimum
> > >>>>>>>>>>>>>>> version to *build* ZooKeeper with. This would not
> > >>>>>>>>>>>>>>> change the runtime requirement, which would remain at
> > >>>>>>>>>>>>>>> JDK 8.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> The only necessary change to make this happen would
> > >>>>>>>>>>>>>>> be to add the minimum Java version to the
> > >>>>>>>>>>>>>>> maven-enforcer-plugin (like
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>
> > >>>> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd1
> > >>>> 1d5dedb146519/pom.xml#L1162-L1164
> > >>>>>>>>>>>>>>> )
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> This would allow ZooKeeper to to streamline its
> > >>>>>>>>>>>>>>> development process a little bit by reducing the
> > >>>>>>>>>>>>>>> amount of CI testing that is done as part of the
> > >>>>>>>>>>>>>>> build. In other words, we can drop the CI builds for
> > >>>>>>>>>>>>>>> JDK 8, which saves on build resources and time. The
> > >>>>>>>>>>>>>>> return on investment is so low for the JDK 8 builds
> > >>>>>>>>>>>>>>> anyway, because of the improved cross-compilation in
> > >>>>>>>>>>>>>>> newer JDKs. So, there's not much value in building on
> > >>>>>>>>>>>>>>> JDK 8 anyway.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Of course, I am only recommending this for *new*
> > >>>>>>>>>>>>>>> release lines, starting with ZooKeeper 3.7.0/master
> > >>>>>>>>>>>>>>> branch, because I would not want to change
> > >>>>>>>>>>>>>>> expectations for users who will build their own
> > >>>>>>>>>>>>>>> 3.5 and 3.6
> > >>>>>>>>>>>>>>> versions as they continue to have patch versions
> > >>>>>>>>>>>>>>> released.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> What do you think?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Kind Regards,
> > >>>>>>>>>>>>>>> Christopher
> > >>>>>>>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >
> > > ________________________________
> > >
> > > CONFIDENTIALITY & PRIVACY NOTICE
> > > This e-mail (including any attachments) is strictly confidential and
> may
> > also contain privileged information. If you are not the intended
> recipient
> > you are not authorised to read, print, save, process or disclose this
> > message. If you have received this message by mistake, please inform the
> > sender immediately and destroy this e-mail, its attachments and any
> copies.
> > Any use, distribution, reproduction or disclosure by any person other
> than
> > the intended recipient is strictly prohibited and the person responsible
> > may incur in penalties.
> > > The use of this e-mail is only for professional purposes; there is no
> > guarantee that the correspondence towards this e-mail will be read only
> by
> > the recipient, because, under certain circumstances, there may be a need
> to
> > access this email by third subjects belonging to the Company.
> >
> >
>

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Szalay-Bekő Máté <sz...@gmail.com>]

Hello All,

A few reflections:

- I don't think that backporting fixes from a JDK 11 only version to a JDK
8 compatible version would be necessarily a harder thing than any regular
backport. It kind of depends on us (whether we use many JDK 11 only
features or not until we drop JDK 8 from all supported versions). Also if
it gets more painful, then we can decide to limit the number of backported
commits (e.g. only strictly to security fixes / CVEs)
- But even if the above point is true (?), I would not do this cut (e.g.
moving entirely to JDK 11 only) in a minor release. I think this should be
a major change in 4.0. (maybe together with other more "risky" changes,
like the separation of the client and server artifacts/code; or some
incompatible changes in the leader election protocol. Although these are
separate discussions)
- So all-in-all I like the JDK 11 only option the most. But I wouldn't do
it in 3.7 (which might happen very soon), but rather in 4.0. The question
for me is wether to do any in-between step in 3.x. (like the options 1 or 2
above in Christopher's mail). I think it mainly should depend on the timing
of 4.0.

Kind regards,
Mate

On Thu, Oct 22, 2020 at 3:05 PM Flavio Junqueira <fp...@apache.org> wrote:

> There are three points that stand out for me in this thread:
>
> - How do we determine how such a change affects our user base?
> - How much effort do the different options induce with respect to
> maintenance?
> - What's the right timeline for changes and how do we communicate them so
> that our users have enough time to prepare?
>
> Someone mentioned a PMC vote, and I don't think this should be a closed
> vote, independent of how the conversation goes.
>
> -Flavio
>
> > On 22 Oct 2020, at 08:39, Alessandro Luccaroni - Diennea
> <al...@diennea.com.INVALID> wrote:
> >
> > Hi all,
> > If I might chime in as a zookeeper user (in multiple products) and a
> follower of the project I think the drop of Java8 support (official and/or
> unofficial) could be a big mistake.
> >
> > From my own company point of view we already support Java11 in all our
> applications so we are not directly impacted (and we have upgrade path for
> older versions to provide to our customers).
> > My worries resides in the (high) probability of a userbase
> fragmentation: in the recent past Zookeeper development picked up speed
> thanks to a bunch of new committers and PMCs after a period of mostly
> maintenance focused works, but the number of active committers and PMCs is
> still very low for a project like this.
> >
> > I foresee the risk of spreading thin the resources of the project if we
> force the userbase to stick to an older version and, in turn, we are forced
> to backport many issue to the 3.6 branch.
> >
> > Alessandro Luccaroni
> > Platform Manager @ Diennea - MagNews
> > Tel.: (+39) 0546 066100 Int. 924 - Mob.: (+39) 393 7273519
> > Viale G.Marconi 30/14 - 48018 Faenza (RA) - Italy
> >
> > -----Messaggio originale-----
> > Da: Christopher <ct...@apache.org>
> > Inviato: giovedì 22 ottobre 2020 05:21
> > A: dev@zookeeper.apache.org
> > Oggetto: Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7
> >
> > I'm happy that this discussion has been so lively! I just want to
> emphasize a few things:
> >
> > I really do understand the desire to continue to support Java 8... I get
> it. But all the conversations around this seem based on what people are
> doing *today*. But, ZK 3.7 is *tomorrow's* version... a
> > *future* release... so it should be based more on reasonable
> expectations for users in the future, and less based on what is happening
> today. I suspect *most* people today are still using 3.4 anyway (it was
> just so stable for so long...), but that shouldn't mean the developers
> should hold back development on 3.5 and 3.6, any more than today's users of
> 3.5/3.6 should hold back 3.7.
> >
> > Some of the opinions expressed in this discussion seem to propose a
> scenario where users are going to be updating to "bleeding edge"
> > versions of ZooKeeper, but are going to insist on using Java 8.
> > Personally, I find this to be implausible. In my experience, people
> either upgrade everything as soon as they are able to, or they upgrade each
> thing individually, only when they are forced to. The first group will be
> happy to move to Java 11 and ZK 3.7. The second group will probably avoid
> 3.7 anyway, and are fine sticking with 3.6, but if they had to update to
> 3.7, they'd also be fine updating to Java 11 if they had to in order to use
> 3.7. I can't imagine the scenario where people are eagerly choosing to
> upgrade to ZK 3.7, but miserly insisting on using Java 8. Perhaps that
> scenario exists, but it's hard for me to imagine. Even so, my proposal
> would still support even that group of people.
> >
> > I think there are now effectively three proposals being discussed in
> this thread:
> >
> > 1. (Christopher's original proposal) passively support Java 8 at runtime
> by making JDK 11 the minimum requirement to build and test.
> > This scenario involves continuing to fix bugs, as they are discovered
> and reported, that affect JDK 8, but passively, rather than proactively.
> This proposal does *not* drop Java 8 support, but merely de-emphasizes it
> in development of what will be 3.7 in the future, and drops the requirement
> to do dedicated testing with Java 8. I think this is low risk, because it
> is very unlikely that the ZK devs would introduce a bug that would affect
> only Java 8 and the compiler wouldn't catch it... because the
> cross-compilation features of newer JDKs are really good.
> >
> > 2. (Enrico's alternate proposal) this variation of my proposal would
> involve continuing to proactively support Java 8 by creating a dedicated
> testing suite to test client code on Java 8. I think this is a good option,
> but since it involves a significantly higher amount of work than option 1,
> I think the cost-benefit analysis would show this to be not worth the
> effort. Also, if it were implemented, it would need to be done carefully to
> avoid requiring developers to have concurrently installed both Java 8 and
> Java 11 in order to perform a build, because requiring Java 8 at build time
> while developing would be worse than we have today.
> >
> > 3. (Andor's preference) move to JDK 11 fully. This would provide no
> support, passive or active, for Java 8 in ZK 3.7. To be honest, this is my
> personal favorite, and is the simplest to implement and communicate clearly
> to end users in release notes. The only reason I proposed a passive support
> of Java 8 instead of this is because I was trying to seek a compromise from
> the start. But, I think by far, this is the best option for the next
> *future* release of ZK. If you wanted to make the change even more visible
> to users, the version could even be bumped to 4.0.
> >
> > If this were to come to a VOTE by the PMC, in order to make a final
> decision, I would recommend they vote on option 3, and then if that fails,
> vote on option 1, and if that fails, keep things the way they are (because
> option 2 is more work).
> >
> > Christopher
> >
> > P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK 11
> and it works just fine there (as long as you add the missing runtime jar
> for javax.activation:javax.activation-api:1.2.0 to its class path, but that
> was fixed in Hadoop 3.3).
> >
> > On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes <ta...@cloudera.com.invalid>
> wrote:
> >>
> >> Hi All,
> >>
> >> I've just talked with a Hadoop/HDFS developer who told me what I
> guessed.
> >> With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would
> >> mean a release of Hadoop4.
> >> Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both. As
> >> of today most of the Hadoop users are still on Hadoop2, he doesn't
> >> expect
> >> Hadoop4 soon.
> >> As many Apache components depend on Hadoop and ZooKeeper they won't
> >> hurry to JDK11 until they have to (they will probably go one-by-one
> >> very slowly), which means if Hadoop stays on JDK8, they would use the
> >> last ZK version which works on JDK8.
> >> Do we want a ZK 3.4 again?
> >>
> >> Regards, Tamaas
> >>
> >>
> >> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <ta...@cloudera.com>
> wrote:
> >>
> >>> Hi All,
> >>>
> >>> Just to add my two cents.
> >>>
> >>> Upgrading to JDK11 looks inevitable sooner or later and I would
> >>> definitely not wait until 2030 or 2026 when the extended support of
> JDK8 ends.
> >>> But on the other side I have to agree with Enrico and Patrick that
> >>> far too many users are tied to JDK8 yet (not because they want to
> >>> use JDK8, but because they have to), some of them are components of
> >>> the Hadoop ecosystem, which would be a loss to tie them to 3.6.x for
> years.
> >>> Do we know the state of Hadoop? It builds with JDK8 at the moment,
> >>> but do we know what are their plans to go to JDK11?
> >>> When they move, we should move too, but I don't think it would be
> >>> wise to do it earlier.
> >>>
> >>> Christopher's option looks like the golden path, but it needs some
> >>> investment on the testing side as Enrico pointed it out.
> >>> Could we agree on it as a compromise?
> >>>
> >>> Regards, Tamaas
> >>>
> >>> On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com>
> wrote:
> >>>
> >>>> I think I was reacting to Enrico's earlier comment of:
> >>>>
> >>>> " ZooKeeper client is used by tons of users and unfortunately many
> >>>> projects are still on JDK8, if we move ZooKeeper to JDK11 the risk
> >>>> is to block users from the adoption, that is that we will see the
> >>>> world to stay on 3.6.x and we will have again a long lived release
> >>>> line, like 3.4."
> >>>>
> >>>> It's a matter of whether or not a long-lived release line is
> >>>> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
> >>>> security/patch-wise (if not feature-wise) for the next N years,
> >>>> then that's a potentially valid approach.  I interpreted that
> >>>> comment as "a long-lived release line is undesirable", but no one
> >>>> explicitly said that, I just read it that way.
> >>>>
> >>>> ~Brent
> >>>>
> >>>> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org>
> wrote:
> >>>>
> >>>>> On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org>
> wrote:
> >>>>>
> >>>>>> As far as I know Hbase, Solr and Kafka are already Java 11 ready.
> >>>>>>
> >>>>>> IMHO contributors of those projects should also put efforts
> >>>>>> into
> >>>> moving
> >>>>>> forward.
> >>>>>>
> >>>>>> We’re not saying that you _have_ to move to Java 11.
> >>>>>> Staying on Java 8? No problem, 3.6 is for you.
> >>>>>> Want the fancy new features of 3.7? Work on it on your side too.
> >>>>>>
> >>>>>>
> >>>>> ppl want things like security fixes. I believe the highlighted
> >>>>> downside
> >>>> is
> >>>>> that we would need to continue to maintain 3.6.x rather than
> >>>>> allowing users, and ourselves, to focus on trunk for production -"64
> percent"
> >>>> would
> >>>>> be blocked.
> >>>>>
> >>>>> Patrick
> >>>>>
> >>>>>
> >>>>>> Andor
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> On 2020. Oct 21., at 17:52, Enrico Olivelli
> >>>>>>> <eo...@gmail.com>
> >>>>> wrote:
> >>>>>>>
> >>>>>>> Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
> >>>>> andor@apache.org>
> >>>>>> ha
> >>>>>>> scritto:
> >>>>>>>
> >>>>>>>> Correct me if I'm wrong, but Oracle gets paid for extended
> support.
> >>>>>>>> Java 8 support until 2030 is not free of charge.
> >>>>>>>>
> >>>>>>>> "ZK may end up with a lot of users potentially locking
> >>>>>>>> themselves
> >>>> to
> >>>>>>>> 3.6.x for a while as Enrico mentioned."
> >>>>>>>>
> >>>>>>>> That's true. What's the downside of that which we should
> >>>>>>>> invest in
> >>>> to
> >>>>>>>> avoid?
> >>>>>>>>
> >>>>>>>
> >>>>>>> I see ZooKeeper used in many many projects, all of the
> >>>>>>> HBase/Pulsar/Kafka/Solr ecosystem...
> >>>>>>> they will have to move to JDK11 in order to move to the new
> >>>>>>> ZK
> >>>> version
> >>>>>>> so probably they will stay on ZK 3.6
> >>>>>>>
> >>>>>>> Probably with Java 17 LTS released the cards will change on
> >>>>>>> the
> >>>> table
> >>>>>>>
> >>>>>>> Enrico
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Andor
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> >>>>>>>>> As a slightly different consideration, if you look at the
> >>>> Long-Term
> >>>>>>>>> Support
> >>>>>>>>> (LTS) roadmaps for Java, currently Java 8 is set to have
> >>>>>>>>> full
> >>>> support
> >>>>>>>>> until
> >>>>>>>>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> >>>>>>>>>
> >>>>>>>>>
> >>>>> https://www.oracle.com/java/technologies/java-se-support-roadmap.
> >>>>> html
> >>>>>>>>> https://en.wikipedia.org/wiki/Java_version_history
> >>>>>>>>>
> >>>>>>>>> My guess is that a number of companies are still heavily
> >>>>>>>>> invested
> >>>> at
> >>>>>>>>> the
> >>>>>>>>> Java 8 level (I know a few) and with that kind of time
> >>>>>>>>> horizon,
> >>>> they
> >>>>>>>>> have
> >>>>>>>>> no real motivation to upgrade for quite a while.  If the
> >>>>>>>>> recent Python 2 deprecation is anything to go by, they
> >>>>>>>>> won't do it until they have to.
> >>>>>>>>>
> >>>>>>>>> Not saying Java 8 isn't *very* old (2014 release it seems
> >>>>>>>>> like?)
> >>>> and
> >>>>>>>>> I'm
> >>>>>>>>> not invested heavily either way, but this might suggest
> >>>>>>>>> that ZK
> >>>> may
> >>>>>>>>> end up
> >>>>>>>>> with a lot of users potentially locking themselves to 3.6.x
> >>>>>>>>> for a while as Enrico mentioned.
> >>>>>>>>>
> >>>>>>>>> (Not a major contributor, but wanted to chime in since I
> >>>>>>>>> just had this conversation with a bunch of people
> >>>>>>>>> professionally recently)
> >>>>>>>>>
> >>>>>>>>> Brent
> >>>>>>>>>
> >>>>>>>>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar
> >>>>>>>>> <an...@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Thanks for the summary.
> >>>>>>>>>>
> >>>>>>>>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully.
> >>>>>>>>>> Other projects will upgrade once they’re JDK11 compliant,
> >>>>>>>>>> otherwise they will
> >>>> stay
> >>>>>>>>>> on 3.5
> >>>>>>>>>> or 3.6. Both version are quite recent in ZooKeeper-terms,
> >>>>>>>>>> we already planned big changes for 3.7.0 and JDK 11 could
> >>>>>>>>>> be one of them.
> >>>>>>>>>>
> >>>>>>>>>> Don’t put extra burden on the ZK community to help others
> >>>>>>>>>> staying on ancient Java versions.
> >>>>>>>>>>
> >>>>>>>>>> Andor
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
> >>>> eolivelli@gmail.com>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Let me recap
> >>>>>>>>>>> - Christopher is proposing to move to JDK11
> >>>>>>>>>>> - ZooKeeper client and server are bundled and coded and
> >>>>>>>>>>> tested together
> >>>>>>>>>> in
> >>>>>>>>>>> zookeeper-server
> >>>>>>>>>>> - Enrico is concerned about the need of testing ZooKeeper
> >>>>>>>>>>> client on JDK8 (not a problem to move the server to
> >>>>>>>>>>> JDK11)
> >>>>>>>>>>>
> >>>>>>>>>>> ZooKeeper client is used by tons of users and
> >>>>>>>>>>> unfortunately many projects are still on JDK8, if we move
> >>>>>>>>>>> ZooKeeper to JDK11 the risk is to block
> >>>>>>>>>> users
> >>>>>>>>>>> from the adoption,
> >>>>>>>>>>> that is that we will see the world to stay on 3.6.x and
> >>>>>>>>>>> we will have
> >>>>>>>>>> again
> >>>>>>>>>>> a long lived release line, like 3.4.
> >>>>>>>>>>>
> >>>>>>>>>>> Testing the client on JDK8 would be possible if we create
> >>>>>>>>>>> some kind of additional module with system tests, then we
> >>>>>>>>>>> can start the
> >>>> server
> >>>>>>>>>>> on
> >>>>>>>>>> docker
> >>>>>>>>>>> on JDK11+ and start a client on JDK8 with Maven toolchain
> >>>>>>>>>>> it should possible to run surefire tests using a separate
> >>>>>>>>>>> JVM.
> >>>>>>>>>>>
> >>>>>>>>>>> So in my vision 2 options:
> >>>>>>>>>>> 1) fully JDK11 - drop JDK8 at all
> >>>>>>>>>>> 2) build with JDK11 - server only on JDK11 - add system
> >>>>>>>>>>> tests with docker and toolchains that ensure the
> >>>>>>>>>>> ZooKeeper client (and all
> >>>>>>>>>>> dependencies)
> >>>>>>>>>>> still work on JDK8
> >>>>>>>>>>>
> >>>>>>>>>>> From my point of view about the ZooKeeper ecosystem
> >>>>>>>>>>> option 2) will be far better, but we need resources to
> >>>>>>>>>>> work on a new test suite.
> >>>>>>>>>>>
> >>>>>>>>>>> Enrico
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> >>>>>>>>>>> andor@apache.org>
> >>>>>>>>>> ha
> >>>>>>>>>>> scritto:
> >>>>>>>>>>>
> >>>>>>>>>>>> Tamas, Enrico,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Sorry I don’t follow. Why do we have to test the client
> >>>>>>>>>>>> with JDK 8 in version 3.7.0?
> >>>>>>>>>>>>
> >>>>>>>>>>>> Andor
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
> >>>>>>>>>>>>> tamaas@cloudera.com.INVALID>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>> Hi Enrico,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Separating ZooKeeper client and server is a huge work,
> >>>>>>>>>>>>> but we might not need it.
> >>>>>>>>>>>>> As you mentioned we have to test ZK client with Java 8,
> >>>>>>>>>>>>> what about separating only the test cases which we need
> >>>>>>>>>>>>> to run with
> >>>>>>>>>>>>> Java8 too?
> >>>>>>>>>>>>> In Curator we have the ZK compatibility tests where we
> >>>>>>>>>>>>> run a limited
> >>>>>>>>>>>> amount
> >>>>>>>>>>>>> of Curator's jUnit tests with a different ZK version.
> >>>>>>>>>>>>> We might be able to do the same here, tag tests which
> >>>>>>>>>>>>> are testing ZK
> >>>>>>>>>>>> client
> >>>>>>>>>>>>> and run them separately with Java 8. The only
> >>>>>>>>>>>>> limitation is that these tests must stay JDK8
> >>>>>>>>>>>>> compatible.
> >>>>>>>>>>>>> But from the tags we will see which ones are those.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Regards, Tamaas
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> >>>>>>>>>>>>> eolivelli@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>> Christopher
> >>>>>>>>>>>>>> I appreciate your idea and I also moved lots of my
> >>>>>>>>>>>>>> projects to work
> >>>>>>>>>> the
> >>>>>>>>>>>> way
> >>>>>>>>>>>>>> you are suggesting.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> We must run tests using real jdk8 to test the
> >>>>>>>>>>>>>> Zookeeper client. We
> >>>>>>>>>> must
> >>>>>>>>>>>>>> ensure that Zookeeper works well, especially while
> >>>>>>>>>>>>>> dealing with
> >>>>>>>>>> security
> >>>>>>>>>>>>>> stuff.
> >>>>>>>>>>>>>> Currently the client is in the same module of the
> >>>>>>>>>>>>>> server and it will
> >>>>>>>>>>>> take a
> >>>>>>>>>>>>>> good (huge) amount of work to separate them
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Enrico
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher
> >>>>>>>>>>>>>> <ct...@apache.org> ha
> >>>>>>>>>> scritto:
> >>>>>>>>>>>>>>> Hi ZK Devs,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> With recent advancements in Java (since Java 9), it
> >>>>>>>>>>>>>>> is now generally no longer necessary to require that
> >>>>>>>>>>>>>>> software be developed on an older JDK in order to
> >>>>>>>>>>>>>>> have confidence that it will run on the older version
> >>>>>>>>>>>>>>> of Java. This is because, as of Java 9, all JDK
> >>>>>>>>>>>>>>> releases have better support for cross-compilation to
> >>>>>>>>>>>>>>> older Java versions.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> What this means is that developers can confidently
> >>>>>>>>>>>>>>> make the build requirements for a project higher than
> >>>>>>>>>>>>>>> the Java version that will actually be supported at
> >>>>>>>>>>>>>>> runtime.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> In fact, ZooKeeper already supports the necessary
> >>>>>>>>>>>>>>> flags in its Maven build configuration to ensure that
> >>>>>>>>>>>>>>> it uses JDK 8 compliance when building on a newer JDK
> >>>>>>>>>>>>>>> (I added this way back in
> >>>>>>>>>>>>>>> ZOOKEEPER-3739 /
> >>>>>>>>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> So, I propose that we make JDK 11 the new minimum
> >>>>>>>>>>>>>>> version to *build* ZooKeeper with. This would not
> >>>>>>>>>>>>>>> change the runtime requirement, which would remain at
> >>>>>>>>>>>>>>> JDK 8.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> The only necessary change to make this happen would
> >>>>>>>>>>>>>>> be to add the minimum Java version to the
> >>>>>>>>>>>>>>> maven-enforcer-plugin (like
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>
> >>>> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd1
> >>>> 1d5dedb146519/pom.xml#L1162-L1164
> >>>>>>>>>>>>>>> )
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> This would allow ZooKeeper to to streamline its
> >>>>>>>>>>>>>>> development process a little bit by reducing the
> >>>>>>>>>>>>>>> amount of CI testing that is done as part of the
> >>>>>>>>>>>>>>> build. In other words, we can drop the CI builds for
> >>>>>>>>>>>>>>> JDK 8, which saves on build resources and time. The
> >>>>>>>>>>>>>>> return on investment is so low for the JDK 8 builds
> >>>>>>>>>>>>>>> anyway, because of the improved cross-compilation in
> >>>>>>>>>>>>>>> newer JDKs. So, there's not much value in building on
> >>>>>>>>>>>>>>> JDK 8 anyway.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Of course, I am only recommending this for *new*
> >>>>>>>>>>>>>>> release lines, starting with ZooKeeper 3.7.0/master
> >>>>>>>>>>>>>>> branch, because I would not want to change
> >>>>>>>>>>>>>>> expectations for users who will build their own
> >>>>>>>>>>>>>>> 3.5 and 3.6
> >>>>>>>>>>>>>>> versions as they continue to have patch versions
> >>>>>>>>>>>>>>> released.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Kind Regards,
> >>>>>>>>>>>>>>> Christopher
> >>>>>>>>>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >
> > ________________________________
> >
> > CONFIDENTIALITY & PRIVACY NOTICE
> > This e-mail (including any attachments) is strictly confidential and may
> also contain privileged information. If you are not the intended recipient
> you are not authorised to read, print, save, process or disclose this
> message. If you have received this message by mistake, please inform the
> sender immediately and destroy this e-mail, its attachments and any copies.
> Any use, distribution, reproduction or disclosure by any person other than
> the intended recipient is strictly prohibited and the person responsible
> may incur in penalties.
> > The use of this e-mail is only for professional purposes; there is no
> guarantee that the correspondence towards this e-mail will be read only by
> the recipient, because, under certain circumstances, there may be a need to
> access this email by third subjects belonging to the Company.
>
>

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Flavio Junqueira <fp...@apache.org>]

There are three points that stand out for me in this thread:

- How do we determine how such a change affects our user base?
- How much effort do the different options induce with respect to maintenance?
- What's the right timeline for changes and how do we communicate them so that our users have enough time to prepare?

Someone mentioned a PMC vote, and I don't think this should be a closed vote, independent of how the conversation goes.

-Flavio

> On 22 Oct 2020, at 08:39, Alessandro Luccaroni - Diennea <al...@diennea.com.INVALID> wrote:
> 
> Hi all,
> If I might chime in as a zookeeper user (in multiple products) and a follower of the project I think the drop of Java8 support (official and/or unofficial) could be a big mistake.
> 
> From my own company point of view we already support Java11 in all our applications so we are not directly impacted (and we have upgrade path for older versions to provide to our customers).
> My worries resides in the (high) probability of a userbase fragmentation: in the recent past Zookeeper development picked up speed thanks to a bunch of new committers and PMCs after a period of mostly maintenance focused works, but the number of active committers and PMCs is still very low for a project like this.
> 
> I foresee the risk of spreading thin the resources of the project if we force the userbase to stick to an older version and, in turn, we are forced to backport many issue to the 3.6 branch.
> 
> Alessandro Luccaroni
> Platform Manager @ Diennea - MagNews
> Tel.: (+39) 0546 066100 Int. 924 - Mob.: (+39) 393 7273519
> Viale G.Marconi 30/14 - 48018 Faenza (RA) - Italy
> 
> -----Messaggio originale-----
> Da: Christopher <ct...@apache.org>
> Inviato: giovedì 22 ottobre 2020 05:21
> A: dev@zookeeper.apache.org
> Oggetto: Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7
> 
> I'm happy that this discussion has been so lively! I just want to emphasize a few things:
> 
> I really do understand the desire to continue to support Java 8... I get it. But all the conversations around this seem based on what people are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
> *future* release... so it should be based more on reasonable expectations for users in the future, and less based on what is happening today. I suspect *most* people today are still using 3.4 anyway (it was just so stable for so long...), but that shouldn't mean the developers should hold back development on 3.5 and 3.6, any more than today's users of 3.5/3.6 should hold back 3.7.
> 
> Some of the opinions expressed in this discussion seem to propose a scenario where users are going to be updating to "bleeding edge"
> versions of ZooKeeper, but are going to insist on using Java 8.
> Personally, I find this to be implausible. In my experience, people either upgrade everything as soon as they are able to, or they upgrade each thing individually, only when they are forced to. The first group will be happy to move to Java 11 and ZK 3.7. The second group will probably avoid 3.7 anyway, and are fine sticking with 3.6, but if they had to update to 3.7, they'd also be fine updating to Java 11 if they had to in order to use 3.7. I can't imagine the scenario where people are eagerly choosing to upgrade to ZK 3.7, but miserly insisting on using Java 8. Perhaps that scenario exists, but it's hard for me to imagine. Even so, my proposal would still support even that group of people.
> 
> I think there are now effectively three proposals being discussed in this thread:
> 
> 1. (Christopher's original proposal) passively support Java 8 at runtime by making JDK 11 the minimum requirement to build and test.
> This scenario involves continuing to fix bugs, as they are discovered and reported, that affect JDK 8, but passively, rather than proactively. This proposal does *not* drop Java 8 support, but merely de-emphasizes it in development of what will be 3.7 in the future, and drops the requirement to do dedicated testing with Java 8. I think this is low risk, because it is very unlikely that the ZK devs would introduce a bug that would affect only Java 8 and the compiler wouldn't catch it... because the cross-compilation features of newer JDKs are really good.
> 
> 2. (Enrico's alternate proposal) this variation of my proposal would involve continuing to proactively support Java 8 by creating a dedicated testing suite to test client code on Java 8. I think this is a good option, but since it involves a significantly higher amount of work than option 1, I think the cost-benefit analysis would show this to be not worth the effort. Also, if it were implemented, it would need to be done carefully to avoid requiring developers to have concurrently installed both Java 8 and Java 11 in order to perform a build, because requiring Java 8 at build time while developing would be worse than we have today.
> 
> 3. (Andor's preference) move to JDK 11 fully. This would provide no support, passive or active, for Java 8 in ZK 3.7. To be honest, this is my personal favorite, and is the simplest to implement and communicate clearly to end users in release notes. The only reason I proposed a passive support of Java 8 instead of this is because I was trying to seek a compromise from the start. But, I think by far, this is the best option for the next *future* release of ZK. If you wanted to make the change even more visible to users, the version could even be bumped to 4.0.
> 
> If this were to come to a VOTE by the PMC, in order to make a final decision, I would recommend they vote on option 3, and then if that fails, vote on option 1, and if that fails, keep things the way they are (because option 2 is more work).
> 
> Christopher
> 
> P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK 11 and it works just fine there (as long as you add the missing runtime jar for javax.activation:javax.activation-api:1.2.0 to its class path, but that was fixed in Hadoop 3.3).
> 
> On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes <ta...@cloudera.com.invalid> wrote:
>> 
>> Hi All,
>> 
>> I've just talked with a Hadoop/HDFS developer who told me what I guessed.
>> With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would
>> mean a release of Hadoop4.
>> Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both. As
>> of today most of the Hadoop users are still on Hadoop2, he doesn't
>> expect
>> Hadoop4 soon.
>> As many Apache components depend on Hadoop and ZooKeeper they won't
>> hurry to JDK11 until they have to (they will probably go one-by-one
>> very slowly), which means if Hadoop stays on JDK8, they would use the
>> last ZK version which works on JDK8.
>> Do we want a ZK 3.4 again?
>> 
>> Regards, Tamaas
>> 
>> 
>> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <ta...@cloudera.com> wrote:
>> 
>>> Hi All,
>>> 
>>> Just to add my two cents.
>>> 
>>> Upgrading to JDK11 looks inevitable sooner or later and I would
>>> definitely not wait until 2030 or 2026 when the extended support of JDK8 ends.
>>> But on the other side I have to agree with Enrico and Patrick that
>>> far too many users are tied to JDK8 yet (not because they want to
>>> use JDK8, but because they have to), some of them are components of
>>> the Hadoop ecosystem, which would be a loss to tie them to 3.6.x for years.
>>> Do we know the state of Hadoop? It builds with JDK8 at the moment,
>>> but do we know what are their plans to go to JDK11?
>>> When they move, we should move too, but I don't think it would be
>>> wise to do it earlier.
>>> 
>>> Christopher's option looks like the golden path, but it needs some
>>> investment on the testing side as Enrico pointed it out.
>>> Could we agree on it as a compromise?
>>> 
>>> Regards, Tamaas
>>> 
>>> On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com> wrote:
>>> 
>>>> I think I was reacting to Enrico's earlier comment of:
>>>> 
>>>> " ZooKeeper client is used by tons of users and unfortunately many
>>>> projects are still on JDK8, if we move ZooKeeper to JDK11 the risk
>>>> is to block users from the adoption, that is that we will see the
>>>> world to stay on 3.6.x and we will have again a long lived release
>>>> line, like 3.4."
>>>> 
>>>> It's a matter of whether or not a long-lived release line is
>>>> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
>>>> security/patch-wise (if not feature-wise) for the next N years,
>>>> then that's a potentially valid approach.  I interpreted that
>>>> comment as "a long-lived release line is undesirable", but no one
>>>> explicitly said that, I just read it that way.
>>>> 
>>>> ~Brent
>>>> 
>>>> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org> wrote:
>>>> 
>>>>> On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org> wrote:
>>>>> 
>>>>>> As far as I know Hbase, Solr and Kafka are already Java 11 ready.
>>>>>> 
>>>>>> IMHO contributors of those projects should also put efforts
>>>>>> into
>>>> moving
>>>>>> forward.
>>>>>> 
>>>>>> We’re not saying that you _have_ to move to Java 11.
>>>>>> Staying on Java 8? No problem, 3.6 is for you.
>>>>>> Want the fancy new features of 3.7? Work on it on your side too.
>>>>>> 
>>>>>> 
>>>>> ppl want things like security fixes. I believe the highlighted
>>>>> downside
>>>> is
>>>>> that we would need to continue to maintain 3.6.x rather than
>>>>> allowing users, and ourselves, to focus on trunk for production -"64 percent"
>>>> would
>>>>> be blocked.
>>>>> 
>>>>> Patrick
>>>>> 
>>>>> 
>>>>>> Andor
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> On 2020. Oct 21., at 17:52, Enrico Olivelli
>>>>>>> <eo...@gmail.com>
>>>>> wrote:
>>>>>>> 
>>>>>>> Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
>>>>> andor@apache.org>
>>>>>> ha
>>>>>>> scritto:
>>>>>>> 
>>>>>>>> Correct me if I'm wrong, but Oracle gets paid for extended support.
>>>>>>>> Java 8 support until 2030 is not free of charge.
>>>>>>>> 
>>>>>>>> "ZK may end up with a lot of users potentially locking
>>>>>>>> themselves
>>>> to
>>>>>>>> 3.6.x for a while as Enrico mentioned."
>>>>>>>> 
>>>>>>>> That's true. What's the downside of that which we should
>>>>>>>> invest in
>>>> to
>>>>>>>> avoid?
>>>>>>>> 
>>>>>>> 
>>>>>>> I see ZooKeeper used in many many projects, all of the
>>>>>>> HBase/Pulsar/Kafka/Solr ecosystem...
>>>>>>> they will have to move to JDK11 in order to move to the new
>>>>>>> ZK
>>>> version
>>>>>>> so probably they will stay on ZK 3.6
>>>>>>> 
>>>>>>> Probably with Java 17 LTS released the cards will change on
>>>>>>> the
>>>> table
>>>>>>> 
>>>>>>> Enrico
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> 
>>>>>>>> Andor
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
>>>>>>>>> As a slightly different consideration, if you look at the
>>>> Long-Term
>>>>>>>>> Support
>>>>>>>>> (LTS) roadmaps for Java, currently Java 8 is set to have
>>>>>>>>> full
>>>> support
>>>>>>>>> until
>>>>>>>>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
>>>>>>>>> 
>>>>>>>>> 
>>>>> https://www.oracle.com/java/technologies/java-se-support-roadmap.
>>>>> html
>>>>>>>>> https://en.wikipedia.org/wiki/Java_version_history
>>>>>>>>> 
>>>>>>>>> My guess is that a number of companies are still heavily
>>>>>>>>> invested
>>>> at
>>>>>>>>> the
>>>>>>>>> Java 8 level (I know a few) and with that kind of time
>>>>>>>>> horizon,
>>>> they
>>>>>>>>> have
>>>>>>>>> no real motivation to upgrade for quite a while.  If the
>>>>>>>>> recent Python 2 deprecation is anything to go by, they
>>>>>>>>> won't do it until they have to.
>>>>>>>>> 
>>>>>>>>> Not saying Java 8 isn't *very* old (2014 release it seems
>>>>>>>>> like?)
>>>> and
>>>>>>>>> I'm
>>>>>>>>> not invested heavily either way, but this might suggest
>>>>>>>>> that ZK
>>>> may
>>>>>>>>> end up
>>>>>>>>> with a lot of users potentially locking themselves to 3.6.x
>>>>>>>>> for a while as Enrico mentioned.
>>>>>>>>> 
>>>>>>>>> (Not a major contributor, but wanted to chime in since I
>>>>>>>>> just had this conversation with a bunch of people
>>>>>>>>> professionally recently)
>>>>>>>>> 
>>>>>>>>> Brent
>>>>>>>>> 
>>>>>>>>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar
>>>>>>>>> <an...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Thanks for the summary.
>>>>>>>>>> 
>>>>>>>>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully.
>>>>>>>>>> Other projects will upgrade once they’re JDK11 compliant,
>>>>>>>>>> otherwise they will
>>>> stay
>>>>>>>>>> on 3.5
>>>>>>>>>> or 3.6. Both version are quite recent in ZooKeeper-terms,
>>>>>>>>>> we already planned big changes for 3.7.0 and JDK 11 could
>>>>>>>>>> be one of them.
>>>>>>>>>> 
>>>>>>>>>> Don’t put extra burden on the ZK community to help others
>>>>>>>>>> staying on ancient Java versions.
>>>>>>>>>> 
>>>>>>>>>> Andor
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
>>>> eolivelli@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Let me recap
>>>>>>>>>>> - Christopher is proposing to move to JDK11
>>>>>>>>>>> - ZooKeeper client and server are bundled and coded and
>>>>>>>>>>> tested together
>>>>>>>>>> in
>>>>>>>>>>> zookeeper-server
>>>>>>>>>>> - Enrico is concerned about the need of testing ZooKeeper
>>>>>>>>>>> client on JDK8 (not a problem to move the server to
>>>>>>>>>>> JDK11)
>>>>>>>>>>> 
>>>>>>>>>>> ZooKeeper client is used by tons of users and
>>>>>>>>>>> unfortunately many projects are still on JDK8, if we move
>>>>>>>>>>> ZooKeeper to JDK11 the risk is to block
>>>>>>>>>> users
>>>>>>>>>>> from the adoption,
>>>>>>>>>>> that is that we will see the world to stay on 3.6.x and
>>>>>>>>>>> we will have
>>>>>>>>>> again
>>>>>>>>>>> a long lived release line, like 3.4.
>>>>>>>>>>> 
>>>>>>>>>>> Testing the client on JDK8 would be possible if we create
>>>>>>>>>>> some kind of additional module with system tests, then we
>>>>>>>>>>> can start the
>>>> server
>>>>>>>>>>> on
>>>>>>>>>> docker
>>>>>>>>>>> on JDK11+ and start a client on JDK8 with Maven toolchain
>>>>>>>>>>> it should possible to run surefire tests using a separate
>>>>>>>>>>> JVM.
>>>>>>>>>>> 
>>>>>>>>>>> So in my vision 2 options:
>>>>>>>>>>> 1) fully JDK11 - drop JDK8 at all
>>>>>>>>>>> 2) build with JDK11 - server only on JDK11 - add system
>>>>>>>>>>> tests with docker and toolchains that ensure the
>>>>>>>>>>> ZooKeeper client (and all
>>>>>>>>>>> dependencies)
>>>>>>>>>>> still work on JDK8
>>>>>>>>>>> 
>>>>>>>>>>> From my point of view about the ZooKeeper ecosystem
>>>>>>>>>>> option 2) will be far better, but we need resources to
>>>>>>>>>>> work on a new test suite.
>>>>>>>>>>> 
>>>>>>>>>>> Enrico
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
>>>>>>>>>>> andor@apache.org>
>>>>>>>>>> ha
>>>>>>>>>>> scritto:
>>>>>>>>>>> 
>>>>>>>>>>>> Tamas, Enrico,
>>>>>>>>>>>> 
>>>>>>>>>>>> Sorry I don’t follow. Why do we have to test the client
>>>>>>>>>>>> with JDK 8 in version 3.7.0?
>>>>>>>>>>>> 
>>>>>>>>>>>> Andor
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
>>>>>>>>>>>>> tamaas@cloudera.com.INVALID>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> Hi Enrico,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Separating ZooKeeper client and server is a huge work,
>>>>>>>>>>>>> but we might not need it.
>>>>>>>>>>>>> As you mentioned we have to test ZK client with Java 8,
>>>>>>>>>>>>> what about separating only the test cases which we need
>>>>>>>>>>>>> to run with
>>>>>>>>>>>>> Java8 too?
>>>>>>>>>>>>> In Curator we have the ZK compatibility tests where we
>>>>>>>>>>>>> run a limited
>>>>>>>>>>>> amount
>>>>>>>>>>>>> of Curator's jUnit tests with a different ZK version.
>>>>>>>>>>>>> We might be able to do the same here, tag tests which
>>>>>>>>>>>>> are testing ZK
>>>>>>>>>>>> client
>>>>>>>>>>>>> and run them separately with Java 8. The only
>>>>>>>>>>>>> limitation is that these tests must stay JDK8
>>>>>>>>>>>>> compatible.
>>>>>>>>>>>>> But from the tags we will see which ones are those.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Regards, Tamaas
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
>>>>>>>>>>>>> eolivelli@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> Christopher
>>>>>>>>>>>>>> I appreciate your idea and I also moved lots of my
>>>>>>>>>>>>>> projects to work
>>>>>>>>>> the
>>>>>>>>>>>> way
>>>>>>>>>>>>>> you are suggesting.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> We must run tests using real jdk8 to test the
>>>>>>>>>>>>>> Zookeeper client. We
>>>>>>>>>> must
>>>>>>>>>>>>>> ensure that Zookeeper works well, especially while
>>>>>>>>>>>>>> dealing with
>>>>>>>>>> security
>>>>>>>>>>>>>> stuff.
>>>>>>>>>>>>>> Currently the client is in the same module of the
>>>>>>>>>>>>>> server and it will
>>>>>>>>>>>> take a
>>>>>>>>>>>>>> good (huge) amount of work to separate them
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Enrico
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher
>>>>>>>>>>>>>> <ct...@apache.org> ha
>>>>>>>>>> scritto:
>>>>>>>>>>>>>>> Hi ZK Devs,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> With recent advancements in Java (since Java 9), it
>>>>>>>>>>>>>>> is now generally no longer necessary to require that
>>>>>>>>>>>>>>> software be developed on an older JDK in order to
>>>>>>>>>>>>>>> have confidence that it will run on the older version
>>>>>>>>>>>>>>> of Java. This is because, as of Java 9, all JDK
>>>>>>>>>>>>>>> releases have better support for cross-compilation to
>>>>>>>>>>>>>>> older Java versions.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> What this means is that developers can confidently
>>>>>>>>>>>>>>> make the build requirements for a project higher than
>>>>>>>>>>>>>>> the Java version that will actually be supported at
>>>>>>>>>>>>>>> runtime.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> In fact, ZooKeeper already supports the necessary
>>>>>>>>>>>>>>> flags in its Maven build configuration to ensure that
>>>>>>>>>>>>>>> it uses JDK 8 compliance when building on a newer JDK
>>>>>>>>>>>>>>> (I added this way back in
>>>>>>>>>>>>>>> ZOOKEEPER-3739 /
>>>>>>>>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> So, I propose that we make JDK 11 the new minimum
>>>>>>>>>>>>>>> version to *build* ZooKeeper with. This would not
>>>>>>>>>>>>>>> change the runtime requirement, which would remain at
>>>>>>>>>>>>>>> JDK 8.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> The only necessary change to make this happen would
>>>>>>>>>>>>>>> be to add the minimum Java version to the
>>>>>>>>>>>>>>> maven-enforcer-plugin (like
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>> 
>>>> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd1
>>>> 1d5dedb146519/pom.xml#L1162-L1164
>>>>>>>>>>>>>>> )
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> This would allow ZooKeeper to to streamline its
>>>>>>>>>>>>>>> development process a little bit by reducing the
>>>>>>>>>>>>>>> amount of CI testing that is done as part of the
>>>>>>>>>>>>>>> build. In other words, we can drop the CI builds for
>>>>>>>>>>>>>>> JDK 8, which saves on build resources and time. The
>>>>>>>>>>>>>>> return on investment is so low for the JDK 8 builds
>>>>>>>>>>>>>>> anyway, because of the improved cross-compilation in
>>>>>>>>>>>>>>> newer JDKs. So, there's not much value in building on
>>>>>>>>>>>>>>> JDK 8 anyway.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Of course, I am only recommending this for *new*
>>>>>>>>>>>>>>> release lines, starting with ZooKeeper 3.7.0/master
>>>>>>>>>>>>>>> branch, because I would not want to change
>>>>>>>>>>>>>>> expectations for users who will build their own
>>>>>>>>>>>>>>> 3.5 and 3.6
>>>>>>>>>>>>>>> versions as they continue to have patch versions
>>>>>>>>>>>>>>> released.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Kind Regards,
>>>>>>>>>>>>>>> Christopher
>>>>>>>>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
> 
> ________________________________
> 
> CONFIDENTIALITY & PRIVACY NOTICE
> This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties.
> The use of this e-mail is only for professional purposes; there is no guarantee that the correspondence towards this e-mail will be read only by the recipient, because, under certain circumstances, there may be a need to access this email by third subjects belonging to the Company.

Re: 3.7.0: JDK 11 vs. JDK 8 (was: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7)

[Ted Dunning <te...@gmail.com>]

Also, software like ZK should lag rather than lead (I realize that moving
to 11 is hardly leading).

There is considerable pain that we inflict when we move JDK requirements
forward.



On Wed, Jan 6, 2021 at 10:20 AM Enrico Olivelli <eo...@gmail.com> wrote:

> In my opinion we can stay on 8, we should change this kind of stuff at the
> beginning of a new iteration and not before the release.
> IIRC We are not in a hurry, having a stable build process is a value.
>
> Enrico
>
> Il Mer 6 Gen 2021, 17:51 Damien Diederen <dd...@apache.org> ha
> scritto:
>
> >
> > Dear ZooKeeper team,
> >
> > Andor just reminded me of this JDK 11 vs. 8 discussion, for which we did
> > not reach a resolution.  Do we want to make a move for the 3.7.0 release?
> >
> > The original proposal, by Christopher, can be found here:
> >
> >
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9Z2TNfAwjZ7etXPF91QmROvUWV+KHnxgPSD_MSeKfpyWA@mail.gmail.com%3e
> >
> > I am explicitly replying to Christopher's nice and useful summary, which
> > you will find directly below.  An archived copy can be found here:
> >
> >
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9ao9-4S9bmzxaYJM8n8=AoLxiDZ-0ZohunVnAyQei0zAg@mail.gmail.com%3e
> >
> > It contains three proposals, to which we can add the trivial one:
> >
> >  1. Compile/test with JDK 11, support JDK 8 "passively";
> >
> >  2. Compile/test with JDK 11, test client only with JDK 8 (requires test
> >     suite adaptations);
> >
> >  3. Move to JDK 11 fully;
> >
> >  4. Postpone any change to 3.8 (or 4.0).
> >
> > Flavio wrote:
> >
> > > [Christopher] mentioned a PMC vote, and I don't think this should be a
> > > closed vote, independent of how the conversation goes.
> >
> > Do we have a better idea of what we want?  Or should we organize a vote?
> >
> > Cheers, -D
> >
> >
> > --8<--------------- original message ------------->8---
> >
> > Christopher <ct...@apache.org> writes:
> > > I'm happy that this discussion has been so lively! I just want to
> > > emphasize a few things:
> > >
> > > I really do understand the desire to continue to support Java 8... I
> > > get it. But all the conversations around this seem based on what
> > > people are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
> > > *future* release... so it should be based more on reasonable
> > > expectations for users in the future, and less based on what is
> > > happening today. I suspect *most* people today are still using 3.4
> > > anyway (it was just so stable for so long...), but that shouldn't mean
> > > the developers should hold back development on 3.5 and 3.6, any more
> > > than today's users of 3.5/3.6 should hold back 3.7.
> > >
> > > Some of the opinions expressed in this discussion seem to propose a
> > > scenario where users are going to be updating to "bleeding edge"
> > > versions of ZooKeeper, but are going to insist on using Java 8.
> > > Personally, I find this to be implausible. In my experience, people
> > > either upgrade everything as soon as they are able to, or they upgrade
> > > each thing individually, only when they are forced to. The first group
> > > will be happy to move to Java 11 and ZK 3.7. The second group will
> > > probably avoid 3.7 anyway, and are fine sticking with 3.6, but if they
> > > had to update to 3.7, they'd also be fine updating to Java 11 if they
> > > had to in order to use 3.7. I can't imagine the scenario where people
> > > are eagerly choosing to upgrade to ZK 3.7, but miserly insisting on
> > > using Java 8. Perhaps that scenario exists, but it's hard for me to
> > > imagine. Even so, my proposal would still support even that group of
> > > people.
> > >
> > > I think there are now effectively three proposals being discussed in
> > > this thread:
> > >
> > > 1. (Christopher's original proposal) passively support Java 8 at
> > > runtime by making JDK 11 the minimum requirement to build and test.
> > > This scenario involves continuing to fix bugs, as they are discovered
> > > and reported, that affect JDK 8, but passively, rather than
> > > proactively. This proposal does *not* drop Java 8 support, but merely
> > > de-emphasizes it in development of what will be 3.7 in the future, and
> > > drops the requirement to do dedicated testing with Java 8. I think
> > > this is low risk, because it is very unlikely that the ZK devs would
> > > introduce a bug that would affect only Java 8 and the compiler
> > > wouldn't catch it... because the cross-compilation features of newer
> > > JDKs are really good.
> > >
> > > 2. (Enrico's alternate proposal) this variation of my proposal would
> > > involve continuing to proactively support Java 8 by creating a
> > > dedicated testing suite to test client code on Java 8. I think this is
> > > a good option, but since it involves a significantly higher amount of
> > > work than option 1, I think the cost-benefit analysis would show this
> > > to be not worth the effort. Also, if it were implemented, it would
> > > need to be done carefully to avoid requiring developers to have
> > > concurrently installed both Java 8 and Java 11 in order to perform a
> > > build, because requiring Java 8 at build time while developing would
> > > be worse than we have today.
> > >
> > > 3. (Andor's preference) move to JDK 11 fully. This would provide no
> > > support, passive or active, for Java 8 in ZK 3.7. To be honest, this
> > > is my personal favorite, and is the simplest to implement and
> > > communicate clearly to end users in release notes. The only reason I
> > > proposed a passive support of Java 8 instead of this is because I was
> > > trying to seek a compromise from the start. But, I think by far, this
> > > is the best option for the next *future* release of ZK. If you wanted
> > > to make the change even more visible to users, the version could even
> > > be bumped to 4.0.
> > >
> > > If this were to come to a VOTE by the PMC, in order to make a final
> > > decision, I would recommend they vote on option 3, and then if that
> > > fails, vote on option 1, and if that fails, keep things the way they
> > > are (because option 2 is more work).
> > >
> > > Christopher
> > >
> > > P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK 11
> > > and it works just fine there (as long as you add the missing runtime
> > > jar for javax.activation:javax.activation-api:1.2.0 to its class path,
> > > but that was fixed in Hadoop 3.3).
> > >
> > > On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes
> > > <ta...@cloudera.com.invalid> wrote:
> > >>
> > >> Hi All,
> > >>
> > >> I've just talked with a Hadoop/HDFS developer who told me what I
> > guessed.
> > >> With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would
> > mean
> > >> a release of Hadoop4.
> > >> Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both. As
> > of
> > >> today most of the Hadoop users are still on Hadoop2, he doesn't expect
> > >> Hadoop4 soon.
> > >> As many Apache components depend on Hadoop and ZooKeeper they won't
> > hurry
> > >> to JDK11 until they have to (they will probably go one-by-one very
> > slowly),
> > >> which means if Hadoop stays on JDK8, they would use the last ZK
> version
> > >> which works on JDK8.
> > >> Do we want a ZK 3.4 again?
> > >>
> > >> Regards, Tamaas
> > >>
> > >>
> > >> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <ta...@cloudera.com>
> > wrote:
> > >>
> > >> > Hi All,
> > >> >
> > >> > Just to add my two cents.
> > >> >
> > >> > Upgrading to JDK11 looks inevitable sooner or later and I would
> > definitely
> > >> > not wait until 2030 or 2026 when the extended support of JDK8 ends.
> > >> > But on the other side I have to agree with Enrico and Patrick that
> > far too
> > >> > many users are tied to JDK8 yet (not because they want to use JDK8,
> > but
> > >> > because they have to), some of them are components of the Hadoop
> > ecosystem,
> > >> > which would be a loss to tie them to 3.6.x for years.
> > >> > Do we know the state of Hadoop? It builds with JDK8 at the moment,
> > but do
> > >> > we know what are their plans to go to JDK11?
> > >> > When they move, we should move too, but I don't think it would be
> > wise to
> > >> > do it earlier.
> > >> >
> > >> > Christopher's option looks like the golden path, but it needs some
> > >> > investment on the testing side as Enrico pointed it out.
> > >> > Could we agree on it as a compromise?
> > >> >
> > >> > Regards, Tamaas
> > >> >
> > >> > On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com>
> > wrote:
> > >> >
> > >> >> I think I was reacting to Enrico's earlier comment of:
> > >> >>
> > >> >> " ZooKeeper client is used by tons of users and unfortunately many
> > >> >> projects
> > >> >> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
> block
> > >> >> users
> > >> >> from the adoption, that is that we will see the world to stay on
> > 3.6.x and
> > >> >> we will have again a long lived release line, like 3.4."
> > >> >>
> > >> >> It's a matter of whether or not a long-lived release line is
> > >> >> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
> > >> >> security/patch-wise (if not feature-wise) for the next N years,
> then
> > >> >> that's
> > >> >> a potentially valid approach.  I interpreted that comment as "a
> > long-lived
> > >> >> release line is undesirable", but no one explicitly said that, I
> > just read
> > >> >> it that way.
> > >> >>
> > >> >> ~Brent
> > >> >>
> > >> >> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org>
> > wrote:
> > >> >>
> > >> >> > On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org>
> > wrote:
> > >> >> >
> > >> >> > > As far as I know Hbase, Solr and Kafka are already Java 11
> ready.
> > >> >> > >
> > >> >> > > IMHO contributors of those projects should also put efforts
> into
> > >> >> moving
> > >> >> > > forward.
> > >> >> > >
> > >> >> > > We’re not saying that you _have_ to move to Java 11.
> > >> >> > > Staying on Java 8? No problem, 3.6 is for you.
> > >> >> > > Want the fancy new features of 3.7? Work on it on your side
> too.
> > >> >> > >
> > >> >> > >
> > >> >> > ppl want things like security fixes. I believe the highlighted
> > downside
> > >> >> is
> > >> >> > that we would need to continue to maintain 3.6.x rather than
> > allowing
> > >> >> > users, and ourselves, to focus on trunk for production -"64
> > percent"
> > >> >> would
> > >> >> > be blocked.
> > >> >> >
> > >> >> > Patrick
> > >> >> >
> > >> >> >
> > >> >> > > Andor
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > > > On 2020. Oct 21., at 17:52, Enrico Olivelli <
> > eolivelli@gmail.com>
> > >> >> > wrote:
> > >> >> > > >
> > >> >> > > > Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
> > >> >> > andor@apache.org>
> > >> >> > > ha
> > >> >> > > > scritto:
> > >> >> > > >
> > >> >> > > >> Correct me if I'm wrong, but Oracle gets paid for extended
> > support.
> > >> >> > > >> Java 8 support until 2030 is not free of charge.
> > >> >> > > >>
> > >> >> > > >> "ZK may end up with a lot of users potentially locking
> > themselves
> > >> >> to
> > >> >> > > >> 3.6.x for a while as Enrico mentioned."
> > >> >> > > >>
> > >> >> > > >> That's true. What's the downside of that which we should
> > invest in
> > >> >> to
> > >> >> > > >> avoid?
> > >> >> > > >>
> > >> >> > > >
> > >> >> > > > I see ZooKeeper used in many many projects, all of the
> > >> >> > > > HBase/Pulsar/Kafka/Solr ecosystem...
> > >> >> > > > they will have to move to JDK11 in order to move to the new
> ZK
> > >> >> version
> > >> >> > > > so probably they will stay on ZK 3.6
> > >> >> > > >
> > >> >> > > > Probably with Java 17 LTS released the cards will change on
> the
> > >> >> table
> > >> >> > > >
> > >> >> > > > Enrico
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >>
> > >> >> > > >> Andor
> > >> >> > > >>
> > >> >> > > >>
> > >> >> > > >>
> > >> >> > > >> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> > >> >> > > >>> As a slightly different consideration, if you look at the
> > >> >> Long-Term
> > >> >> > > >>> Support
> > >> >> > > >>> (LTS) roadmaps for Java, currently Java 8 is set to have
> full
> > >> >> support
> > >> >> > > >>> until
> > >> >> > > >>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> > >> >> > > >>>
> > >> >> > > >>>
> > >> >> >
> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> > >> >> > > >>> https://en.wikipedia.org/wiki/Java_version_history
> > >> >> > > >>>
> > >> >> > > >>> My guess is that a number of companies are still heavily
> > invested
> > >> >> at
> > >> >> > > >>> the
> > >> >> > > >>> Java 8 level (I know a few) and with that kind of time
> > horizon,
> > >> >> they
> > >> >> > > >>> have
> > >> >> > > >>> no real motivation to upgrade for quite a while.  If the
> > recent
> > >> >> > > >>> Python 2
> > >> >> > > >>> deprecation is anything to go by, they won't do it until
> > they have
> > >> >> > > >>> to.
> > >> >> > > >>>
> > >> >> > > >>> Not saying Java 8 isn't *very* old (2014 release it seems
> > like?)
> > >> >> and
> > >> >> > > >>> I'm
> > >> >> > > >>> not invested heavily either way, but this might suggest
> that
> > ZK
> > >> >> may
> > >> >> > > >>> end up
> > >> >> > > >>> with a lot of users potentially locking themselves to 3.6.x
> > for a
> > >> >> > > >>> while as
> > >> >> > > >>> Enrico mentioned.
> > >> >> > > >>>
> > >> >> > > >>> (Not a major contributor, but wanted to chime in since I
> > just had
> > >> >> > > >>> this
> > >> >> > > >>> conversation with a bunch of people professionally
> recently)
> > >> >> > > >>>
> > >> >> > > >>> Brent
> > >> >> > > >>>
> > >> >> > > >>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <
> > andor@apache.org>
> > >> >> > > >>> wrote:
> > >> >> > > >>>
> > >> >> > > >>>> Thanks for the summary.
> > >> >> > > >>>>
> > >> >> > > >>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully.
> > Other
> > >> >> > > >>>> projects
> > >> >> > > >>>> will upgrade once they’re JDK11 compliant, otherwise they
> > will
> > >> >> stay
> > >> >> > > >>>> on 3.5
> > >> >> > > >>>> or 3.6. Both version are quite recent in ZooKeeper-terms,
> we
> > >> >> > > >>>> already
> > >> >> > > >>>> planned big changes for 3.7.0 and JDK 11 could be one of
> > them.
> > >> >> > > >>>>
> > >> >> > > >>>> Don’t put extra burden on the ZK community to help others
> > staying
> > >> >> > > >>>> on
> > >> >> > > >>>> ancient Java versions.
> > >> >> > > >>>>
> > >> >> > > >>>> Andor
> > >> >> > > >>>>
> > >> >> > > >>>>
> > >> >> > > >>>>
> > >> >> > > >>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
> > >> >> eolivelli@gmail.com>
> > >> >> > > >>>>> wrote:
> > >> >> > > >>>>>
> > >> >> > > >>>>> Let me recap
> > >> >> > > >>>>> - Christopher is proposing to move to JDK11
> > >> >> > > >>>>> - ZooKeeper client and server are bundled and coded and
> > tested
> > >> >> > > >>>>> together
> > >> >> > > >>>> in
> > >> >> > > >>>>> zookeeper-server
> > >> >> > > >>>>> - Enrico is concerned about the need of testing ZooKeeper
> > client
> > >> >> > > >>>>> on JDK8
> > >> >> > > >>>>> (not a problem to move the server to JDK11)
> > >> >> > > >>>>>
> > >> >> > > >>>>> ZooKeeper client is used by tons of users and
> > unfortunately many
> > >> >> > > >>>>> projects
> > >> >> > > >>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk
> > is to
> > >> >> > > >>>>> block
> > >> >> > > >>>> users
> > >> >> > > >>>>> from the adoption,
> > >> >> > > >>>>> that is that we will see the world to stay on 3.6.x and
> we
> > will
> > >> >> > > >>>>> have
> > >> >> > > >>>> again
> > >> >> > > >>>>> a long lived release line, like 3.4.
> > >> >> > > >>>>>
> > >> >> > > >>>>> Testing the client on JDK8 would be possible if we create
> > some
> > >> >> > > >>>>> kind of
> > >> >> > > >>>>> additional module with system tests, then we can start
> the
> > >> >> server
> > >> >> > > >>>>> on
> > >> >> > > >>>> docker
> > >> >> > > >>>>> on JDK11+ and start a client on JDK8
> > >> >> > > >>>>> with Maven toolchain it should possible to run surefire
> > tests
> > >> >> > > >>>>> using a
> > >> >> > > >>>>> separate JVM.
> > >> >> > > >>>>>
> > >> >> > > >>>>> So in my vision 2 options:
> > >> >> > > >>>>> 1) fully JDK11 - drop JDK8 at all
> > >> >> > > >>>>> 2) build with JDK11 - server only on JDK11 - add system
> > tests
> > >> >> > > >>>>> with docker
> > >> >> > > >>>>> and toolchains that ensure the ZooKeeper client (and all
> > >> >> > > >>>>> dependencies)
> > >> >> > > >>>>> still work on JDK8
> > >> >> > > >>>>>
> > >> >> > > >>>>> From my point of view about the ZooKeeper ecosystem
> option
> > 2)
> > >> >> > > >>>>> will be far
> > >> >> > > >>>>> better, but we need resources to work on a new test
> suite.
> > >> >> > > >>>>>
> > >> >> > > >>>>> Enrico
> > >> >> > > >>>>>
> > >> >> > > >>>>>
> > >> >> > > >>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> > >> >> > > >>>>> andor@apache.org>
> > >> >> > > >>>> ha
> > >> >> > > >>>>> scritto:
> > >> >> > > >>>>>
> > >> >> > > >>>>>> Tamas, Enrico,
> > >> >> > > >>>>>>
> > >> >> > > >>>>>> Sorry I don’t follow. Why do we have to test the client
> > with
> > >> >> > > >>>>>> JDK 8 in
> > >> >> > > >>>>>> version 3.7.0?
> > >> >> > > >>>>>>
> > >> >> > > >>>>>> Andor
> > >> >> > > >>>>>>
> > >> >> > > >>>>>>
> > >> >> > > >>>>>>
> > >> >> > > >>>>>>
> > >> >> > > >>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
> > >> >> > > >>>>>>> tamaas@cloudera.com.INVALID>
> > >> >> > > >>>>>> wrote:
> > >> >> > > >>>>>>> Hi Enrico,
> > >> >> > > >>>>>>>
> > >> >> > > >>>>>>> Separating ZooKeeper client and server is a huge work,
> > but we
> > >> >> > > >>>>>>> might not
> > >> >> > > >>>>>>> need it.
> > >> >> > > >>>>>>> As you mentioned we have to test ZK client with Java 8,
> > what
> > >> >> > > >>>>>>> about
> > >> >> > > >>>>>>> separating only the test cases which we need to run
> with
> > >> >> > > >>>>>>> Java8 too?
> > >> >> > > >>>>>>> In Curator we have the ZK compatibility tests where we
> > run a
> > >> >> > > >>>>>>> limited
> > >> >> > > >>>>>> amount
> > >> >> > > >>>>>>> of Curator's jUnit tests with a different ZK version.
> > >> >> > > >>>>>>> We might be able to do the same here, tag tests which
> are
> > >> >> > > >>>>>>> testing ZK
> > >> >> > > >>>>>> client
> > >> >> > > >>>>>>> and run them separately with Java 8. The only
> limitation
> > is
> > >> >> > > >>>>>>> that these
> > >> >> > > >>>>>>> tests must stay JDK8 compatible.
> > >> >> > > >>>>>>> But from the tags we will see which ones are those.
> > >> >> > > >>>>>>>
> > >> >> > > >>>>>>> What do you think?
> > >> >> > > >>>>>>>
> > >> >> > > >>>>>>> Regards, Tamaas
> > >> >> > > >>>>>>>
> > >> >> > > >>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> > >> >> > > >>>>>>> eolivelli@gmail.com>
> > >> >> > > >>>>>> wrote:
> > >> >> > > >>>>>>>> Christopher
> > >> >> > > >>>>>>>> I appreciate your idea and I also moved lots of my
> > projects
> > >> >> > > >>>>>>>> to work
> > >> >> > > >>>> the
> > >> >> > > >>>>>> way
> > >> >> > > >>>>>>>> you are suggesting.
> > >> >> > > >>>>>>>>
> > >> >> > > >>>>>>>>
> > >> >> > > >>>>>>>> We must run tests using real jdk8 to test the
> Zookeeper
> > >> >> > > >>>>>>>> client. We
> > >> >> > > >>>> must
> > >> >> > > >>>>>>>> ensure that Zookeeper works well, especially while
> > dealing
> > >> >> > > >>>>>>>> with
> > >> >> > > >>>> security
> > >> >> > > >>>>>>>> stuff.
> > >> >> > > >>>>>>>> Currently the client is in the same module of the
> server
> > >> >> > > >>>>>>>> and it will
> > >> >> > > >>>>>> take a
> > >> >> > > >>>>>>>> good (huge) amount of work to separate them
> > >> >> > > >>>>>>>>
> > >> >> > > >>>>>>>>
> > >> >> > > >>>>>>>> Enrico
> > >> >> > > >>>>>>>>
> > >> >> > > >>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher <
> > ctubbsii@apache.org>
> > >> >> > > >>>>>>>> ha
> > >> >> > > >>>> scritto:
> > >> >> > > >>>>>>>>> Hi ZK Devs,
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> With recent advancements in Java (since Java 9), it
> is
> > >> >> > > >>>>>>>>> now generally
> > >> >> > > >>>>>>>>> no longer necessary to require that software be
> > developed
> > >> >> > > >>>>>>>>> on an older
> > >> >> > > >>>>>>>>> JDK in order to have confidence that it will run on
> the
> > >> >> > > >>>>>>>>> older version
> > >> >> > > >>>>>>>>> of Java. This is because, as of Java 9, all JDK
> > releases
> > >> >> > > >>>>>>>>> have better
> > >> >> > > >>>>>>>>> support for cross-compilation to older Java versions.
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> What this means is that developers can confidently
> make
> > >> >> > > >>>>>>>>> the build
> > >> >> > > >>>>>>>>> requirements for a project higher than the Java
> version
> > >> >> > > >>>>>>>>> that will
> > >> >> > > >>>>>>>>> actually be supported at runtime.
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> In fact, ZooKeeper already supports the necessary
> flags
> > >> >> > > >>>>>>>>> in its Maven
> > >> >> > > >>>>>>>>> build configuration to ensure that it uses JDK 8
> > >> >> > > >>>>>>>>> compliance when
> > >> >> > > >>>>>>>>> building on a newer JDK (I added this way back in
> > >> >> > > >>>>>>>>> ZOOKEEPER-3739 /
> > >> >> > > >>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> So, I propose that we make JDK 11 the new minimum
> > version
> > >> >> > > >>>>>>>>> to *build*
> > >> >> > > >>>>>>>>> ZooKeeper with. This would not change the runtime
> > >> >> > > >>>>>>>>> requirement, which
> > >> >> > > >>>>>>>>> would remain at JDK 8.
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> The only necessary change to make this happen would
> be
> > to
> > >> >> > > >>>>>>>>> add the
> > >> >> > > >>>>>>>>> minimum Java version to the maven-enforcer-plugin
> (like
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>
> > >> >> > > >>
> > >> >> > >
> > >> >> >
> > >> >>
> > >
> >
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> > >> >> > > >>>>>>>>> )
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> This would allow ZooKeeper to to streamline its
> > >> >> > > >>>>>>>>> development process a
> > >> >> > > >>>>>>>>> little bit by reducing the amount of CI testing that
> is
> > >> >> > > >>>>>>>>> done as part
> > >> >> > > >>>>>>>>> of the build. In other words, we can drop the CI
> builds
> > >> >> > > >>>>>>>>> for JDK 8,
> > >> >> > > >>>>>>>>> which saves on build resources and time. The return
> on
> > >> >> > > >>>>>>>>> investment is
> > >> >> > > >>>>>>>>> so low for the JDK 8 builds anyway, because of the
> > >> >> > > >>>>>>>>> improved
> > >> >> > > >>>>>>>>> cross-compilation in newer JDKs. So, there's not much
> > >> >> > > >>>>>>>>> value in
> > >> >> > > >>>>>>>>> building on JDK 8 anyway.
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> Of course, I am only recommending this for *new*
> > release
> > >> >> > > >>>>>>>>> lines,
> > >> >> > > >>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because
> I
> > >> >> > > >>>>>>>>> would not want
> > >> >> > > >>>>>>>>> to change expectations for users who will build their
> > own
> > >> >> > > >>>>>>>>> 3.5 and 3.6
> > >> >> > > >>>>>>>>> versions as they continue to have patch versions
> > >> >> > > >>>>>>>>> released.
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> What do you think?
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> Kind Regards,
> > >> >> > > >>>>>>>>> Christopher
> > >> >> > > >>>>>>>>>
> > >> >> > > >>
> > >> >> > > >>
> > >> >> > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >
> >
>

Re: 3.7.0: JDK 11 vs. JDK 8 (was: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7)

[Norbert Kalmar <nk...@cloudera.com.INVALID>]

I'd say there are quite a few tasks aimed at 4.0. I just answered a thread
about jute.maxbuffer error, which could be improved for example. Or better
yet, throw jute out and use a standardized serialization library.
But there's also the issue of separating client and server code. And I'm
sure there's many more I can't recall right now.

But this is worth a separate vote thread, that's for sure.

- Norbert

On Fri, Jan 8, 2021 at 1:33 PM Andor Molnar <an...@apache.org> wrote:

> Okay, let’s stay on JDK 8 with 3.7.0 release and do the transition in 4.0.
> Not sure if we want 3.8 release or make the master 4.0 from now on.
>
> Andor
>
>
>
> > On 2021. Jan 6., at 22:35, Christopher <ct...@apache.org> wrote:
> >
> > I agree with Enrico on this point. If the ZK PMC is considering a 3.7
> > release, now would not be the right time to make this change, and it
> would
> > be better to make a change at the beginning of the next iteration.
> >
> > That said, I think switching to builds with JDK 11 and supporting JDK 8
> > "passively" is the right thing to do for 3.8, and switching to JDK 11
> fully
> > would be the right thing to do if the PMC decides to do a major version
> > bump to 4.0 instead of a 3.8.
> >
> > On Wed, Jan 6, 2021 at 1:20 PM Enrico Olivelli <eo...@gmail.com>
> wrote:
> >
> >> In my opinion we can stay on 8, we should change this kind of stuff at
> the
> >> beginning of a new iteration and not before the release.
> >> IIRC We are not in a hurry, having a stable build process is a value.
> >>
> >> Enrico
> >>
> >> Il Mer 6 Gen 2021, 17:51 Damien Diederen <dd...@apache.org> ha
> >> scritto:
> >>
> >>>
> >>> Dear ZooKeeper team,
> >>>
> >>> Andor just reminded me of this JDK 11 vs. 8 discussion, for which we
> did
> >>> not reach a resolution.  Do we want to make a move for the 3.7.0
> release?
> >>>
> >>> The original proposal, by Christopher, can be found here:
> >>>
> >>>
> >>>
> >>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9Z2TNfAwjZ7etXPF91QmROvUWV+KHnxgPSD_MSeKfpyWA@mail.gmail.com%3e
> >>>
> >>> I am explicitly replying to Christopher's nice and useful summary,
> which
> >>> you will find directly below.  An archived copy can be found here:
> >>>
> >>>
> >>>
> >>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9ao9-4S9bmzxaYJM8n8=AoLxiDZ-0ZohunVnAyQei0zAg@mail.gmail.com%3e
> >>>
> >>> It contains three proposals, to which we can add the trivial one:
> >>>
> >>> 1. Compile/test with JDK 11, support JDK 8 "passively";
> >>>
> >>> 2. Compile/test with JDK 11, test client only with JDK 8 (requires test
> >>>    suite adaptations);
> >>>
> >>> 3. Move to JDK 11 fully;
> >>>
> >>> 4. Postpone any change to 3.8 (or 4.0).
> >>>
> >>> Flavio wrote:
> >>>
> >>>> [Christopher] mentioned a PMC vote, and I don't think this should be a
> >>>> closed vote, independent of how the conversation goes.
> >>>
> >>> Do we have a better idea of what we want?  Or should we organize a
> vote?
> >>>
> >>> Cheers, -D
> >>>
> >>>
> >>> --8<--------------- original message ------------->8---
> >>>
> >>> Christopher <ct...@apache.org> writes:
> >>>> I'm happy that this discussion has been so lively! I just want to
> >>>> emphasize a few things:
> >>>>
> >>>> I really do understand the desire to continue to support Java 8... I
> >>>> get it. But all the conversations around this seem based on what
> >>>> people are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
> >>>> *future* release... so it should be based more on reasonable
> >>>> expectations for users in the future, and less based on what is
> >>>> happening today. I suspect *most* people today are still using 3.4
> >>>> anyway (it was just so stable for so long...), but that shouldn't mean
> >>>> the developers should hold back development on 3.5 and 3.6, any more
> >>>> than today's users of 3.5/3.6 should hold back 3.7.
> >>>>
> >>>> Some of the opinions expressed in this discussion seem to propose a
> >>>> scenario where users are going to be updating to "bleeding edge"
> >>>> versions of ZooKeeper, but are going to insist on using Java 8.
> >>>> Personally, I find this to be implausible. In my experience, people
> >>>> either upgrade everything as soon as they are able to, or they upgrade
> >>>> each thing individually, only when they are forced to. The first group
> >>>> will be happy to move to Java 11 and ZK 3.7. The second group will
> >>>> probably avoid 3.7 anyway, and are fine sticking with 3.6, but if they
> >>>> had to update to 3.7, they'd also be fine updating to Java 11 if they
> >>>> had to in order to use 3.7. I can't imagine the scenario where people
> >>>> are eagerly choosing to upgrade to ZK 3.7, but miserly insisting on
> >>>> using Java 8. Perhaps that scenario exists, but it's hard for me to
> >>>> imagine. Even so, my proposal would still support even that group of
> >>>> people.
> >>>>
> >>>> I think there are now effectively three proposals being discussed in
> >>>> this thread:
> >>>>
> >>>> 1. (Christopher's original proposal) passively support Java 8 at
> >>>> runtime by making JDK 11 the minimum requirement to build and test.
> >>>> This scenario involves continuing to fix bugs, as they are discovered
> >>>> and reported, that affect JDK 8, but passively, rather than
> >>>> proactively. This proposal does *not* drop Java 8 support, but merely
> >>>> de-emphasizes it in development of what will be 3.7 in the future, and
> >>>> drops the requirement to do dedicated testing with Java 8. I think
> >>>> this is low risk, because it is very unlikely that the ZK devs would
> >>>> introduce a bug that would affect only Java 8 and the compiler
> >>>> wouldn't catch it... because the cross-compilation features of newer
> >>>> JDKs are really good.
> >>>>
> >>>> 2. (Enrico's alternate proposal) this variation of my proposal would
> >>>> involve continuing to proactively support Java 8 by creating a
> >>>> dedicated testing suite to test client code on Java 8. I think this is
> >>>> a good option, but since it involves a significantly higher amount of
> >>>> work than option 1, I think the cost-benefit analysis would show this
> >>>> to be not worth the effort. Also, if it were implemented, it would
> >>>> need to be done carefully to avoid requiring developers to have
> >>>> concurrently installed both Java 8 and Java 11 in order to perform a
> >>>> build, because requiring Java 8 at build time while developing would
> >>>> be worse than we have today.
> >>>>
> >>>> 3. (Andor's preference) move to JDK 11 fully. This would provide no
> >>>> support, passive or active, for Java 8 in ZK 3.7. To be honest, this
> >>>> is my personal favorite, and is the simplest to implement and
> >>>> communicate clearly to end users in release notes. The only reason I
> >>>> proposed a passive support of Java 8 instead of this is because I was
> >>>> trying to seek a compromise from the start. But, I think by far, this
> >>>> is the best option for the next *future* release of ZK. If you wanted
> >>>> to make the change even more visible to users, the version could even
> >>>> be bumped to 4.0.
> >>>>
> >>>> If this were to come to a VOTE by the PMC, in order to make a final
> >>>> decision, I would recommend they vote on option 3, and then if that
> >>>> fails, vote on option 1, and if that fails, keep things the way they
> >>>> are (because option 2 is more work).
> >>>>
> >>>> Christopher
> >>>>
> >>>> P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK 11
> >>>> and it works just fine there (as long as you add the missing runtime
> >>>> jar for javax.activation:javax.activation-api:1.2.0 to its class path,
> >>>> but that was fixed in Hadoop 3.3).
> >>>>
> >>>> On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes
> >>>> <ta...@cloudera.com.invalid> wrote:
> >>>>>
> >>>>> Hi All,
> >>>>>
> >>>>> I've just talked with a Hadoop/HDFS developer who told me what I
> >>> guessed.
> >>>>> With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would
> >>> mean
> >>>>> a release of Hadoop4.
> >>>>> Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both.
> As
> >>> of
> >>>>> today most of the Hadoop users are still on Hadoop2, he doesn't
> expect
> >>>>> Hadoop4 soon.
> >>>>> As many Apache components depend on Hadoop and ZooKeeper they won't
> >>> hurry
> >>>>> to JDK11 until they have to (they will probably go one-by-one very
> >>> slowly),
> >>>>> which means if Hadoop stays on JDK8, they would use the last ZK
> >> version
> >>>>> which works on JDK8.
> >>>>> Do we want a ZK 3.4 again?
> >>>>>
> >>>>> Regards, Tamaas
> >>>>>
> >>>>>
> >>>>> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <ta...@cloudera.com>
> >>> wrote:
> >>>>>
> >>>>>> Hi All,
> >>>>>>
> >>>>>> Just to add my two cents.
> >>>>>>
> >>>>>> Upgrading to JDK11 looks inevitable sooner or later and I would
> >>> definitely
> >>>>>> not wait until 2030 or 2026 when the extended support of JDK8 ends.
> >>>>>> But on the other side I have to agree with Enrico and Patrick that
> >>> far too
> >>>>>> many users are tied to JDK8 yet (not because they want to use JDK8,
> >>> but
> >>>>>> because they have to), some of them are components of the Hadoop
> >>> ecosystem,
> >>>>>> which would be a loss to tie them to 3.6.x for years.
> >>>>>> Do we know the state of Hadoop? It builds with JDK8 at the moment,
> >>> but do
> >>>>>> we know what are their plans to go to JDK11?
> >>>>>> When they move, we should move too, but I don't think it would be
> >>> wise to
> >>>>>> do it earlier.
> >>>>>>
> >>>>>> Christopher's option looks like the golden path, but it needs some
> >>>>>> investment on the testing side as Enrico pointed it out.
> >>>>>> Could we agree on it as a compromise?
> >>>>>>
> >>>>>> Regards, Tamaas
> >>>>>>
> >>>>>> On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com>
> >>> wrote:
> >>>>>>
> >>>>>>> I think I was reacting to Enrico's earlier comment of:
> >>>>>>>
> >>>>>>> " ZooKeeper client is used by tons of users and unfortunately many
> >>>>>>> projects
> >>>>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
> >> block
> >>>>>>> users
> >>>>>>> from the adoption, that is that we will see the world to stay on
> >>> 3.6.x and
> >>>>>>> we will have again a long lived release line, like 3.4."
> >>>>>>>
> >>>>>>> It's a matter of whether or not a long-lived release line is
> >>>>>>> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
> >>>>>>> security/patch-wise (if not feature-wise) for the next N years,
> >> then
> >>>>>>> that's
> >>>>>>> a potentially valid approach.  I interpreted that comment as "a
> >>> long-lived
> >>>>>>> release line is undesirable", but no one explicitly said that, I
> >>> just read
> >>>>>>> it that way.
> >>>>>>>
> >>>>>>> ~Brent
> >>>>>>>
> >>>>>>> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org>
> >>> wrote:
> >>>>>>>
> >>>>>>>> On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org>
> >>> wrote:
> >>>>>>>>
> >>>>>>>>> As far as I know Hbase, Solr and Kafka are already Java 11
> >> ready.
> >>>>>>>>>
> >>>>>>>>> IMHO contributors of those projects should also put efforts
> >> into
> >>>>>>> moving
> >>>>>>>>> forward.
> >>>>>>>>>
> >>>>>>>>> We’re not saying that you _have_ to move to Java 11.
> >>>>>>>>> Staying on Java 8? No problem, 3.6 is for you.
> >>>>>>>>> Want the fancy new features of 3.7? Work on it on your side
> >> too.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>> ppl want things like security fixes. I believe the highlighted
> >>> downside
> >>>>>>> is
> >>>>>>>> that we would need to continue to maintain 3.6.x rather than
> >>> allowing
> >>>>>>>> users, and ourselves, to focus on trunk for production -"64
> >>> percent"
> >>>>>>> would
> >>>>>>>> be blocked.
> >>>>>>>>
> >>>>>>>> Patrick
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> Andor
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> On 2020. Oct 21., at 17:52, Enrico Olivelli <
> >>> eolivelli@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
> >>>>>>>> andor@apache.org>
> >>>>>>>>> ha
> >>>>>>>>>> scritto:
> >>>>>>>>>>
> >>>>>>>>>>> Correct me if I'm wrong, but Oracle gets paid for extended
> >>> support.
> >>>>>>>>>>> Java 8 support until 2030 is not free of charge.
> >>>>>>>>>>>
> >>>>>>>>>>> "ZK may end up with a lot of users potentially locking
> >>> themselves
> >>>>>>> to
> >>>>>>>>>>> 3.6.x for a while as Enrico mentioned."
> >>>>>>>>>>>
> >>>>>>>>>>> That's true. What's the downside of that which we should
> >>> invest in
> >>>>>>> to
> >>>>>>>>>>> avoid?
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> I see ZooKeeper used in many many projects, all of the
> >>>>>>>>>> HBase/Pulsar/Kafka/Solr ecosystem...
> >>>>>>>>>> they will have to move to JDK11 in order to move to the new
> >> ZK
> >>>>>>> version
> >>>>>>>>>> so probably they will stay on ZK 3.6
> >>>>>>>>>>
> >>>>>>>>>> Probably with Java 17 LTS released the cards will change on
> >> the
> >>>>>>> table
> >>>>>>>>>>
> >>>>>>>>>> Enrico
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Andor
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> >>>>>>>>>>>> As a slightly different consideration, if you look at the
> >>>>>>> Long-Term
> >>>>>>>>>>>> Support
> >>>>>>>>>>>> (LTS) roadmaps for Java, currently Java 8 is set to have
> >> full
> >>>>>>> support
> >>>>>>>>>>>> until
> >>>>>>>>>>>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>
> >>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> >>>>>>>>>>>> https://en.wikipedia.org/wiki/Java_version_history
> >>>>>>>>>>>>
> >>>>>>>>>>>> My guess is that a number of companies are still heavily
> >>> invested
> >>>>>>> at
> >>>>>>>>>>>> the
> >>>>>>>>>>>> Java 8 level (I know a few) and with that kind of time
> >>> horizon,
> >>>>>>> they
> >>>>>>>>>>>> have
> >>>>>>>>>>>> no real motivation to upgrade for quite a while.  If the
> >>> recent
> >>>>>>>>>>>> Python 2
> >>>>>>>>>>>> deprecation is anything to go by, they won't do it until
> >>> they have
> >>>>>>>>>>>> to.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Not saying Java 8 isn't *very* old (2014 release it seems
> >>> like?)
> >>>>>>> and
> >>>>>>>>>>>> I'm
> >>>>>>>>>>>> not invested heavily either way, but this might suggest
> >> that
> >>> ZK
> >>>>>>> may
> >>>>>>>>>>>> end up
> >>>>>>>>>>>> with a lot of users potentially locking themselves to 3.6.x
> >>> for a
> >>>>>>>>>>>> while as
> >>>>>>>>>>>> Enrico mentioned.
> >>>>>>>>>>>>
> >>>>>>>>>>>> (Not a major contributor, but wanted to chime in since I
> >>> just had
> >>>>>>>>>>>> this
> >>>>>>>>>>>> conversation with a bunch of people professionally
> >> recently)
> >>>>>>>>>>>>
> >>>>>>>>>>>> Brent
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <
> >>> andor@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Thanks for the summary.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully.
> >>> Other
> >>>>>>>>>>>>> projects
> >>>>>>>>>>>>> will upgrade once they’re JDK11 compliant, otherwise they
> >>> will
> >>>>>>> stay
> >>>>>>>>>>>>> on 3.5
> >>>>>>>>>>>>> or 3.6. Both version are quite recent in ZooKeeper-terms,
> >> we
> >>>>>>>>>>>>> already
> >>>>>>>>>>>>> planned big changes for 3.7.0 and JDK 11 could be one of
> >>> them.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Don’t put extra burden on the ZK community to help others
> >>> staying
> >>>>>>>>>>>>> on
> >>>>>>>>>>>>> ancient Java versions.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Andor
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
> >>>>>>> eolivelli@gmail.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Let me recap
> >>>>>>>>>>>>>> - Christopher is proposing to move to JDK11
> >>>>>>>>>>>>>> - ZooKeeper client and server are bundled and coded and
> >>> tested
> >>>>>>>>>>>>>> together
> >>>>>>>>>>>>> in
> >>>>>>>>>>>>>> zookeeper-server
> >>>>>>>>>>>>>> - Enrico is concerned about the need of testing ZooKeeper
> >>> client
> >>>>>>>>>>>>>> on JDK8
> >>>>>>>>>>>>>> (not a problem to move the server to JDK11)
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> ZooKeeper client is used by tons of users and
> >>> unfortunately many
> >>>>>>>>>>>>>> projects
> >>>>>>>>>>>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk
> >>> is to
> >>>>>>>>>>>>>> block
> >>>>>>>>>>>>> users
> >>>>>>>>>>>>>> from the adoption,
> >>>>>>>>>>>>>> that is that we will see the world to stay on 3.6.x and
> >> we
> >>> will
> >>>>>>>>>>>>>> have
> >>>>>>>>>>>>> again
> >>>>>>>>>>>>>> a long lived release line, like 3.4.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Testing the client on JDK8 would be possible if we create
> >>> some
> >>>>>>>>>>>>>> kind of
> >>>>>>>>>>>>>> additional module with system tests, then we can start
> >> the
> >>>>>>> server
> >>>>>>>>>>>>>> on
> >>>>>>>>>>>>> docker
> >>>>>>>>>>>>>> on JDK11+ and start a client on JDK8
> >>>>>>>>>>>>>> with Maven toolchain it should possible to run surefire
> >>> tests
> >>>>>>>>>>>>>> using a
> >>>>>>>>>>>>>> separate JVM.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> So in my vision 2 options:
> >>>>>>>>>>>>>> 1) fully JDK11 - drop JDK8 at all
> >>>>>>>>>>>>>> 2) build with JDK11 - server only on JDK11 - add system
> >>> tests
> >>>>>>>>>>>>>> with docker
> >>>>>>>>>>>>>> and toolchains that ensure the ZooKeeper client (and all
> >>>>>>>>>>>>>> dependencies)
> >>>>>>>>>>>>>> still work on JDK8
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> From my point of view about the ZooKeeper ecosystem
> >> option
> >>> 2)
> >>>>>>>>>>>>>> will be far
> >>>>>>>>>>>>>> better, but we need resources to work on a new test
> >> suite.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Enrico
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> >>>>>>>>>>>>>> andor@apache.org>
> >>>>>>>>>>>>> ha
> >>>>>>>>>>>>>> scritto:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Tamas, Enrico,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Sorry I don’t follow. Why do we have to test the client
> >>> with
> >>>>>>>>>>>>>>> JDK 8 in
> >>>>>>>>>>>>>>> version 3.7.0?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Andor
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
> >>>>>>>>>>>>>>>> tamaas@cloudera.com.INVALID>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>> Hi Enrico,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Separating ZooKeeper client and server is a huge work,
> >>> but we
> >>>>>>>>>>>>>>>> might not
> >>>>>>>>>>>>>>>> need it.
> >>>>>>>>>>>>>>>> As you mentioned we have to test ZK client with Java 8,
> >>> what
> >>>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>> separating only the test cases which we need to run
> >> with
> >>>>>>>>>>>>>>>> Java8 too?
> >>>>>>>>>>>>>>>> In Curator we have the ZK compatibility tests where we
> >>> run a
> >>>>>>>>>>>>>>>> limited
> >>>>>>>>>>>>>>> amount
> >>>>>>>>>>>>>>>> of Curator's jUnit tests with a different ZK version.
> >>>>>>>>>>>>>>>> We might be able to do the same here, tag tests which
> >> are
> >>>>>>>>>>>>>>>> testing ZK
> >>>>>>>>>>>>>>> client
> >>>>>>>>>>>>>>>> and run them separately with Java 8. The only
> >> limitation
> >>> is
> >>>>>>>>>>>>>>>> that these
> >>>>>>>>>>>>>>>> tests must stay JDK8 compatible.
> >>>>>>>>>>>>>>>> But from the tags we will see which ones are those.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Regards, Tamaas
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> >>>>>>>>>>>>>>>> eolivelli@gmail.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>> Christopher
> >>>>>>>>>>>>>>>>> I appreciate your idea and I also moved lots of my
> >>> projects
> >>>>>>>>>>>>>>>>> to work
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> way
> >>>>>>>>>>>>>>>>> you are suggesting.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> We must run tests using real jdk8 to test the
> >> Zookeeper
> >>>>>>>>>>>>>>>>> client. We
> >>>>>>>>>>>>> must
> >>>>>>>>>>>>>>>>> ensure that Zookeeper works well, especially while
> >>> dealing
> >>>>>>>>>>>>>>>>> with
> >>>>>>>>>>>>> security
> >>>>>>>>>>>>>>>>> stuff.
> >>>>>>>>>>>>>>>>> Currently the client is in the same module of the
> >> server
> >>>>>>>>>>>>>>>>> and it will
> >>>>>>>>>>>>>>> take a
> >>>>>>>>>>>>>>>>> good (huge) amount of work to separate them
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Enrico
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher <
> >>> ctubbsii@apache.org>
> >>>>>>>>>>>>>>>>> ha
> >>>>>>>>>>>>> scritto:
> >>>>>>>>>>>>>>>>>> Hi ZK Devs,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> With recent advancements in Java (since Java 9), it
> >> is
> >>>>>>>>>>>>>>>>>> now generally
> >>>>>>>>>>>>>>>>>> no longer necessary to require that software be
> >>> developed
> >>>>>>>>>>>>>>>>>> on an older
> >>>>>>>>>>>>>>>>>> JDK in order to have confidence that it will run on
> >> the
> >>>>>>>>>>>>>>>>>> older version
> >>>>>>>>>>>>>>>>>> of Java. This is because, as of Java 9, all JDK
> >>> releases
> >>>>>>>>>>>>>>>>>> have better
> >>>>>>>>>>>>>>>>>> support for cross-compilation to older Java versions.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> What this means is that developers can confidently
> >> make
> >>>>>>>>>>>>>>>>>> the build
> >>>>>>>>>>>>>>>>>> requirements for a project higher than the Java
> >> version
> >>>>>>>>>>>>>>>>>> that will
> >>>>>>>>>>>>>>>>>> actually be supported at runtime.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> In fact, ZooKeeper already supports the necessary
> >> flags
> >>>>>>>>>>>>>>>>>> in its Maven
> >>>>>>>>>>>>>>>>>> build configuration to ensure that it uses JDK 8
> >>>>>>>>>>>>>>>>>> compliance when
> >>>>>>>>>>>>>>>>>> building on a newer JDK (I added this way back in
> >>>>>>>>>>>>>>>>>> ZOOKEEPER-3739 /
> >>>>>>>>>>>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> So, I propose that we make JDK 11 the new minimum
> >>> version
> >>>>>>>>>>>>>>>>>> to *build*
> >>>>>>>>>>>>>>>>>> ZooKeeper with. This would not change the runtime
> >>>>>>>>>>>>>>>>>> requirement, which
> >>>>>>>>>>>>>>>>>> would remain at JDK 8.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> The only necessary change to make this happen would
> >> be
> >>> to
> >>>>>>>>>>>>>>>>>> add the
> >>>>>>>>>>>>>>>>>> minimum Java version to the maven-enforcer-plugin
> >> (like
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>
> >>>
> >>
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> >>>>>>>>>>>>>>>>>> )
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> This would allow ZooKeeper to to streamline its
> >>>>>>>>>>>>>>>>>> development process a
> >>>>>>>>>>>>>>>>>> little bit by reducing the amount of CI testing that
> >> is
> >>>>>>>>>>>>>>>>>> done as part
> >>>>>>>>>>>>>>>>>> of the build. In other words, we can drop the CI
> >> builds
> >>>>>>>>>>>>>>>>>> for JDK 8,
> >>>>>>>>>>>>>>>>>> which saves on build resources and time. The return
> >> on
> >>>>>>>>>>>>>>>>>> investment is
> >>>>>>>>>>>>>>>>>> so low for the JDK 8 builds anyway, because of the
> >>>>>>>>>>>>>>>>>> improved
> >>>>>>>>>>>>>>>>>> cross-compilation in newer JDKs. So, there's not much
> >>>>>>>>>>>>>>>>>> value in
> >>>>>>>>>>>>>>>>>> building on JDK 8 anyway.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Of course, I am only recommending this for *new*
> >>> release
> >>>>>>>>>>>>>>>>>> lines,
> >>>>>>>>>>>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because
> >> I
> >>>>>>>>>>>>>>>>>> would not want
> >>>>>>>>>>>>>>>>>> to change expectations for users who will build their
> >>> own
> >>>>>>>>>>>>>>>>>> 3.5 and 3.6
> >>>>>>>>>>>>>>>>>> versions as they continue to have patch versions
> >>>>>>>>>>>>>>>>>> released.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Kind Regards,
> >>>>>>>>>>>>>>>>>> Christopher
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>
> >>
>
>

Re: 3.7.0: JDK 11 vs. JDK 8

[Damien Diederen <dd...@apache.org>]

Hi Andor, all,

Okay—a clear consensus has emerged in this thread.  3.7+ will follow the
same "JDK policy" as 3.6.

Thank you all for your input.

Cheers, -D



Andor Molnar <an...@apache.org> writes:
> Okay, let’s stay on JDK 8 with 3.7.0 release and do the transition in 4.0.
> Not sure if we want 3.8 release or make the master 4.0 from now on.
>
> Andor
>
>
>
>> On 2021. Jan 6., at 22:35, Christopher <ct...@apache.org> wrote:
>> 
>> I agree with Enrico on this point. If the ZK PMC is considering a 3.7
>> release, now would not be the right time to make this change, and it would
>> be better to make a change at the beginning of the next iteration.
>> 
>> That said, I think switching to builds with JDK 11 and supporting JDK 8
>> "passively" is the right thing to do for 3.8, and switching to JDK 11 fully
>> would be the right thing to do if the PMC decides to do a major version
>> bump to 4.0 instead of a 3.8.
>> 
>> On Wed, Jan 6, 2021 at 1:20 PM Enrico Olivelli <eo...@gmail.com> wrote:
>> 
>>> In my opinion we can stay on 8, we should change this kind of stuff at the
>>> beginning of a new iteration and not before the release.
>>> IIRC We are not in a hurry, having a stable build process is a value.
>>> 
>>> Enrico
>>> 
>>> Il Mer 6 Gen 2021, 17:51 Damien Diederen <dd...@apache.org> ha
>>> scritto:
>>> 
>>>> 
>>>> Dear ZooKeeper team,
>>>> 
>>>> Andor just reminded me of this JDK 11 vs. 8 discussion, for which we did
>>>> not reach a resolution.  Do we want to make a move for the 3.7.0 release?
>>>> 
>>>> The original proposal, by Christopher, can be found here:
>>>> 
>>>> 
>>>> 
>>>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9Z2TNfAwjZ7etXPF91QmROvUWV+KHnxgPSD_MSeKfpyWA@mail.gmail.com%3e
>>>> 
>>>> I am explicitly replying to Christopher's nice and useful summary, which
>>>> you will find directly below.  An archived copy can be found here:
>>>> 
>>>> 
>>>> 
>>>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9ao9-4S9bmzxaYJM8n8=AoLxiDZ-0ZohunVnAyQei0zAg@mail.gmail.com%3e
>>>> 
>>>> It contains three proposals, to which we can add the trivial one:
>>>> 
>>>> 1. Compile/test with JDK 11, support JDK 8 "passively";
>>>> 
>>>> 2. Compile/test with JDK 11, test client only with JDK 8 (requires test
>>>>    suite adaptations);
>>>> 
>>>> 3. Move to JDK 11 fully;
>>>> 
>>>> 4. Postpone any change to 3.8 (or 4.0).
>>>> 
>>>> Flavio wrote:
>>>> 
>>>>> [Christopher] mentioned a PMC vote, and I don't think this should be a
>>>>> closed vote, independent of how the conversation goes.
>>>> 
>>>> Do we have a better idea of what we want?  Or should we organize a vote?
>>>> 
>>>> Cheers, -D
>>>> 
>>>> 
>>>> --8<--------------- original message ------------->8---
>>>> 
>>>> Christopher <ct...@apache.org> writes:
>>>>> I'm happy that this discussion has been so lively! I just want to
>>>>> emphasize a few things:
>>>>> 
>>>>> I really do understand the desire to continue to support Java 8... I
>>>>> get it. But all the conversations around this seem based on what
>>>>> people are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
>>>>> *future* release... so it should be based more on reasonable
>>>>> expectations for users in the future, and less based on what is
>>>>> happening today. I suspect *most* people today are still using 3.4
>>>>> anyway (it was just so stable for so long...), but that shouldn't mean
>>>>> the developers should hold back development on 3.5 and 3.6, any more
>>>>> than today's users of 3.5/3.6 should hold back 3.7.
>>>>> 
>>>>> Some of the opinions expressed in this discussion seem to propose a
>>>>> scenario where users are going to be updating to "bleeding edge"
>>>>> versions of ZooKeeper, but are going to insist on using Java 8.
>>>>> Personally, I find this to be implausible. In my experience, people
>>>>> either upgrade everything as soon as they are able to, or they upgrade
>>>>> each thing individually, only when they are forced to. The first group
>>>>> will be happy to move to Java 11 and ZK 3.7. The second group will
>>>>> probably avoid 3.7 anyway, and are fine sticking with 3.6, but if they
>>>>> had to update to 3.7, they'd also be fine updating to Java 11 if they
>>>>> had to in order to use 3.7. I can't imagine the scenario where people
>>>>> are eagerly choosing to upgrade to ZK 3.7, but miserly insisting on
>>>>> using Java 8. Perhaps that scenario exists, but it's hard for me to
>>>>> imagine. Even so, my proposal would still support even that group of
>>>>> people.
>>>>> 
>>>>> I think there are now effectively three proposals being discussed in
>>>>> this thread:
>>>>> 
>>>>> 1. (Christopher's original proposal) passively support Java 8 at
>>>>> runtime by making JDK 11 the minimum requirement to build and test.
>>>>> This scenario involves continuing to fix bugs, as they are discovered
>>>>> and reported, that affect JDK 8, but passively, rather than
>>>>> proactively. This proposal does *not* drop Java 8 support, but merely
>>>>> de-emphasizes it in development of what will be 3.7 in the future, and
>>>>> drops the requirement to do dedicated testing with Java 8. I think
>>>>> this is low risk, because it is very unlikely that the ZK devs would
>>>>> introduce a bug that would affect only Java 8 and the compiler
>>>>> wouldn't catch it... because the cross-compilation features of newer
>>>>> JDKs are really good.
>>>>> 
>>>>> 2. (Enrico's alternate proposal) this variation of my proposal would
>>>>> involve continuing to proactively support Java 8 by creating a
>>>>> dedicated testing suite to test client code on Java 8. I think this is
>>>>> a good option, but since it involves a significantly higher amount of
>>>>> work than option 1, I think the cost-benefit analysis would show this
>>>>> to be not worth the effort. Also, if it were implemented, it would
>>>>> need to be done carefully to avoid requiring developers to have
>>>>> concurrently installed both Java 8 and Java 11 in order to perform a
>>>>> build, because requiring Java 8 at build time while developing would
>>>>> be worse than we have today.
>>>>> 
>>>>> 3. (Andor's preference) move to JDK 11 fully. This would provide no
>>>>> support, passive or active, for Java 8 in ZK 3.7. To be honest, this
>>>>> is my personal favorite, and is the simplest to implement and
>>>>> communicate clearly to end users in release notes. The only reason I
>>>>> proposed a passive support of Java 8 instead of this is because I was
>>>>> trying to seek a compromise from the start. But, I think by far, this
>>>>> is the best option for the next *future* release of ZK. If you wanted
>>>>> to make the change even more visible to users, the version could even
>>>>> be bumped to 4.0.
>>>>> 
>>>>> If this were to come to a VOTE by the PMC, in order to make a final
>>>>> decision, I would recommend they vote on option 3, and then if that
>>>>> fails, vote on option 1, and if that fails, keep things the way they
>>>>> are (because option 2 is more work).
>>>>> 
>>>>> Christopher
>>>>> 
>>>>> P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK 11
>>>>> and it works just fine there (as long as you add the missing runtime
>>>>> jar for javax.activation:javax.activation-api:1.2.0 to its class path,
>>>>> but that was fixed in Hadoop 3.3).
>>>>> 
>>>>> On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes
>>>>> <ta...@cloudera.com.invalid> wrote:
>>>>>> 
>>>>>> Hi All,
>>>>>> 
>>>>>> I've just talked with a Hadoop/HDFS developer who told me what I
>>>> guessed.
>>>>>> With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would
>>>> mean
>>>>>> a release of Hadoop4.
>>>>>> Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both. As
>>>> of
>>>>>> today most of the Hadoop users are still on Hadoop2, he doesn't expect
>>>>>> Hadoop4 soon.
>>>>>> As many Apache components depend on Hadoop and ZooKeeper they won't
>>>> hurry
>>>>>> to JDK11 until they have to (they will probably go one-by-one very
>>>> slowly),
>>>>>> which means if Hadoop stays on JDK8, they would use the last ZK
>>> version
>>>>>> which works on JDK8.
>>>>>> Do we want a ZK 3.4 again?
>>>>>> 
>>>>>> Regards, Tamaas
>>>>>> 
>>>>>> 
>>>>>> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <ta...@cloudera.com>
>>>> wrote:
>>>>>> 
>>>>>>> Hi All,
>>>>>>> 
>>>>>>> Just to add my two cents.
>>>>>>> 
>>>>>>> Upgrading to JDK11 looks inevitable sooner or later and I would
>>>> definitely
>>>>>>> not wait until 2030 or 2026 when the extended support of JDK8 ends.
>>>>>>> But on the other side I have to agree with Enrico and Patrick that
>>>> far too
>>>>>>> many users are tied to JDK8 yet (not because they want to use JDK8,
>>>> but
>>>>>>> because they have to), some of them are components of the Hadoop
>>>> ecosystem,
>>>>>>> which would be a loss to tie them to 3.6.x for years.
>>>>>>> Do we know the state of Hadoop? It builds with JDK8 at the moment,
>>>> but do
>>>>>>> we know what are their plans to go to JDK11?
>>>>>>> When they move, we should move too, but I don't think it would be
>>>> wise to
>>>>>>> do it earlier.
>>>>>>> 
>>>>>>> Christopher's option looks like the golden path, but it needs some
>>>>>>> investment on the testing side as Enrico pointed it out.
>>>>>>> Could we agree on it as a compromise?
>>>>>>> 
>>>>>>> Regards, Tamaas
>>>>>>> 
>>>>>>> On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com>
>>>> wrote:
>>>>>>> 
>>>>>>>> I think I was reacting to Enrico's earlier comment of:
>>>>>>>> 
>>>>>>>> " ZooKeeper client is used by tons of users and unfortunately many
>>>>>>>> projects
>>>>>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
>>> block
>>>>>>>> users
>>>>>>>> from the adoption, that is that we will see the world to stay on
>>>> 3.6.x and
>>>>>>>> we will have again a long lived release line, like 3.4."
>>>>>>>> 
>>>>>>>> It's a matter of whether or not a long-lived release line is
>>>>>>>> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
>>>>>>>> security/patch-wise (if not feature-wise) for the next N years,
>>> then
>>>>>>>> that's
>>>>>>>> a potentially valid approach.  I interpreted that comment as "a
>>>> long-lived
>>>>>>>> release line is undesirable", but no one explicitly said that, I
>>>> just read
>>>>>>>> it that way.
>>>>>>>> 
>>>>>>>> ~Brent
>>>>>>>> 
>>>>>>>> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org>
>>>> wrote:
>>>>>>>> 
>>>>>>>>> On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org>
>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> As far as I know Hbase, Solr and Kafka are already Java 11
>>> ready.
>>>>>>>>>> 
>>>>>>>>>> IMHO contributors of those projects should also put efforts
>>> into
>>>>>>>> moving
>>>>>>>>>> forward.
>>>>>>>>>> 
>>>>>>>>>> We’re not saying that you _have_ to move to Java 11.
>>>>>>>>>> Staying on Java 8? No problem, 3.6 is for you.
>>>>>>>>>> Want the fancy new features of 3.7? Work on it on your side
>>> too.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> ppl want things like security fixes. I believe the highlighted
>>>> downside
>>>>>>>> is
>>>>>>>>> that we would need to continue to maintain 3.6.x rather than
>>>> allowing
>>>>>>>>> users, and ourselves, to focus on trunk for production -"64
>>>> percent"
>>>>>>>> would
>>>>>>>>> be blocked.
>>>>>>>>> 
>>>>>>>>> Patrick
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> Andor
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> On 2020. Oct 21., at 17:52, Enrico Olivelli <
>>>> eolivelli@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
>>>>>>>>> andor@apache.org>
>>>>>>>>>> ha
>>>>>>>>>>> scritto:
>>>>>>>>>>> 
>>>>>>>>>>>> Correct me if I'm wrong, but Oracle gets paid for extended
>>>> support.
>>>>>>>>>>>> Java 8 support until 2030 is not free of charge.
>>>>>>>>>>>> 
>>>>>>>>>>>> "ZK may end up with a lot of users potentially locking
>>>> themselves
>>>>>>>> to
>>>>>>>>>>>> 3.6.x for a while as Enrico mentioned."
>>>>>>>>>>>> 
>>>>>>>>>>>> That's true. What's the downside of that which we should
>>>> invest in
>>>>>>>> to
>>>>>>>>>>>> avoid?
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> I see ZooKeeper used in many many projects, all of the
>>>>>>>>>>> HBase/Pulsar/Kafka/Solr ecosystem...
>>>>>>>>>>> they will have to move to JDK11 in order to move to the new
>>> ZK
>>>>>>>> version
>>>>>>>>>>> so probably they will stay on ZK 3.6
>>>>>>>>>>> 
>>>>>>>>>>> Probably with Java 17 LTS released the cards will change on
>>> the
>>>>>>>> table
>>>>>>>>>>> 
>>>>>>>>>>> Enrico
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> Andor
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
>>>>>>>>>>>>> As a slightly different consideration, if you look at the
>>>>>>>> Long-Term
>>>>>>>>>>>>> Support
>>>>>>>>>>>>> (LTS) roadmaps for Java, currently Java 8 is set to have
>>> full
>>>>>>>> support
>>>>>>>>>>>>> until
>>>>>>>>>>>>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>> 
>>>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>>>>>>>>>>>>> https://en.wikipedia.org/wiki/Java_version_history
>>>>>>>>>>>>> 
>>>>>>>>>>>>> My guess is that a number of companies are still heavily
>>>> invested
>>>>>>>> at
>>>>>>>>>>>>> the
>>>>>>>>>>>>> Java 8 level (I know a few) and with that kind of time
>>>> horizon,
>>>>>>>> they
>>>>>>>>>>>>> have
>>>>>>>>>>>>> no real motivation to upgrade for quite a while.  If the
>>>> recent
>>>>>>>>>>>>> Python 2
>>>>>>>>>>>>> deprecation is anything to go by, they won't do it until
>>>> they have
>>>>>>>>>>>>> to.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Not saying Java 8 isn't *very* old (2014 release it seems
>>>> like?)
>>>>>>>> and
>>>>>>>>>>>>> I'm
>>>>>>>>>>>>> not invested heavily either way, but this might suggest
>>> that
>>>> ZK
>>>>>>>> may
>>>>>>>>>>>>> end up
>>>>>>>>>>>>> with a lot of users potentially locking themselves to 3.6.x
>>>> for a
>>>>>>>>>>>>> while as
>>>>>>>>>>>>> Enrico mentioned.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> (Not a major contributor, but wanted to chime in since I
>>>> just had
>>>>>>>>>>>>> this
>>>>>>>>>>>>> conversation with a bunch of people professionally
>>> recently)
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Brent
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <
>>>> andor@apache.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Thanks for the summary.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully.
>>>> Other
>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>> will upgrade once they’re JDK11 compliant, otherwise they
>>>> will
>>>>>>>> stay
>>>>>>>>>>>>>> on 3.5
>>>>>>>>>>>>>> or 3.6. Both version are quite recent in ZooKeeper-terms,
>>> we
>>>>>>>>>>>>>> already
>>>>>>>>>>>>>> planned big changes for 3.7.0 and JDK 11 could be one of
>>>> them.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Don’t put extra burden on the ZK community to help others
>>>> staying
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>> ancient Java versions.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Andor
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
>>>>>>>> eolivelli@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Let me recap
>>>>>>>>>>>>>>> - Christopher is proposing to move to JDK11
>>>>>>>>>>>>>>> - ZooKeeper client and server are bundled and coded and
>>>> tested
>>>>>>>>>>>>>>> together
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> zookeeper-server
>>>>>>>>>>>>>>> - Enrico is concerned about the need of testing ZooKeeper
>>>> client
>>>>>>>>>>>>>>> on JDK8
>>>>>>>>>>>>>>> (not a problem to move the server to JDK11)
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> ZooKeeper client is used by tons of users and
>>>> unfortunately many
>>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk
>>>> is to
>>>>>>>>>>>>>>> block
>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>> from the adoption,
>>>>>>>>>>>>>>> that is that we will see the world to stay on 3.6.x and
>>> we
>>>> will
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>> again
>>>>>>>>>>>>>>> a long lived release line, like 3.4.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Testing the client on JDK8 would be possible if we create
>>>> some
>>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>>> additional module with system tests, then we can start
>>> the
>>>>>>>> server
>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>> docker
>>>>>>>>>>>>>>> on JDK11+ and start a client on JDK8
>>>>>>>>>>>>>>> with Maven toolchain it should possible to run surefire
>>>> tests
>>>>>>>>>>>>>>> using a
>>>>>>>>>>>>>>> separate JVM.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> So in my vision 2 options:
>>>>>>>>>>>>>>> 1) fully JDK11 - drop JDK8 at all
>>>>>>>>>>>>>>> 2) build with JDK11 - server only on JDK11 - add system
>>>> tests
>>>>>>>>>>>>>>> with docker
>>>>>>>>>>>>>>> and toolchains that ensure the ZooKeeper client (and all
>>>>>>>>>>>>>>> dependencies)
>>>>>>>>>>>>>>> still work on JDK8
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> From my point of view about the ZooKeeper ecosystem
>>> option
>>>> 2)
>>>>>>>>>>>>>>> will be far
>>>>>>>>>>>>>>> better, but we need resources to work on a new test
>>> suite.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Enrico
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
>>>>>>>>>>>>>>> andor@apache.org>
>>>>>>>>>>>>>> ha
>>>>>>>>>>>>>>> scritto:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Tamas, Enrico,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Sorry I don’t follow. Why do we have to test the client
>>>> with
>>>>>>>>>>>>>>>> JDK 8 in
>>>>>>>>>>>>>>>> version 3.7.0?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Andor
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
>>>>>>>>>>>>>>>>> tamaas@cloudera.com.INVALID>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> Hi Enrico,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Separating ZooKeeper client and server is a huge work,
>>>> but we
>>>>>>>>>>>>>>>>> might not
>>>>>>>>>>>>>>>>> need it.
>>>>>>>>>>>>>>>>> As you mentioned we have to test ZK client with Java 8,
>>>> what
>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>> separating only the test cases which we need to run
>>> with
>>>>>>>>>>>>>>>>> Java8 too?
>>>>>>>>>>>>>>>>> In Curator we have the ZK compatibility tests where we
>>>> run a
>>>>>>>>>>>>>>>>> limited
>>>>>>>>>>>>>>>> amount
>>>>>>>>>>>>>>>>> of Curator's jUnit tests with a different ZK version.
>>>>>>>>>>>>>>>>> We might be able to do the same here, tag tests which
>>> are
>>>>>>>>>>>>>>>>> testing ZK
>>>>>>>>>>>>>>>> client
>>>>>>>>>>>>>>>>> and run them separately with Java 8. The only
>>> limitation
>>>> is
>>>>>>>>>>>>>>>>> that these
>>>>>>>>>>>>>>>>> tests must stay JDK8 compatible.
>>>>>>>>>>>>>>>>> But from the tags we will see which ones are those.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Regards, Tamaas
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
>>>>>>>>>>>>>>>>> eolivelli@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> Christopher
>>>>>>>>>>>>>>>>>> I appreciate your idea and I also moved lots of my
>>>> projects
>>>>>>>>>>>>>>>>>> to work
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>>> you are suggesting.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> We must run tests using real jdk8 to test the
>>> Zookeeper
>>>>>>>>>>>>>>>>>> client. We
>>>>>>>>>>>>>> must
>>>>>>>>>>>>>>>>>> ensure that Zookeeper works well, especially while
>>>> dealing
>>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>> security
>>>>>>>>>>>>>>>>>> stuff.
>>>>>>>>>>>>>>>>>> Currently the client is in the same module of the
>>> server
>>>>>>>>>>>>>>>>>> and it will
>>>>>>>>>>>>>>>> take a
>>>>>>>>>>>>>>>>>> good (huge) amount of work to separate them
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Enrico
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher <
>>>> ctubbsii@apache.org>
>>>>>>>>>>>>>>>>>> ha
>>>>>>>>>>>>>> scritto:
>>>>>>>>>>>>>>>>>>> Hi ZK Devs,
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> With recent advancements in Java (since Java 9), it
>>> is
>>>>>>>>>>>>>>>>>>> now generally
>>>>>>>>>>>>>>>>>>> no longer necessary to require that software be
>>>> developed
>>>>>>>>>>>>>>>>>>> on an older
>>>>>>>>>>>>>>>>>>> JDK in order to have confidence that it will run on
>>> the
>>>>>>>>>>>>>>>>>>> older version
>>>>>>>>>>>>>>>>>>> of Java. This is because, as of Java 9, all JDK
>>>> releases
>>>>>>>>>>>>>>>>>>> have better
>>>>>>>>>>>>>>>>>>> support for cross-compilation to older Java versions.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> What this means is that developers can confidently
>>> make
>>>>>>>>>>>>>>>>>>> the build
>>>>>>>>>>>>>>>>>>> requirements for a project higher than the Java
>>> version
>>>>>>>>>>>>>>>>>>> that will
>>>>>>>>>>>>>>>>>>> actually be supported at runtime.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> In fact, ZooKeeper already supports the necessary
>>> flags
>>>>>>>>>>>>>>>>>>> in its Maven
>>>>>>>>>>>>>>>>>>> build configuration to ensure that it uses JDK 8
>>>>>>>>>>>>>>>>>>> compliance when
>>>>>>>>>>>>>>>>>>> building on a newer JDK (I added this way back in
>>>>>>>>>>>>>>>>>>> ZOOKEEPER-3739 /
>>>>>>>>>>>>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> So, I propose that we make JDK 11 the new minimum
>>>> version
>>>>>>>>>>>>>>>>>>> to *build*
>>>>>>>>>>>>>>>>>>> ZooKeeper with. This would not change the runtime
>>>>>>>>>>>>>>>>>>> requirement, which
>>>>>>>>>>>>>>>>>>> would remain at JDK 8.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> The only necessary change to make this happen would
>>> be
>>>> to
>>>>>>>>>>>>>>>>>>> add the
>>>>>>>>>>>>>>>>>>> minimum Java version to the maven-enforcer-plugin
>>> (like
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>> 
>>>> 
>>>
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
>>>>>>>>>>>>>>>>>>> )
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> This would allow ZooKeeper to to streamline its
>>>>>>>>>>>>>>>>>>> development process a
>>>>>>>>>>>>>>>>>>> little bit by reducing the amount of CI testing that
>>> is
>>>>>>>>>>>>>>>>>>> done as part
>>>>>>>>>>>>>>>>>>> of the build. In other words, we can drop the CI
>>> builds
>>>>>>>>>>>>>>>>>>> for JDK 8,
>>>>>>>>>>>>>>>>>>> which saves on build resources and time. The return
>>> on
>>>>>>>>>>>>>>>>>>> investment is
>>>>>>>>>>>>>>>>>>> so low for the JDK 8 builds anyway, because of the
>>>>>>>>>>>>>>>>>>> improved
>>>>>>>>>>>>>>>>>>> cross-compilation in newer JDKs. So, there's not much
>>>>>>>>>>>>>>>>>>> value in
>>>>>>>>>>>>>>>>>>> building on JDK 8 anyway.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Of course, I am only recommending this for *new*
>>>> release
>>>>>>>>>>>>>>>>>>> lines,
>>>>>>>>>>>>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because
>>> I
>>>>>>>>>>>>>>>>>>> would not want
>>>>>>>>>>>>>>>>>>> to change expectations for users who will build their
>>>> own
>>>>>>>>>>>>>>>>>>> 3.5 and 3.6
>>>>>>>>>>>>>>>>>>> versions as they continue to have patch versions
>>>>>>>>>>>>>>>>>>> released.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Kind Regards,
>>>>>>>>>>>>>>>>>>> Christopher
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>> 
>>> 

Re: 3.7.0: JDK 11 vs. JDK 8 (was: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7)

[Andor Molnar <an...@apache.org>]

Okay, let’s stay on JDK 8 with 3.7.0 release and do the transition in 4.0.
Not sure if we want 3.8 release or make the master 4.0 from now on.

Andor



> On 2021. Jan 6., at 22:35, Christopher <ct...@apache.org> wrote:
> 
> I agree with Enrico on this point. If the ZK PMC is considering a 3.7
> release, now would not be the right time to make this change, and it would
> be better to make a change at the beginning of the next iteration.
> 
> That said, I think switching to builds with JDK 11 and supporting JDK 8
> "passively" is the right thing to do for 3.8, and switching to JDK 11 fully
> would be the right thing to do if the PMC decides to do a major version
> bump to 4.0 instead of a 3.8.
> 
> On Wed, Jan 6, 2021 at 1:20 PM Enrico Olivelli <eo...@gmail.com> wrote:
> 
>> In my opinion we can stay on 8, we should change this kind of stuff at the
>> beginning of a new iteration and not before the release.
>> IIRC We are not in a hurry, having a stable build process is a value.
>> 
>> Enrico
>> 
>> Il Mer 6 Gen 2021, 17:51 Damien Diederen <dd...@apache.org> ha
>> scritto:
>> 
>>> 
>>> Dear ZooKeeper team,
>>> 
>>> Andor just reminded me of this JDK 11 vs. 8 discussion, for which we did
>>> not reach a resolution.  Do we want to make a move for the 3.7.0 release?
>>> 
>>> The original proposal, by Christopher, can be found here:
>>> 
>>> 
>>> 
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9Z2TNfAwjZ7etXPF91QmROvUWV+KHnxgPSD_MSeKfpyWA@mail.gmail.com%3e
>>> 
>>> I am explicitly replying to Christopher's nice and useful summary, which
>>> you will find directly below.  An archived copy can be found here:
>>> 
>>> 
>>> 
>> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9ao9-4S9bmzxaYJM8n8=AoLxiDZ-0ZohunVnAyQei0zAg@mail.gmail.com%3e
>>> 
>>> It contains three proposals, to which we can add the trivial one:
>>> 
>>> 1. Compile/test with JDK 11, support JDK 8 "passively";
>>> 
>>> 2. Compile/test with JDK 11, test client only with JDK 8 (requires test
>>>    suite adaptations);
>>> 
>>> 3. Move to JDK 11 fully;
>>> 
>>> 4. Postpone any change to 3.8 (or 4.0).
>>> 
>>> Flavio wrote:
>>> 
>>>> [Christopher] mentioned a PMC vote, and I don't think this should be a
>>>> closed vote, independent of how the conversation goes.
>>> 
>>> Do we have a better idea of what we want?  Or should we organize a vote?
>>> 
>>> Cheers, -D
>>> 
>>> 
>>> --8<--------------- original message ------------->8---
>>> 
>>> Christopher <ct...@apache.org> writes:
>>>> I'm happy that this discussion has been so lively! I just want to
>>>> emphasize a few things:
>>>> 
>>>> I really do understand the desire to continue to support Java 8... I
>>>> get it. But all the conversations around this seem based on what
>>>> people are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
>>>> *future* release... so it should be based more on reasonable
>>>> expectations for users in the future, and less based on what is
>>>> happening today. I suspect *most* people today are still using 3.4
>>>> anyway (it was just so stable for so long...), but that shouldn't mean
>>>> the developers should hold back development on 3.5 and 3.6, any more
>>>> than today's users of 3.5/3.6 should hold back 3.7.
>>>> 
>>>> Some of the opinions expressed in this discussion seem to propose a
>>>> scenario where users are going to be updating to "bleeding edge"
>>>> versions of ZooKeeper, but are going to insist on using Java 8.
>>>> Personally, I find this to be implausible. In my experience, people
>>>> either upgrade everything as soon as they are able to, or they upgrade
>>>> each thing individually, only when they are forced to. The first group
>>>> will be happy to move to Java 11 and ZK 3.7. The second group will
>>>> probably avoid 3.7 anyway, and are fine sticking with 3.6, but if they
>>>> had to update to 3.7, they'd also be fine updating to Java 11 if they
>>>> had to in order to use 3.7. I can't imagine the scenario where people
>>>> are eagerly choosing to upgrade to ZK 3.7, but miserly insisting on
>>>> using Java 8. Perhaps that scenario exists, but it's hard for me to
>>>> imagine. Even so, my proposal would still support even that group of
>>>> people.
>>>> 
>>>> I think there are now effectively three proposals being discussed in
>>>> this thread:
>>>> 
>>>> 1. (Christopher's original proposal) passively support Java 8 at
>>>> runtime by making JDK 11 the minimum requirement to build and test.
>>>> This scenario involves continuing to fix bugs, as they are discovered
>>>> and reported, that affect JDK 8, but passively, rather than
>>>> proactively. This proposal does *not* drop Java 8 support, but merely
>>>> de-emphasizes it in development of what will be 3.7 in the future, and
>>>> drops the requirement to do dedicated testing with Java 8. I think
>>>> this is low risk, because it is very unlikely that the ZK devs would
>>>> introduce a bug that would affect only Java 8 and the compiler
>>>> wouldn't catch it... because the cross-compilation features of newer
>>>> JDKs are really good.
>>>> 
>>>> 2. (Enrico's alternate proposal) this variation of my proposal would
>>>> involve continuing to proactively support Java 8 by creating a
>>>> dedicated testing suite to test client code on Java 8. I think this is
>>>> a good option, but since it involves a significantly higher amount of
>>>> work than option 1, I think the cost-benefit analysis would show this
>>>> to be not worth the effort. Also, if it were implemented, it would
>>>> need to be done carefully to avoid requiring developers to have
>>>> concurrently installed both Java 8 and Java 11 in order to perform a
>>>> build, because requiring Java 8 at build time while developing would
>>>> be worse than we have today.
>>>> 
>>>> 3. (Andor's preference) move to JDK 11 fully. This would provide no
>>>> support, passive or active, for Java 8 in ZK 3.7. To be honest, this
>>>> is my personal favorite, and is the simplest to implement and
>>>> communicate clearly to end users in release notes. The only reason I
>>>> proposed a passive support of Java 8 instead of this is because I was
>>>> trying to seek a compromise from the start. But, I think by far, this
>>>> is the best option for the next *future* release of ZK. If you wanted
>>>> to make the change even more visible to users, the version could even
>>>> be bumped to 4.0.
>>>> 
>>>> If this were to come to a VOTE by the PMC, in order to make a final
>>>> decision, I would recommend they vote on option 3, and then if that
>>>> fails, vote on option 1, and if that fails, keep things the way they
>>>> are (because option 2 is more work).
>>>> 
>>>> Christopher
>>>> 
>>>> P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK 11
>>>> and it works just fine there (as long as you add the missing runtime
>>>> jar for javax.activation:javax.activation-api:1.2.0 to its class path,
>>>> but that was fixed in Hadoop 3.3).
>>>> 
>>>> On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes
>>>> <ta...@cloudera.com.invalid> wrote:
>>>>> 
>>>>> Hi All,
>>>>> 
>>>>> I've just talked with a Hadoop/HDFS developer who told me what I
>>> guessed.
>>>>> With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would
>>> mean
>>>>> a release of Hadoop4.
>>>>> Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both. As
>>> of
>>>>> today most of the Hadoop users are still on Hadoop2, he doesn't expect
>>>>> Hadoop4 soon.
>>>>> As many Apache components depend on Hadoop and ZooKeeper they won't
>>> hurry
>>>>> to JDK11 until they have to (they will probably go one-by-one very
>>> slowly),
>>>>> which means if Hadoop stays on JDK8, they would use the last ZK
>> version
>>>>> which works on JDK8.
>>>>> Do we want a ZK 3.4 again?
>>>>> 
>>>>> Regards, Tamaas
>>>>> 
>>>>> 
>>>>> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <ta...@cloudera.com>
>>> wrote:
>>>>> 
>>>>>> Hi All,
>>>>>> 
>>>>>> Just to add my two cents.
>>>>>> 
>>>>>> Upgrading to JDK11 looks inevitable sooner or later and I would
>>> definitely
>>>>>> not wait until 2030 or 2026 when the extended support of JDK8 ends.
>>>>>> But on the other side I have to agree with Enrico and Patrick that
>>> far too
>>>>>> many users are tied to JDK8 yet (not because they want to use JDK8,
>>> but
>>>>>> because they have to), some of them are components of the Hadoop
>>> ecosystem,
>>>>>> which would be a loss to tie them to 3.6.x for years.
>>>>>> Do we know the state of Hadoop? It builds with JDK8 at the moment,
>>> but do
>>>>>> we know what are their plans to go to JDK11?
>>>>>> When they move, we should move too, but I don't think it would be
>>> wise to
>>>>>> do it earlier.
>>>>>> 
>>>>>> Christopher's option looks like the golden path, but it needs some
>>>>>> investment on the testing side as Enrico pointed it out.
>>>>>> Could we agree on it as a compromise?
>>>>>> 
>>>>>> Regards, Tamaas
>>>>>> 
>>>>>> On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com>
>>> wrote:
>>>>>> 
>>>>>>> I think I was reacting to Enrico's earlier comment of:
>>>>>>> 
>>>>>>> " ZooKeeper client is used by tons of users and unfortunately many
>>>>>>> projects
>>>>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
>> block
>>>>>>> users
>>>>>>> from the adoption, that is that we will see the world to stay on
>>> 3.6.x and
>>>>>>> we will have again a long lived release line, like 3.4."
>>>>>>> 
>>>>>>> It's a matter of whether or not a long-lived release line is
>>>>>>> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
>>>>>>> security/patch-wise (if not feature-wise) for the next N years,
>> then
>>>>>>> that's
>>>>>>> a potentially valid approach.  I interpreted that comment as "a
>>> long-lived
>>>>>>> release line is undesirable", but no one explicitly said that, I
>>> just read
>>>>>>> it that way.
>>>>>>> 
>>>>>>> ~Brent
>>>>>>> 
>>>>>>> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org>
>>> wrote:
>>>>>>> 
>>>>>>>> On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org>
>>> wrote:
>>>>>>>> 
>>>>>>>>> As far as I know Hbase, Solr and Kafka are already Java 11
>> ready.
>>>>>>>>> 
>>>>>>>>> IMHO contributors of those projects should also put efforts
>> into
>>>>>>> moving
>>>>>>>>> forward.
>>>>>>>>> 
>>>>>>>>> We’re not saying that you _have_ to move to Java 11.
>>>>>>>>> Staying on Java 8? No problem, 3.6 is for you.
>>>>>>>>> Want the fancy new features of 3.7? Work on it on your side
>> too.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> ppl want things like security fixes. I believe the highlighted
>>> downside
>>>>>>> is
>>>>>>>> that we would need to continue to maintain 3.6.x rather than
>>> allowing
>>>>>>>> users, and ourselves, to focus on trunk for production -"64
>>> percent"
>>>>>>> would
>>>>>>>> be blocked.
>>>>>>>> 
>>>>>>>> Patrick
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> Andor
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>>> On 2020. Oct 21., at 17:52, Enrico Olivelli <
>>> eolivelli@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
>>>>>>>> andor@apache.org>
>>>>>>>>> ha
>>>>>>>>>> scritto:
>>>>>>>>>> 
>>>>>>>>>>> Correct me if I'm wrong, but Oracle gets paid for extended
>>> support.
>>>>>>>>>>> Java 8 support until 2030 is not free of charge.
>>>>>>>>>>> 
>>>>>>>>>>> "ZK may end up with a lot of users potentially locking
>>> themselves
>>>>>>> to
>>>>>>>>>>> 3.6.x for a while as Enrico mentioned."
>>>>>>>>>>> 
>>>>>>>>>>> That's true. What's the downside of that which we should
>>> invest in
>>>>>>> to
>>>>>>>>>>> avoid?
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> I see ZooKeeper used in many many projects, all of the
>>>>>>>>>> HBase/Pulsar/Kafka/Solr ecosystem...
>>>>>>>>>> they will have to move to JDK11 in order to move to the new
>> ZK
>>>>>>> version
>>>>>>>>>> so probably they will stay on ZK 3.6
>>>>>>>>>> 
>>>>>>>>>> Probably with Java 17 LTS released the cards will change on
>> the
>>>>>>> table
>>>>>>>>>> 
>>>>>>>>>> Enrico
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> Andor
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
>>>>>>>>>>>> As a slightly different consideration, if you look at the
>>>>>>> Long-Term
>>>>>>>>>>>> Support
>>>>>>>>>>>> (LTS) roadmaps for Java, currently Java 8 is set to have
>> full
>>>>>>> support
>>>>>>>>>>>> until
>>>>>>>>>>>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>> 
>>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>>>>>>>>>>>> https://en.wikipedia.org/wiki/Java_version_history
>>>>>>>>>>>> 
>>>>>>>>>>>> My guess is that a number of companies are still heavily
>>> invested
>>>>>>> at
>>>>>>>>>>>> the
>>>>>>>>>>>> Java 8 level (I know a few) and with that kind of time
>>> horizon,
>>>>>>> they
>>>>>>>>>>>> have
>>>>>>>>>>>> no real motivation to upgrade for quite a while.  If the
>>> recent
>>>>>>>>>>>> Python 2
>>>>>>>>>>>> deprecation is anything to go by, they won't do it until
>>> they have
>>>>>>>>>>>> to.
>>>>>>>>>>>> 
>>>>>>>>>>>> Not saying Java 8 isn't *very* old (2014 release it seems
>>> like?)
>>>>>>> and
>>>>>>>>>>>> I'm
>>>>>>>>>>>> not invested heavily either way, but this might suggest
>> that
>>> ZK
>>>>>>> may
>>>>>>>>>>>> end up
>>>>>>>>>>>> with a lot of users potentially locking themselves to 3.6.x
>>> for a
>>>>>>>>>>>> while as
>>>>>>>>>>>> Enrico mentioned.
>>>>>>>>>>>> 
>>>>>>>>>>>> (Not a major contributor, but wanted to chime in since I
>>> just had
>>>>>>>>>>>> this
>>>>>>>>>>>> conversation with a bunch of people professionally
>> recently)
>>>>>>>>>>>> 
>>>>>>>>>>>> Brent
>>>>>>>>>>>> 
>>>>>>>>>>>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <
>>> andor@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks for the summary.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully.
>>> Other
>>>>>>>>>>>>> projects
>>>>>>>>>>>>> will upgrade once they’re JDK11 compliant, otherwise they
>>> will
>>>>>>> stay
>>>>>>>>>>>>> on 3.5
>>>>>>>>>>>>> or 3.6. Both version are quite recent in ZooKeeper-terms,
>> we
>>>>>>>>>>>>> already
>>>>>>>>>>>>> planned big changes for 3.7.0 and JDK 11 could be one of
>>> them.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Don’t put extra burden on the ZK community to help others
>>> staying
>>>>>>>>>>>>> on
>>>>>>>>>>>>> ancient Java versions.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Andor
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
>>>>>>> eolivelli@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Let me recap
>>>>>>>>>>>>>> - Christopher is proposing to move to JDK11
>>>>>>>>>>>>>> - ZooKeeper client and server are bundled and coded and
>>> tested
>>>>>>>>>>>>>> together
>>>>>>>>>>>>> in
>>>>>>>>>>>>>> zookeeper-server
>>>>>>>>>>>>>> - Enrico is concerned about the need of testing ZooKeeper
>>> client
>>>>>>>>>>>>>> on JDK8
>>>>>>>>>>>>>> (not a problem to move the server to JDK11)
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> ZooKeeper client is used by tons of users and
>>> unfortunately many
>>>>>>>>>>>>>> projects
>>>>>>>>>>>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk
>>> is to
>>>>>>>>>>>>>> block
>>>>>>>>>>>>> users
>>>>>>>>>>>>>> from the adoption,
>>>>>>>>>>>>>> that is that we will see the world to stay on 3.6.x and
>> we
>>> will
>>>>>>>>>>>>>> have
>>>>>>>>>>>>> again
>>>>>>>>>>>>>> a long lived release line, like 3.4.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Testing the client on JDK8 would be possible if we create
>>> some
>>>>>>>>>>>>>> kind of
>>>>>>>>>>>>>> additional module with system tests, then we can start
>> the
>>>>>>> server
>>>>>>>>>>>>>> on
>>>>>>>>>>>>> docker
>>>>>>>>>>>>>> on JDK11+ and start a client on JDK8
>>>>>>>>>>>>>> with Maven toolchain it should possible to run surefire
>>> tests
>>>>>>>>>>>>>> using a
>>>>>>>>>>>>>> separate JVM.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> So in my vision 2 options:
>>>>>>>>>>>>>> 1) fully JDK11 - drop JDK8 at all
>>>>>>>>>>>>>> 2) build with JDK11 - server only on JDK11 - add system
>>> tests
>>>>>>>>>>>>>> with docker
>>>>>>>>>>>>>> and toolchains that ensure the ZooKeeper client (and all
>>>>>>>>>>>>>> dependencies)
>>>>>>>>>>>>>> still work on JDK8
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> From my point of view about the ZooKeeper ecosystem
>> option
>>> 2)
>>>>>>>>>>>>>> will be far
>>>>>>>>>>>>>> better, but we need resources to work on a new test
>> suite.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Enrico
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
>>>>>>>>>>>>>> andor@apache.org>
>>>>>>>>>>>>> ha
>>>>>>>>>>>>>> scritto:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Tamas, Enrico,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Sorry I don’t follow. Why do we have to test the client
>>> with
>>>>>>>>>>>>>>> JDK 8 in
>>>>>>>>>>>>>>> version 3.7.0?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Andor
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
>>>>>>>>>>>>>>>> tamaas@cloudera.com.INVALID>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> Hi Enrico,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Separating ZooKeeper client and server is a huge work,
>>> but we
>>>>>>>>>>>>>>>> might not
>>>>>>>>>>>>>>>> need it.
>>>>>>>>>>>>>>>> As you mentioned we have to test ZK client with Java 8,
>>> what
>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>> separating only the test cases which we need to run
>> with
>>>>>>>>>>>>>>>> Java8 too?
>>>>>>>>>>>>>>>> In Curator we have the ZK compatibility tests where we
>>> run a
>>>>>>>>>>>>>>>> limited
>>>>>>>>>>>>>>> amount
>>>>>>>>>>>>>>>> of Curator's jUnit tests with a different ZK version.
>>>>>>>>>>>>>>>> We might be able to do the same here, tag tests which
>> are
>>>>>>>>>>>>>>>> testing ZK
>>>>>>>>>>>>>>> client
>>>>>>>>>>>>>>>> and run them separately with Java 8. The only
>> limitation
>>> is
>>>>>>>>>>>>>>>> that these
>>>>>>>>>>>>>>>> tests must stay JDK8 compatible.
>>>>>>>>>>>>>>>> But from the tags we will see which ones are those.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Regards, Tamaas
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
>>>>>>>>>>>>>>>> eolivelli@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> Christopher
>>>>>>>>>>>>>>>>> I appreciate your idea and I also moved lots of my
>>> projects
>>>>>>>>>>>>>>>>> to work
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> way
>>>>>>>>>>>>>>>>> you are suggesting.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> We must run tests using real jdk8 to test the
>> Zookeeper
>>>>>>>>>>>>>>>>> client. We
>>>>>>>>>>>>> must
>>>>>>>>>>>>>>>>> ensure that Zookeeper works well, especially while
>>> dealing
>>>>>>>>>>>>>>>>> with
>>>>>>>>>>>>> security
>>>>>>>>>>>>>>>>> stuff.
>>>>>>>>>>>>>>>>> Currently the client is in the same module of the
>> server
>>>>>>>>>>>>>>>>> and it will
>>>>>>>>>>>>>>> take a
>>>>>>>>>>>>>>>>> good (huge) amount of work to separate them
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Enrico
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher <
>>> ctubbsii@apache.org>
>>>>>>>>>>>>>>>>> ha
>>>>>>>>>>>>> scritto:
>>>>>>>>>>>>>>>>>> Hi ZK Devs,
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> With recent advancements in Java (since Java 9), it
>> is
>>>>>>>>>>>>>>>>>> now generally
>>>>>>>>>>>>>>>>>> no longer necessary to require that software be
>>> developed
>>>>>>>>>>>>>>>>>> on an older
>>>>>>>>>>>>>>>>>> JDK in order to have confidence that it will run on
>> the
>>>>>>>>>>>>>>>>>> older version
>>>>>>>>>>>>>>>>>> of Java. This is because, as of Java 9, all JDK
>>> releases
>>>>>>>>>>>>>>>>>> have better
>>>>>>>>>>>>>>>>>> support for cross-compilation to older Java versions.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> What this means is that developers can confidently
>> make
>>>>>>>>>>>>>>>>>> the build
>>>>>>>>>>>>>>>>>> requirements for a project higher than the Java
>> version
>>>>>>>>>>>>>>>>>> that will
>>>>>>>>>>>>>>>>>> actually be supported at runtime.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> In fact, ZooKeeper already supports the necessary
>> flags
>>>>>>>>>>>>>>>>>> in its Maven
>>>>>>>>>>>>>>>>>> build configuration to ensure that it uses JDK 8
>>>>>>>>>>>>>>>>>> compliance when
>>>>>>>>>>>>>>>>>> building on a newer JDK (I added this way back in
>>>>>>>>>>>>>>>>>> ZOOKEEPER-3739 /
>>>>>>>>>>>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> So, I propose that we make JDK 11 the new minimum
>>> version
>>>>>>>>>>>>>>>>>> to *build*
>>>>>>>>>>>>>>>>>> ZooKeeper with. This would not change the runtime
>>>>>>>>>>>>>>>>>> requirement, which
>>>>>>>>>>>>>>>>>> would remain at JDK 8.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> The only necessary change to make this happen would
>> be
>>> to
>>>>>>>>>>>>>>>>>> add the
>>>>>>>>>>>>>>>>>> minimum Java version to the maven-enforcer-plugin
>> (like
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>> 
>>> 
>> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
>>>>>>>>>>>>>>>>>> )
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> This would allow ZooKeeper to to streamline its
>>>>>>>>>>>>>>>>>> development process a
>>>>>>>>>>>>>>>>>> little bit by reducing the amount of CI testing that
>> is
>>>>>>>>>>>>>>>>>> done as part
>>>>>>>>>>>>>>>>>> of the build. In other words, we can drop the CI
>> builds
>>>>>>>>>>>>>>>>>> for JDK 8,
>>>>>>>>>>>>>>>>>> which saves on build resources and time. The return
>> on
>>>>>>>>>>>>>>>>>> investment is
>>>>>>>>>>>>>>>>>> so low for the JDK 8 builds anyway, because of the
>>>>>>>>>>>>>>>>>> improved
>>>>>>>>>>>>>>>>>> cross-compilation in newer JDKs. So, there's not much
>>>>>>>>>>>>>>>>>> value in
>>>>>>>>>>>>>>>>>> building on JDK 8 anyway.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Of course, I am only recommending this for *new*
>>> release
>>>>>>>>>>>>>>>>>> lines,
>>>>>>>>>>>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because
>> I
>>>>>>>>>>>>>>>>>> would not want
>>>>>>>>>>>>>>>>>> to change expectations for users who will build their
>>> own
>>>>>>>>>>>>>>>>>> 3.5 and 3.6
>>>>>>>>>>>>>>>>>> versions as they continue to have patch versions
>>>>>>>>>>>>>>>>>> released.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Kind Regards,
>>>>>>>>>>>>>>>>>> Christopher
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>> 
>> 

Re: 3.7.0: JDK 11 vs. JDK 8 (was: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7)

[Christopher <ct...@apache.org>]

I agree with Enrico on this point. If the ZK PMC is considering a 3.7
release, now would not be the right time to make this change, and it would
be better to make a change at the beginning of the next iteration.

That said, I think switching to builds with JDK 11 and supporting JDK 8
"passively" is the right thing to do for 3.8, and switching to JDK 11 fully
would be the right thing to do if the PMC decides to do a major version
bump to 4.0 instead of a 3.8.

On Wed, Jan 6, 2021 at 1:20 PM Enrico Olivelli <eo...@gmail.com> wrote:

> In my opinion we can stay on 8, we should change this kind of stuff at the
> beginning of a new iteration and not before the release.
> IIRC We are not in a hurry, having a stable build process is a value.
>
> Enrico
>
> Il Mer 6 Gen 2021, 17:51 Damien Diederen <dd...@apache.org> ha
> scritto:
>
> >
> > Dear ZooKeeper team,
> >
> > Andor just reminded me of this JDK 11 vs. 8 discussion, for which we did
> > not reach a resolution.  Do we want to make a move for the 3.7.0 release?
> >
> > The original proposal, by Christopher, can be found here:
> >
> >
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9Z2TNfAwjZ7etXPF91QmROvUWV+KHnxgPSD_MSeKfpyWA@mail.gmail.com%3e
> >
> > I am explicitly replying to Christopher's nice and useful summary, which
> > you will find directly below.  An archived copy can be found here:
> >
> >
> >
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9ao9-4S9bmzxaYJM8n8=AoLxiDZ-0ZohunVnAyQei0zAg@mail.gmail.com%3e
> >
> > It contains three proposals, to which we can add the trivial one:
> >
> >  1. Compile/test with JDK 11, support JDK 8 "passively";
> >
> >  2. Compile/test with JDK 11, test client only with JDK 8 (requires test
> >     suite adaptations);
> >
> >  3. Move to JDK 11 fully;
> >
> >  4. Postpone any change to 3.8 (or 4.0).
> >
> > Flavio wrote:
> >
> > > [Christopher] mentioned a PMC vote, and I don't think this should be a
> > > closed vote, independent of how the conversation goes.
> >
> > Do we have a better idea of what we want?  Or should we organize a vote?
> >
> > Cheers, -D
> >
> >
> > --8<--------------- original message ------------->8---
> >
> > Christopher <ct...@apache.org> writes:
> > > I'm happy that this discussion has been so lively! I just want to
> > > emphasize a few things:
> > >
> > > I really do understand the desire to continue to support Java 8... I
> > > get it. But all the conversations around this seem based on what
> > > people are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
> > > *future* release... so it should be based more on reasonable
> > > expectations for users in the future, and less based on what is
> > > happening today. I suspect *most* people today are still using 3.4
> > > anyway (it was just so stable for so long...), but that shouldn't mean
> > > the developers should hold back development on 3.5 and 3.6, any more
> > > than today's users of 3.5/3.6 should hold back 3.7.
> > >
> > > Some of the opinions expressed in this discussion seem to propose a
> > > scenario where users are going to be updating to "bleeding edge"
> > > versions of ZooKeeper, but are going to insist on using Java 8.
> > > Personally, I find this to be implausible. In my experience, people
> > > either upgrade everything as soon as they are able to, or they upgrade
> > > each thing individually, only when they are forced to. The first group
> > > will be happy to move to Java 11 and ZK 3.7. The second group will
> > > probably avoid 3.7 anyway, and are fine sticking with 3.6, but if they
> > > had to update to 3.7, they'd also be fine updating to Java 11 if they
> > > had to in order to use 3.7. I can't imagine the scenario where people
> > > are eagerly choosing to upgrade to ZK 3.7, but miserly insisting on
> > > using Java 8. Perhaps that scenario exists, but it's hard for me to
> > > imagine. Even so, my proposal would still support even that group of
> > > people.
> > >
> > > I think there are now effectively three proposals being discussed in
> > > this thread:
> > >
> > > 1. (Christopher's original proposal) passively support Java 8 at
> > > runtime by making JDK 11 the minimum requirement to build and test.
> > > This scenario involves continuing to fix bugs, as they are discovered
> > > and reported, that affect JDK 8, but passively, rather than
> > > proactively. This proposal does *not* drop Java 8 support, but merely
> > > de-emphasizes it in development of what will be 3.7 in the future, and
> > > drops the requirement to do dedicated testing with Java 8. I think
> > > this is low risk, because it is very unlikely that the ZK devs would
> > > introduce a bug that would affect only Java 8 and the compiler
> > > wouldn't catch it... because the cross-compilation features of newer
> > > JDKs are really good.
> > >
> > > 2. (Enrico's alternate proposal) this variation of my proposal would
> > > involve continuing to proactively support Java 8 by creating a
> > > dedicated testing suite to test client code on Java 8. I think this is
> > > a good option, but since it involves a significantly higher amount of
> > > work than option 1, I think the cost-benefit analysis would show this
> > > to be not worth the effort. Also, if it were implemented, it would
> > > need to be done carefully to avoid requiring developers to have
> > > concurrently installed both Java 8 and Java 11 in order to perform a
> > > build, because requiring Java 8 at build time while developing would
> > > be worse than we have today.
> > >
> > > 3. (Andor's preference) move to JDK 11 fully. This would provide no
> > > support, passive or active, for Java 8 in ZK 3.7. To be honest, this
> > > is my personal favorite, and is the simplest to implement and
> > > communicate clearly to end users in release notes. The only reason I
> > > proposed a passive support of Java 8 instead of this is because I was
> > > trying to seek a compromise from the start. But, I think by far, this
> > > is the best option for the next *future* release of ZK. If you wanted
> > > to make the change even more visible to users, the version could even
> > > be bumped to 4.0.
> > >
> > > If this were to come to a VOTE by the PMC, in order to make a final
> > > decision, I would recommend they vote on option 3, and then if that
> > > fails, vote on option 1, and if that fails, keep things the way they
> > > are (because option 2 is more work).
> > >
> > > Christopher
> > >
> > > P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK 11
> > > and it works just fine there (as long as you add the missing runtime
> > > jar for javax.activation:javax.activation-api:1.2.0 to its class path,
> > > but that was fixed in Hadoop 3.3).
> > >
> > > On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes
> > > <ta...@cloudera.com.invalid> wrote:
> > >>
> > >> Hi All,
> > >>
> > >> I've just talked with a Hadoop/HDFS developer who told me what I
> > guessed.
> > >> With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would
> > mean
> > >> a release of Hadoop4.
> > >> Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both. As
> > of
> > >> today most of the Hadoop users are still on Hadoop2, he doesn't expect
> > >> Hadoop4 soon.
> > >> As many Apache components depend on Hadoop and ZooKeeper they won't
> > hurry
> > >> to JDK11 until they have to (they will probably go one-by-one very
> > slowly),
> > >> which means if Hadoop stays on JDK8, they would use the last ZK
> version
> > >> which works on JDK8.
> > >> Do we want a ZK 3.4 again?
> > >>
> > >> Regards, Tamaas
> > >>
> > >>
> > >> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <ta...@cloudera.com>
> > wrote:
> > >>
> > >> > Hi All,
> > >> >
> > >> > Just to add my two cents.
> > >> >
> > >> > Upgrading to JDK11 looks inevitable sooner or later and I would
> > definitely
> > >> > not wait until 2030 or 2026 when the extended support of JDK8 ends.
> > >> > But on the other side I have to agree with Enrico and Patrick that
> > far too
> > >> > many users are tied to JDK8 yet (not because they want to use JDK8,
> > but
> > >> > because they have to), some of them are components of the Hadoop
> > ecosystem,
> > >> > which would be a loss to tie them to 3.6.x for years.
> > >> > Do we know the state of Hadoop? It builds with JDK8 at the moment,
> > but do
> > >> > we know what are their plans to go to JDK11?
> > >> > When they move, we should move too, but I don't think it would be
> > wise to
> > >> > do it earlier.
> > >> >
> > >> > Christopher's option looks like the golden path, but it needs some
> > >> > investment on the testing side as Enrico pointed it out.
> > >> > Could we agree on it as a compromise?
> > >> >
> > >> > Regards, Tamaas
> > >> >
> > >> > On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com>
> > wrote:
> > >> >
> > >> >> I think I was reacting to Enrico's earlier comment of:
> > >> >>
> > >> >> " ZooKeeper client is used by tons of users and unfortunately many
> > >> >> projects
> > >> >> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
> block
> > >> >> users
> > >> >> from the adoption, that is that we will see the world to stay on
> > 3.6.x and
> > >> >> we will have again a long lived release line, like 3.4."
> > >> >>
> > >> >> It's a matter of whether or not a long-lived release line is
> > >> >> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
> > >> >> security/patch-wise (if not feature-wise) for the next N years,
> then
> > >> >> that's
> > >> >> a potentially valid approach.  I interpreted that comment as "a
> > long-lived
> > >> >> release line is undesirable", but no one explicitly said that, I
> > just read
> > >> >> it that way.
> > >> >>
> > >> >> ~Brent
> > >> >>
> > >> >> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org>
> > wrote:
> > >> >>
> > >> >> > On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org>
> > wrote:
> > >> >> >
> > >> >> > > As far as I know Hbase, Solr and Kafka are already Java 11
> ready.
> > >> >> > >
> > >> >> > > IMHO contributors of those projects should also put efforts
> into
> > >> >> moving
> > >> >> > > forward.
> > >> >> > >
> > >> >> > > We’re not saying that you _have_ to move to Java 11.
> > >> >> > > Staying on Java 8? No problem, 3.6 is for you.
> > >> >> > > Want the fancy new features of 3.7? Work on it on your side
> too.
> > >> >> > >
> > >> >> > >
> > >> >> > ppl want things like security fixes. I believe the highlighted
> > downside
> > >> >> is
> > >> >> > that we would need to continue to maintain 3.6.x rather than
> > allowing
> > >> >> > users, and ourselves, to focus on trunk for production -"64
> > percent"
> > >> >> would
> > >> >> > be blocked.
> > >> >> >
> > >> >> > Patrick
> > >> >> >
> > >> >> >
> > >> >> > > Andor
> > >> >> > >
> > >> >> > >
> > >> >> > >
> > >> >> > > > On 2020. Oct 21., at 17:52, Enrico Olivelli <
> > eolivelli@gmail.com>
> > >> >> > wrote:
> > >> >> > > >
> > >> >> > > > Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
> > >> >> > andor@apache.org>
> > >> >> > > ha
> > >> >> > > > scritto:
> > >> >> > > >
> > >> >> > > >> Correct me if I'm wrong, but Oracle gets paid for extended
> > support.
> > >> >> > > >> Java 8 support until 2030 is not free of charge.
> > >> >> > > >>
> > >> >> > > >> "ZK may end up with a lot of users potentially locking
> > themselves
> > >> >> to
> > >> >> > > >> 3.6.x for a while as Enrico mentioned."
> > >> >> > > >>
> > >> >> > > >> That's true. What's the downside of that which we should
> > invest in
> > >> >> to
> > >> >> > > >> avoid?
> > >> >> > > >>
> > >> >> > > >
> > >> >> > > > I see ZooKeeper used in many many projects, all of the
> > >> >> > > > HBase/Pulsar/Kafka/Solr ecosystem...
> > >> >> > > > they will have to move to JDK11 in order to move to the new
> ZK
> > >> >> version
> > >> >> > > > so probably they will stay on ZK 3.6
> > >> >> > > >
> > >> >> > > > Probably with Java 17 LTS released the cards will change on
> the
> > >> >> table
> > >> >> > > >
> > >> >> > > > Enrico
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >>
> > >> >> > > >> Andor
> > >> >> > > >>
> > >> >> > > >>
> > >> >> > > >>
> > >> >> > > >> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> > >> >> > > >>> As a slightly different consideration, if you look at the
> > >> >> Long-Term
> > >> >> > > >>> Support
> > >> >> > > >>> (LTS) roadmaps for Java, currently Java 8 is set to have
> full
> > >> >> support
> > >> >> > > >>> until
> > >> >> > > >>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> > >> >> > > >>>
> > >> >> > > >>>
> > >> >> >
> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> > >> >> > > >>> https://en.wikipedia.org/wiki/Java_version_history
> > >> >> > > >>>
> > >> >> > > >>> My guess is that a number of companies are still heavily
> > invested
> > >> >> at
> > >> >> > > >>> the
> > >> >> > > >>> Java 8 level (I know a few) and with that kind of time
> > horizon,
> > >> >> they
> > >> >> > > >>> have
> > >> >> > > >>> no real motivation to upgrade for quite a while.  If the
> > recent
> > >> >> > > >>> Python 2
> > >> >> > > >>> deprecation is anything to go by, they won't do it until
> > they have
> > >> >> > > >>> to.
> > >> >> > > >>>
> > >> >> > > >>> Not saying Java 8 isn't *very* old (2014 release it seems
> > like?)
> > >> >> and
> > >> >> > > >>> I'm
> > >> >> > > >>> not invested heavily either way, but this might suggest
> that
> > ZK
> > >> >> may
> > >> >> > > >>> end up
> > >> >> > > >>> with a lot of users potentially locking themselves to 3.6.x
> > for a
> > >> >> > > >>> while as
> > >> >> > > >>> Enrico mentioned.
> > >> >> > > >>>
> > >> >> > > >>> (Not a major contributor, but wanted to chime in since I
> > just had
> > >> >> > > >>> this
> > >> >> > > >>> conversation with a bunch of people professionally
> recently)
> > >> >> > > >>>
> > >> >> > > >>> Brent
> > >> >> > > >>>
> > >> >> > > >>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <
> > andor@apache.org>
> > >> >> > > >>> wrote:
> > >> >> > > >>>
> > >> >> > > >>>> Thanks for the summary.
> > >> >> > > >>>>
> > >> >> > > >>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully.
> > Other
> > >> >> > > >>>> projects
> > >> >> > > >>>> will upgrade once they’re JDK11 compliant, otherwise they
> > will
> > >> >> stay
> > >> >> > > >>>> on 3.5
> > >> >> > > >>>> or 3.6. Both version are quite recent in ZooKeeper-terms,
> we
> > >> >> > > >>>> already
> > >> >> > > >>>> planned big changes for 3.7.0 and JDK 11 could be one of
> > them.
> > >> >> > > >>>>
> > >> >> > > >>>> Don’t put extra burden on the ZK community to help others
> > staying
> > >> >> > > >>>> on
> > >> >> > > >>>> ancient Java versions.
> > >> >> > > >>>>
> > >> >> > > >>>> Andor
> > >> >> > > >>>>
> > >> >> > > >>>>
> > >> >> > > >>>>
> > >> >> > > >>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
> > >> >> eolivelli@gmail.com>
> > >> >> > > >>>>> wrote:
> > >> >> > > >>>>>
> > >> >> > > >>>>> Let me recap
> > >> >> > > >>>>> - Christopher is proposing to move to JDK11
> > >> >> > > >>>>> - ZooKeeper client and server are bundled and coded and
> > tested
> > >> >> > > >>>>> together
> > >> >> > > >>>> in
> > >> >> > > >>>>> zookeeper-server
> > >> >> > > >>>>> - Enrico is concerned about the need of testing ZooKeeper
> > client
> > >> >> > > >>>>> on JDK8
> > >> >> > > >>>>> (not a problem to move the server to JDK11)
> > >> >> > > >>>>>
> > >> >> > > >>>>> ZooKeeper client is used by tons of users and
> > unfortunately many
> > >> >> > > >>>>> projects
> > >> >> > > >>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk
> > is to
> > >> >> > > >>>>> block
> > >> >> > > >>>> users
> > >> >> > > >>>>> from the adoption,
> > >> >> > > >>>>> that is that we will see the world to stay on 3.6.x and
> we
> > will
> > >> >> > > >>>>> have
> > >> >> > > >>>> again
> > >> >> > > >>>>> a long lived release line, like 3.4.
> > >> >> > > >>>>>
> > >> >> > > >>>>> Testing the client on JDK8 would be possible if we create
> > some
> > >> >> > > >>>>> kind of
> > >> >> > > >>>>> additional module with system tests, then we can start
> the
> > >> >> server
> > >> >> > > >>>>> on
> > >> >> > > >>>> docker
> > >> >> > > >>>>> on JDK11+ and start a client on JDK8
> > >> >> > > >>>>> with Maven toolchain it should possible to run surefire
> > tests
> > >> >> > > >>>>> using a
> > >> >> > > >>>>> separate JVM.
> > >> >> > > >>>>>
> > >> >> > > >>>>> So in my vision 2 options:
> > >> >> > > >>>>> 1) fully JDK11 - drop JDK8 at all
> > >> >> > > >>>>> 2) build with JDK11 - server only on JDK11 - add system
> > tests
> > >> >> > > >>>>> with docker
> > >> >> > > >>>>> and toolchains that ensure the ZooKeeper client (and all
> > >> >> > > >>>>> dependencies)
> > >> >> > > >>>>> still work on JDK8
> > >> >> > > >>>>>
> > >> >> > > >>>>> From my point of view about the ZooKeeper ecosystem
> option
> > 2)
> > >> >> > > >>>>> will be far
> > >> >> > > >>>>> better, but we need resources to work on a new test
> suite.
> > >> >> > > >>>>>
> > >> >> > > >>>>> Enrico
> > >> >> > > >>>>>
> > >> >> > > >>>>>
> > >> >> > > >>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> > >> >> > > >>>>> andor@apache.org>
> > >> >> > > >>>> ha
> > >> >> > > >>>>> scritto:
> > >> >> > > >>>>>
> > >> >> > > >>>>>> Tamas, Enrico,
> > >> >> > > >>>>>>
> > >> >> > > >>>>>> Sorry I don’t follow. Why do we have to test the client
> > with
> > >> >> > > >>>>>> JDK 8 in
> > >> >> > > >>>>>> version 3.7.0?
> > >> >> > > >>>>>>
> > >> >> > > >>>>>> Andor
> > >> >> > > >>>>>>
> > >> >> > > >>>>>>
> > >> >> > > >>>>>>
> > >> >> > > >>>>>>
> > >> >> > > >>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
> > >> >> > > >>>>>>> tamaas@cloudera.com.INVALID>
> > >> >> > > >>>>>> wrote:
> > >> >> > > >>>>>>> Hi Enrico,
> > >> >> > > >>>>>>>
> > >> >> > > >>>>>>> Separating ZooKeeper client and server is a huge work,
> > but we
> > >> >> > > >>>>>>> might not
> > >> >> > > >>>>>>> need it.
> > >> >> > > >>>>>>> As you mentioned we have to test ZK client with Java 8,
> > what
> > >> >> > > >>>>>>> about
> > >> >> > > >>>>>>> separating only the test cases which we need to run
> with
> > >> >> > > >>>>>>> Java8 too?
> > >> >> > > >>>>>>> In Curator we have the ZK compatibility tests where we
> > run a
> > >> >> > > >>>>>>> limited
> > >> >> > > >>>>>> amount
> > >> >> > > >>>>>>> of Curator's jUnit tests with a different ZK version.
> > >> >> > > >>>>>>> We might be able to do the same here, tag tests which
> are
> > >> >> > > >>>>>>> testing ZK
> > >> >> > > >>>>>> client
> > >> >> > > >>>>>>> and run them separately with Java 8. The only
> limitation
> > is
> > >> >> > > >>>>>>> that these
> > >> >> > > >>>>>>> tests must stay JDK8 compatible.
> > >> >> > > >>>>>>> But from the tags we will see which ones are those.
> > >> >> > > >>>>>>>
> > >> >> > > >>>>>>> What do you think?
> > >> >> > > >>>>>>>
> > >> >> > > >>>>>>> Regards, Tamaas
> > >> >> > > >>>>>>>
> > >> >> > > >>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> > >> >> > > >>>>>>> eolivelli@gmail.com>
> > >> >> > > >>>>>> wrote:
> > >> >> > > >>>>>>>> Christopher
> > >> >> > > >>>>>>>> I appreciate your idea and I also moved lots of my
> > projects
> > >> >> > > >>>>>>>> to work
> > >> >> > > >>>> the
> > >> >> > > >>>>>> way
> > >> >> > > >>>>>>>> you are suggesting.
> > >> >> > > >>>>>>>>
> > >> >> > > >>>>>>>>
> > >> >> > > >>>>>>>> We must run tests using real jdk8 to test the
> Zookeeper
> > >> >> > > >>>>>>>> client. We
> > >> >> > > >>>> must
> > >> >> > > >>>>>>>> ensure that Zookeeper works well, especially while
> > dealing
> > >> >> > > >>>>>>>> with
> > >> >> > > >>>> security
> > >> >> > > >>>>>>>> stuff.
> > >> >> > > >>>>>>>> Currently the client is in the same module of the
> server
> > >> >> > > >>>>>>>> and it will
> > >> >> > > >>>>>> take a
> > >> >> > > >>>>>>>> good (huge) amount of work to separate them
> > >> >> > > >>>>>>>>
> > >> >> > > >>>>>>>>
> > >> >> > > >>>>>>>> Enrico
> > >> >> > > >>>>>>>>
> > >> >> > > >>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher <
> > ctubbsii@apache.org>
> > >> >> > > >>>>>>>> ha
> > >> >> > > >>>> scritto:
> > >> >> > > >>>>>>>>> Hi ZK Devs,
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> With recent advancements in Java (since Java 9), it
> is
> > >> >> > > >>>>>>>>> now generally
> > >> >> > > >>>>>>>>> no longer necessary to require that software be
> > developed
> > >> >> > > >>>>>>>>> on an older
> > >> >> > > >>>>>>>>> JDK in order to have confidence that it will run on
> the
> > >> >> > > >>>>>>>>> older version
> > >> >> > > >>>>>>>>> of Java. This is because, as of Java 9, all JDK
> > releases
> > >> >> > > >>>>>>>>> have better
> > >> >> > > >>>>>>>>> support for cross-compilation to older Java versions.
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> What this means is that developers can confidently
> make
> > >> >> > > >>>>>>>>> the build
> > >> >> > > >>>>>>>>> requirements for a project higher than the Java
> version
> > >> >> > > >>>>>>>>> that will
> > >> >> > > >>>>>>>>> actually be supported at runtime.
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> In fact, ZooKeeper already supports the necessary
> flags
> > >> >> > > >>>>>>>>> in its Maven
> > >> >> > > >>>>>>>>> build configuration to ensure that it uses JDK 8
> > >> >> > > >>>>>>>>> compliance when
> > >> >> > > >>>>>>>>> building on a newer JDK (I added this way back in
> > >> >> > > >>>>>>>>> ZOOKEEPER-3739 /
> > >> >> > > >>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> So, I propose that we make JDK 11 the new minimum
> > version
> > >> >> > > >>>>>>>>> to *build*
> > >> >> > > >>>>>>>>> ZooKeeper with. This would not change the runtime
> > >> >> > > >>>>>>>>> requirement, which
> > >> >> > > >>>>>>>>> would remain at JDK 8.
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> The only necessary change to make this happen would
> be
> > to
> > >> >> > > >>>>>>>>> add the
> > >> >> > > >>>>>>>>> minimum Java version to the maven-enforcer-plugin
> (like
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>
> > >> >> > > >>
> > >> >> > >
> > >> >> >
> > >> >>
> > >
> >
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> > >> >> > > >>>>>>>>> )
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> This would allow ZooKeeper to to streamline its
> > >> >> > > >>>>>>>>> development process a
> > >> >> > > >>>>>>>>> little bit by reducing the amount of CI testing that
> is
> > >> >> > > >>>>>>>>> done as part
> > >> >> > > >>>>>>>>> of the build. In other words, we can drop the CI
> builds
> > >> >> > > >>>>>>>>> for JDK 8,
> > >> >> > > >>>>>>>>> which saves on build resources and time. The return
> on
> > >> >> > > >>>>>>>>> investment is
> > >> >> > > >>>>>>>>> so low for the JDK 8 builds anyway, because of the
> > >> >> > > >>>>>>>>> improved
> > >> >> > > >>>>>>>>> cross-compilation in newer JDKs. So, there's not much
> > >> >> > > >>>>>>>>> value in
> > >> >> > > >>>>>>>>> building on JDK 8 anyway.
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> Of course, I am only recommending this for *new*
> > release
> > >> >> > > >>>>>>>>> lines,
> > >> >> > > >>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because
> I
> > >> >> > > >>>>>>>>> would not want
> > >> >> > > >>>>>>>>> to change expectations for users who will build their
> > own
> > >> >> > > >>>>>>>>> 3.5 and 3.6
> > >> >> > > >>>>>>>>> versions as they continue to have patch versions
> > >> >> > > >>>>>>>>> released.
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> What do you think?
> > >> >> > > >>>>>>>>>
> > >> >> > > >>>>>>>>> Kind Regards,
> > >> >> > > >>>>>>>>> Christopher
> > >> >> > > >>>>>>>>>
> > >> >> > > >>
> > >> >> > > >>
> > >> >> > >
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >
> >
>

Re: 3.7.0: JDK 11 vs. JDK 8 (was: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7)

[Enrico Olivelli <eo...@gmail.com>]

In my opinion we can stay on 8, we should change this kind of stuff at the
beginning of a new iteration and not before the release.
IIRC We are not in a hurry, having a stable build process is a value.

Enrico

Il Mer 6 Gen 2021, 17:51 Damien Diederen <dd...@apache.org> ha scritto:

>
> Dear ZooKeeper team,
>
> Andor just reminded me of this JDK 11 vs. 8 discussion, for which we did
> not reach a resolution.  Do we want to make a move for the 3.7.0 release?
>
> The original proposal, by Christopher, can be found here:
>
>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9Z2TNfAwjZ7etXPF91QmROvUWV+KHnxgPSD_MSeKfpyWA@mail.gmail.com%3e
>
> I am explicitly replying to Christopher's nice and useful summary, which
> you will find directly below.  An archived copy can be found here:
>
>
> https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9ao9-4S9bmzxaYJM8n8=AoLxiDZ-0ZohunVnAyQei0zAg@mail.gmail.com%3e
>
> It contains three proposals, to which we can add the trivial one:
>
>  1. Compile/test with JDK 11, support JDK 8 "passively";
>
>  2. Compile/test with JDK 11, test client only with JDK 8 (requires test
>     suite adaptations);
>
>  3. Move to JDK 11 fully;
>
>  4. Postpone any change to 3.8 (or 4.0).
>
> Flavio wrote:
>
> > [Christopher] mentioned a PMC vote, and I don't think this should be a
> > closed vote, independent of how the conversation goes.
>
> Do we have a better idea of what we want?  Or should we organize a vote?
>
> Cheers, -D
>
>
> --8<--------------- original message ------------->8---
>
> Christopher <ct...@apache.org> writes:
> > I'm happy that this discussion has been so lively! I just want to
> > emphasize a few things:
> >
> > I really do understand the desire to continue to support Java 8... I
> > get it. But all the conversations around this seem based on what
> > people are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
> > *future* release... so it should be based more on reasonable
> > expectations for users in the future, and less based on what is
> > happening today. I suspect *most* people today are still using 3.4
> > anyway (it was just so stable for so long...), but that shouldn't mean
> > the developers should hold back development on 3.5 and 3.6, any more
> > than today's users of 3.5/3.6 should hold back 3.7.
> >
> > Some of the opinions expressed in this discussion seem to propose a
> > scenario where users are going to be updating to "bleeding edge"
> > versions of ZooKeeper, but are going to insist on using Java 8.
> > Personally, I find this to be implausible. In my experience, people
> > either upgrade everything as soon as they are able to, or they upgrade
> > each thing individually, only when they are forced to. The first group
> > will be happy to move to Java 11 and ZK 3.7. The second group will
> > probably avoid 3.7 anyway, and are fine sticking with 3.6, but if they
> > had to update to 3.7, they'd also be fine updating to Java 11 if they
> > had to in order to use 3.7. I can't imagine the scenario where people
> > are eagerly choosing to upgrade to ZK 3.7, but miserly insisting on
> > using Java 8. Perhaps that scenario exists, but it's hard for me to
> > imagine. Even so, my proposal would still support even that group of
> > people.
> >
> > I think there are now effectively three proposals being discussed in
> > this thread:
> >
> > 1. (Christopher's original proposal) passively support Java 8 at
> > runtime by making JDK 11 the minimum requirement to build and test.
> > This scenario involves continuing to fix bugs, as they are discovered
> > and reported, that affect JDK 8, but passively, rather than
> > proactively. This proposal does *not* drop Java 8 support, but merely
> > de-emphasizes it in development of what will be 3.7 in the future, and
> > drops the requirement to do dedicated testing with Java 8. I think
> > this is low risk, because it is very unlikely that the ZK devs would
> > introduce a bug that would affect only Java 8 and the compiler
> > wouldn't catch it... because the cross-compilation features of newer
> > JDKs are really good.
> >
> > 2. (Enrico's alternate proposal) this variation of my proposal would
> > involve continuing to proactively support Java 8 by creating a
> > dedicated testing suite to test client code on Java 8. I think this is
> > a good option, but since it involves a significantly higher amount of
> > work than option 1, I think the cost-benefit analysis would show this
> > to be not worth the effort. Also, if it were implemented, it would
> > need to be done carefully to avoid requiring developers to have
> > concurrently installed both Java 8 and Java 11 in order to perform a
> > build, because requiring Java 8 at build time while developing would
> > be worse than we have today.
> >
> > 3. (Andor's preference) move to JDK 11 fully. This would provide no
> > support, passive or active, for Java 8 in ZK 3.7. To be honest, this
> > is my personal favorite, and is the simplest to implement and
> > communicate clearly to end users in release notes. The only reason I
> > proposed a passive support of Java 8 instead of this is because I was
> > trying to seek a compromise from the start. But, I think by far, this
> > is the best option for the next *future* release of ZK. If you wanted
> > to make the change even more visible to users, the version could even
> > be bumped to 4.0.
> >
> > If this were to come to a VOTE by the PMC, in order to make a final
> > decision, I would recommend they vote on option 3, and then if that
> > fails, vote on option 1, and if that fails, keep things the way they
> > are (because option 2 is more work).
> >
> > Christopher
> >
> > P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK 11
> > and it works just fine there (as long as you add the missing runtime
> > jar for javax.activation:javax.activation-api:1.2.0 to its class path,
> > but that was fixed in Hadoop 3.3).
> >
> > On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes
> > <ta...@cloudera.com.invalid> wrote:
> >>
> >> Hi All,
> >>
> >> I've just talked with a Hadoop/HDFS developer who told me what I
> guessed.
> >> With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would
> mean
> >> a release of Hadoop4.
> >> Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both. As
> of
> >> today most of the Hadoop users are still on Hadoop2, he doesn't expect
> >> Hadoop4 soon.
> >> As many Apache components depend on Hadoop and ZooKeeper they won't
> hurry
> >> to JDK11 until they have to (they will probably go one-by-one very
> slowly),
> >> which means if Hadoop stays on JDK8, they would use the last ZK version
> >> which works on JDK8.
> >> Do we want a ZK 3.4 again?
> >>
> >> Regards, Tamaas
> >>
> >>
> >> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <ta...@cloudera.com>
> wrote:
> >>
> >> > Hi All,
> >> >
> >> > Just to add my two cents.
> >> >
> >> > Upgrading to JDK11 looks inevitable sooner or later and I would
> definitely
> >> > not wait until 2030 or 2026 when the extended support of JDK8 ends.
> >> > But on the other side I have to agree with Enrico and Patrick that
> far too
> >> > many users are tied to JDK8 yet (not because they want to use JDK8,
> but
> >> > because they have to), some of them are components of the Hadoop
> ecosystem,
> >> > which would be a loss to tie them to 3.6.x for years.
> >> > Do we know the state of Hadoop? It builds with JDK8 at the moment,
> but do
> >> > we know what are their plans to go to JDK11?
> >> > When they move, we should move too, but I don't think it would be
> wise to
> >> > do it earlier.
> >> >
> >> > Christopher's option looks like the golden path, but it needs some
> >> > investment on the testing side as Enrico pointed it out.
> >> > Could we agree on it as a compromise?
> >> >
> >> > Regards, Tamaas
> >> >
> >> > On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com>
> wrote:
> >> >
> >> >> I think I was reacting to Enrico's earlier comment of:
> >> >>
> >> >> " ZooKeeper client is used by tons of users and unfortunately many
> >> >> projects
> >> >> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to block
> >> >> users
> >> >> from the adoption, that is that we will see the world to stay on
> 3.6.x and
> >> >> we will have again a long lived release line, like 3.4."
> >> >>
> >> >> It's a matter of whether or not a long-lived release line is
> >> >> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
> >> >> security/patch-wise (if not feature-wise) for the next N years, then
> >> >> that's
> >> >> a potentially valid approach.  I interpreted that comment as "a
> long-lived
> >> >> release line is undesirable", but no one explicitly said that, I
> just read
> >> >> it that way.
> >> >>
> >> >> ~Brent
> >> >>
> >> >> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org>
> wrote:
> >> >>
> >> >> > On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org>
> wrote:
> >> >> >
> >> >> > > As far as I know Hbase, Solr and Kafka are already Java 11 ready.
> >> >> > >
> >> >> > > IMHO contributors of those projects should also put efforts into
> >> >> moving
> >> >> > > forward.
> >> >> > >
> >> >> > > We’re not saying that you _have_ to move to Java 11.
> >> >> > > Staying on Java 8? No problem, 3.6 is for you.
> >> >> > > Want the fancy new features of 3.7? Work on it on your side too.
> >> >> > >
> >> >> > >
> >> >> > ppl want things like security fixes. I believe the highlighted
> downside
> >> >> is
> >> >> > that we would need to continue to maintain 3.6.x rather than
> allowing
> >> >> > users, and ourselves, to focus on trunk for production -"64
> percent"
> >> >> would
> >> >> > be blocked.
> >> >> >
> >> >> > Patrick
> >> >> >
> >> >> >
> >> >> > > Andor
> >> >> > >
> >> >> > >
> >> >> > >
> >> >> > > > On 2020. Oct 21., at 17:52, Enrico Olivelli <
> eolivelli@gmail.com>
> >> >> > wrote:
> >> >> > > >
> >> >> > > > Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
> >> >> > andor@apache.org>
> >> >> > > ha
> >> >> > > > scritto:
> >> >> > > >
> >> >> > > >> Correct me if I'm wrong, but Oracle gets paid for extended
> support.
> >> >> > > >> Java 8 support until 2030 is not free of charge.
> >> >> > > >>
> >> >> > > >> "ZK may end up with a lot of users potentially locking
> themselves
> >> >> to
> >> >> > > >> 3.6.x for a while as Enrico mentioned."
> >> >> > > >>
> >> >> > > >> That's true. What's the downside of that which we should
> invest in
> >> >> to
> >> >> > > >> avoid?
> >> >> > > >>
> >> >> > > >
> >> >> > > > I see ZooKeeper used in many many projects, all of the
> >> >> > > > HBase/Pulsar/Kafka/Solr ecosystem...
> >> >> > > > they will have to move to JDK11 in order to move to the new ZK
> >> >> version
> >> >> > > > so probably they will stay on ZK 3.6
> >> >> > > >
> >> >> > > > Probably with Java 17 LTS released the cards will change on the
> >> >> table
> >> >> > > >
> >> >> > > > Enrico
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > >>
> >> >> > > >> Andor
> >> >> > > >>
> >> >> > > >>
> >> >> > > >>
> >> >> > > >> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> >> >> > > >>> As a slightly different consideration, if you look at the
> >> >> Long-Term
> >> >> > > >>> Support
> >> >> > > >>> (LTS) roadmaps for Java, currently Java 8 is set to have full
> >> >> support
> >> >> > > >>> until
> >> >> > > >>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> >> >> > > >>>
> >> >> > > >>>
> >> >> >
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> >> >> > > >>> https://en.wikipedia.org/wiki/Java_version_history
> >> >> > > >>>
> >> >> > > >>> My guess is that a number of companies are still heavily
> invested
> >> >> at
> >> >> > > >>> the
> >> >> > > >>> Java 8 level (I know a few) and with that kind of time
> horizon,
> >> >> they
> >> >> > > >>> have
> >> >> > > >>> no real motivation to upgrade for quite a while.  If the
> recent
> >> >> > > >>> Python 2
> >> >> > > >>> deprecation is anything to go by, they won't do it until
> they have
> >> >> > > >>> to.
> >> >> > > >>>
> >> >> > > >>> Not saying Java 8 isn't *very* old (2014 release it seems
> like?)
> >> >> and
> >> >> > > >>> I'm
> >> >> > > >>> not invested heavily either way, but this might suggest that
> ZK
> >> >> may
> >> >> > > >>> end up
> >> >> > > >>> with a lot of users potentially locking themselves to 3.6.x
> for a
> >> >> > > >>> while as
> >> >> > > >>> Enrico mentioned.
> >> >> > > >>>
> >> >> > > >>> (Not a major contributor, but wanted to chime in since I
> just had
> >> >> > > >>> this
> >> >> > > >>> conversation with a bunch of people professionally recently)
> >> >> > > >>>
> >> >> > > >>> Brent
> >> >> > > >>>
> >> >> > > >>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <
> andor@apache.org>
> >> >> > > >>> wrote:
> >> >> > > >>>
> >> >> > > >>>> Thanks for the summary.
> >> >> > > >>>>
> >> >> > > >>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully.
> Other
> >> >> > > >>>> projects
> >> >> > > >>>> will upgrade once they’re JDK11 compliant, otherwise they
> will
> >> >> stay
> >> >> > > >>>> on 3.5
> >> >> > > >>>> or 3.6. Both version are quite recent in ZooKeeper-terms, we
> >> >> > > >>>> already
> >> >> > > >>>> planned big changes for 3.7.0 and JDK 11 could be one of
> them.
> >> >> > > >>>>
> >> >> > > >>>> Don’t put extra burden on the ZK community to help others
> staying
> >> >> > > >>>> on
> >> >> > > >>>> ancient Java versions.
> >> >> > > >>>>
> >> >> > > >>>> Andor
> >> >> > > >>>>
> >> >> > > >>>>
> >> >> > > >>>>
> >> >> > > >>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
> >> >> eolivelli@gmail.com>
> >> >> > > >>>>> wrote:
> >> >> > > >>>>>
> >> >> > > >>>>> Let me recap
> >> >> > > >>>>> - Christopher is proposing to move to JDK11
> >> >> > > >>>>> - ZooKeeper client and server are bundled and coded and
> tested
> >> >> > > >>>>> together
> >> >> > > >>>> in
> >> >> > > >>>>> zookeeper-server
> >> >> > > >>>>> - Enrico is concerned about the need of testing ZooKeeper
> client
> >> >> > > >>>>> on JDK8
> >> >> > > >>>>> (not a problem to move the server to JDK11)
> >> >> > > >>>>>
> >> >> > > >>>>> ZooKeeper client is used by tons of users and
> unfortunately many
> >> >> > > >>>>> projects
> >> >> > > >>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk
> is to
> >> >> > > >>>>> block
> >> >> > > >>>> users
> >> >> > > >>>>> from the adoption,
> >> >> > > >>>>> that is that we will see the world to stay on 3.6.x and we
> will
> >> >> > > >>>>> have
> >> >> > > >>>> again
> >> >> > > >>>>> a long lived release line, like 3.4.
> >> >> > > >>>>>
> >> >> > > >>>>> Testing the client on JDK8 would be possible if we create
> some
> >> >> > > >>>>> kind of
> >> >> > > >>>>> additional module with system tests, then we can start the
> >> >> server
> >> >> > > >>>>> on
> >> >> > > >>>> docker
> >> >> > > >>>>> on JDK11+ and start a client on JDK8
> >> >> > > >>>>> with Maven toolchain it should possible to run surefire
> tests
> >> >> > > >>>>> using a
> >> >> > > >>>>> separate JVM.
> >> >> > > >>>>>
> >> >> > > >>>>> So in my vision 2 options:
> >> >> > > >>>>> 1) fully JDK11 - drop JDK8 at all
> >> >> > > >>>>> 2) build with JDK11 - server only on JDK11 - add system
> tests
> >> >> > > >>>>> with docker
> >> >> > > >>>>> and toolchains that ensure the ZooKeeper client (and all
> >> >> > > >>>>> dependencies)
> >> >> > > >>>>> still work on JDK8
> >> >> > > >>>>>
> >> >> > > >>>>> From my point of view about the ZooKeeper ecosystem option
> 2)
> >> >> > > >>>>> will be far
> >> >> > > >>>>> better, but we need resources to work on a new test suite.
> >> >> > > >>>>>
> >> >> > > >>>>> Enrico
> >> >> > > >>>>>
> >> >> > > >>>>>
> >> >> > > >>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> >> >> > > >>>>> andor@apache.org>
> >> >> > > >>>> ha
> >> >> > > >>>>> scritto:
> >> >> > > >>>>>
> >> >> > > >>>>>> Tamas, Enrico,
> >> >> > > >>>>>>
> >> >> > > >>>>>> Sorry I don’t follow. Why do we have to test the client
> with
> >> >> > > >>>>>> JDK 8 in
> >> >> > > >>>>>> version 3.7.0?
> >> >> > > >>>>>>
> >> >> > > >>>>>> Andor
> >> >> > > >>>>>>
> >> >> > > >>>>>>
> >> >> > > >>>>>>
> >> >> > > >>>>>>
> >> >> > > >>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
> >> >> > > >>>>>>> tamaas@cloudera.com.INVALID>
> >> >> > > >>>>>> wrote:
> >> >> > > >>>>>>> Hi Enrico,
> >> >> > > >>>>>>>
> >> >> > > >>>>>>> Separating ZooKeeper client and server is a huge work,
> but we
> >> >> > > >>>>>>> might not
> >> >> > > >>>>>>> need it.
> >> >> > > >>>>>>> As you mentioned we have to test ZK client with Java 8,
> what
> >> >> > > >>>>>>> about
> >> >> > > >>>>>>> separating only the test cases which we need to run with
> >> >> > > >>>>>>> Java8 too?
> >> >> > > >>>>>>> In Curator we have the ZK compatibility tests where we
> run a
> >> >> > > >>>>>>> limited
> >> >> > > >>>>>> amount
> >> >> > > >>>>>>> of Curator's jUnit tests with a different ZK version.
> >> >> > > >>>>>>> We might be able to do the same here, tag tests which are
> >> >> > > >>>>>>> testing ZK
> >> >> > > >>>>>> client
> >> >> > > >>>>>>> and run them separately with Java 8. The only limitation
> is
> >> >> > > >>>>>>> that these
> >> >> > > >>>>>>> tests must stay JDK8 compatible.
> >> >> > > >>>>>>> But from the tags we will see which ones are those.
> >> >> > > >>>>>>>
> >> >> > > >>>>>>> What do you think?
> >> >> > > >>>>>>>
> >> >> > > >>>>>>> Regards, Tamaas
> >> >> > > >>>>>>>
> >> >> > > >>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> >> >> > > >>>>>>> eolivelli@gmail.com>
> >> >> > > >>>>>> wrote:
> >> >> > > >>>>>>>> Christopher
> >> >> > > >>>>>>>> I appreciate your idea and I also moved lots of my
> projects
> >> >> > > >>>>>>>> to work
> >> >> > > >>>> the
> >> >> > > >>>>>> way
> >> >> > > >>>>>>>> you are suggesting.
> >> >> > > >>>>>>>>
> >> >> > > >>>>>>>>
> >> >> > > >>>>>>>> We must run tests using real jdk8 to test the Zookeeper
> >> >> > > >>>>>>>> client. We
> >> >> > > >>>> must
> >> >> > > >>>>>>>> ensure that Zookeeper works well, especially while
> dealing
> >> >> > > >>>>>>>> with
> >> >> > > >>>> security
> >> >> > > >>>>>>>> stuff.
> >> >> > > >>>>>>>> Currently the client is in the same module of the server
> >> >> > > >>>>>>>> and it will
> >> >> > > >>>>>> take a
> >> >> > > >>>>>>>> good (huge) amount of work to separate them
> >> >> > > >>>>>>>>
> >> >> > > >>>>>>>>
> >> >> > > >>>>>>>> Enrico
> >> >> > > >>>>>>>>
> >> >> > > >>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher <
> ctubbsii@apache.org>
> >> >> > > >>>>>>>> ha
> >> >> > > >>>> scritto:
> >> >> > > >>>>>>>>> Hi ZK Devs,
> >> >> > > >>>>>>>>>
> >> >> > > >>>>>>>>> With recent advancements in Java (since Java 9), it is
> >> >> > > >>>>>>>>> now generally
> >> >> > > >>>>>>>>> no longer necessary to require that software be
> developed
> >> >> > > >>>>>>>>> on an older
> >> >> > > >>>>>>>>> JDK in order to have confidence that it will run on the
> >> >> > > >>>>>>>>> older version
> >> >> > > >>>>>>>>> of Java. This is because, as of Java 9, all JDK
> releases
> >> >> > > >>>>>>>>> have better
> >> >> > > >>>>>>>>> support for cross-compilation to older Java versions.
> >> >> > > >>>>>>>>>
> >> >> > > >>>>>>>>> What this means is that developers can confidently make
> >> >> > > >>>>>>>>> the build
> >> >> > > >>>>>>>>> requirements for a project higher than the Java version
> >> >> > > >>>>>>>>> that will
> >> >> > > >>>>>>>>> actually be supported at runtime.
> >> >> > > >>>>>>>>>
> >> >> > > >>>>>>>>> In fact, ZooKeeper already supports the necessary flags
> >> >> > > >>>>>>>>> in its Maven
> >> >> > > >>>>>>>>> build configuration to ensure that it uses JDK 8
> >> >> > > >>>>>>>>> compliance when
> >> >> > > >>>>>>>>> building on a newer JDK (I added this way back in
> >> >> > > >>>>>>>>> ZOOKEEPER-3739 /
> >> >> > > >>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
> >> >> > > >>>>>>>>>
> >> >> > > >>>>>>>>> So, I propose that we make JDK 11 the new minimum
> version
> >> >> > > >>>>>>>>> to *build*
> >> >> > > >>>>>>>>> ZooKeeper with. This would not change the runtime
> >> >> > > >>>>>>>>> requirement, which
> >> >> > > >>>>>>>>> would remain at JDK 8.
> >> >> > > >>>>>>>>>
> >> >> > > >>>>>>>>> The only necessary change to make this happen would be
> to
> >> >> > > >>>>>>>>> add the
> >> >> > > >>>>>>>>> minimum Java version to the maven-enforcer-plugin (like
> >> >> > > >>>>>>>>>
> >> >> > > >>>>>>>>>
> >> >> > > >>>>
> >> >> > > >>
> >> >> > >
> >> >> >
> >> >>
> >
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> >> >> > > >>>>>>>>> )
> >> >> > > >>>>>>>>>
> >> >> > > >>>>>>>>> This would allow ZooKeeper to to streamline its
> >> >> > > >>>>>>>>> development process a
> >> >> > > >>>>>>>>> little bit by reducing the amount of CI testing that is
> >> >> > > >>>>>>>>> done as part
> >> >> > > >>>>>>>>> of the build. In other words, we can drop the CI builds
> >> >> > > >>>>>>>>> for JDK 8,
> >> >> > > >>>>>>>>> which saves on build resources and time. The return on
> >> >> > > >>>>>>>>> investment is
> >> >> > > >>>>>>>>> so low for the JDK 8 builds anyway, because of the
> >> >> > > >>>>>>>>> improved
> >> >> > > >>>>>>>>> cross-compilation in newer JDKs. So, there's not much
> >> >> > > >>>>>>>>> value in
> >> >> > > >>>>>>>>> building on JDK 8 anyway.
> >> >> > > >>>>>>>>>
> >> >> > > >>>>>>>>> Of course, I am only recommending this for *new*
> release
> >> >> > > >>>>>>>>> lines,
> >> >> > > >>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because I
> >> >> > > >>>>>>>>> would not want
> >> >> > > >>>>>>>>> to change expectations for users who will build their
> own
> >> >> > > >>>>>>>>> 3.5 and 3.6
> >> >> > > >>>>>>>>> versions as they continue to have patch versions
> >> >> > > >>>>>>>>> released.
> >> >> > > >>>>>>>>>
> >> >> > > >>>>>>>>> What do you think?
> >> >> > > >>>>>>>>>
> >> >> > > >>>>>>>>> Kind Regards,
> >> >> > > >>>>>>>>> Christopher
> >> >> > > >>>>>>>>>
> >> >> > > >>
> >> >> > > >>
> >> >> > >
> >> >> > >
> >> >> >
> >> >>
> >> >
>

3.7.0: JDK 11 vs. JDK 8 (was: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7)

[Damien Diederen <dd...@apache.org>]

Dear ZooKeeper team,

Andor just reminded me of this JDK 11 vs. 8 discussion, for which we did
not reach a resolution.  Do we want to make a move for the 3.7.0 release?

The original proposal, by Christopher, can be found here:

  https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9Z2TNfAwjZ7etXPF91QmROvUWV+KHnxgPSD_MSeKfpyWA@mail.gmail.com%3e

I am explicitly replying to Christopher's nice and useful summary, which
you will find directly below.  An archived copy can be found here:

  https://mail-archives.apache.org/mod_mbox/zookeeper-dev/202010.mbox/%3cCAL5zq9ao9-4S9bmzxaYJM8n8=AoLxiDZ-0ZohunVnAyQei0zAg@mail.gmail.com%3e

It contains three proposals, to which we can add the trivial one:

 1. Compile/test with JDK 11, support JDK 8 "passively";

 2. Compile/test with JDK 11, test client only with JDK 8 (requires test
    suite adaptations);

 3. Move to JDK 11 fully;

 4. Postpone any change to 3.8 (or 4.0).

Flavio wrote:

> [Christopher] mentioned a PMC vote, and I don't think this should be a
> closed vote, independent of how the conversation goes.

Do we have a better idea of what we want?  Or should we organize a vote?

Cheers, -D


--8<--------------- original message ------------->8---

Christopher <ct...@apache.org> writes:
> I'm happy that this discussion has been so lively! I just want to
> emphasize a few things:
>
> I really do understand the desire to continue to support Java 8... I
> get it. But all the conversations around this seem based on what
> people are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
> *future* release... so it should be based more on reasonable
> expectations for users in the future, and less based on what is
> happening today. I suspect *most* people today are still using 3.4
> anyway (it was just so stable for so long...), but that shouldn't mean
> the developers should hold back development on 3.5 and 3.6, any more
> than today's users of 3.5/3.6 should hold back 3.7.
>
> Some of the opinions expressed in this discussion seem to propose a
> scenario where users are going to be updating to "bleeding edge"
> versions of ZooKeeper, but are going to insist on using Java 8.
> Personally, I find this to be implausible. In my experience, people
> either upgrade everything as soon as they are able to, or they upgrade
> each thing individually, only when they are forced to. The first group
> will be happy to move to Java 11 and ZK 3.7. The second group will
> probably avoid 3.7 anyway, and are fine sticking with 3.6, but if they
> had to update to 3.7, they'd also be fine updating to Java 11 if they
> had to in order to use 3.7. I can't imagine the scenario where people
> are eagerly choosing to upgrade to ZK 3.7, but miserly insisting on
> using Java 8. Perhaps that scenario exists, but it's hard for me to
> imagine. Even so, my proposal would still support even that group of
> people.
>
> I think there are now effectively three proposals being discussed in
> this thread:
>
> 1. (Christopher's original proposal) passively support Java 8 at
> runtime by making JDK 11 the minimum requirement to build and test.
> This scenario involves continuing to fix bugs, as they are discovered
> and reported, that affect JDK 8, but passively, rather than
> proactively. This proposal does *not* drop Java 8 support, but merely
> de-emphasizes it in development of what will be 3.7 in the future, and
> drops the requirement to do dedicated testing with Java 8. I think
> this is low risk, because it is very unlikely that the ZK devs would
> introduce a bug that would affect only Java 8 and the compiler
> wouldn't catch it... because the cross-compilation features of newer
> JDKs are really good.
>
> 2. (Enrico's alternate proposal) this variation of my proposal would
> involve continuing to proactively support Java 8 by creating a
> dedicated testing suite to test client code on Java 8. I think this is
> a good option, but since it involves a significantly higher amount of
> work than option 1, I think the cost-benefit analysis would show this
> to be not worth the effort. Also, if it were implemented, it would
> need to be done carefully to avoid requiring developers to have
> concurrently installed both Java 8 and Java 11 in order to perform a
> build, because requiring Java 8 at build time while developing would
> be worse than we have today.
>
> 3. (Andor's preference) move to JDK 11 fully. This would provide no
> support, passive or active, for Java 8 in ZK 3.7. To be honest, this
> is my personal favorite, and is the simplest to implement and
> communicate clearly to end users in release notes. The only reason I
> proposed a passive support of Java 8 instead of this is because I was
> trying to seek a compromise from the start. But, I think by far, this
> is the best option for the next *future* release of ZK. If you wanted
> to make the change even more visible to users, the version could even
> be bumped to 4.0.
>
> If this were to come to a VOTE by the PMC, in order to make a final
> decision, I would recommend they vote on option 3, and then if that
> fails, vote on option 1, and if that fails, keep things the way they
> are (because option 2 is more work).
>
> Christopher
>
> P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK 11
> and it works just fine there (as long as you add the missing runtime
> jar for javax.activation:javax.activation-api:1.2.0 to its class path,
> but that was fixed in Hadoop 3.3).
>
> On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes
> <ta...@cloudera.com.invalid> wrote:
>>
>> Hi All,
>>
>> I've just talked with a Hadoop/HDFS developer who told me what I guessed.
>> With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would mean
>> a release of Hadoop4.
>> Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both. As of
>> today most of the Hadoop users are still on Hadoop2, he doesn't expect
>> Hadoop4 soon.
>> As many Apache components depend on Hadoop and ZooKeeper they won't hurry
>> to JDK11 until they have to (they will probably go one-by-one very slowly),
>> which means if Hadoop stays on JDK8, they would use the last ZK version
>> which works on JDK8.
>> Do we want a ZK 3.4 again?
>>
>> Regards, Tamaas
>>
>>
>> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <ta...@cloudera.com> wrote:
>>
>> > Hi All,
>> >
>> > Just to add my two cents.
>> >
>> > Upgrading to JDK11 looks inevitable sooner or later and I would definitely
>> > not wait until 2030 or 2026 when the extended support of JDK8 ends.
>> > But on the other side I have to agree with Enrico and Patrick that far too
>> > many users are tied to JDK8 yet (not because they want to use JDK8, but
>> > because they have to), some of them are components of the Hadoop ecosystem,
>> > which would be a loss to tie them to 3.6.x for years.
>> > Do we know the state of Hadoop? It builds with JDK8 at the moment, but do
>> > we know what are their plans to go to JDK11?
>> > When they move, we should move too, but I don't think it would be wise to
>> > do it earlier.
>> >
>> > Christopher's option looks like the golden path, but it needs some
>> > investment on the testing side as Enrico pointed it out.
>> > Could we agree on it as a compromise?
>> >
>> > Regards, Tamaas
>> >
>> > On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com> wrote:
>> >
>> >> I think I was reacting to Enrico's earlier comment of:
>> >>
>> >> " ZooKeeper client is used by tons of users and unfortunately many
>> >> projects
>> >> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to block
>> >> users
>> >> from the adoption, that is that we will see the world to stay on 3.6.x and
>> >> we will have again a long lived release line, like 3.4."
>> >>
>> >> It's a matter of whether or not a long-lived release line is
>> >> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
>> >> security/patch-wise (if not feature-wise) for the next N years, then
>> >> that's
>> >> a potentially valid approach.  I interpreted that comment as "a long-lived
>> >> release line is undesirable", but no one explicitly said that, I just read
>> >> it that way.
>> >>
>> >> ~Brent
>> >>
>> >> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org> wrote:
>> >>
>> >> > On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org> wrote:
>> >> >
>> >> > > As far as I know Hbase, Solr and Kafka are already Java 11 ready.
>> >> > >
>> >> > > IMHO contributors of those projects should also put efforts into
>> >> moving
>> >> > > forward.
>> >> > >
>> >> > > We’re not saying that you _have_ to move to Java 11.
>> >> > > Staying on Java 8? No problem, 3.6 is for you.
>> >> > > Want the fancy new features of 3.7? Work on it on your side too.
>> >> > >
>> >> > >
>> >> > ppl want things like security fixes. I believe the highlighted downside
>> >> is
>> >> > that we would need to continue to maintain 3.6.x rather than allowing
>> >> > users, and ourselves, to focus on trunk for production -"64 percent"
>> >> would
>> >> > be blocked.
>> >> >
>> >> > Patrick
>> >> >
>> >> >
>> >> > > Andor
>> >> > >
>> >> > >
>> >> > >
>> >> > > > On 2020. Oct 21., at 17:52, Enrico Olivelli <eo...@gmail.com>
>> >> > wrote:
>> >> > > >
>> >> > > > Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
>> >> > andor@apache.org>
>> >> > > ha
>> >> > > > scritto:
>> >> > > >
>> >> > > >> Correct me if I'm wrong, but Oracle gets paid for extended support.
>> >> > > >> Java 8 support until 2030 is not free of charge.
>> >> > > >>
>> >> > > >> "ZK may end up with a lot of users potentially locking themselves
>> >> to
>> >> > > >> 3.6.x for a while as Enrico mentioned."
>> >> > > >>
>> >> > > >> That's true. What's the downside of that which we should invest in
>> >> to
>> >> > > >> avoid?
>> >> > > >>
>> >> > > >
>> >> > > > I see ZooKeeper used in many many projects, all of the
>> >> > > > HBase/Pulsar/Kafka/Solr ecosystem...
>> >> > > > they will have to move to JDK11 in order to move to the new ZK
>> >> version
>> >> > > > so probably they will stay on ZK 3.6
>> >> > > >
>> >> > > > Probably with Java 17 LTS released the cards will change on the
>> >> table
>> >> > > >
>> >> > > > Enrico
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > >>
>> >> > > >> Andor
>> >> > > >>
>> >> > > >>
>> >> > > >>
>> >> > > >> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
>> >> > > >>> As a slightly different consideration, if you look at the
>> >> Long-Term
>> >> > > >>> Support
>> >> > > >>> (LTS) roadmaps for Java, currently Java 8 is set to have full
>> >> support
>> >> > > >>> until
>> >> > > >>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
>> >> > > >>>
>> >> > > >>>
>> >> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>> >> > > >>> https://en.wikipedia.org/wiki/Java_version_history
>> >> > > >>>
>> >> > > >>> My guess is that a number of companies are still heavily invested
>> >> at
>> >> > > >>> the
>> >> > > >>> Java 8 level (I know a few) and with that kind of time horizon,
>> >> they
>> >> > > >>> have
>> >> > > >>> no real motivation to upgrade for quite a while.  If the recent
>> >> > > >>> Python 2
>> >> > > >>> deprecation is anything to go by, they won't do it until they have
>> >> > > >>> to.
>> >> > > >>>
>> >> > > >>> Not saying Java 8 isn't *very* old (2014 release it seems like?)
>> >> and
>> >> > > >>> I'm
>> >> > > >>> not invested heavily either way, but this might suggest that ZK
>> >> may
>> >> > > >>> end up
>> >> > > >>> with a lot of users potentially locking themselves to 3.6.x for a
>> >> > > >>> while as
>> >> > > >>> Enrico mentioned.
>> >> > > >>>
>> >> > > >>> (Not a major contributor, but wanted to chime in since I just had
>> >> > > >>> this
>> >> > > >>> conversation with a bunch of people professionally recently)
>> >> > > >>>
>> >> > > >>> Brent
>> >> > > >>>
>> >> > > >>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <an...@apache.org>
>> >> > > >>> wrote:
>> >> > > >>>
>> >> > > >>>> Thanks for the summary.
>> >> > > >>>>
>> >> > > >>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully. Other
>> >> > > >>>> projects
>> >> > > >>>> will upgrade once they’re JDK11 compliant, otherwise they will
>> >> stay
>> >> > > >>>> on 3.5
>> >> > > >>>> or 3.6. Both version are quite recent in ZooKeeper-terms, we
>> >> > > >>>> already
>> >> > > >>>> planned big changes for 3.7.0 and JDK 11 could be one of them.
>> >> > > >>>>
>> >> > > >>>> Don’t put extra burden on the ZK community to help others staying
>> >> > > >>>> on
>> >> > > >>>> ancient Java versions.
>> >> > > >>>>
>> >> > > >>>> Andor
>> >> > > >>>>
>> >> > > >>>>
>> >> > > >>>>
>> >> > > >>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
>> >> eolivelli@gmail.com>
>> >> > > >>>>> wrote:
>> >> > > >>>>>
>> >> > > >>>>> Let me recap
>> >> > > >>>>> - Christopher is proposing to move to JDK11
>> >> > > >>>>> - ZooKeeper client and server are bundled and coded and tested
>> >> > > >>>>> together
>> >> > > >>>> in
>> >> > > >>>>> zookeeper-server
>> >> > > >>>>> - Enrico is concerned about the need of testing ZooKeeper client
>> >> > > >>>>> on JDK8
>> >> > > >>>>> (not a problem to move the server to JDK11)
>> >> > > >>>>>
>> >> > > >>>>> ZooKeeper client is used by tons of users and unfortunately many
>> >> > > >>>>> projects
>> >> > > >>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
>> >> > > >>>>> block
>> >> > > >>>> users
>> >> > > >>>>> from the adoption,
>> >> > > >>>>> that is that we will see the world to stay on 3.6.x and we will
>> >> > > >>>>> have
>> >> > > >>>> again
>> >> > > >>>>> a long lived release line, like 3.4.
>> >> > > >>>>>
>> >> > > >>>>> Testing the client on JDK8 would be possible if we create some
>> >> > > >>>>> kind of
>> >> > > >>>>> additional module with system tests, then we can start the
>> >> server
>> >> > > >>>>> on
>> >> > > >>>> docker
>> >> > > >>>>> on JDK11+ and start a client on JDK8
>> >> > > >>>>> with Maven toolchain it should possible to run surefire tests
>> >> > > >>>>> using a
>> >> > > >>>>> separate JVM.
>> >> > > >>>>>
>> >> > > >>>>> So in my vision 2 options:
>> >> > > >>>>> 1) fully JDK11 - drop JDK8 at all
>> >> > > >>>>> 2) build with JDK11 - server only on JDK11 - add system tests
>> >> > > >>>>> with docker
>> >> > > >>>>> and toolchains that ensure the ZooKeeper client (and all
>> >> > > >>>>> dependencies)
>> >> > > >>>>> still work on JDK8
>> >> > > >>>>>
>> >> > > >>>>> From my point of view about the ZooKeeper ecosystem option 2)
>> >> > > >>>>> will be far
>> >> > > >>>>> better, but we need resources to work on a new test suite.
>> >> > > >>>>>
>> >> > > >>>>> Enrico
>> >> > > >>>>>
>> >> > > >>>>>
>> >> > > >>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
>> >> > > >>>>> andor@apache.org>
>> >> > > >>>> ha
>> >> > > >>>>> scritto:
>> >> > > >>>>>
>> >> > > >>>>>> Tamas, Enrico,
>> >> > > >>>>>>
>> >> > > >>>>>> Sorry I don’t follow. Why do we have to test the client with
>> >> > > >>>>>> JDK 8 in
>> >> > > >>>>>> version 3.7.0?
>> >> > > >>>>>>
>> >> > > >>>>>> Andor
>> >> > > >>>>>>
>> >> > > >>>>>>
>> >> > > >>>>>>
>> >> > > >>>>>>
>> >> > > >>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
>> >> > > >>>>>>> tamaas@cloudera.com.INVALID>
>> >> > > >>>>>> wrote:
>> >> > > >>>>>>> Hi Enrico,
>> >> > > >>>>>>>
>> >> > > >>>>>>> Separating ZooKeeper client and server is a huge work, but we
>> >> > > >>>>>>> might not
>> >> > > >>>>>>> need it.
>> >> > > >>>>>>> As you mentioned we have to test ZK client with Java 8, what
>> >> > > >>>>>>> about
>> >> > > >>>>>>> separating only the test cases which we need to run with
>> >> > > >>>>>>> Java8 too?
>> >> > > >>>>>>> In Curator we have the ZK compatibility tests where we run a
>> >> > > >>>>>>> limited
>> >> > > >>>>>> amount
>> >> > > >>>>>>> of Curator's jUnit tests with a different ZK version.
>> >> > > >>>>>>> We might be able to do the same here, tag tests which are
>> >> > > >>>>>>> testing ZK
>> >> > > >>>>>> client
>> >> > > >>>>>>> and run them separately with Java 8. The only limitation is
>> >> > > >>>>>>> that these
>> >> > > >>>>>>> tests must stay JDK8 compatible.
>> >> > > >>>>>>> But from the tags we will see which ones are those.
>> >> > > >>>>>>>
>> >> > > >>>>>>> What do you think?
>> >> > > >>>>>>>
>> >> > > >>>>>>> Regards, Tamaas
>> >> > > >>>>>>>
>> >> > > >>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
>> >> > > >>>>>>> eolivelli@gmail.com>
>> >> > > >>>>>> wrote:
>> >> > > >>>>>>>> Christopher
>> >> > > >>>>>>>> I appreciate your idea and I also moved lots of my projects
>> >> > > >>>>>>>> to work
>> >> > > >>>> the
>> >> > > >>>>>> way
>> >> > > >>>>>>>> you are suggesting.
>> >> > > >>>>>>>>
>> >> > > >>>>>>>>
>> >> > > >>>>>>>> We must run tests using real jdk8 to test the Zookeeper
>> >> > > >>>>>>>> client. We
>> >> > > >>>> must
>> >> > > >>>>>>>> ensure that Zookeeper works well, especially while dealing
>> >> > > >>>>>>>> with
>> >> > > >>>> security
>> >> > > >>>>>>>> stuff.
>> >> > > >>>>>>>> Currently the client is in the same module of the server
>> >> > > >>>>>>>> and it will
>> >> > > >>>>>> take a
>> >> > > >>>>>>>> good (huge) amount of work to separate them
>> >> > > >>>>>>>>
>> >> > > >>>>>>>>
>> >> > > >>>>>>>> Enrico
>> >> > > >>>>>>>>
>> >> > > >>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org>
>> >> > > >>>>>>>> ha
>> >> > > >>>> scritto:
>> >> > > >>>>>>>>> Hi ZK Devs,
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> With recent advancements in Java (since Java 9), it is
>> >> > > >>>>>>>>> now generally
>> >> > > >>>>>>>>> no longer necessary to require that software be developed
>> >> > > >>>>>>>>> on an older
>> >> > > >>>>>>>>> JDK in order to have confidence that it will run on the
>> >> > > >>>>>>>>> older version
>> >> > > >>>>>>>>> of Java. This is because, as of Java 9, all JDK releases
>> >> > > >>>>>>>>> have better
>> >> > > >>>>>>>>> support for cross-compilation to older Java versions.
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> What this means is that developers can confidently make
>> >> > > >>>>>>>>> the build
>> >> > > >>>>>>>>> requirements for a project higher than the Java version
>> >> > > >>>>>>>>> that will
>> >> > > >>>>>>>>> actually be supported at runtime.
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> In fact, ZooKeeper already supports the necessary flags
>> >> > > >>>>>>>>> in its Maven
>> >> > > >>>>>>>>> build configuration to ensure that it uses JDK 8
>> >> > > >>>>>>>>> compliance when
>> >> > > >>>>>>>>> building on a newer JDK (I added this way back in
>> >> > > >>>>>>>>> ZOOKEEPER-3739 /
>> >> > > >>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> So, I propose that we make JDK 11 the new minimum version
>> >> > > >>>>>>>>> to *build*
>> >> > > >>>>>>>>> ZooKeeper with. This would not change the runtime
>> >> > > >>>>>>>>> requirement, which
>> >> > > >>>>>>>>> would remain at JDK 8.
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> The only necessary change to make this happen would be to
>> >> > > >>>>>>>>> add the
>> >> > > >>>>>>>>> minimum Java version to the maven-enforcer-plugin (like
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>>
>> >> > > >>>>
>> >> > > >>
>> >> > >
>> >> >
>> >>
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
>> >> > > >>>>>>>>> )
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> This would allow ZooKeeper to to streamline its
>> >> > > >>>>>>>>> development process a
>> >> > > >>>>>>>>> little bit by reducing the amount of CI testing that is
>> >> > > >>>>>>>>> done as part
>> >> > > >>>>>>>>> of the build. In other words, we can drop the CI builds
>> >> > > >>>>>>>>> for JDK 8,
>> >> > > >>>>>>>>> which saves on build resources and time. The return on
>> >> > > >>>>>>>>> investment is
>> >> > > >>>>>>>>> so low for the JDK 8 builds anyway, because of the
>> >> > > >>>>>>>>> improved
>> >> > > >>>>>>>>> cross-compilation in newer JDKs. So, there's not much
>> >> > > >>>>>>>>> value in
>> >> > > >>>>>>>>> building on JDK 8 anyway.
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> Of course, I am only recommending this for *new* release
>> >> > > >>>>>>>>> lines,
>> >> > > >>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because I
>> >> > > >>>>>>>>> would not want
>> >> > > >>>>>>>>> to change expectations for users who will build their own
>> >> > > >>>>>>>>> 3.5 and 3.6
>> >> > > >>>>>>>>> versions as they continue to have patch versions
>> >> > > >>>>>>>>> released.
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> What do you think?
>> >> > > >>>>>>>>>
>> >> > > >>>>>>>>> Kind Regards,
>> >> > > >>>>>>>>> Christopher
>> >> > > >>>>>>>>>
>> >> > > >>
>> >> > > >>
>> >> > >
>> >> > >
>> >> >
>> >>
>> >

R: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Alessandro Luccaroni - Diennea <al...@diennea.com.INVALID>]

Hi all,
If I might chime in as a zookeeper user (in multiple products) and a follower of the project I think the drop of Java8 support (official and/or unofficial) could be a big mistake.

From my own company point of view we already support Java11 in all our applications so we are not directly impacted (and we have upgrade path for older versions to provide to our customers).
My worries resides in the (high) probability of a userbase fragmentation: in the recent past Zookeeper development picked up speed thanks to a bunch of new committers and PMCs after a period of mostly maintenance focused works, but the number of active committers and PMCs is still very low for a project like this.

I foresee the risk of spreading thin the resources of the project if we force the userbase to stick to an older version and, in turn, we are forced to backport many issue to the 3.6 branch.

Alessandro Luccaroni
Platform Manager @ Diennea - MagNews
Tel.: (+39) 0546 066100 Int. 924 - Mob.: (+39) 393 7273519
Viale G.Marconi 30/14 - 48018 Faenza (RA) - Italy

-----Messaggio originale-----
Da: Christopher <ct...@apache.org>
Inviato: giovedì 22 ottobre 2020 05:21
A: dev@zookeeper.apache.org
Oggetto: Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

I'm happy that this discussion has been so lively! I just want to emphasize a few things:

I really do understand the desire to continue to support Java 8... I get it. But all the conversations around this seem based on what people are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
*future* release... so it should be based more on reasonable expectations for users in the future, and less based on what is happening today. I suspect *most* people today are still using 3.4 anyway (it was just so stable for so long...), but that shouldn't mean the developers should hold back development on 3.5 and 3.6, any more than today's users of 3.5/3.6 should hold back 3.7.

Some of the opinions expressed in this discussion seem to propose a scenario where users are going to be updating to "bleeding edge"
versions of ZooKeeper, but are going to insist on using Java 8.
Personally, I find this to be implausible. In my experience, people either upgrade everything as soon as they are able to, or they upgrade each thing individually, only when they are forced to. The first group will be happy to move to Java 11 and ZK 3.7. The second group will probably avoid 3.7 anyway, and are fine sticking with 3.6, but if they had to update to 3.7, they'd also be fine updating to Java 11 if they had to in order to use 3.7. I can't imagine the scenario where people are eagerly choosing to upgrade to ZK 3.7, but miserly insisting on using Java 8. Perhaps that scenario exists, but it's hard for me to imagine. Even so, my proposal would still support even that group of people.

I think there are now effectively three proposals being discussed in this thread:

1. (Christopher's original proposal) passively support Java 8 at runtime by making JDK 11 the minimum requirement to build and test.
This scenario involves continuing to fix bugs, as they are discovered and reported, that affect JDK 8, but passively, rather than proactively. This proposal does *not* drop Java 8 support, but merely de-emphasizes it in development of what will be 3.7 in the future, and drops the requirement to do dedicated testing with Java 8. I think this is low risk, because it is very unlikely that the ZK devs would introduce a bug that would affect only Java 8 and the compiler wouldn't catch it... because the cross-compilation features of newer JDKs are really good.

2. (Enrico's alternate proposal) this variation of my proposal would involve continuing to proactively support Java 8 by creating a dedicated testing suite to test client code on Java 8. I think this is a good option, but since it involves a significantly higher amount of work than option 1, I think the cost-benefit analysis would show this to be not worth the effort. Also, if it were implemented, it would need to be done carefully to avoid requiring developers to have concurrently installed both Java 8 and Java 11 in order to perform a build, because requiring Java 8 at build time while developing would be worse than we have today.

3. (Andor's preference) move to JDK 11 fully. This would provide no support, passive or active, for Java 8 in ZK 3.7. To be honest, this is my personal favorite, and is the simplest to implement and communicate clearly to end users in release notes. The only reason I proposed a passive support of Java 8 instead of this is because I was trying to seek a compromise from the start. But, I think by far, this is the best option for the next *future* release of ZK. If you wanted to make the change even more visible to users, the version could even be bumped to 4.0.

If this were to come to a VOTE by the PMC, in order to make a final decision, I would recommend they vote on option 3, and then if that fails, vote on option 1, and if that fails, keep things the way they are (because option 2 is more work).

Christopher

P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK 11 and it works just fine there (as long as you add the missing runtime jar for javax.activation:javax.activation-api:1.2.0 to its class path, but that was fixed in Hadoop 3.3).

On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes <ta...@cloudera.com.invalid> wrote:
>
> Hi All,
>
> I've just talked with a Hadoop/HDFS developer who told me what I guessed.
> With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would
> mean a release of Hadoop4.
> Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both. As
> of today most of the Hadoop users are still on Hadoop2, he doesn't
> expect
> Hadoop4 soon.
> As many Apache components depend on Hadoop and ZooKeeper they won't
> hurry to JDK11 until they have to (they will probably go one-by-one
> very slowly), which means if Hadoop stays on JDK8, they would use the
> last ZK version which works on JDK8.
> Do we want a ZK 3.4 again?
>
> Regards, Tamaas
>
>
> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <ta...@cloudera.com> wrote:
>
> > Hi All,
> >
> > Just to add my two cents.
> >
> > Upgrading to JDK11 looks inevitable sooner or later and I would
> > definitely not wait until 2030 or 2026 when the extended support of JDK8 ends.
> > But on the other side I have to agree with Enrico and Patrick that
> > far too many users are tied to JDK8 yet (not because they want to
> > use JDK8, but because they have to), some of them are components of
> > the Hadoop ecosystem, which would be a loss to tie them to 3.6.x for years.
> > Do we know the state of Hadoop? It builds with JDK8 at the moment,
> > but do we know what are their plans to go to JDK11?
> > When they move, we should move too, but I don't think it would be
> > wise to do it earlier.
> >
> > Christopher's option looks like the golden path, but it needs some
> > investment on the testing side as Enrico pointed it out.
> > Could we agree on it as a compromise?
> >
> > Regards, Tamaas
> >
> > On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com> wrote:
> >
> >> I think I was reacting to Enrico's earlier comment of:
> >>
> >> " ZooKeeper client is used by tons of users and unfortunately many
> >> projects are still on JDK8, if we move ZooKeeper to JDK11 the risk
> >> is to block users from the adoption, that is that we will see the
> >> world to stay on 3.6.x and we will have again a long lived release
> >> line, like 3.4."
> >>
> >> It's a matter of whether or not a long-lived release line is
> >> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
> >> security/patch-wise (if not feature-wise) for the next N years,
> >> then that's a potentially valid approach.  I interpreted that
> >> comment as "a long-lived release line is undesirable", but no one
> >> explicitly said that, I just read it that way.
> >>
> >> ~Brent
> >>
> >> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org> wrote:
> >>
> >> > On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org> wrote:
> >> >
> >> > > As far as I know Hbase, Solr and Kafka are already Java 11 ready.
> >> > >
> >> > > IMHO contributors of those projects should also put efforts
> >> > > into
> >> moving
> >> > > forward.
> >> > >
> >> > > We’re not saying that you _have_ to move to Java 11.
> >> > > Staying on Java 8? No problem, 3.6 is for you.
> >> > > Want the fancy new features of 3.7? Work on it on your side too.
> >> > >
> >> > >
> >> > ppl want things like security fixes. I believe the highlighted
> >> > downside
> >> is
> >> > that we would need to continue to maintain 3.6.x rather than
> >> > allowing users, and ourselves, to focus on trunk for production -"64 percent"
> >> would
> >> > be blocked.
> >> >
> >> > Patrick
> >> >
> >> >
> >> > > Andor
> >> > >
> >> > >
> >> > >
> >> > > > On 2020. Oct 21., at 17:52, Enrico Olivelli
> >> > > > <eo...@gmail.com>
> >> > wrote:
> >> > > >
> >> > > > Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
> >> > andor@apache.org>
> >> > > ha
> >> > > > scritto:
> >> > > >
> >> > > >> Correct me if I'm wrong, but Oracle gets paid for extended support.
> >> > > >> Java 8 support until 2030 is not free of charge.
> >> > > >>
> >> > > >> "ZK may end up with a lot of users potentially locking
> >> > > >> themselves
> >> to
> >> > > >> 3.6.x for a while as Enrico mentioned."
> >> > > >>
> >> > > >> That's true. What's the downside of that which we should
> >> > > >> invest in
> >> to
> >> > > >> avoid?
> >> > > >>
> >> > > >
> >> > > > I see ZooKeeper used in many many projects, all of the
> >> > > > HBase/Pulsar/Kafka/Solr ecosystem...
> >> > > > they will have to move to JDK11 in order to move to the new
> >> > > > ZK
> >> version
> >> > > > so probably they will stay on ZK 3.6
> >> > > >
> >> > > > Probably with Java 17 LTS released the cards will change on
> >> > > > the
> >> table
> >> > > >
> >> > > > Enrico
> >> > > >
> >> > > >
> >> > > >
> >> > > >>
> >> > > >> Andor
> >> > > >>
> >> > > >>
> >> > > >>
> >> > > >> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> >> > > >>> As a slightly different consideration, if you look at the
> >> Long-Term
> >> > > >>> Support
> >> > > >>> (LTS) roadmaps for Java, currently Java 8 is set to have
> >> > > >>> full
> >> support
> >> > > >>> until
> >> > > >>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> >> > > >>>
> >> > > >>>
> >> > https://www.oracle.com/java/technologies/java-se-support-roadmap.
> >> > html
> >> > > >>> https://en.wikipedia.org/wiki/Java_version_history
> >> > > >>>
> >> > > >>> My guess is that a number of companies are still heavily
> >> > > >>> invested
> >> at
> >> > > >>> the
> >> > > >>> Java 8 level (I know a few) and with that kind of time
> >> > > >>> horizon,
> >> they
> >> > > >>> have
> >> > > >>> no real motivation to upgrade for quite a while.  If the
> >> > > >>> recent Python 2 deprecation is anything to go by, they
> >> > > >>> won't do it until they have to.
> >> > > >>>
> >> > > >>> Not saying Java 8 isn't *very* old (2014 release it seems
> >> > > >>> like?)
> >> and
> >> > > >>> I'm
> >> > > >>> not invested heavily either way, but this might suggest
> >> > > >>> that ZK
> >> may
> >> > > >>> end up
> >> > > >>> with a lot of users potentially locking themselves to 3.6.x
> >> > > >>> for a while as Enrico mentioned.
> >> > > >>>
> >> > > >>> (Not a major contributor, but wanted to chime in since I
> >> > > >>> just had this conversation with a bunch of people
> >> > > >>> professionally recently)
> >> > > >>>
> >> > > >>> Brent
> >> > > >>>
> >> > > >>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar
> >> > > >>> <an...@apache.org>
> >> > > >>> wrote:
> >> > > >>>
> >> > > >>>> Thanks for the summary.
> >> > > >>>>
> >> > > >>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully.
> >> > > >>>> Other projects will upgrade once they’re JDK11 compliant,
> >> > > >>>> otherwise they will
> >> stay
> >> > > >>>> on 3.5
> >> > > >>>> or 3.6. Both version are quite recent in ZooKeeper-terms,
> >> > > >>>> we already planned big changes for 3.7.0 and JDK 11 could
> >> > > >>>> be one of them.
> >> > > >>>>
> >> > > >>>> Don’t put extra burden on the ZK community to help others
> >> > > >>>> staying on ancient Java versions.
> >> > > >>>>
> >> > > >>>> Andor
> >> > > >>>>
> >> > > >>>>
> >> > > >>>>
> >> > > >>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
> >> eolivelli@gmail.com>
> >> > > >>>>> wrote:
> >> > > >>>>>
> >> > > >>>>> Let me recap
> >> > > >>>>> - Christopher is proposing to move to JDK11
> >> > > >>>>> - ZooKeeper client and server are bundled and coded and
> >> > > >>>>> tested together
> >> > > >>>> in
> >> > > >>>>> zookeeper-server
> >> > > >>>>> - Enrico is concerned about the need of testing ZooKeeper
> >> > > >>>>> client on JDK8 (not a problem to move the server to
> >> > > >>>>> JDK11)
> >> > > >>>>>
> >> > > >>>>> ZooKeeper client is used by tons of users and
> >> > > >>>>> unfortunately many projects are still on JDK8, if we move
> >> > > >>>>> ZooKeeper to JDK11 the risk is to block
> >> > > >>>> users
> >> > > >>>>> from the adoption,
> >> > > >>>>> that is that we will see the world to stay on 3.6.x and
> >> > > >>>>> we will have
> >> > > >>>> again
> >> > > >>>>> a long lived release line, like 3.4.
> >> > > >>>>>
> >> > > >>>>> Testing the client on JDK8 would be possible if we create
> >> > > >>>>> some kind of additional module with system tests, then we
> >> > > >>>>> can start the
> >> server
> >> > > >>>>> on
> >> > > >>>> docker
> >> > > >>>>> on JDK11+ and start a client on JDK8 with Maven toolchain
> >> > > >>>>> it should possible to run surefire tests using a separate
> >> > > >>>>> JVM.
> >> > > >>>>>
> >> > > >>>>> So in my vision 2 options:
> >> > > >>>>> 1) fully JDK11 - drop JDK8 at all
> >> > > >>>>> 2) build with JDK11 - server only on JDK11 - add system
> >> > > >>>>> tests with docker and toolchains that ensure the
> >> > > >>>>> ZooKeeper client (and all
> >> > > >>>>> dependencies)
> >> > > >>>>> still work on JDK8
> >> > > >>>>>
> >> > > >>>>> From my point of view about the ZooKeeper ecosystem
> >> > > >>>>> option 2) will be far better, but we need resources to
> >> > > >>>>> work on a new test suite.
> >> > > >>>>>
> >> > > >>>>> Enrico
> >> > > >>>>>
> >> > > >>>>>
> >> > > >>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> >> > > >>>>> andor@apache.org>
> >> > > >>>> ha
> >> > > >>>>> scritto:
> >> > > >>>>>
> >> > > >>>>>> Tamas, Enrico,
> >> > > >>>>>>
> >> > > >>>>>> Sorry I don’t follow. Why do we have to test the client
> >> > > >>>>>> with JDK 8 in version 3.7.0?
> >> > > >>>>>>
> >> > > >>>>>> Andor
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
> >> > > >>>>>>> tamaas@cloudera.com.INVALID>
> >> > > >>>>>> wrote:
> >> > > >>>>>>> Hi Enrico,
> >> > > >>>>>>>
> >> > > >>>>>>> Separating ZooKeeper client and server is a huge work,
> >> > > >>>>>>> but we might not need it.
> >> > > >>>>>>> As you mentioned we have to test ZK client with Java 8,
> >> > > >>>>>>> what about separating only the test cases which we need
> >> > > >>>>>>> to run with
> >> > > >>>>>>> Java8 too?
> >> > > >>>>>>> In Curator we have the ZK compatibility tests where we
> >> > > >>>>>>> run a limited
> >> > > >>>>>> amount
> >> > > >>>>>>> of Curator's jUnit tests with a different ZK version.
> >> > > >>>>>>> We might be able to do the same here, tag tests which
> >> > > >>>>>>> are testing ZK
> >> > > >>>>>> client
> >> > > >>>>>>> and run them separately with Java 8. The only
> >> > > >>>>>>> limitation is that these tests must stay JDK8
> >> > > >>>>>>> compatible.
> >> > > >>>>>>> But from the tags we will see which ones are those.
> >> > > >>>>>>>
> >> > > >>>>>>> What do you think?
> >> > > >>>>>>>
> >> > > >>>>>>> Regards, Tamaas
> >> > > >>>>>>>
> >> > > >>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> >> > > >>>>>>> eolivelli@gmail.com>
> >> > > >>>>>> wrote:
> >> > > >>>>>>>> Christopher
> >> > > >>>>>>>> I appreciate your idea and I also moved lots of my
> >> > > >>>>>>>> projects to work
> >> > > >>>> the
> >> > > >>>>>> way
> >> > > >>>>>>>> you are suggesting.
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>> We must run tests using real jdk8 to test the
> >> > > >>>>>>>> Zookeeper client. We
> >> > > >>>> must
> >> > > >>>>>>>> ensure that Zookeeper works well, especially while
> >> > > >>>>>>>> dealing with
> >> > > >>>> security
> >> > > >>>>>>>> stuff.
> >> > > >>>>>>>> Currently the client is in the same module of the
> >> > > >>>>>>>> server and it will
> >> > > >>>>>> take a
> >> > > >>>>>>>> good (huge) amount of work to separate them
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>> Enrico
> >> > > >>>>>>>>
> >> > > >>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher
> >> > > >>>>>>>> <ct...@apache.org> ha
> >> > > >>>> scritto:
> >> > > >>>>>>>>> Hi ZK Devs,
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> With recent advancements in Java (since Java 9), it
> >> > > >>>>>>>>> is now generally no longer necessary to require that
> >> > > >>>>>>>>> software be developed on an older JDK in order to
> >> > > >>>>>>>>> have confidence that it will run on the older version
> >> > > >>>>>>>>> of Java. This is because, as of Java 9, all JDK
> >> > > >>>>>>>>> releases have better support for cross-compilation to
> >> > > >>>>>>>>> older Java versions.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> What this means is that developers can confidently
> >> > > >>>>>>>>> make the build requirements for a project higher than
> >> > > >>>>>>>>> the Java version that will actually be supported at
> >> > > >>>>>>>>> runtime.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> In fact, ZooKeeper already supports the necessary
> >> > > >>>>>>>>> flags in its Maven build configuration to ensure that
> >> > > >>>>>>>>> it uses JDK 8 compliance when building on a newer JDK
> >> > > >>>>>>>>> (I added this way back in
> >> > > >>>>>>>>> ZOOKEEPER-3739 /
> >> > > >>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> So, I propose that we make JDK 11 the new minimum
> >> > > >>>>>>>>> version to *build* ZooKeeper with. This would not
> >> > > >>>>>>>>> change the runtime requirement, which would remain at
> >> > > >>>>>>>>> JDK 8.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> The only necessary change to make this happen would
> >> > > >>>>>>>>> be to add the minimum Java version to the
> >> > > >>>>>>>>> maven-enforcer-plugin (like
> >> > > >>>>>>>>>
> >> > > >>>>>>>>>
> >> > > >>>>
> >> > > >>
> >> > >
> >> >
> >> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd1
> >> 1d5dedb146519/pom.xml#L1162-L1164
> >> > > >>>>>>>>> )
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> This would allow ZooKeeper to to streamline its
> >> > > >>>>>>>>> development process a little bit by reducing the
> >> > > >>>>>>>>> amount of CI testing that is done as part of the
> >> > > >>>>>>>>> build. In other words, we can drop the CI builds for
> >> > > >>>>>>>>> JDK 8, which saves on build resources and time. The
> >> > > >>>>>>>>> return on investment is so low for the JDK 8 builds
> >> > > >>>>>>>>> anyway, because of the improved cross-compilation in
> >> > > >>>>>>>>> newer JDKs. So, there's not much value in building on
> >> > > >>>>>>>>> JDK 8 anyway.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> Of course, I am only recommending this for *new*
> >> > > >>>>>>>>> release lines, starting with ZooKeeper 3.7.0/master
> >> > > >>>>>>>>> branch, because I would not want to change
> >> > > >>>>>>>>> expectations for users who will build their own
> >> > > >>>>>>>>> 3.5 and 3.6
> >> > > >>>>>>>>> versions as they continue to have patch versions
> >> > > >>>>>>>>> released.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> What do you think?
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> Kind Regards,
> >> > > >>>>>>>>> Christopher
> >> > > >>>>>>>>>
> >> > > >>
> >> > > >>
> >> > >
> >> > >
> >> >
> >>
> >

________________________________

CONFIDENTIALITY & PRIVACY NOTICE
This e-mail (including any attachments) is strictly confidential and may also contain privileged information. If you are not the intended recipient you are not authorised to read, print, save, process or disclose this message. If you have received this message by mistake, please inform the sender immediately and destroy this e-mail, its attachments and any copies. Any use, distribution, reproduction or disclosure by any person other than the intended recipient is strictly prohibited and the person responsible may incur in penalties.
The use of this e-mail is only for professional purposes; there is no guarantee that the correspondence towards this e-mail will be read only by the recipient, because, under certain circumstances, there may be a need to access this email by third subjects belonging to the Company.

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Christopher <ct...@apache.org>]

I'm happy that this discussion has been so lively! I just want to
emphasize a few things:

I really do understand the desire to continue to support Java 8... I
get it. But all the conversations around this seem based on what
people are doing *today*. But, ZK 3.7 is *tomorrow's* version... a
*future* release... so it should be based more on reasonable
expectations for users in the future, and less based on what is
happening today. I suspect *most* people today are still using 3.4
anyway (it was just so stable for so long...), but that shouldn't mean
the developers should hold back development on 3.5 and 3.6, any more
than today's users of 3.5/3.6 should hold back 3.7.

Some of the opinions expressed in this discussion seem to propose a
scenario where users are going to be updating to "bleeding edge"
versions of ZooKeeper, but are going to insist on using Java 8.
Personally, I find this to be implausible. In my experience, people
either upgrade everything as soon as they are able to, or they upgrade
each thing individually, only when they are forced to. The first group
will be happy to move to Java 11 and ZK 3.7. The second group will
probably avoid 3.7 anyway, and are fine sticking with 3.6, but if they
had to update to 3.7, they'd also be fine updating to Java 11 if they
had to in order to use 3.7. I can't imagine the scenario where people
are eagerly choosing to upgrade to ZK 3.7, but miserly insisting on
using Java 8. Perhaps that scenario exists, but it's hard for me to
imagine. Even so, my proposal would still support even that group of
people.

I think there are now effectively three proposals being discussed in
this thread:

1. (Christopher's original proposal) passively support Java 8 at
runtime by making JDK 11 the minimum requirement to build and test.
This scenario involves continuing to fix bugs, as they are discovered
and reported, that affect JDK 8, but passively, rather than
proactively. This proposal does *not* drop Java 8 support, but merely
de-emphasizes it in development of what will be 3.7 in the future, and
drops the requirement to do dedicated testing with Java 8. I think
this is low risk, because it is very unlikely that the ZK devs would
introduce a bug that would affect only Java 8 and the compiler
wouldn't catch it... because the cross-compilation features of newer
JDKs are really good.

2. (Enrico's alternate proposal) this variation of my proposal would
involve continuing to proactively support Java 8 by creating a
dedicated testing suite to test client code on Java 8. I think this is
a good option, but since it involves a significantly higher amount of
work than option 1, I think the cost-benefit analysis would show this
to be not worth the effort. Also, if it were implemented, it would
need to be done carefully to avoid requiring developers to have
concurrently installed both Java 8 and Java 11 in order to perform a
build, because requiring Java 8 at build time while developing would
be worse than we have today.

3. (Andor's preference) move to JDK 11 fully. This would provide no
support, passive or active, for Java 8 in ZK 3.7. To be honest, this
is my personal favorite, and is the simplest to implement and
communicate clearly to end users in release notes. The only reason I
proposed a passive support of Java 8 instead of this is because I was
trying to seek a compromise from the start. But, I think by far, this
is the best option for the next *future* release of ZK. If you wanted
to make the change even more visible to users, the version could even
be bumped to 4.0.

If this were to come to a VOTE by the PMC, in order to make a final
decision, I would recommend they vote on option 3, and then if that
fails, vote on option 1, and if that fails, keep things the way they
are (because option 2 is more work).

Christopher

P.S. as for Hadoop on Java 11... I've been running Hadoop 3 on JDK 11
and it works just fine there (as long as you add the missing runtime
jar for javax.activation:javax.activation-api:1.2.0 to its class path,
but that was fixed in Hadoop 3.3).

On Wed, Oct 21, 2020 at 5:42 PM Tamas Penzes
<ta...@cloudera.com.invalid> wrote:
>
> Hi All,
>
> I've just talked with a Hadoop/HDFS developer who told me what I guessed.
> With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would mean
> a release of Hadoop4.
> Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both. As of
> today most of the Hadoop users are still on Hadoop2, he doesn't expect
> Hadoop4 soon.
> As many Apache components depend on Hadoop and ZooKeeper they won't hurry
> to JDK11 until they have to (they will probably go one-by-one very slowly),
> which means if Hadoop stays on JDK8, they would use the last ZK version
> which works on JDK8.
> Do we want a ZK 3.4 again?
>
> Regards, Tamaas
>
>
> On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <ta...@cloudera.com> wrote:
>
> > Hi All,
> >
> > Just to add my two cents.
> >
> > Upgrading to JDK11 looks inevitable sooner or later and I would definitely
> > not wait until 2030 or 2026 when the extended support of JDK8 ends.
> > But on the other side I have to agree with Enrico and Patrick that far too
> > many users are tied to JDK8 yet (not because they want to use JDK8, but
> > because they have to), some of them are components of the Hadoop ecosystem,
> > which would be a loss to tie them to 3.6.x for years.
> > Do we know the state of Hadoop? It builds with JDK8 at the moment, but do
> > we know what are their plans to go to JDK11?
> > When they move, we should move too, but I don't think it would be wise to
> > do it earlier.
> >
> > Christopher's option looks like the golden path, but it needs some
> > investment on the testing side as Enrico pointed it out.
> > Could we agree on it as a compromise?
> >
> > Regards, Tamaas
> >
> > On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com> wrote:
> >
> >> I think I was reacting to Enrico's earlier comment of:
> >>
> >> " ZooKeeper client is used by tons of users and unfortunately many
> >> projects
> >> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to block
> >> users
> >> from the adoption, that is that we will see the world to stay on 3.6.x and
> >> we will have again a long lived release line, like 3.4."
> >>
> >> It's a matter of whether or not a long-lived release line is
> >> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
> >> security/patch-wise (if not feature-wise) for the next N years, then
> >> that's
> >> a potentially valid approach.  I interpreted that comment as "a long-lived
> >> release line is undesirable", but no one explicitly said that, I just read
> >> it that way.
> >>
> >> ~Brent
> >>
> >> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org> wrote:
> >>
> >> > On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org> wrote:
> >> >
> >> > > As far as I know Hbase, Solr and Kafka are already Java 11 ready.
> >> > >
> >> > > IMHO contributors of those projects should also put efforts into
> >> moving
> >> > > forward.
> >> > >
> >> > > We’re not saying that you _have_ to move to Java 11.
> >> > > Staying on Java 8? No problem, 3.6 is for you.
> >> > > Want the fancy new features of 3.7? Work on it on your side too.
> >> > >
> >> > >
> >> > ppl want things like security fixes. I believe the highlighted downside
> >> is
> >> > that we would need to continue to maintain 3.6.x rather than allowing
> >> > users, and ourselves, to focus on trunk for production -"64 percent"
> >> would
> >> > be blocked.
> >> >
> >> > Patrick
> >> >
> >> >
> >> > > Andor
> >> > >
> >> > >
> >> > >
> >> > > > On 2020. Oct 21., at 17:52, Enrico Olivelli <eo...@gmail.com>
> >> > wrote:
> >> > > >
> >> > > > Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
> >> > andor@apache.org>
> >> > > ha
> >> > > > scritto:
> >> > > >
> >> > > >> Correct me if I'm wrong, but Oracle gets paid for extended support.
> >> > > >> Java 8 support until 2030 is not free of charge.
> >> > > >>
> >> > > >> "ZK may end up with a lot of users potentially locking themselves
> >> to
> >> > > >> 3.6.x for a while as Enrico mentioned."
> >> > > >>
> >> > > >> That's true. What's the downside of that which we should invest in
> >> to
> >> > > >> avoid?
> >> > > >>
> >> > > >
> >> > > > I see ZooKeeper used in many many projects, all of the
> >> > > > HBase/Pulsar/Kafka/Solr ecosystem...
> >> > > > they will have to move to JDK11 in order to move to the new ZK
> >> version
> >> > > > so probably they will stay on ZK 3.6
> >> > > >
> >> > > > Probably with Java 17 LTS released the cards will change on the
> >> table
> >> > > >
> >> > > > Enrico
> >> > > >
> >> > > >
> >> > > >
> >> > > >>
> >> > > >> Andor
> >> > > >>
> >> > > >>
> >> > > >>
> >> > > >> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> >> > > >>> As a slightly different consideration, if you look at the
> >> Long-Term
> >> > > >>> Support
> >> > > >>> (LTS) roadmaps for Java, currently Java 8 is set to have full
> >> support
> >> > > >>> until
> >> > > >>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> >> > > >>>
> >> > > >>>
> >> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> >> > > >>> https://en.wikipedia.org/wiki/Java_version_history
> >> > > >>>
> >> > > >>> My guess is that a number of companies are still heavily invested
> >> at
> >> > > >>> the
> >> > > >>> Java 8 level (I know a few) and with that kind of time horizon,
> >> they
> >> > > >>> have
> >> > > >>> no real motivation to upgrade for quite a while.  If the recent
> >> > > >>> Python 2
> >> > > >>> deprecation is anything to go by, they won't do it until they have
> >> > > >>> to.
> >> > > >>>
> >> > > >>> Not saying Java 8 isn't *very* old (2014 release it seems like?)
> >> and
> >> > > >>> I'm
> >> > > >>> not invested heavily either way, but this might suggest that ZK
> >> may
> >> > > >>> end up
> >> > > >>> with a lot of users potentially locking themselves to 3.6.x for a
> >> > > >>> while as
> >> > > >>> Enrico mentioned.
> >> > > >>>
> >> > > >>> (Not a major contributor, but wanted to chime in since I just had
> >> > > >>> this
> >> > > >>> conversation with a bunch of people professionally recently)
> >> > > >>>
> >> > > >>> Brent
> >> > > >>>
> >> > > >>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <an...@apache.org>
> >> > > >>> wrote:
> >> > > >>>
> >> > > >>>> Thanks for the summary.
> >> > > >>>>
> >> > > >>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully. Other
> >> > > >>>> projects
> >> > > >>>> will upgrade once they’re JDK11 compliant, otherwise they will
> >> stay
> >> > > >>>> on 3.5
> >> > > >>>> or 3.6. Both version are quite recent in ZooKeeper-terms, we
> >> > > >>>> already
> >> > > >>>> planned big changes for 3.7.0 and JDK 11 could be one of them.
> >> > > >>>>
> >> > > >>>> Don’t put extra burden on the ZK community to help others staying
> >> > > >>>> on
> >> > > >>>> ancient Java versions.
> >> > > >>>>
> >> > > >>>> Andor
> >> > > >>>>
> >> > > >>>>
> >> > > >>>>
> >> > > >>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
> >> eolivelli@gmail.com>
> >> > > >>>>> wrote:
> >> > > >>>>>
> >> > > >>>>> Let me recap
> >> > > >>>>> - Christopher is proposing to move to JDK11
> >> > > >>>>> - ZooKeeper client and server are bundled and coded and tested
> >> > > >>>>> together
> >> > > >>>> in
> >> > > >>>>> zookeeper-server
> >> > > >>>>> - Enrico is concerned about the need of testing ZooKeeper client
> >> > > >>>>> on JDK8
> >> > > >>>>> (not a problem to move the server to JDK11)
> >> > > >>>>>
> >> > > >>>>> ZooKeeper client is used by tons of users and unfortunately many
> >> > > >>>>> projects
> >> > > >>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
> >> > > >>>>> block
> >> > > >>>> users
> >> > > >>>>> from the adoption,
> >> > > >>>>> that is that we will see the world to stay on 3.6.x and we will
> >> > > >>>>> have
> >> > > >>>> again
> >> > > >>>>> a long lived release line, like 3.4.
> >> > > >>>>>
> >> > > >>>>> Testing the client on JDK8 would be possible if we create some
> >> > > >>>>> kind of
> >> > > >>>>> additional module with system tests, then we can start the
> >> server
> >> > > >>>>> on
> >> > > >>>> docker
> >> > > >>>>> on JDK11+ and start a client on JDK8
> >> > > >>>>> with Maven toolchain it should possible to run surefire tests
> >> > > >>>>> using a
> >> > > >>>>> separate JVM.
> >> > > >>>>>
> >> > > >>>>> So in my vision 2 options:
> >> > > >>>>> 1) fully JDK11 - drop JDK8 at all
> >> > > >>>>> 2) build with JDK11 - server only on JDK11 - add system tests
> >> > > >>>>> with docker
> >> > > >>>>> and toolchains that ensure the ZooKeeper client (and all
> >> > > >>>>> dependencies)
> >> > > >>>>> still work on JDK8
> >> > > >>>>>
> >> > > >>>>> From my point of view about the ZooKeeper ecosystem option 2)
> >> > > >>>>> will be far
> >> > > >>>>> better, but we need resources to work on a new test suite.
> >> > > >>>>>
> >> > > >>>>> Enrico
> >> > > >>>>>
> >> > > >>>>>
> >> > > >>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> >> > > >>>>> andor@apache.org>
> >> > > >>>> ha
> >> > > >>>>> scritto:
> >> > > >>>>>
> >> > > >>>>>> Tamas, Enrico,
> >> > > >>>>>>
> >> > > >>>>>> Sorry I don’t follow. Why do we have to test the client with
> >> > > >>>>>> JDK 8 in
> >> > > >>>>>> version 3.7.0?
> >> > > >>>>>>
> >> > > >>>>>> Andor
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>>
> >> > > >>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
> >> > > >>>>>>> tamaas@cloudera.com.INVALID>
> >> > > >>>>>> wrote:
> >> > > >>>>>>> Hi Enrico,
> >> > > >>>>>>>
> >> > > >>>>>>> Separating ZooKeeper client and server is a huge work, but we
> >> > > >>>>>>> might not
> >> > > >>>>>>> need it.
> >> > > >>>>>>> As you mentioned we have to test ZK client with Java 8, what
> >> > > >>>>>>> about
> >> > > >>>>>>> separating only the test cases which we need to run with
> >> > > >>>>>>> Java8 too?
> >> > > >>>>>>> In Curator we have the ZK compatibility tests where we run a
> >> > > >>>>>>> limited
> >> > > >>>>>> amount
> >> > > >>>>>>> of Curator's jUnit tests with a different ZK version.
> >> > > >>>>>>> We might be able to do the same here, tag tests which are
> >> > > >>>>>>> testing ZK
> >> > > >>>>>> client
> >> > > >>>>>>> and run them separately with Java 8. The only limitation is
> >> > > >>>>>>> that these
> >> > > >>>>>>> tests must stay JDK8 compatible.
> >> > > >>>>>>> But from the tags we will see which ones are those.
> >> > > >>>>>>>
> >> > > >>>>>>> What do you think?
> >> > > >>>>>>>
> >> > > >>>>>>> Regards, Tamaas
> >> > > >>>>>>>
> >> > > >>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> >> > > >>>>>>> eolivelli@gmail.com>
> >> > > >>>>>> wrote:
> >> > > >>>>>>>> Christopher
> >> > > >>>>>>>> I appreciate your idea and I also moved lots of my projects
> >> > > >>>>>>>> to work
> >> > > >>>> the
> >> > > >>>>>> way
> >> > > >>>>>>>> you are suggesting.
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>> We must run tests using real jdk8 to test the Zookeeper
> >> > > >>>>>>>> client. We
> >> > > >>>> must
> >> > > >>>>>>>> ensure that Zookeeper works well, especially while dealing
> >> > > >>>>>>>> with
> >> > > >>>> security
> >> > > >>>>>>>> stuff.
> >> > > >>>>>>>> Currently the client is in the same module of the server
> >> > > >>>>>>>> and it will
> >> > > >>>>>> take a
> >> > > >>>>>>>> good (huge) amount of work to separate them
> >> > > >>>>>>>>
> >> > > >>>>>>>>
> >> > > >>>>>>>> Enrico
> >> > > >>>>>>>>
> >> > > >>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org>
> >> > > >>>>>>>> ha
> >> > > >>>> scritto:
> >> > > >>>>>>>>> Hi ZK Devs,
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> With recent advancements in Java (since Java 9), it is
> >> > > >>>>>>>>> now generally
> >> > > >>>>>>>>> no longer necessary to require that software be developed
> >> > > >>>>>>>>> on an older
> >> > > >>>>>>>>> JDK in order to have confidence that it will run on the
> >> > > >>>>>>>>> older version
> >> > > >>>>>>>>> of Java. This is because, as of Java 9, all JDK releases
> >> > > >>>>>>>>> have better
> >> > > >>>>>>>>> support for cross-compilation to older Java versions.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> What this means is that developers can confidently make
> >> > > >>>>>>>>> the build
> >> > > >>>>>>>>> requirements for a project higher than the Java version
> >> > > >>>>>>>>> that will
> >> > > >>>>>>>>> actually be supported at runtime.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> In fact, ZooKeeper already supports the necessary flags
> >> > > >>>>>>>>> in its Maven
> >> > > >>>>>>>>> build configuration to ensure that it uses JDK 8
> >> > > >>>>>>>>> compliance when
> >> > > >>>>>>>>> building on a newer JDK (I added this way back in
> >> > > >>>>>>>>> ZOOKEEPER-3739 /
> >> > > >>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> So, I propose that we make JDK 11 the new minimum version
> >> > > >>>>>>>>> to *build*
> >> > > >>>>>>>>> ZooKeeper with. This would not change the runtime
> >> > > >>>>>>>>> requirement, which
> >> > > >>>>>>>>> would remain at JDK 8.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> The only necessary change to make this happen would be to
> >> > > >>>>>>>>> add the
> >> > > >>>>>>>>> minimum Java version to the maven-enforcer-plugin (like
> >> > > >>>>>>>>>
> >> > > >>>>>>>>>
> >> > > >>>>
> >> > > >>
> >> > >
> >> >
> >> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> >> > > >>>>>>>>> )
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> This would allow ZooKeeper to to streamline its
> >> > > >>>>>>>>> development process a
> >> > > >>>>>>>>> little bit by reducing the amount of CI testing that is
> >> > > >>>>>>>>> done as part
> >> > > >>>>>>>>> of the build. In other words, we can drop the CI builds
> >> > > >>>>>>>>> for JDK 8,
> >> > > >>>>>>>>> which saves on build resources and time. The return on
> >> > > >>>>>>>>> investment is
> >> > > >>>>>>>>> so low for the JDK 8 builds anyway, because of the
> >> > > >>>>>>>>> improved
> >> > > >>>>>>>>> cross-compilation in newer JDKs. So, there's not much
> >> > > >>>>>>>>> value in
> >> > > >>>>>>>>> building on JDK 8 anyway.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> Of course, I am only recommending this for *new* release
> >> > > >>>>>>>>> lines,
> >> > > >>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because I
> >> > > >>>>>>>>> would not want
> >> > > >>>>>>>>> to change expectations for users who will build their own
> >> > > >>>>>>>>> 3.5 and 3.6
> >> > > >>>>>>>>> versions as they continue to have patch versions
> >> > > >>>>>>>>> released.
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> What do you think?
> >> > > >>>>>>>>>
> >> > > >>>>>>>>> Kind Regards,
> >> > > >>>>>>>>> Christopher
> >> > > >>>>>>>>>
> >> > > >>
> >> > > >>
> >> > >
> >> > >
> >> >
> >>
> >

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Tamas Penzes <ta...@cloudera.com.INVALID>]

Hi All,

I've just talked with a Hadoop/HDFS developer who told me what I guessed.
With Hadoop3 they have just dropped JDK7 support, dropping JDK8 would mean
a release of Hadoop4.
Since HADOOP-15338 is finished, they test with JDK8 and JDK11 both. As of
today most of the Hadoop users are still on Hadoop2, he doesn't expect
Hadoop4 soon.
As many Apache components depend on Hadoop and ZooKeeper they won't hurry
to JDK11 until they have to (they will probably go one-by-one very slowly),
which means if Hadoop stays on JDK8, they would use the last ZK version
which works on JDK8.
Do we want a ZK 3.4 again?

Regards, Tamaas


On Wed, Oct 21, 2020 at 11:23 PM Tamas Penzes <ta...@cloudera.com> wrote:

> Hi All,
>
> Just to add my two cents.
>
> Upgrading to JDK11 looks inevitable sooner or later and I would definitely
> not wait until 2030 or 2026 when the extended support of JDK8 ends.
> But on the other side I have to agree with Enrico and Patrick that far too
> many users are tied to JDK8 yet (not because they want to use JDK8, but
> because they have to), some of them are components of the Hadoop ecosystem,
> which would be a loss to tie them to 3.6.x for years.
> Do we know the state of Hadoop? It builds with JDK8 at the moment, but do
> we know what are their plans to go to JDK11?
> When they move, we should move too, but I don't think it would be wise to
> do it earlier.
>
> Christopher's option looks like the golden path, but it needs some
> investment on the testing side as Enrico pointed it out.
> Could we agree on it as a compromise?
>
> Regards, Tamaas
>
> On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com> wrote:
>
>> I think I was reacting to Enrico's earlier comment of:
>>
>> " ZooKeeper client is used by tons of users and unfortunately many
>> projects
>> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to block
>> users
>> from the adoption, that is that we will see the world to stay on 3.6.x and
>> we will have again a long lived release line, like 3.4."
>>
>> It's a matter of whether or not a long-lived release line is
>> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
>> security/patch-wise (if not feature-wise) for the next N years, then
>> that's
>> a potentially valid approach.  I interpreted that comment as "a long-lived
>> release line is undesirable", but no one explicitly said that, I just read
>> it that way.
>>
>> ~Brent
>>
>> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org> wrote:
>>
>> > On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org> wrote:
>> >
>> > > As far as I know Hbase, Solr and Kafka are already Java 11 ready.
>> > >
>> > > IMHO contributors of those projects should also put efforts into
>> moving
>> > > forward.
>> > >
>> > > We’re not saying that you _have_ to move to Java 11.
>> > > Staying on Java 8? No problem, 3.6 is for you.
>> > > Want the fancy new features of 3.7? Work on it on your side too.
>> > >
>> > >
>> > ppl want things like security fixes. I believe the highlighted downside
>> is
>> > that we would need to continue to maintain 3.6.x rather than allowing
>> > users, and ourselves, to focus on trunk for production -"64 percent"
>> would
>> > be blocked.
>> >
>> > Patrick
>> >
>> >
>> > > Andor
>> > >
>> > >
>> > >
>> > > > On 2020. Oct 21., at 17:52, Enrico Olivelli <eo...@gmail.com>
>> > wrote:
>> > > >
>> > > > Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
>> > andor@apache.org>
>> > > ha
>> > > > scritto:
>> > > >
>> > > >> Correct me if I'm wrong, but Oracle gets paid for extended support.
>> > > >> Java 8 support until 2030 is not free of charge.
>> > > >>
>> > > >> "ZK may end up with a lot of users potentially locking themselves
>> to
>> > > >> 3.6.x for a while as Enrico mentioned."
>> > > >>
>> > > >> That's true. What's the downside of that which we should invest in
>> to
>> > > >> avoid?
>> > > >>
>> > > >
>> > > > I see ZooKeeper used in many many projects, all of the
>> > > > HBase/Pulsar/Kafka/Solr ecosystem...
>> > > > they will have to move to JDK11 in order to move to the new ZK
>> version
>> > > > so probably they will stay on ZK 3.6
>> > > >
>> > > > Probably with Java 17 LTS released the cards will change on the
>> table
>> > > >
>> > > > Enrico
>> > > >
>> > > >
>> > > >
>> > > >>
>> > > >> Andor
>> > > >>
>> > > >>
>> > > >>
>> > > >> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
>> > > >>> As a slightly different consideration, if you look at the
>> Long-Term
>> > > >>> Support
>> > > >>> (LTS) roadmaps for Java, currently Java 8 is set to have full
>> support
>> > > >>> until
>> > > >>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
>> > > >>>
>> > > >>>
>> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>> > > >>> https://en.wikipedia.org/wiki/Java_version_history
>> > > >>>
>> > > >>> My guess is that a number of companies are still heavily invested
>> at
>> > > >>> the
>> > > >>> Java 8 level (I know a few) and with that kind of time horizon,
>> they
>> > > >>> have
>> > > >>> no real motivation to upgrade for quite a while.  If the recent
>> > > >>> Python 2
>> > > >>> deprecation is anything to go by, they won't do it until they have
>> > > >>> to.
>> > > >>>
>> > > >>> Not saying Java 8 isn't *very* old (2014 release it seems like?)
>> and
>> > > >>> I'm
>> > > >>> not invested heavily either way, but this might suggest that ZK
>> may
>> > > >>> end up
>> > > >>> with a lot of users potentially locking themselves to 3.6.x for a
>> > > >>> while as
>> > > >>> Enrico mentioned.
>> > > >>>
>> > > >>> (Not a major contributor, but wanted to chime in since I just had
>> > > >>> this
>> > > >>> conversation with a bunch of people professionally recently)
>> > > >>>
>> > > >>> Brent
>> > > >>>
>> > > >>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <an...@apache.org>
>> > > >>> wrote:
>> > > >>>
>> > > >>>> Thanks for the summary.
>> > > >>>>
>> > > >>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully. Other
>> > > >>>> projects
>> > > >>>> will upgrade once they’re JDK11 compliant, otherwise they will
>> stay
>> > > >>>> on 3.5
>> > > >>>> or 3.6. Both version are quite recent in ZooKeeper-terms, we
>> > > >>>> already
>> > > >>>> planned big changes for 3.7.0 and JDK 11 could be one of them.
>> > > >>>>
>> > > >>>> Don’t put extra burden on the ZK community to help others staying
>> > > >>>> on
>> > > >>>> ancient Java versions.
>> > > >>>>
>> > > >>>> Andor
>> > > >>>>
>> > > >>>>
>> > > >>>>
>> > > >>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <
>> eolivelli@gmail.com>
>> > > >>>>> wrote:
>> > > >>>>>
>> > > >>>>> Let me recap
>> > > >>>>> - Christopher is proposing to move to JDK11
>> > > >>>>> - ZooKeeper client and server are bundled and coded and tested
>> > > >>>>> together
>> > > >>>> in
>> > > >>>>> zookeeper-server
>> > > >>>>> - Enrico is concerned about the need of testing ZooKeeper client
>> > > >>>>> on JDK8
>> > > >>>>> (not a problem to move the server to JDK11)
>> > > >>>>>
>> > > >>>>> ZooKeeper client is used by tons of users and unfortunately many
>> > > >>>>> projects
>> > > >>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
>> > > >>>>> block
>> > > >>>> users
>> > > >>>>> from the adoption,
>> > > >>>>> that is that we will see the world to stay on 3.6.x and we will
>> > > >>>>> have
>> > > >>>> again
>> > > >>>>> a long lived release line, like 3.4.
>> > > >>>>>
>> > > >>>>> Testing the client on JDK8 would be possible if we create some
>> > > >>>>> kind of
>> > > >>>>> additional module with system tests, then we can start the
>> server
>> > > >>>>> on
>> > > >>>> docker
>> > > >>>>> on JDK11+ and start a client on JDK8
>> > > >>>>> with Maven toolchain it should possible to run surefire tests
>> > > >>>>> using a
>> > > >>>>> separate JVM.
>> > > >>>>>
>> > > >>>>> So in my vision 2 options:
>> > > >>>>> 1) fully JDK11 - drop JDK8 at all
>> > > >>>>> 2) build with JDK11 - server only on JDK11 - add system tests
>> > > >>>>> with docker
>> > > >>>>> and toolchains that ensure the ZooKeeper client (and all
>> > > >>>>> dependencies)
>> > > >>>>> still work on JDK8
>> > > >>>>>
>> > > >>>>> From my point of view about the ZooKeeper ecosystem option 2)
>> > > >>>>> will be far
>> > > >>>>> better, but we need resources to work on a new test suite.
>> > > >>>>>
>> > > >>>>> Enrico
>> > > >>>>>
>> > > >>>>>
>> > > >>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
>> > > >>>>> andor@apache.org>
>> > > >>>> ha
>> > > >>>>> scritto:
>> > > >>>>>
>> > > >>>>>> Tamas, Enrico,
>> > > >>>>>>
>> > > >>>>>> Sorry I don’t follow. Why do we have to test the client with
>> > > >>>>>> JDK 8 in
>> > > >>>>>> version 3.7.0?
>> > > >>>>>>
>> > > >>>>>> Andor
>> > > >>>>>>
>> > > >>>>>>
>> > > >>>>>>
>> > > >>>>>>
>> > > >>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
>> > > >>>>>>> tamaas@cloudera.com.INVALID>
>> > > >>>>>> wrote:
>> > > >>>>>>> Hi Enrico,
>> > > >>>>>>>
>> > > >>>>>>> Separating ZooKeeper client and server is a huge work, but we
>> > > >>>>>>> might not
>> > > >>>>>>> need it.
>> > > >>>>>>> As you mentioned we have to test ZK client with Java 8, what
>> > > >>>>>>> about
>> > > >>>>>>> separating only the test cases which we need to run with
>> > > >>>>>>> Java8 too?
>> > > >>>>>>> In Curator we have the ZK compatibility tests where we run a
>> > > >>>>>>> limited
>> > > >>>>>> amount
>> > > >>>>>>> of Curator's jUnit tests with a different ZK version.
>> > > >>>>>>> We might be able to do the same here, tag tests which are
>> > > >>>>>>> testing ZK
>> > > >>>>>> client
>> > > >>>>>>> and run them separately with Java 8. The only limitation is
>> > > >>>>>>> that these
>> > > >>>>>>> tests must stay JDK8 compatible.
>> > > >>>>>>> But from the tags we will see which ones are those.
>> > > >>>>>>>
>> > > >>>>>>> What do you think?
>> > > >>>>>>>
>> > > >>>>>>> Regards, Tamaas
>> > > >>>>>>>
>> > > >>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
>> > > >>>>>>> eolivelli@gmail.com>
>> > > >>>>>> wrote:
>> > > >>>>>>>> Christopher
>> > > >>>>>>>> I appreciate your idea and I also moved lots of my projects
>> > > >>>>>>>> to work
>> > > >>>> the
>> > > >>>>>> way
>> > > >>>>>>>> you are suggesting.
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>> We must run tests using real jdk8 to test the Zookeeper
>> > > >>>>>>>> client. We
>> > > >>>> must
>> > > >>>>>>>> ensure that Zookeeper works well, especially while dealing
>> > > >>>>>>>> with
>> > > >>>> security
>> > > >>>>>>>> stuff.
>> > > >>>>>>>> Currently the client is in the same module of the server
>> > > >>>>>>>> and it will
>> > > >>>>>> take a
>> > > >>>>>>>> good (huge) amount of work to separate them
>> > > >>>>>>>>
>> > > >>>>>>>>
>> > > >>>>>>>> Enrico
>> > > >>>>>>>>
>> > > >>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org>
>> > > >>>>>>>> ha
>> > > >>>> scritto:
>> > > >>>>>>>>> Hi ZK Devs,
>> > > >>>>>>>>>
>> > > >>>>>>>>> With recent advancements in Java (since Java 9), it is
>> > > >>>>>>>>> now generally
>> > > >>>>>>>>> no longer necessary to require that software be developed
>> > > >>>>>>>>> on an older
>> > > >>>>>>>>> JDK in order to have confidence that it will run on the
>> > > >>>>>>>>> older version
>> > > >>>>>>>>> of Java. This is because, as of Java 9, all JDK releases
>> > > >>>>>>>>> have better
>> > > >>>>>>>>> support for cross-compilation to older Java versions.
>> > > >>>>>>>>>
>> > > >>>>>>>>> What this means is that developers can confidently make
>> > > >>>>>>>>> the build
>> > > >>>>>>>>> requirements for a project higher than the Java version
>> > > >>>>>>>>> that will
>> > > >>>>>>>>> actually be supported at runtime.
>> > > >>>>>>>>>
>> > > >>>>>>>>> In fact, ZooKeeper already supports the necessary flags
>> > > >>>>>>>>> in its Maven
>> > > >>>>>>>>> build configuration to ensure that it uses JDK 8
>> > > >>>>>>>>> compliance when
>> > > >>>>>>>>> building on a newer JDK (I added this way back in
>> > > >>>>>>>>> ZOOKEEPER-3739 /
>> > > >>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
>> > > >>>>>>>>>
>> > > >>>>>>>>> So, I propose that we make JDK 11 the new minimum version
>> > > >>>>>>>>> to *build*
>> > > >>>>>>>>> ZooKeeper with. This would not change the runtime
>> > > >>>>>>>>> requirement, which
>> > > >>>>>>>>> would remain at JDK 8.
>> > > >>>>>>>>>
>> > > >>>>>>>>> The only necessary change to make this happen would be to
>> > > >>>>>>>>> add the
>> > > >>>>>>>>> minimum Java version to the maven-enforcer-plugin (like
>> > > >>>>>>>>>
>> > > >>>>>>>>>
>> > > >>>>
>> > > >>
>> > >
>> >
>> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
>> > > >>>>>>>>> )
>> > > >>>>>>>>>
>> > > >>>>>>>>> This would allow ZooKeeper to to streamline its
>> > > >>>>>>>>> development process a
>> > > >>>>>>>>> little bit by reducing the amount of CI testing that is
>> > > >>>>>>>>> done as part
>> > > >>>>>>>>> of the build. In other words, we can drop the CI builds
>> > > >>>>>>>>> for JDK 8,
>> > > >>>>>>>>> which saves on build resources and time. The return on
>> > > >>>>>>>>> investment is
>> > > >>>>>>>>> so low for the JDK 8 builds anyway, because of the
>> > > >>>>>>>>> improved
>> > > >>>>>>>>> cross-compilation in newer JDKs. So, there's not much
>> > > >>>>>>>>> value in
>> > > >>>>>>>>> building on JDK 8 anyway.
>> > > >>>>>>>>>
>> > > >>>>>>>>> Of course, I am only recommending this for *new* release
>> > > >>>>>>>>> lines,
>> > > >>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because I
>> > > >>>>>>>>> would not want
>> > > >>>>>>>>> to change expectations for users who will build their own
>> > > >>>>>>>>> 3.5 and 3.6
>> > > >>>>>>>>> versions as they continue to have patch versions
>> > > >>>>>>>>> released.
>> > > >>>>>>>>>
>> > > >>>>>>>>> What do you think?
>> > > >>>>>>>>>
>> > > >>>>>>>>> Kind Regards,
>> > > >>>>>>>>> Christopher
>> > > >>>>>>>>>
>> > > >>
>> > > >>
>> > >
>> > >
>> >
>>
>

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Tamas Penzes <ta...@cloudera.com.INVALID>]

Hi All,

Just to add my two cents.

Upgrading to JDK11 looks inevitable sooner or later and I would definitely
not wait until 2030 or 2026 when the extended support of JDK8 ends.
But on the other side I have to agree with Enrico and Patrick that far too
many users are tied to JDK8 yet (not because they want to use JDK8, but
because they have to), some of them are components of the Hadoop ecosystem,
which would be a loss to tie them to 3.6.x for years.
Do we know the state of Hadoop? It builds with JDK8 at the moment, but do
we know what are their plans to go to JDK11?
When they move, we should move too, but I don't think it would be wise to
do it earlier.

Christopher's option looks like the golden path, but it needs some
investment on the testing side as Enrico pointed it out.
Could we agree on it as a compromise?

Regards, Tamaas

On Wed, Oct 21, 2020 at 7:11 PM Brent <br...@gmail.com> wrote:

> I think I was reacting to Enrico's earlier comment of:
>
> " ZooKeeper client is used by tons of users and unfortunately many projects
> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to block users
> from the adoption, that is that we will see the world to stay on 3.6.x and
> we will have again a long lived release line, like 3.4."
>
> It's a matter of whether or not a long-lived release line is
> desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
> security/patch-wise (if not feature-wise) for the next N years, then that's
> a potentially valid approach.  I interpreted that comment as "a long-lived
> release line is undesirable", but no one explicitly said that, I just read
> it that way.
>
> ~Brent
>
> On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org> wrote:
>
> > On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org> wrote:
> >
> > > As far as I know Hbase, Solr and Kafka are already Java 11 ready.
> > >
> > > IMHO contributors of those projects should also put efforts into moving
> > > forward.
> > >
> > > We’re not saying that you _have_ to move to Java 11.
> > > Staying on Java 8? No problem, 3.6 is for you.
> > > Want the fancy new features of 3.7? Work on it on your side too.
> > >
> > >
> > ppl want things like security fixes. I believe the highlighted downside
> is
> > that we would need to continue to maintain 3.6.x rather than allowing
> > users, and ourselves, to focus on trunk for production -"64 percent"
> would
> > be blocked.
> >
> > Patrick
> >
> >
> > > Andor
> > >
> > >
> > >
> > > > On 2020. Oct 21., at 17:52, Enrico Olivelli <eo...@gmail.com>
> > wrote:
> > > >
> > > > Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
> > andor@apache.org>
> > > ha
> > > > scritto:
> > > >
> > > >> Correct me if I'm wrong, but Oracle gets paid for extended support.
> > > >> Java 8 support until 2030 is not free of charge.
> > > >>
> > > >> "ZK may end up with a lot of users potentially locking themselves to
> > > >> 3.6.x for a while as Enrico mentioned."
> > > >>
> > > >> That's true. What's the downside of that which we should invest in
> to
> > > >> avoid?
> > > >>
> > > >
> > > > I see ZooKeeper used in many many projects, all of the
> > > > HBase/Pulsar/Kafka/Solr ecosystem...
> > > > they will have to move to JDK11 in order to move to the new ZK
> version
> > > > so probably they will stay on ZK 3.6
> > > >
> > > > Probably with Java 17 LTS released the cards will change on the table
> > > >
> > > > Enrico
> > > >
> > > >
> > > >
> > > >>
> > > >> Andor
> > > >>
> > > >>
> > > >>
> > > >> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> > > >>> As a slightly different consideration, if you look at the Long-Term
> > > >>> Support
> > > >>> (LTS) roadmaps for Java, currently Java 8 is set to have full
> support
> > > >>> until
> > > >>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> > > >>>
> > > >>>
> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> > > >>> https://en.wikipedia.org/wiki/Java_version_history
> > > >>>
> > > >>> My guess is that a number of companies are still heavily invested
> at
> > > >>> the
> > > >>> Java 8 level (I know a few) and with that kind of time horizon,
> they
> > > >>> have
> > > >>> no real motivation to upgrade for quite a while.  If the recent
> > > >>> Python 2
> > > >>> deprecation is anything to go by, they won't do it until they have
> > > >>> to.
> > > >>>
> > > >>> Not saying Java 8 isn't *very* old (2014 release it seems like?)
> and
> > > >>> I'm
> > > >>> not invested heavily either way, but this might suggest that ZK may
> > > >>> end up
> > > >>> with a lot of users potentially locking themselves to 3.6.x for a
> > > >>> while as
> > > >>> Enrico mentioned.
> > > >>>
> > > >>> (Not a major contributor, but wanted to chime in since I just had
> > > >>> this
> > > >>> conversation with a bunch of people professionally recently)
> > > >>>
> > > >>> Brent
> > > >>>
> > > >>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <an...@apache.org>
> > > >>> wrote:
> > > >>>
> > > >>>> Thanks for the summary.
> > > >>>>
> > > >>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully. Other
> > > >>>> projects
> > > >>>> will upgrade once they’re JDK11 compliant, otherwise they will
> stay
> > > >>>> on 3.5
> > > >>>> or 3.6. Both version are quite recent in ZooKeeper-terms, we
> > > >>>> already
> > > >>>> planned big changes for 3.7.0 and JDK 11 could be one of them.
> > > >>>>
> > > >>>> Don’t put extra burden on the ZK community to help others staying
> > > >>>> on
> > > >>>> ancient Java versions.
> > > >>>>
> > > >>>> Andor
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <eolivelli@gmail.com
> >
> > > >>>>> wrote:
> > > >>>>>
> > > >>>>> Let me recap
> > > >>>>> - Christopher is proposing to move to JDK11
> > > >>>>> - ZooKeeper client and server are bundled and coded and tested
> > > >>>>> together
> > > >>>> in
> > > >>>>> zookeeper-server
> > > >>>>> - Enrico is concerned about the need of testing ZooKeeper client
> > > >>>>> on JDK8
> > > >>>>> (not a problem to move the server to JDK11)
> > > >>>>>
> > > >>>>> ZooKeeper client is used by tons of users and unfortunately many
> > > >>>>> projects
> > > >>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
> > > >>>>> block
> > > >>>> users
> > > >>>>> from the adoption,
> > > >>>>> that is that we will see the world to stay on 3.6.x and we will
> > > >>>>> have
> > > >>>> again
> > > >>>>> a long lived release line, like 3.4.
> > > >>>>>
> > > >>>>> Testing the client on JDK8 would be possible if we create some
> > > >>>>> kind of
> > > >>>>> additional module with system tests, then we can start the server
> > > >>>>> on
> > > >>>> docker
> > > >>>>> on JDK11+ and start a client on JDK8
> > > >>>>> with Maven toolchain it should possible to run surefire tests
> > > >>>>> using a
> > > >>>>> separate JVM.
> > > >>>>>
> > > >>>>> So in my vision 2 options:
> > > >>>>> 1) fully JDK11 - drop JDK8 at all
> > > >>>>> 2) build with JDK11 - server only on JDK11 - add system tests
> > > >>>>> with docker
> > > >>>>> and toolchains that ensure the ZooKeeper client (and all
> > > >>>>> dependencies)
> > > >>>>> still work on JDK8
> > > >>>>>
> > > >>>>> From my point of view about the ZooKeeper ecosystem option 2)
> > > >>>>> will be far
> > > >>>>> better, but we need resources to work on a new test suite.
> > > >>>>>
> > > >>>>> Enrico
> > > >>>>>
> > > >>>>>
> > > >>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> > > >>>>> andor@apache.org>
> > > >>>> ha
> > > >>>>> scritto:
> > > >>>>>
> > > >>>>>> Tamas, Enrico,
> > > >>>>>>
> > > >>>>>> Sorry I don’t follow. Why do we have to test the client with
> > > >>>>>> JDK 8 in
> > > >>>>>> version 3.7.0?
> > > >>>>>>
> > > >>>>>> Andor
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>
> > > >>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
> > > >>>>>>> tamaas@cloudera.com.INVALID>
> > > >>>>>> wrote:
> > > >>>>>>> Hi Enrico,
> > > >>>>>>>
> > > >>>>>>> Separating ZooKeeper client and server is a huge work, but we
> > > >>>>>>> might not
> > > >>>>>>> need it.
> > > >>>>>>> As you mentioned we have to test ZK client with Java 8, what
> > > >>>>>>> about
> > > >>>>>>> separating only the test cases which we need to run with
> > > >>>>>>> Java8 too?
> > > >>>>>>> In Curator we have the ZK compatibility tests where we run a
> > > >>>>>>> limited
> > > >>>>>> amount
> > > >>>>>>> of Curator's jUnit tests with a different ZK version.
> > > >>>>>>> We might be able to do the same here, tag tests which are
> > > >>>>>>> testing ZK
> > > >>>>>> client
> > > >>>>>>> and run them separately with Java 8. The only limitation is
> > > >>>>>>> that these
> > > >>>>>>> tests must stay JDK8 compatible.
> > > >>>>>>> But from the tags we will see which ones are those.
> > > >>>>>>>
> > > >>>>>>> What do you think?
> > > >>>>>>>
> > > >>>>>>> Regards, Tamaas
> > > >>>>>>>
> > > >>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> > > >>>>>>> eolivelli@gmail.com>
> > > >>>>>> wrote:
> > > >>>>>>>> Christopher
> > > >>>>>>>> I appreciate your idea and I also moved lots of my projects
> > > >>>>>>>> to work
> > > >>>> the
> > > >>>>>> way
> > > >>>>>>>> you are suggesting.
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> We must run tests using real jdk8 to test the Zookeeper
> > > >>>>>>>> client. We
> > > >>>> must
> > > >>>>>>>> ensure that Zookeeper works well, especially while dealing
> > > >>>>>>>> with
> > > >>>> security
> > > >>>>>>>> stuff.
> > > >>>>>>>> Currently the client is in the same module of the server
> > > >>>>>>>> and it will
> > > >>>>>> take a
> > > >>>>>>>> good (huge) amount of work to separate them
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> Enrico
> > > >>>>>>>>
> > > >>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org>
> > > >>>>>>>> ha
> > > >>>> scritto:
> > > >>>>>>>>> Hi ZK Devs,
> > > >>>>>>>>>
> > > >>>>>>>>> With recent advancements in Java (since Java 9), it is
> > > >>>>>>>>> now generally
> > > >>>>>>>>> no longer necessary to require that software be developed
> > > >>>>>>>>> on an older
> > > >>>>>>>>> JDK in order to have confidence that it will run on the
> > > >>>>>>>>> older version
> > > >>>>>>>>> of Java. This is because, as of Java 9, all JDK releases
> > > >>>>>>>>> have better
> > > >>>>>>>>> support for cross-compilation to older Java versions.
> > > >>>>>>>>>
> > > >>>>>>>>> What this means is that developers can confidently make
> > > >>>>>>>>> the build
> > > >>>>>>>>> requirements for a project higher than the Java version
> > > >>>>>>>>> that will
> > > >>>>>>>>> actually be supported at runtime.
> > > >>>>>>>>>
> > > >>>>>>>>> In fact, ZooKeeper already supports the necessary flags
> > > >>>>>>>>> in its Maven
> > > >>>>>>>>> build configuration to ensure that it uses JDK 8
> > > >>>>>>>>> compliance when
> > > >>>>>>>>> building on a newer JDK (I added this way back in
> > > >>>>>>>>> ZOOKEEPER-3739 /
> > > >>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
> > > >>>>>>>>>
> > > >>>>>>>>> So, I propose that we make JDK 11 the new minimum version
> > > >>>>>>>>> to *build*
> > > >>>>>>>>> ZooKeeper with. This would not change the runtime
> > > >>>>>>>>> requirement, which
> > > >>>>>>>>> would remain at JDK 8.
> > > >>>>>>>>>
> > > >>>>>>>>> The only necessary change to make this happen would be to
> > > >>>>>>>>> add the
> > > >>>>>>>>> minimum Java version to the maven-enforcer-plugin (like
> > > >>>>>>>>>
> > > >>>>>>>>>
> > > >>>>
> > > >>
> > >
> >
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> > > >>>>>>>>> )
> > > >>>>>>>>>
> > > >>>>>>>>> This would allow ZooKeeper to to streamline its
> > > >>>>>>>>> development process a
> > > >>>>>>>>> little bit by reducing the amount of CI testing that is
> > > >>>>>>>>> done as part
> > > >>>>>>>>> of the build. In other words, we can drop the CI builds
> > > >>>>>>>>> for JDK 8,
> > > >>>>>>>>> which saves on build resources and time. The return on
> > > >>>>>>>>> investment is
> > > >>>>>>>>> so low for the JDK 8 builds anyway, because of the
> > > >>>>>>>>> improved
> > > >>>>>>>>> cross-compilation in newer JDKs. So, there's not much
> > > >>>>>>>>> value in
> > > >>>>>>>>> building on JDK 8 anyway.
> > > >>>>>>>>>
> > > >>>>>>>>> Of course, I am only recommending this for *new* release
> > > >>>>>>>>> lines,
> > > >>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because I
> > > >>>>>>>>> would not want
> > > >>>>>>>>> to change expectations for users who will build their own
> > > >>>>>>>>> 3.5 and 3.6
> > > >>>>>>>>> versions as they continue to have patch versions
> > > >>>>>>>>> released.
> > > >>>>>>>>>
> > > >>>>>>>>> What do you think?
> > > >>>>>>>>>
> > > >>>>>>>>> Kind Regards,
> > > >>>>>>>>> Christopher
> > > >>>>>>>>>
> > > >>
> > > >>
> > >
> > >
> >
>

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Brent <br...@gmail.com>]

I think I was reacting to Enrico's earlier comment of:

" ZooKeeper client is used by tons of users and unfortunately many projects
are still on JDK8, if we move ZooKeeper to JDK11 the risk is to block users
from the adoption, that is that we will see the world to stay on 3.6.x and
we will have again a long lived release line, like 3.4."

It's a matter of whether or not a long-lived release line is
desirable/undesirable.  If everyone is OK keeping 3.6.x up-to-date
security/patch-wise (if not feature-wise) for the next N years, then that's
a potentially valid approach.  I interpreted that comment as "a long-lived
release line is undesirable", but no one explicitly said that, I just read
it that way.

~Brent

On Wed, Oct 21, 2020 at 9:49 AM Patrick Hunt <ph...@apache.org> wrote:

> On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org> wrote:
>
> > As far as I know Hbase, Solr and Kafka are already Java 11 ready.
> >
> > IMHO contributors of those projects should also put efforts into moving
> > forward.
> >
> > We’re not saying that you _have_ to move to Java 11.
> > Staying on Java 8? No problem, 3.6 is for you.
> > Want the fancy new features of 3.7? Work on it on your side too.
> >
> >
> ppl want things like security fixes. I believe the highlighted downside is
> that we would need to continue to maintain 3.6.x rather than allowing
> users, and ourselves, to focus on trunk for production -"64 percent" would
> be blocked.
>
> Patrick
>
>
> > Andor
> >
> >
> >
> > > On 2020. Oct 21., at 17:52, Enrico Olivelli <eo...@gmail.com>
> wrote:
> > >
> > > Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <
> andor@apache.org>
> > ha
> > > scritto:
> > >
> > >> Correct me if I'm wrong, but Oracle gets paid for extended support.
> > >> Java 8 support until 2030 is not free of charge.
> > >>
> > >> "ZK may end up with a lot of users potentially locking themselves to
> > >> 3.6.x for a while as Enrico mentioned."
> > >>
> > >> That's true. What's the downside of that which we should invest in to
> > >> avoid?
> > >>
> > >
> > > I see ZooKeeper used in many many projects, all of the
> > > HBase/Pulsar/Kafka/Solr ecosystem...
> > > they will have to move to JDK11 in order to move to the new ZK version
> > > so probably they will stay on ZK 3.6
> > >
> > > Probably with Java 17 LTS released the cards will change on the table
> > >
> > > Enrico
> > >
> > >
> > >
> > >>
> > >> Andor
> > >>
> > >>
> > >>
> > >> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> > >>> As a slightly different consideration, if you look at the Long-Term
> > >>> Support
> > >>> (LTS) roadmaps for Java, currently Java 8 is set to have full support
> > >>> until
> > >>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> > >>>
> > >>>
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> > >>> https://en.wikipedia.org/wiki/Java_version_history
> > >>>
> > >>> My guess is that a number of companies are still heavily invested at
> > >>> the
> > >>> Java 8 level (I know a few) and with that kind of time horizon, they
> > >>> have
> > >>> no real motivation to upgrade for quite a while.  If the recent
> > >>> Python 2
> > >>> deprecation is anything to go by, they won't do it until they have
> > >>> to.
> > >>>
> > >>> Not saying Java 8 isn't *very* old (2014 release it seems like?)  and
> > >>> I'm
> > >>> not invested heavily either way, but this might suggest that ZK may
> > >>> end up
> > >>> with a lot of users potentially locking themselves to 3.6.x for a
> > >>> while as
> > >>> Enrico mentioned.
> > >>>
> > >>> (Not a major contributor, but wanted to chime in since I just had
> > >>> this
> > >>> conversation with a bunch of people professionally recently)
> > >>>
> > >>> Brent
> > >>>
> > >>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <an...@apache.org>
> > >>> wrote:
> > >>>
> > >>>> Thanks for the summary.
> > >>>>
> > >>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully. Other
> > >>>> projects
> > >>>> will upgrade once they’re JDK11 compliant, otherwise they will stay
> > >>>> on 3.5
> > >>>> or 3.6. Both version are quite recent in ZooKeeper-terms, we
> > >>>> already
> > >>>> planned big changes for 3.7.0 and JDK 11 could be one of them.
> > >>>>
> > >>>> Don’t put extra burden on the ZK community to help others staying
> > >>>> on
> > >>>> ancient Java versions.
> > >>>>
> > >>>> Andor
> > >>>>
> > >>>>
> > >>>>
> > >>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <eo...@gmail.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>> Let me recap
> > >>>>> - Christopher is proposing to move to JDK11
> > >>>>> - ZooKeeper client and server are bundled and coded and tested
> > >>>>> together
> > >>>> in
> > >>>>> zookeeper-server
> > >>>>> - Enrico is concerned about the need of testing ZooKeeper client
> > >>>>> on JDK8
> > >>>>> (not a problem to move the server to JDK11)
> > >>>>>
> > >>>>> ZooKeeper client is used by tons of users and unfortunately many
> > >>>>> projects
> > >>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
> > >>>>> block
> > >>>> users
> > >>>>> from the adoption,
> > >>>>> that is that we will see the world to stay on 3.6.x and we will
> > >>>>> have
> > >>>> again
> > >>>>> a long lived release line, like 3.4.
> > >>>>>
> > >>>>> Testing the client on JDK8 would be possible if we create some
> > >>>>> kind of
> > >>>>> additional module with system tests, then we can start the server
> > >>>>> on
> > >>>> docker
> > >>>>> on JDK11+ and start a client on JDK8
> > >>>>> with Maven toolchain it should possible to run surefire tests
> > >>>>> using a
> > >>>>> separate JVM.
> > >>>>>
> > >>>>> So in my vision 2 options:
> > >>>>> 1) fully JDK11 - drop JDK8 at all
> > >>>>> 2) build with JDK11 - server only on JDK11 - add system tests
> > >>>>> with docker
> > >>>>> and toolchains that ensure the ZooKeeper client (and all
> > >>>>> dependencies)
> > >>>>> still work on JDK8
> > >>>>>
> > >>>>> From my point of view about the ZooKeeper ecosystem option 2)
> > >>>>> will be far
> > >>>>> better, but we need resources to work on a new test suite.
> > >>>>>
> > >>>>> Enrico
> > >>>>>
> > >>>>>
> > >>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> > >>>>> andor@apache.org>
> > >>>> ha
> > >>>>> scritto:
> > >>>>>
> > >>>>>> Tamas, Enrico,
> > >>>>>>
> > >>>>>> Sorry I don’t follow. Why do we have to test the client with
> > >>>>>> JDK 8 in
> > >>>>>> version 3.7.0?
> > >>>>>>
> > >>>>>> Andor
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
> > >>>>>>> tamaas@cloudera.com.INVALID>
> > >>>>>> wrote:
> > >>>>>>> Hi Enrico,
> > >>>>>>>
> > >>>>>>> Separating ZooKeeper client and server is a huge work, but we
> > >>>>>>> might not
> > >>>>>>> need it.
> > >>>>>>> As you mentioned we have to test ZK client with Java 8, what
> > >>>>>>> about
> > >>>>>>> separating only the test cases which we need to run with
> > >>>>>>> Java8 too?
> > >>>>>>> In Curator we have the ZK compatibility tests where we run a
> > >>>>>>> limited
> > >>>>>> amount
> > >>>>>>> of Curator's jUnit tests with a different ZK version.
> > >>>>>>> We might be able to do the same here, tag tests which are
> > >>>>>>> testing ZK
> > >>>>>> client
> > >>>>>>> and run them separately with Java 8. The only limitation is
> > >>>>>>> that these
> > >>>>>>> tests must stay JDK8 compatible.
> > >>>>>>> But from the tags we will see which ones are those.
> > >>>>>>>
> > >>>>>>> What do you think?
> > >>>>>>>
> > >>>>>>> Regards, Tamaas
> > >>>>>>>
> > >>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> > >>>>>>> eolivelli@gmail.com>
> > >>>>>> wrote:
> > >>>>>>>> Christopher
> > >>>>>>>> I appreciate your idea and I also moved lots of my projects
> > >>>>>>>> to work
> > >>>> the
> > >>>>>> way
> > >>>>>>>> you are suggesting.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> We must run tests using real jdk8 to test the Zookeeper
> > >>>>>>>> client. We
> > >>>> must
> > >>>>>>>> ensure that Zookeeper works well, especially while dealing
> > >>>>>>>> with
> > >>>> security
> > >>>>>>>> stuff.
> > >>>>>>>> Currently the client is in the same module of the server
> > >>>>>>>> and it will
> > >>>>>> take a
> > >>>>>>>> good (huge) amount of work to separate them
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Enrico
> > >>>>>>>>
> > >>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org>
> > >>>>>>>> ha
> > >>>> scritto:
> > >>>>>>>>> Hi ZK Devs,
> > >>>>>>>>>
> > >>>>>>>>> With recent advancements in Java (since Java 9), it is
> > >>>>>>>>> now generally
> > >>>>>>>>> no longer necessary to require that software be developed
> > >>>>>>>>> on an older
> > >>>>>>>>> JDK in order to have confidence that it will run on the
> > >>>>>>>>> older version
> > >>>>>>>>> of Java. This is because, as of Java 9, all JDK releases
> > >>>>>>>>> have better
> > >>>>>>>>> support for cross-compilation to older Java versions.
> > >>>>>>>>>
> > >>>>>>>>> What this means is that developers can confidently make
> > >>>>>>>>> the build
> > >>>>>>>>> requirements for a project higher than the Java version
> > >>>>>>>>> that will
> > >>>>>>>>> actually be supported at runtime.
> > >>>>>>>>>
> > >>>>>>>>> In fact, ZooKeeper already supports the necessary flags
> > >>>>>>>>> in its Maven
> > >>>>>>>>> build configuration to ensure that it uses JDK 8
> > >>>>>>>>> compliance when
> > >>>>>>>>> building on a newer JDK (I added this way back in
> > >>>>>>>>> ZOOKEEPER-3739 /
> > >>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
> > >>>>>>>>>
> > >>>>>>>>> So, I propose that we make JDK 11 the new minimum version
> > >>>>>>>>> to *build*
> > >>>>>>>>> ZooKeeper with. This would not change the runtime
> > >>>>>>>>> requirement, which
> > >>>>>>>>> would remain at JDK 8.
> > >>>>>>>>>
> > >>>>>>>>> The only necessary change to make this happen would be to
> > >>>>>>>>> add the
> > >>>>>>>>> minimum Java version to the maven-enforcer-plugin (like
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>
> > >>
> >
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> > >>>>>>>>> )
> > >>>>>>>>>
> > >>>>>>>>> This would allow ZooKeeper to to streamline its
> > >>>>>>>>> development process a
> > >>>>>>>>> little bit by reducing the amount of CI testing that is
> > >>>>>>>>> done as part
> > >>>>>>>>> of the build. In other words, we can drop the CI builds
> > >>>>>>>>> for JDK 8,
> > >>>>>>>>> which saves on build resources and time. The return on
> > >>>>>>>>> investment is
> > >>>>>>>>> so low for the JDK 8 builds anyway, because of the
> > >>>>>>>>> improved
> > >>>>>>>>> cross-compilation in newer JDKs. So, there's not much
> > >>>>>>>>> value in
> > >>>>>>>>> building on JDK 8 anyway.
> > >>>>>>>>>
> > >>>>>>>>> Of course, I am only recommending this for *new* release
> > >>>>>>>>> lines,
> > >>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because I
> > >>>>>>>>> would not want
> > >>>>>>>>> to change expectations for users who will build their own
> > >>>>>>>>> 3.5 and 3.6
> > >>>>>>>>> versions as they continue to have patch versions
> > >>>>>>>>> released.
> > >>>>>>>>>
> > >>>>>>>>> What do you think?
> > >>>>>>>>>
> > >>>>>>>>> Kind Regards,
> > >>>>>>>>> Christopher
> > >>>>>>>>>
> > >>
> > >>
> >
> >
>

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Patrick Hunt <ph...@apache.org>]

On Wed, Oct 21, 2020 at 9:03 AM Andor Molnar <an...@apache.org> wrote:

> As far as I know Hbase, Solr and Kafka are already Java 11 ready.
>
> IMHO contributors of those projects should also put efforts into moving
> forward.
>
> We’re not saying that you _have_ to move to Java 11.
> Staying on Java 8? No problem, 3.6 is for you.
> Want the fancy new features of 3.7? Work on it on your side too.
>
>
ppl want things like security fixes. I believe the highlighted downside is
that we would need to continue to maintain 3.6.x rather than allowing
users, and ourselves, to focus on trunk for production -"64 percent" would
be blocked.

Patrick


> Andor
>
>
>
> > On 2020. Oct 21., at 17:52, Enrico Olivelli <eo...@gmail.com> wrote:
> >
> > Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <an...@apache.org>
> ha
> > scritto:
> >
> >> Correct me if I'm wrong, but Oracle gets paid for extended support.
> >> Java 8 support until 2030 is not free of charge.
> >>
> >> "ZK may end up with a lot of users potentially locking themselves to
> >> 3.6.x for a while as Enrico mentioned."
> >>
> >> That's true. What's the downside of that which we should invest in to
> >> avoid?
> >>
> >
> > I see ZooKeeper used in many many projects, all of the
> > HBase/Pulsar/Kafka/Solr ecosystem...
> > they will have to move to JDK11 in order to move to the new ZK version
> > so probably they will stay on ZK 3.6
> >
> > Probably with Java 17 LTS released the cards will change on the table
> >
> > Enrico
> >
> >
> >
> >>
> >> Andor
> >>
> >>
> >>
> >> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> >>> As a slightly different consideration, if you look at the Long-Term
> >>> Support
> >>> (LTS) roadmaps for Java, currently Java 8 is set to have full support
> >>> until
> >>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> >>>
> >>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> >>> https://en.wikipedia.org/wiki/Java_version_history
> >>>
> >>> My guess is that a number of companies are still heavily invested at
> >>> the
> >>> Java 8 level (I know a few) and with that kind of time horizon, they
> >>> have
> >>> no real motivation to upgrade for quite a while.  If the recent
> >>> Python 2
> >>> deprecation is anything to go by, they won't do it until they have
> >>> to.
> >>>
> >>> Not saying Java 8 isn't *very* old (2014 release it seems like?)  and
> >>> I'm
> >>> not invested heavily either way, but this might suggest that ZK may
> >>> end up
> >>> with a lot of users potentially locking themselves to 3.6.x for a
> >>> while as
> >>> Enrico mentioned.
> >>>
> >>> (Not a major contributor, but wanted to chime in since I just had
> >>> this
> >>> conversation with a bunch of people professionally recently)
> >>>
> >>> Brent
> >>>
> >>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <an...@apache.org>
> >>> wrote:
> >>>
> >>>> Thanks for the summary.
> >>>>
> >>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully. Other
> >>>> projects
> >>>> will upgrade once they’re JDK11 compliant, otherwise they will stay
> >>>> on 3.5
> >>>> or 3.6. Both version are quite recent in ZooKeeper-terms, we
> >>>> already
> >>>> planned big changes for 3.7.0 and JDK 11 could be one of them.
> >>>>
> >>>> Don’t put extra burden on the ZK community to help others staying
> >>>> on
> >>>> ancient Java versions.
> >>>>
> >>>> Andor
> >>>>
> >>>>
> >>>>
> >>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <eo...@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>> Let me recap
> >>>>> - Christopher is proposing to move to JDK11
> >>>>> - ZooKeeper client and server are bundled and coded and tested
> >>>>> together
> >>>> in
> >>>>> zookeeper-server
> >>>>> - Enrico is concerned about the need of testing ZooKeeper client
> >>>>> on JDK8
> >>>>> (not a problem to move the server to JDK11)
> >>>>>
> >>>>> ZooKeeper client is used by tons of users and unfortunately many
> >>>>> projects
> >>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
> >>>>> block
> >>>> users
> >>>>> from the adoption,
> >>>>> that is that we will see the world to stay on 3.6.x and we will
> >>>>> have
> >>>> again
> >>>>> a long lived release line, like 3.4.
> >>>>>
> >>>>> Testing the client on JDK8 would be possible if we create some
> >>>>> kind of
> >>>>> additional module with system tests, then we can start the server
> >>>>> on
> >>>> docker
> >>>>> on JDK11+ and start a client on JDK8
> >>>>> with Maven toolchain it should possible to run surefire tests
> >>>>> using a
> >>>>> separate JVM.
> >>>>>
> >>>>> So in my vision 2 options:
> >>>>> 1) fully JDK11 - drop JDK8 at all
> >>>>> 2) build with JDK11 - server only on JDK11 - add system tests
> >>>>> with docker
> >>>>> and toolchains that ensure the ZooKeeper client (and all
> >>>>> dependencies)
> >>>>> still work on JDK8
> >>>>>
> >>>>> From my point of view about the ZooKeeper ecosystem option 2)
> >>>>> will be far
> >>>>> better, but we need resources to work on a new test suite.
> >>>>>
> >>>>> Enrico
> >>>>>
> >>>>>
> >>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> >>>>> andor@apache.org>
> >>>> ha
> >>>>> scritto:
> >>>>>
> >>>>>> Tamas, Enrico,
> >>>>>>
> >>>>>> Sorry I don’t follow. Why do we have to test the client with
> >>>>>> JDK 8 in
> >>>>>> version 3.7.0?
> >>>>>>
> >>>>>> Andor
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
> >>>>>>> tamaas@cloudera.com.INVALID>
> >>>>>> wrote:
> >>>>>>> Hi Enrico,
> >>>>>>>
> >>>>>>> Separating ZooKeeper client and server is a huge work, but we
> >>>>>>> might not
> >>>>>>> need it.
> >>>>>>> As you mentioned we have to test ZK client with Java 8, what
> >>>>>>> about
> >>>>>>> separating only the test cases which we need to run with
> >>>>>>> Java8 too?
> >>>>>>> In Curator we have the ZK compatibility tests where we run a
> >>>>>>> limited
> >>>>>> amount
> >>>>>>> of Curator's jUnit tests with a different ZK version.
> >>>>>>> We might be able to do the same here, tag tests which are
> >>>>>>> testing ZK
> >>>>>> client
> >>>>>>> and run them separately with Java 8. The only limitation is
> >>>>>>> that these
> >>>>>>> tests must stay JDK8 compatible.
> >>>>>>> But from the tags we will see which ones are those.
> >>>>>>>
> >>>>>>> What do you think?
> >>>>>>>
> >>>>>>> Regards, Tamaas
> >>>>>>>
> >>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> >>>>>>> eolivelli@gmail.com>
> >>>>>> wrote:
> >>>>>>>> Christopher
> >>>>>>>> I appreciate your idea and I also moved lots of my projects
> >>>>>>>> to work
> >>>> the
> >>>>>> way
> >>>>>>>> you are suggesting.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> We must run tests using real jdk8 to test the Zookeeper
> >>>>>>>> client. We
> >>>> must
> >>>>>>>> ensure that Zookeeper works well, especially while dealing
> >>>>>>>> with
> >>>> security
> >>>>>>>> stuff.
> >>>>>>>> Currently the client is in the same module of the server
> >>>>>>>> and it will
> >>>>>> take a
> >>>>>>>> good (huge) amount of work to separate them
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> Enrico
> >>>>>>>>
> >>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org>
> >>>>>>>> ha
> >>>> scritto:
> >>>>>>>>> Hi ZK Devs,
> >>>>>>>>>
> >>>>>>>>> With recent advancements in Java (since Java 9), it is
> >>>>>>>>> now generally
> >>>>>>>>> no longer necessary to require that software be developed
> >>>>>>>>> on an older
> >>>>>>>>> JDK in order to have confidence that it will run on the
> >>>>>>>>> older version
> >>>>>>>>> of Java. This is because, as of Java 9, all JDK releases
> >>>>>>>>> have better
> >>>>>>>>> support for cross-compilation to older Java versions.
> >>>>>>>>>
> >>>>>>>>> What this means is that developers can confidently make
> >>>>>>>>> the build
> >>>>>>>>> requirements for a project higher than the Java version
> >>>>>>>>> that will
> >>>>>>>>> actually be supported at runtime.
> >>>>>>>>>
> >>>>>>>>> In fact, ZooKeeper already supports the necessary flags
> >>>>>>>>> in its Maven
> >>>>>>>>> build configuration to ensure that it uses JDK 8
> >>>>>>>>> compliance when
> >>>>>>>>> building on a newer JDK (I added this way back in
> >>>>>>>>> ZOOKEEPER-3739 /
> >>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
> >>>>>>>>>
> >>>>>>>>> So, I propose that we make JDK 11 the new minimum version
> >>>>>>>>> to *build*
> >>>>>>>>> ZooKeeper with. This would not change the runtime
> >>>>>>>>> requirement, which
> >>>>>>>>> would remain at JDK 8.
> >>>>>>>>>
> >>>>>>>>> The only necessary change to make this happen would be to
> >>>>>>>>> add the
> >>>>>>>>> minimum Java version to the maven-enforcer-plugin (like
> >>>>>>>>>
> >>>>>>>>>
> >>>>
> >>
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> >>>>>>>>> )
> >>>>>>>>>
> >>>>>>>>> This would allow ZooKeeper to to streamline its
> >>>>>>>>> development process a
> >>>>>>>>> little bit by reducing the amount of CI testing that is
> >>>>>>>>> done as part
> >>>>>>>>> of the build. In other words, we can drop the CI builds
> >>>>>>>>> for JDK 8,
> >>>>>>>>> which saves on build resources and time. The return on
> >>>>>>>>> investment is
> >>>>>>>>> so low for the JDK 8 builds anyway, because of the
> >>>>>>>>> improved
> >>>>>>>>> cross-compilation in newer JDKs. So, there's not much
> >>>>>>>>> value in
> >>>>>>>>> building on JDK 8 anyway.
> >>>>>>>>>
> >>>>>>>>> Of course, I am only recommending this for *new* release
> >>>>>>>>> lines,
> >>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because I
> >>>>>>>>> would not want
> >>>>>>>>> to change expectations for users who will build their own
> >>>>>>>>> 3.5 and 3.6
> >>>>>>>>> versions as they continue to have patch versions
> >>>>>>>>> released.
> >>>>>>>>>
> >>>>>>>>> What do you think?
> >>>>>>>>>
> >>>>>>>>> Kind Regards,
> >>>>>>>>> Christopher
> >>>>>>>>>
> >>
> >>
>
>

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Andor Molnar <an...@apache.org>]

As far as I know Hbase, Solr and Kafka are already Java 11 ready.

IMHO contributors of those projects should also put efforts into moving forward.

We’re not saying that you _have_ to move to Java 11. 
Staying on Java 8? No problem, 3.6 is for you. 
Want the fancy new features of 3.7? Work on it on your side too.

Andor



> On 2020. Oct 21., at 17:52, Enrico Olivelli <eo...@gmail.com> wrote:
> 
> Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <an...@apache.org> ha
> scritto:
> 
>> Correct me if I'm wrong, but Oracle gets paid for extended support.
>> Java 8 support until 2030 is not free of charge.
>> 
>> "ZK may end up with a lot of users potentially locking themselves to
>> 3.6.x for a while as Enrico mentioned."
>> 
>> That's true. What's the downside of that which we should invest in to
>> avoid?
>> 
> 
> I see ZooKeeper used in many many projects, all of the
> HBase/Pulsar/Kafka/Solr ecosystem...
> they will have to move to JDK11 in order to move to the new ZK version
> so probably they will stay on ZK 3.6
> 
> Probably with Java 17 LTS released the cards will change on the table
> 
> Enrico
> 
> 
> 
>> 
>> Andor
>> 
>> 
>> 
>> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
>>> As a slightly different consideration, if you look at the Long-Term
>>> Support
>>> (LTS) roadmaps for Java, currently Java 8 is set to have full support
>>> until
>>> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
>>> 
>>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>>> https://en.wikipedia.org/wiki/Java_version_history
>>> 
>>> My guess is that a number of companies are still heavily invested at
>>> the
>>> Java 8 level (I know a few) and with that kind of time horizon, they
>>> have
>>> no real motivation to upgrade for quite a while.  If the recent
>>> Python 2
>>> deprecation is anything to go by, they won't do it until they have
>>> to.
>>> 
>>> Not saying Java 8 isn't *very* old (2014 release it seems like?)  and
>>> I'm
>>> not invested heavily either way, but this might suggest that ZK may
>>> end up
>>> with a lot of users potentially locking themselves to 3.6.x for a
>>> while as
>>> Enrico mentioned.
>>> 
>>> (Not a major contributor, but wanted to chime in since I just had
>>> this
>>> conversation with a bunch of people professionally recently)
>>> 
>>> Brent
>>> 
>>> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <an...@apache.org>
>>> wrote:
>>> 
>>>> Thanks for the summary.
>>>> 
>>>> I still vote for option 1). Move 3.7.0 to JDK 11 fully. Other
>>>> projects
>>>> will upgrade once they’re JDK11 compliant, otherwise they will stay
>>>> on 3.5
>>>> or 3.6. Both version are quite recent in ZooKeeper-terms, we
>>>> already
>>>> planned big changes for 3.7.0 and JDK 11 could be one of them.
>>>> 
>>>> Don’t put extra burden on the ZK community to help others staying
>>>> on
>>>> ancient Java versions.
>>>> 
>>>> Andor
>>>> 
>>>> 
>>>> 
>>>>> On 2020. Oct 21., at 10:57, Enrico Olivelli <eo...@gmail.com>
>>>>> wrote:
>>>>> 
>>>>> Let me recap
>>>>> - Christopher is proposing to move to JDK11
>>>>> - ZooKeeper client and server are bundled and coded and tested
>>>>> together
>>>> in
>>>>> zookeeper-server
>>>>> - Enrico is concerned about the need of testing ZooKeeper client
>>>>> on JDK8
>>>>> (not a problem to move the server to JDK11)
>>>>> 
>>>>> ZooKeeper client is used by tons of users and unfortunately many
>>>>> projects
>>>>> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
>>>>> block
>>>> users
>>>>> from the adoption,
>>>>> that is that we will see the world to stay on 3.6.x and we will
>>>>> have
>>>> again
>>>>> a long lived release line, like 3.4.
>>>>> 
>>>>> Testing the client on JDK8 would be possible if we create some
>>>>> kind of
>>>>> additional module with system tests, then we can start the server
>>>>> on
>>>> docker
>>>>> on JDK11+ and start a client on JDK8
>>>>> with Maven toolchain it should possible to run surefire tests
>>>>> using a
>>>>> separate JVM.
>>>>> 
>>>>> So in my vision 2 options:
>>>>> 1) fully JDK11 - drop JDK8 at all
>>>>> 2) build with JDK11 - server only on JDK11 - add system tests
>>>>> with docker
>>>>> and toolchains that ensure the ZooKeeper client (and all
>>>>> dependencies)
>>>>> still work on JDK8
>>>>> 
>>>>> From my point of view about the ZooKeeper ecosystem option 2)
>>>>> will be far
>>>>> better, but we need resources to work on a new test suite.
>>>>> 
>>>>> Enrico
>>>>> 
>>>>> 
>>>>> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
>>>>> andor@apache.org>
>>>> ha
>>>>> scritto:
>>>>> 
>>>>>> Tamas, Enrico,
>>>>>> 
>>>>>> Sorry I don’t follow. Why do we have to test the client with
>>>>>> JDK 8 in
>>>>>> version 3.7.0?
>>>>>> 
>>>>>> Andor
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> On 2020. Oct 20., at 22:29, Tamas Penzes <
>>>>>>> tamaas@cloudera.com.INVALID>
>>>>>> wrote:
>>>>>>> Hi Enrico,
>>>>>>> 
>>>>>>> Separating ZooKeeper client and server is a huge work, but we
>>>>>>> might not
>>>>>>> need it.
>>>>>>> As you mentioned we have to test ZK client with Java 8, what
>>>>>>> about
>>>>>>> separating only the test cases which we need to run with
>>>>>>> Java8 too?
>>>>>>> In Curator we have the ZK compatibility tests where we run a
>>>>>>> limited
>>>>>> amount
>>>>>>> of Curator's jUnit tests with a different ZK version.
>>>>>>> We might be able to do the same here, tag tests which are
>>>>>>> testing ZK
>>>>>> client
>>>>>>> and run them separately with Java 8. The only limitation is
>>>>>>> that these
>>>>>>> tests must stay JDK8 compatible.
>>>>>>> But from the tags we will see which ones are those.
>>>>>>> 
>>>>>>> What do you think?
>>>>>>> 
>>>>>>> Regards, Tamaas
>>>>>>> 
>>>>>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
>>>>>>> eolivelli@gmail.com>
>>>>>> wrote:
>>>>>>>> Christopher
>>>>>>>> I appreciate your idea and I also moved lots of my projects
>>>>>>>> to work
>>>> the
>>>>>> way
>>>>>>>> you are suggesting.
>>>>>>>> 
>>>>>>>> 
>>>>>>>> We must run tests using real jdk8 to test the Zookeeper
>>>>>>>> client. We
>>>> must
>>>>>>>> ensure that Zookeeper works well, especially while dealing
>>>>>>>> with
>>>> security
>>>>>>>> stuff.
>>>>>>>> Currently the client is in the same module of the server
>>>>>>>> and it will
>>>>>> take a
>>>>>>>> good (huge) amount of work to separate them
>>>>>>>> 
>>>>>>>> 
>>>>>>>> Enrico
>>>>>>>> 
>>>>>>>> Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org>
>>>>>>>> ha
>>>> scritto:
>>>>>>>>> Hi ZK Devs,
>>>>>>>>> 
>>>>>>>>> With recent advancements in Java (since Java 9), it is
>>>>>>>>> now generally
>>>>>>>>> no longer necessary to require that software be developed
>>>>>>>>> on an older
>>>>>>>>> JDK in order to have confidence that it will run on the
>>>>>>>>> older version
>>>>>>>>> of Java. This is because, as of Java 9, all JDK releases
>>>>>>>>> have better
>>>>>>>>> support for cross-compilation to older Java versions.
>>>>>>>>> 
>>>>>>>>> What this means is that developers can confidently make
>>>>>>>>> the build
>>>>>>>>> requirements for a project higher than the Java version
>>>>>>>>> that will
>>>>>>>>> actually be supported at runtime.
>>>>>>>>> 
>>>>>>>>> In fact, ZooKeeper already supports the necessary flags
>>>>>>>>> in its Maven
>>>>>>>>> build configuration to ensure that it uses JDK 8
>>>>>>>>> compliance when
>>>>>>>>> building on a newer JDK (I added this way back in
>>>>>>>>> ZOOKEEPER-3739 /
>>>>>>>>> https://github.com/apache/zookeeper/pull/1269)
>>>>>>>>> 
>>>>>>>>> So, I propose that we make JDK 11 the new minimum version
>>>>>>>>> to *build*
>>>>>>>>> ZooKeeper with. This would not change the runtime
>>>>>>>>> requirement, which
>>>>>>>>> would remain at JDK 8.
>>>>>>>>> 
>>>>>>>>> The only necessary change to make this happen would be to
>>>>>>>>> add the
>>>>>>>>> minimum Java version to the maven-enforcer-plugin (like
>>>>>>>>> 
>>>>>>>>> 
>>>> 
>> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
>>>>>>>>> )
>>>>>>>>> 
>>>>>>>>> This would allow ZooKeeper to to streamline its
>>>>>>>>> development process a
>>>>>>>>> little bit by reducing the amount of CI testing that is
>>>>>>>>> done as part
>>>>>>>>> of the build. In other words, we can drop the CI builds
>>>>>>>>> for JDK 8,
>>>>>>>>> which saves on build resources and time. The return on
>>>>>>>>> investment is
>>>>>>>>> so low for the JDK 8 builds anyway, because of the
>>>>>>>>> improved
>>>>>>>>> cross-compilation in newer JDKs. So, there's not much
>>>>>>>>> value in
>>>>>>>>> building on JDK 8 anyway.
>>>>>>>>> 
>>>>>>>>> Of course, I am only recommending this for *new* release
>>>>>>>>> lines,
>>>>>>>>> starting with ZooKeeper 3.7.0/master branch, because I
>>>>>>>>> would not want
>>>>>>>>> to change expectations for users who will build their own
>>>>>>>>> 3.5 and 3.6
>>>>>>>>> versions as they continue to have patch versions
>>>>>>>>> released.
>>>>>>>>> 
>>>>>>>>> What do you think?
>>>>>>>>> 
>>>>>>>>> Kind Regards,
>>>>>>>>> Christopher
>>>>>>>>> 
>> 
>> 

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Enrico Olivelli <eo...@gmail.com>]

Il giorno mer 21 ott 2020 alle ore 17:49 Andor Molnar <an...@apache.org> ha
scritto:

> Correct me if I'm wrong, but Oracle gets paid for extended support.
> Java 8 support until 2030 is not free of charge.
>
> "ZK may end up with a lot of users potentially locking themselves to
> 3.6.x for a while as Enrico mentioned."
>
> That's true. What's the downside of that which we should invest in to
> avoid?
>

I see ZooKeeper used in many many projects, all of the
HBase/Pulsar/Kafka/Solr ecosystem...
they will have to move to JDK11 in order to move to the new ZK version
so probably they will stay on ZK 3.6

Probably with Java 17 LTS released the cards will change on the table

Enrico



>
> Andor
>
>
>
> On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> > As a slightly different consideration, if you look at the Long-Term
> > Support
> > (LTS) roadmaps for Java, currently Java 8 is set to have full support
> > until
> > 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> >
> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> > https://en.wikipedia.org/wiki/Java_version_history
> >
> > My guess is that a number of companies are still heavily invested at
> > the
> > Java 8 level (I know a few) and with that kind of time horizon, they
> > have
> > no real motivation to upgrade for quite a while.  If the recent
> > Python 2
> > deprecation is anything to go by, they won't do it until they have
> > to.
> >
> > Not saying Java 8 isn't *very* old (2014 release it seems like?)  and
> > I'm
> > not invested heavily either way, but this might suggest that ZK may
> > end up
> > with a lot of users potentially locking themselves to 3.6.x for a
> > while as
> > Enrico mentioned.
> >
> > (Not a major contributor, but wanted to chime in since I just had
> > this
> > conversation with a bunch of people professionally recently)
> >
> > Brent
> >
> > On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <an...@apache.org>
> > wrote:
> >
> > > Thanks for the summary.
> > >
> > > I still vote for option 1). Move 3.7.0 to JDK 11 fully. Other
> > > projects
> > > will upgrade once they’re JDK11 compliant, otherwise they will stay
> > > on 3.5
> > > or 3.6. Both version are quite recent in ZooKeeper-terms, we
> > > already
> > > planned big changes for 3.7.0 and JDK 11 could be one of them.
> > >
> > > Don’t put extra burden on the ZK community to help others staying
> > > on
> > > ancient Java versions.
> > >
> > > Andor
> > >
> > >
> > >
> > > > On 2020. Oct 21., at 10:57, Enrico Olivelli <eo...@gmail.com>
> > > > wrote:
> > > >
> > > > Let me recap
> > > > - Christopher is proposing to move to JDK11
> > > > - ZooKeeper client and server are bundled and coded and tested
> > > > together
> > > in
> > > > zookeeper-server
> > > > - Enrico is concerned about the need of testing ZooKeeper client
> > > > on JDK8
> > > > (not a problem to move the server to JDK11)
> > > >
> > > > ZooKeeper client is used by tons of users and unfortunately many
> > > > projects
> > > > are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
> > > > block
> > > users
> > > > from the adoption,
> > > > that is that we will see the world to stay on 3.6.x and we will
> > > > have
> > > again
> > > > a long lived release line, like 3.4.
> > > >
> > > > Testing the client on JDK8 would be possible if we create some
> > > > kind of
> > > > additional module with system tests, then we can start the server
> > > > on
> > > docker
> > > > on JDK11+ and start a client on JDK8
> > > > with Maven toolchain it should possible to run surefire tests
> > > > using a
> > > > separate JVM.
> > > >
> > > > So in my vision 2 options:
> > > > 1) fully JDK11 - drop JDK8 at all
> > > > 2) build with JDK11 - server only on JDK11 - add system tests
> > > > with docker
> > > > and toolchains that ensure the ZooKeeper client (and all
> > > > dependencies)
> > > > still work on JDK8
> > > >
> > > > From my point of view about the ZooKeeper ecosystem option 2)
> > > > will be far
> > > > better, but we need resources to work on a new test suite.
> > > >
> > > > Enrico
> > > >
> > > >
> > > > Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> > > > andor@apache.org>
> > > ha
> > > > scritto:
> > > >
> > > > > Tamas, Enrico,
> > > > >
> > > > > Sorry I don’t follow. Why do we have to test the client with
> > > > > JDK 8 in
> > > > > version 3.7.0?
> > > > >
> > > > > Andor
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > On 2020. Oct 20., at 22:29, Tamas Penzes <
> > > > > > tamaas@cloudera.com.INVALID>
> > > > > wrote:
> > > > > > Hi Enrico,
> > > > > >
> > > > > > Separating ZooKeeper client and server is a huge work, but we
> > > > > > might not
> > > > > > need it.
> > > > > > As you mentioned we have to test ZK client with Java 8, what
> > > > > > about
> > > > > > separating only the test cases which we need to run with
> > > > > > Java8 too?
> > > > > > In Curator we have the ZK compatibility tests where we run a
> > > > > > limited
> > > > > amount
> > > > > > of Curator's jUnit tests with a different ZK version.
> > > > > > We might be able to do the same here, tag tests which are
> > > > > > testing ZK
> > > > > client
> > > > > > and run them separately with Java 8. The only limitation is
> > > > > > that these
> > > > > > tests must stay JDK8 compatible.
> > > > > > But from the tags we will see which ones are those.
> > > > > >
> > > > > > What do you think?
> > > > > >
> > > > > > Regards, Tamaas
> > > > > >
> > > > > > On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> > > > > > eolivelli@gmail.com>
> > > > > wrote:
> > > > > > > Christopher
> > > > > > > I appreciate your idea and I also moved lots of my projects
> > > > > > > to work
> > > the
> > > > > way
> > > > > > > you are suggesting.
> > > > > > >
> > > > > > >
> > > > > > > We must run tests using real jdk8 to test the Zookeeper
> > > > > > > client. We
> > > must
> > > > > > > ensure that Zookeeper works well, especially while dealing
> > > > > > > with
> > > security
> > > > > > > stuff.
> > > > > > > Currently the client is in the same module of the server
> > > > > > > and it will
> > > > > take a
> > > > > > > good (huge) amount of work to separate them
> > > > > > >
> > > > > > >
> > > > > > > Enrico
> > > > > > >
> > > > > > > Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org>
> > > > > > > ha
> > > scritto:
> > > > > > > > Hi ZK Devs,
> > > > > > > >
> > > > > > > > With recent advancements in Java (since Java 9), it is
> > > > > > > > now generally
> > > > > > > > no longer necessary to require that software be developed
> > > > > > > > on an older
> > > > > > > > JDK in order to have confidence that it will run on the
> > > > > > > > older version
> > > > > > > > of Java. This is because, as of Java 9, all JDK releases
> > > > > > > > have better
> > > > > > > > support for cross-compilation to older Java versions.
> > > > > > > >
> > > > > > > > What this means is that developers can confidently make
> > > > > > > > the build
> > > > > > > > requirements for a project higher than the Java version
> > > > > > > > that will
> > > > > > > > actually be supported at runtime.
> > > > > > > >
> > > > > > > > In fact, ZooKeeper already supports the necessary flags
> > > > > > > > in its Maven
> > > > > > > > build configuration to ensure that it uses JDK 8
> > > > > > > > compliance when
> > > > > > > > building on a newer JDK (I added this way back in
> > > > > > > > ZOOKEEPER-3739 /
> > > > > > > > https://github.com/apache/zookeeper/pull/1269)
> > > > > > > >
> > > > > > > > So, I propose that we make JDK 11 the new minimum version
> > > > > > > > to *build*
> > > > > > > > ZooKeeper with. This would not change the runtime
> > > > > > > > requirement, which
> > > > > > > > would remain at JDK 8.
> > > > > > > >
> > > > > > > > The only necessary change to make this happen would be to
> > > > > > > > add the
> > > > > > > > minimum Java version to the maven-enforcer-plugin (like
> > > > > > > >
> > > > > > > >
> > >
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> > > > > > > > )
> > > > > > > >
> > > > > > > > This would allow ZooKeeper to to streamline its
> > > > > > > > development process a
> > > > > > > > little bit by reducing the amount of CI testing that is
> > > > > > > > done as part
> > > > > > > > of the build. In other words, we can drop the CI builds
> > > > > > > > for JDK 8,
> > > > > > > > which saves on build resources and time. The return on
> > > > > > > > investment is
> > > > > > > > so low for the JDK 8 builds anyway, because of the
> > > > > > > > improved
> > > > > > > > cross-compilation in newer JDKs. So, there's not much
> > > > > > > > value in
> > > > > > > > building on JDK 8 anyway.
> > > > > > > >
> > > > > > > > Of course, I am only recommending this for *new* release
> > > > > > > > lines,
> > > > > > > > starting with ZooKeeper 3.7.0/master branch, because I
> > > > > > > > would not want
> > > > > > > > to change expectations for users who will build their own
> > > > > > > > 3.5 and 3.6
> > > > > > > > versions as they continue to have patch versions
> > > > > > > > released.
> > > > > > > >
> > > > > > > > What do you think?
> > > > > > > >
> > > > > > > > Kind Regards,
> > > > > > > > Christopher
> > > > > > > >
>
>

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Andor Molnar <an...@apache.org>]

Correct me if I'm wrong, but Oracle gets paid for extended support. 
Java 8 support until 2030 is not free of charge.

"ZK may end up with a lot of users potentially locking themselves to
3.6.x for a while as Enrico mentioned."

That's true. What's the downside of that which we should invest in to
avoid?

Andor



On Wed, 2020-10-21 at 08:03 -0700, Brent wrote:
> As a slightly different consideration, if you look at the Long-Term
> Support
> (LTS) roadmaps for Java, currently Java 8 is set to have full support
> until
> 2030 from Oracle and at least 2026 from OpenJDK & Corretto:
> 
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> https://en.wikipedia.org/wiki/Java_version_history
> 
> My guess is that a number of companies are still heavily invested at
> the
> Java 8 level (I know a few) and with that kind of time horizon, they
> have
> no real motivation to upgrade for quite a while.  If the recent
> Python 2
> deprecation is anything to go by, they won't do it until they have
> to.
> 
> Not saying Java 8 isn't *very* old (2014 release it seems like?)  and
> I'm
> not invested heavily either way, but this might suggest that ZK may
> end up
> with a lot of users potentially locking themselves to 3.6.x for a
> while as
> Enrico mentioned.
> 
> (Not a major contributor, but wanted to chime in since I just had
> this
> conversation with a bunch of people professionally recently)
> 
> Brent
> 
> On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <an...@apache.org>
> wrote:
> 
> > Thanks for the summary.
> > 
> > I still vote for option 1). Move 3.7.0 to JDK 11 fully. Other
> > projects
> > will upgrade once they’re JDK11 compliant, otherwise they will stay
> > on 3.5
> > or 3.6. Both version are quite recent in ZooKeeper-terms, we
> > already
> > planned big changes for 3.7.0 and JDK 11 could be one of them.
> > 
> > Don’t put extra burden on the ZK community to help others staying
> > on
> > ancient Java versions.
> > 
> > Andor
> > 
> > 
> > 
> > > On 2020. Oct 21., at 10:57, Enrico Olivelli <eo...@gmail.com>
> > > wrote:
> > > 
> > > Let me recap
> > > - Christopher is proposing to move to JDK11
> > > - ZooKeeper client and server are bundled and coded and tested
> > > together
> > in
> > > zookeeper-server
> > > - Enrico is concerned about the need of testing ZooKeeper client
> > > on JDK8
> > > (not a problem to move the server to JDK11)
> > > 
> > > ZooKeeper client is used by tons of users and unfortunately many
> > > projects
> > > are still on JDK8, if we move ZooKeeper to JDK11 the risk is to
> > > block
> > users
> > > from the adoption,
> > > that is that we will see the world to stay on 3.6.x and we will
> > > have
> > again
> > > a long lived release line, like 3.4.
> > > 
> > > Testing the client on JDK8 would be possible if we create some
> > > kind of
> > > additional module with system tests, then we can start the server
> > > on
> > docker
> > > on JDK11+ and start a client on JDK8
> > > with Maven toolchain it should possible to run surefire tests
> > > using a
> > > separate JVM.
> > > 
> > > So in my vision 2 options:
> > > 1) fully JDK11 - drop JDK8 at all
> > > 2) build with JDK11 - server only on JDK11 - add system tests
> > > with docker
> > > and toolchains that ensure the ZooKeeper client (and all
> > > dependencies)
> > > still work on JDK8
> > > 
> > > From my point of view about the ZooKeeper ecosystem option 2)
> > > will be far
> > > better, but we need resources to work on a new test suite.
> > > 
> > > Enrico
> > > 
> > > 
> > > Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <
> > > andor@apache.org>
> > ha
> > > scritto:
> > > 
> > > > Tamas, Enrico,
> > > > 
> > > > Sorry I don’t follow. Why do we have to test the client with
> > > > JDK 8 in
> > > > version 3.7.0?
> > > > 
> > > > Andor
> > > > 
> > > > 
> > > > 
> > > > 
> > > > > On 2020. Oct 20., at 22:29, Tamas Penzes <
> > > > > tamaas@cloudera.com.INVALID>
> > > > wrote:
> > > > > Hi Enrico,
> > > > > 
> > > > > Separating ZooKeeper client and server is a huge work, but we
> > > > > might not
> > > > > need it.
> > > > > As you mentioned we have to test ZK client with Java 8, what
> > > > > about
> > > > > separating only the test cases which we need to run with
> > > > > Java8 too?
> > > > > In Curator we have the ZK compatibility tests where we run a
> > > > > limited
> > > > amount
> > > > > of Curator's jUnit tests with a different ZK version.
> > > > > We might be able to do the same here, tag tests which are
> > > > > testing ZK
> > > > client
> > > > > and run them separately with Java 8. The only limitation is
> > > > > that these
> > > > > tests must stay JDK8 compatible.
> > > > > But from the tags we will see which ones are those.
> > > > > 
> > > > > What do you think?
> > > > > 
> > > > > Regards, Tamaas
> > > > > 
> > > > > On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <
> > > > > eolivelli@gmail.com>
> > > > wrote:
> > > > > > Christopher
> > > > > > I appreciate your idea and I also moved lots of my projects
> > > > > > to work
> > the
> > > > way
> > > > > > you are suggesting.
> > > > > > 
> > > > > > 
> > > > > > We must run tests using real jdk8 to test the Zookeeper
> > > > > > client. We
> > must
> > > > > > ensure that Zookeeper works well, especially while dealing
> > > > > > with
> > security
> > > > > > stuff.
> > > > > > Currently the client is in the same module of the server
> > > > > > and it will
> > > > take a
> > > > > > good (huge) amount of work to separate them
> > > > > > 
> > > > > > 
> > > > > > Enrico
> > > > > > 
> > > > > > Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org>
> > > > > > ha
> > scritto:
> > > > > > > Hi ZK Devs,
> > > > > > > 
> > > > > > > With recent advancements in Java (since Java 9), it is
> > > > > > > now generally
> > > > > > > no longer necessary to require that software be developed
> > > > > > > on an older
> > > > > > > JDK in order to have confidence that it will run on the
> > > > > > > older version
> > > > > > > of Java. This is because, as of Java 9, all JDK releases
> > > > > > > have better
> > > > > > > support for cross-compilation to older Java versions.
> > > > > > > 
> > > > > > > What this means is that developers can confidently make
> > > > > > > the build
> > > > > > > requirements for a project higher than the Java version
> > > > > > > that will
> > > > > > > actually be supported at runtime.
> > > > > > > 
> > > > > > > In fact, ZooKeeper already supports the necessary flags
> > > > > > > in its Maven
> > > > > > > build configuration to ensure that it uses JDK 8
> > > > > > > compliance when
> > > > > > > building on a newer JDK (I added this way back in
> > > > > > > ZOOKEEPER-3739 /
> > > > > > > https://github.com/apache/zookeeper/pull/1269)
> > > > > > > 
> > > > > > > So, I propose that we make JDK 11 the new minimum version
> > > > > > > to *build*
> > > > > > > ZooKeeper with. This would not change the runtime
> > > > > > > requirement, which
> > > > > > > would remain at JDK 8.
> > > > > > > 
> > > > > > > The only necessary change to make this happen would be to
> > > > > > > add the
> > > > > > > minimum Java version to the maven-enforcer-plugin (like
> > > > > > > 
> > > > > > > 
> > https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> > > > > > > )
> > > > > > > 
> > > > > > > This would allow ZooKeeper to to streamline its
> > > > > > > development process a
> > > > > > > little bit by reducing the amount of CI testing that is
> > > > > > > done as part
> > > > > > > of the build. In other words, we can drop the CI builds
> > > > > > > for JDK 8,
> > > > > > > which saves on build resources and time. The return on
> > > > > > > investment is
> > > > > > > so low for the JDK 8 builds anyway, because of the
> > > > > > > improved
> > > > > > > cross-compilation in newer JDKs. So, there's not much
> > > > > > > value in
> > > > > > > building on JDK 8 anyway.
> > > > > > > 
> > > > > > > Of course, I am only recommending this for *new* release
> > > > > > > lines,
> > > > > > > starting with ZooKeeper 3.7.0/master branch, because I
> > > > > > > would not want
> > > > > > > to change expectations for users who will build their own
> > > > > > > 3.5 and 3.6
> > > > > > > versions as they continue to have patch versions
> > > > > > > released.
> > > > > > > 
> > > > > > > What do you think?
> > > > > > > 
> > > > > > > Kind Regards,
> > > > > > > Christopher
> > > > > > > 

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Brent <br...@gmail.com>]

As a slightly different consideration, if you look at the Long-Term Support
(LTS) roadmaps for Java, currently Java 8 is set to have full support until
2030 from Oracle and at least 2026 from OpenJDK & Corretto:

https://www.oracle.com/java/technologies/java-se-support-roadmap.html
https://en.wikipedia.org/wiki/Java_version_history

My guess is that a number of companies are still heavily invested at the
Java 8 level (I know a few) and with that kind of time horizon, they have
no real motivation to upgrade for quite a while.  If the recent Python 2
deprecation is anything to go by, they won't do it until they have to.

Not saying Java 8 isn't *very* old (2014 release it seems like?)  and I'm
not invested heavily either way, but this might suggest that ZK may end up
with a lot of users potentially locking themselves to 3.6.x for a while as
Enrico mentioned.

(Not a major contributor, but wanted to chime in since I just had this
conversation with a bunch of people professionally recently)

Brent

On Wed, Oct 21, 2020 at 2:07 AM Andor Molnar <an...@apache.org> wrote:

> Thanks for the summary.
>
> I still vote for option 1). Move 3.7.0 to JDK 11 fully. Other projects
> will upgrade once they’re JDK11 compliant, otherwise they will stay on 3.5
> or 3.6. Both version are quite recent in ZooKeeper-terms, we already
> planned big changes for 3.7.0 and JDK 11 could be one of them.
>
> Don’t put extra burden on the ZK community to help others staying on
> ancient Java versions.
>
> Andor
>
>
>
> > On 2020. Oct 21., at 10:57, Enrico Olivelli <eo...@gmail.com> wrote:
> >
> > Let me recap
> > - Christopher is proposing to move to JDK11
> > - ZooKeeper client and server are bundled and coded and tested together
> in
> > zookeeper-server
> > - Enrico is concerned about the need of testing ZooKeeper client on JDK8
> > (not a problem to move the server to JDK11)
> >
> > ZooKeeper client is used by tons of users and unfortunately many projects
> > are still on JDK8, if we move ZooKeeper to JDK11 the risk is to block
> users
> > from the adoption,
> > that is that we will see the world to stay on 3.6.x and we will have
> again
> > a long lived release line, like 3.4.
> >
> > Testing the client on JDK8 would be possible if we create some kind of
> > additional module with system tests, then we can start the server on
> docker
> > on JDK11+ and start a client on JDK8
> > with Maven toolchain it should possible to run surefire tests using a
> > separate JVM.
> >
> > So in my vision 2 options:
> > 1) fully JDK11 - drop JDK8 at all
> > 2) build with JDK11 - server only on JDK11 - add system tests with docker
> > and toolchains that ensure the ZooKeeper client (and all dependencies)
> > still work on JDK8
> >
> > From my point of view about the ZooKeeper ecosystem option 2) will be far
> > better, but we need resources to work on a new test suite.
> >
> > Enrico
> >
> >
> > Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <an...@apache.org>
> ha
> > scritto:
> >
> >> Tamas, Enrico,
> >>
> >> Sorry I don’t follow. Why do we have to test the client with JDK 8 in
> >> version 3.7.0?
> >>
> >> Andor
> >>
> >>
> >>
> >>
> >>> On 2020. Oct 20., at 22:29, Tamas Penzes <ta...@cloudera.com.INVALID>
> >> wrote:
> >>>
> >>> Hi Enrico,
> >>>
> >>> Separating ZooKeeper client and server is a huge work, but we might not
> >>> need it.
> >>> As you mentioned we have to test ZK client with Java 8, what about
> >>> separating only the test cases which we need to run with Java8 too?
> >>> In Curator we have the ZK compatibility tests where we run a limited
> >> amount
> >>> of Curator's jUnit tests with a different ZK version.
> >>> We might be able to do the same here, tag tests which are testing ZK
> >> client
> >>> and run them separately with Java 8. The only limitation is that these
> >>> tests must stay JDK8 compatible.
> >>> But from the tags we will see which ones are those.
> >>>
> >>> What do you think?
> >>>
> >>> Regards, Tamaas
> >>>
> >>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <eo...@gmail.com>
> >> wrote:
> >>>
> >>>> Christopher
> >>>> I appreciate your idea and I also moved lots of my projects to work
> the
> >> way
> >>>> you are suggesting.
> >>>>
> >>>>
> >>>> We must run tests using real jdk8 to test the Zookeeper client. We
> must
> >>>> ensure that Zookeeper works well, especially while dealing with
> security
> >>>> stuff.
> >>>> Currently the client is in the same module of the server and it will
> >> take a
> >>>> good (huge) amount of work to separate them
> >>>>
> >>>>
> >>>> Enrico
> >>>>
> >>>> Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org> ha
> scritto:
> >>>>
> >>>>> Hi ZK Devs,
> >>>>>
> >>>>> With recent advancements in Java (since Java 9), it is now generally
> >>>>> no longer necessary to require that software be developed on an older
> >>>>> JDK in order to have confidence that it will run on the older version
> >>>>> of Java. This is because, as of Java 9, all JDK releases have better
> >>>>> support for cross-compilation to older Java versions.
> >>>>>
> >>>>> What this means is that developers can confidently make the build
> >>>>> requirements for a project higher than the Java version that will
> >>>>> actually be supported at runtime.
> >>>>>
> >>>>> In fact, ZooKeeper already supports the necessary flags in its Maven
> >>>>> build configuration to ensure that it uses JDK 8 compliance when
> >>>>> building on a newer JDK (I added this way back in ZOOKEEPER-3739 /
> >>>>> https://github.com/apache/zookeeper/pull/1269)
> >>>>>
> >>>>> So, I propose that we make JDK 11 the new minimum version to *build*
> >>>>> ZooKeeper with. This would not change the runtime requirement, which
> >>>>> would remain at JDK 8.
> >>>>>
> >>>>> The only necessary change to make this happen would be to add the
> >>>>> minimum Java version to the maven-enforcer-plugin (like
> >>>>>
> >>>>>
> >>>>
> >>
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> >>>>> )
> >>>>>
> >>>>> This would allow ZooKeeper to to streamline its development process a
> >>>>> little bit by reducing the amount of CI testing that is done as part
> >>>>> of the build. In other words, we can drop the CI builds for JDK 8,
> >>>>> which saves on build resources and time. The return on investment is
> >>>>> so low for the JDK 8 builds anyway, because of the improved
> >>>>> cross-compilation in newer JDKs. So, there's not much value in
> >>>>> building on JDK 8 anyway.
> >>>>>
> >>>>> Of course, I am only recommending this for *new* release lines,
> >>>>> starting with ZooKeeper 3.7.0/master branch, because I would not want
> >>>>> to change expectations for users who will build their own 3.5 and 3.6
> >>>>> versions as they continue to have patch versions released.
> >>>>>
> >>>>> What do you think?
> >>>>>
> >>>>> Kind Regards,
> >>>>> Christopher
> >>>>>
> >>>>
> >>
> >>
>
>

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Andor Molnar <an...@apache.org>]

Thanks for the summary.

I still vote for option 1). Move 3.7.0 to JDK 11 fully. Other projects will upgrade once they’re JDK11 compliant, otherwise they will stay on 3.5 or 3.6. Both version are quite recent in ZooKeeper-terms, we already planned big changes for 3.7.0 and JDK 11 could be one of them.

Don’t put extra burden on the ZK community to help others staying on ancient Java versions.

Andor



> On 2020. Oct 21., at 10:57, Enrico Olivelli <eo...@gmail.com> wrote:
> 
> Let me recap
> - Christopher is proposing to move to JDK11
> - ZooKeeper client and server are bundled and coded and tested together in
> zookeeper-server
> - Enrico is concerned about the need of testing ZooKeeper client on JDK8
> (not a problem to move the server to JDK11)
> 
> ZooKeeper client is used by tons of users and unfortunately many projects
> are still on JDK8, if we move ZooKeeper to JDK11 the risk is to block users
> from the adoption,
> that is that we will see the world to stay on 3.6.x and we will have again
> a long lived release line, like 3.4.
> 
> Testing the client on JDK8 would be possible if we create some kind of
> additional module with system tests, then we can start the server on docker
> on JDK11+ and start a client on JDK8
> with Maven toolchain it should possible to run surefire tests using a
> separate JVM.
> 
> So in my vision 2 options:
> 1) fully JDK11 - drop JDK8 at all
> 2) build with JDK11 - server only on JDK11 - add system tests with docker
> and toolchains that ensure the ZooKeeper client (and all dependencies)
> still work on JDK8
> 
> From my point of view about the ZooKeeper ecosystem option 2) will be far
> better, but we need resources to work on a new test suite.
> 
> Enrico
> 
> 
> Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <an...@apache.org> ha
> scritto:
> 
>> Tamas, Enrico,
>> 
>> Sorry I don’t follow. Why do we have to test the client with JDK 8 in
>> version 3.7.0?
>> 
>> Andor
>> 
>> 
>> 
>> 
>>> On 2020. Oct 20., at 22:29, Tamas Penzes <ta...@cloudera.com.INVALID>
>> wrote:
>>> 
>>> Hi Enrico,
>>> 
>>> Separating ZooKeeper client and server is a huge work, but we might not
>>> need it.
>>> As you mentioned we have to test ZK client with Java 8, what about
>>> separating only the test cases which we need to run with Java8 too?
>>> In Curator we have the ZK compatibility tests where we run a limited
>> amount
>>> of Curator's jUnit tests with a different ZK version.
>>> We might be able to do the same here, tag tests which are testing ZK
>> client
>>> and run them separately with Java 8. The only limitation is that these
>>> tests must stay JDK8 compatible.
>>> But from the tags we will see which ones are those.
>>> 
>>> What do you think?
>>> 
>>> Regards, Tamaas
>>> 
>>> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <eo...@gmail.com>
>> wrote:
>>> 
>>>> Christopher
>>>> I appreciate your idea and I also moved lots of my projects to work the
>> way
>>>> you are suggesting.
>>>> 
>>>> 
>>>> We must run tests using real jdk8 to test the Zookeeper client. We must
>>>> ensure that Zookeeper works well, especially while dealing with security
>>>> stuff.
>>>> Currently the client is in the same module of the server and it will
>> take a
>>>> good (huge) amount of work to separate them
>>>> 
>>>> 
>>>> Enrico
>>>> 
>>>> Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org> ha scritto:
>>>> 
>>>>> Hi ZK Devs,
>>>>> 
>>>>> With recent advancements in Java (since Java 9), it is now generally
>>>>> no longer necessary to require that software be developed on an older
>>>>> JDK in order to have confidence that it will run on the older version
>>>>> of Java. This is because, as of Java 9, all JDK releases have better
>>>>> support for cross-compilation to older Java versions.
>>>>> 
>>>>> What this means is that developers can confidently make the build
>>>>> requirements for a project higher than the Java version that will
>>>>> actually be supported at runtime.
>>>>> 
>>>>> In fact, ZooKeeper already supports the necessary flags in its Maven
>>>>> build configuration to ensure that it uses JDK 8 compliance when
>>>>> building on a newer JDK (I added this way back in ZOOKEEPER-3739 /
>>>>> https://github.com/apache/zookeeper/pull/1269)
>>>>> 
>>>>> So, I propose that we make JDK 11 the new minimum version to *build*
>>>>> ZooKeeper with. This would not change the runtime requirement, which
>>>>> would remain at JDK 8.
>>>>> 
>>>>> The only necessary change to make this happen would be to add the
>>>>> minimum Java version to the maven-enforcer-plugin (like
>>>>> 
>>>>> 
>>>> 
>> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
>>>>> )
>>>>> 
>>>>> This would allow ZooKeeper to to streamline its development process a
>>>>> little bit by reducing the amount of CI testing that is done as part
>>>>> of the build. In other words, we can drop the CI builds for JDK 8,
>>>>> which saves on build resources and time. The return on investment is
>>>>> so low for the JDK 8 builds anyway, because of the improved
>>>>> cross-compilation in newer JDKs. So, there's not much value in
>>>>> building on JDK 8 anyway.
>>>>> 
>>>>> Of course, I am only recommending this for *new* release lines,
>>>>> starting with ZooKeeper 3.7.0/master branch, because I would not want
>>>>> to change expectations for users who will build their own 3.5 and 3.6
>>>>> versions as they continue to have patch versions released.
>>>>> 
>>>>> What do you think?
>>>>> 
>>>>> Kind Regards,
>>>>> Christopher
>>>>> 
>>>> 
>> 
>> 

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Enrico Olivelli <eo...@gmail.com>]

Let me recap
- Christopher is proposing to move to JDK11
- ZooKeeper client and server are bundled and coded and tested together in
zookeeper-server
- Enrico is concerned about the need of testing ZooKeeper client on JDK8
(not a problem to move the server to JDK11)

ZooKeeper client is used by tons of users and unfortunately many projects
are still on JDK8, if we move ZooKeeper to JDK11 the risk is to block users
from the adoption,
that is that we will see the world to stay on 3.6.x and we will have again
a long lived release line, like 3.4.

Testing the client on JDK8 would be possible if we create some kind of
additional module with system tests, then we can start the server on docker
on JDK11+ and start a client on JDK8
with Maven toolchain it should possible to run surefire tests using a
separate JVM.

So in my vision 2 options:
1) fully JDK11 - drop JDK8 at all
2) build with JDK11 - server only on JDK11 - add system tests with docker
and toolchains that ensure the ZooKeeper client (and all dependencies)
still work on JDK8

From my point of view about the ZooKeeper ecosystem option 2) will be far
better, but we need resources to work on a new test suite.

Enrico


Il giorno mer 21 ott 2020 alle ore 10:43 Andor Molnar <an...@apache.org> ha
scritto:

> Tamas, Enrico,
>
> Sorry I don’t follow. Why do we have to test the client with JDK 8 in
> version 3.7.0?
>
> Andor
>
>
>
>
> > On 2020. Oct 20., at 22:29, Tamas Penzes <ta...@cloudera.com.INVALID>
> wrote:
> >
> > Hi Enrico,
> >
> > Separating ZooKeeper client and server is a huge work, but we might not
> > need it.
> > As you mentioned we have to test ZK client with Java 8, what about
> > separating only the test cases which we need to run with Java8 too?
> > In Curator we have the ZK compatibility tests where we run a limited
> amount
> > of Curator's jUnit tests with a different ZK version.
> > We might be able to do the same here, tag tests which are testing ZK
> client
> > and run them separately with Java 8. The only limitation is that these
> > tests must stay JDK8 compatible.
> > But from the tags we will see which ones are those.
> >
> > What do you think?
> >
> > Regards, Tamaas
> >
> > On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <eo...@gmail.com>
> wrote:
> >
> >> Christopher
> >> I appreciate your idea and I also moved lots of my projects to work the
> way
> >> you are suggesting.
> >>
> >>
> >> We must run tests using real jdk8 to test the Zookeeper client. We must
> >> ensure that Zookeeper works well, especially while dealing with security
> >> stuff.
> >> Currently the client is in the same module of the server and it will
> take a
> >> good (huge) amount of work to separate them
> >>
> >>
> >> Enrico
> >>
> >> Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org> ha scritto:
> >>
> >>> Hi ZK Devs,
> >>>
> >>> With recent advancements in Java (since Java 9), it is now generally
> >>> no longer necessary to require that software be developed on an older
> >>> JDK in order to have confidence that it will run on the older version
> >>> of Java. This is because, as of Java 9, all JDK releases have better
> >>> support for cross-compilation to older Java versions.
> >>>
> >>> What this means is that developers can confidently make the build
> >>> requirements for a project higher than the Java version that will
> >>> actually be supported at runtime.
> >>>
> >>> In fact, ZooKeeper already supports the necessary flags in its Maven
> >>> build configuration to ensure that it uses JDK 8 compliance when
> >>> building on a newer JDK (I added this way back in ZOOKEEPER-3739 /
> >>> https://github.com/apache/zookeeper/pull/1269)
> >>>
> >>> So, I propose that we make JDK 11 the new minimum version to *build*
> >>> ZooKeeper with. This would not change the runtime requirement, which
> >>> would remain at JDK 8.
> >>>
> >>> The only necessary change to make this happen would be to add the
> >>> minimum Java version to the maven-enforcer-plugin (like
> >>>
> >>>
> >>
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> >>> )
> >>>
> >>> This would allow ZooKeeper to to streamline its development process a
> >>> little bit by reducing the amount of CI testing that is done as part
> >>> of the build. In other words, we can drop the CI builds for JDK 8,
> >>> which saves on build resources and time. The return on investment is
> >>> so low for the JDK 8 builds anyway, because of the improved
> >>> cross-compilation in newer JDKs. So, there's not much value in
> >>> building on JDK 8 anyway.
> >>>
> >>> Of course, I am only recommending this for *new* release lines,
> >>> starting with ZooKeeper 3.7.0/master branch, because I would not want
> >>> to change expectations for users who will build their own 3.5 and 3.6
> >>> versions as they continue to have patch versions released.
> >>>
> >>> What do you think?
> >>>
> >>> Kind Regards,
> >>> Christopher
> >>>
> >>
>
>

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Andor Molnar <an...@apache.org>]

Tamas, Enrico, 

Sorry I don’t follow. Why do we have to test the client with JDK 8 in version 3.7.0?

Andor




> On 2020. Oct 20., at 22:29, Tamas Penzes <ta...@cloudera.com.INVALID> wrote:
> 
> Hi Enrico,
> 
> Separating ZooKeeper client and server is a huge work, but we might not
> need it.
> As you mentioned we have to test ZK client with Java 8, what about
> separating only the test cases which we need to run with Java8 too?
> In Curator we have the ZK compatibility tests where we run a limited amount
> of Curator's jUnit tests with a different ZK version.
> We might be able to do the same here, tag tests which are testing ZK client
> and run them separately with Java 8. The only limitation is that these
> tests must stay JDK8 compatible.
> But from the tags we will see which ones are those.
> 
> What do you think?
> 
> Regards, Tamaas
> 
> On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <eo...@gmail.com> wrote:
> 
>> Christopher
>> I appreciate your idea and I also moved lots of my projects to work the way
>> you are suggesting.
>> 
>> 
>> We must run tests using real jdk8 to test the Zookeeper client. We must
>> ensure that Zookeeper works well, especially while dealing with security
>> stuff.
>> Currently the client is in the same module of the server and it will take a
>> good (huge) amount of work to separate them
>> 
>> 
>> Enrico
>> 
>> Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org> ha scritto:
>> 
>>> Hi ZK Devs,
>>> 
>>> With recent advancements in Java (since Java 9), it is now generally
>>> no longer necessary to require that software be developed on an older
>>> JDK in order to have confidence that it will run on the older version
>>> of Java. This is because, as of Java 9, all JDK releases have better
>>> support for cross-compilation to older Java versions.
>>> 
>>> What this means is that developers can confidently make the build
>>> requirements for a project higher than the Java version that will
>>> actually be supported at runtime.
>>> 
>>> In fact, ZooKeeper already supports the necessary flags in its Maven
>>> build configuration to ensure that it uses JDK 8 compliance when
>>> building on a newer JDK (I added this way back in ZOOKEEPER-3739 /
>>> https://github.com/apache/zookeeper/pull/1269)
>>> 
>>> So, I propose that we make JDK 11 the new minimum version to *build*
>>> ZooKeeper with. This would not change the runtime requirement, which
>>> would remain at JDK 8.
>>> 
>>> The only necessary change to make this happen would be to add the
>>> minimum Java version to the maven-enforcer-plugin (like
>>> 
>>> 
>> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
>>> )
>>> 
>>> This would allow ZooKeeper to to streamline its development process a
>>> little bit by reducing the amount of CI testing that is done as part
>>> of the build. In other words, we can drop the CI builds for JDK 8,
>>> which saves on build resources and time. The return on investment is
>>> so low for the JDK 8 builds anyway, because of the improved
>>> cross-compilation in newer JDKs. So, there's not much value in
>>> building on JDK 8 anyway.
>>> 
>>> Of course, I am only recommending this for *new* release lines,
>>> starting with ZooKeeper 3.7.0/master branch, because I would not want
>>> to change expectations for users who will build their own 3.5 and 3.6
>>> versions as they continue to have patch versions released.
>>> 
>>> What do you think?
>>> 
>>> Kind Regards,
>>> Christopher
>>> 
>> 

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Tamas Penzes <ta...@cloudera.com.INVALID>]

Hi Enrico,

Separating ZooKeeper client and server is a huge work, but we might not
need it.
As you mentioned we have to test ZK client with Java 8, what about
separating only the test cases which we need to run with Java8 too?
In Curator we have the ZK compatibility tests where we run a limited amount
of Curator's jUnit tests with a different ZK version.
We might be able to do the same here, tag tests which are testing ZK client
and run them separately with Java 8. The only limitation is that these
tests must stay JDK8 compatible.
But from the tags we will see which ones are those.

What do you think?

Regards, Tamaas

On Sat, Oct 17, 2020 at 7:45 AM Enrico Olivelli <eo...@gmail.com> wrote:

> Christopher
> I appreciate your idea and I also moved lots of my projects to work the way
> you are suggesting.
>
>
> We must run tests using real jdk8 to test the Zookeeper client. We must
> ensure that Zookeeper works well, especially while dealing with security
> stuff.
> Currently the client is in the same module of the server and it will take a
> good (huge) amount of work to separate them
>
>
> Enrico
>
> Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org> ha scritto:
>
> > Hi ZK Devs,
> >
> > With recent advancements in Java (since Java 9), it is now generally
> > no longer necessary to require that software be developed on an older
> > JDK in order to have confidence that it will run on the older version
> > of Java. This is because, as of Java 9, all JDK releases have better
> > support for cross-compilation to older Java versions.
> >
> > What this means is that developers can confidently make the build
> > requirements for a project higher than the Java version that will
> > actually be supported at runtime.
> >
> > In fact, ZooKeeper already supports the necessary flags in its Maven
> > build configuration to ensure that it uses JDK 8 compliance when
> > building on a newer JDK (I added this way back in ZOOKEEPER-3739 /
> > https://github.com/apache/zookeeper/pull/1269)
> >
> > So, I propose that we make JDK 11 the new minimum version to *build*
> > ZooKeeper with. This would not change the runtime requirement, which
> > would remain at JDK 8.
> >
> > The only necessary change to make this happen would be to add the
> > minimum Java version to the maven-enforcer-plugin (like
> >
> >
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> > )
> >
> > This would allow ZooKeeper to to streamline its development process a
> > little bit by reducing the amount of CI testing that is done as part
> > of the build. In other words, we can drop the CI builds for JDK 8,
> > which saves on build resources and time. The return on investment is
> > so low for the JDK 8 builds anyway, because of the improved
> > cross-compilation in newer JDKs. So, there's not much value in
> > building on JDK 8 anyway.
> >
> > Of course, I am only recommending this for *new* release lines,
> > starting with ZooKeeper 3.7.0/master branch, because I would not want
> > to change expectations for users who will build their own 3.5 and 3.6
> > versions as they continue to have patch versions released.
> >
> > What do you think?
> >
> > Kind Regards,
> > Christopher
> >
>

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Andor Molnar <an...@apache.org>]

Hi,

Looks like Chris is very good at convincing people. :)

+1 for moving to JDK 11. Let’s go ahead.

Andor



> On 2020. Oct 17., at 12:50, Christopher <ct...@apache.org> wrote:
> 
> On Sat, Oct 17, 2020 at 1:45 AM Enrico Olivelli <eo...@gmail.com> wrote:
>> We must run tests using real jdk8 to test the Zookeeper client. We must
> 
> Hi Enrico,
> 
> I sort of understand your perspective, as it pertains to the
> *existing* versions that are already released with Java 8 support
> (3.5.x and 3.6.x), especially since the runtime testing workflow is so
> tightly coupled to the build. After all, you don't want to change user
> expectations in a patch release for those versions.
> 
> However, I don't understand continuing to have the same expectations
> for the *next*, currently not released, version of ZooKeeper (3.7.x)
> that is still under development. Now (before the ".0" release) is the
> perfect time to change the requirements and set new user expectations
> (to either bump it for the build only, as in my proposal, or to also
> bump it for the runtime requirement). Is there a good justification
> for holding back 3.7 to Java 8 and maintaining the same user
> expectations and requirements for 3.7 as 3.5/3.6?
> 
> My main concerns about holding back the build requirement to JDK 8 are:
>  1) extra testing burden, which will get worse when JDK 17 is
> released (which will probably happen while 3.7 is still being
> supported)
>  2) supporting build compatibility, runtime compatibility, and all
> the testing across three LTS versions of Java (8, 11, and 17), not to
> mention any intermediate non-LTS Java versions, is an enormous
> commitment for a small, volunteer-driven, open source project; I would
> generally expect the open source project to be more narrowly focused
> on fewer commitments, and letting downstream vendors/packagers pick up
> those additional commitments, and
>  3) Maven plugin developers aren't going to support JDK 8 forever.
> Many have already begun migrating to Java 11 as their minimum, which
> means those plugins won't be usable by people building their projects
> with JDK 8. It won't be long before a serious bug or security
> vulnerability in a build plugin affects projects' ability to build
> with JDK 8. In the interests of security and stability, it would be
> good to keep the build tooling up to date.
> 
> If the ZK project isn't ready to move to Java 11 for 3.7.0, not even
> for the build requirement, then when will it be ready (for either the
> build or runtime)? Has that already been discussed and decided by this
> project's PMC in a previous thread on the mailing lists? If so, where?
> 
> Regards,
> Christopher

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Christopher <ct...@apache.org>]

On Sat, Oct 17, 2020 at 1:45 AM Enrico Olivelli <eo...@gmail.com> wrote:
> We must run tests using real jdk8 to test the Zookeeper client. We must

Hi Enrico,

I sort of understand your perspective, as it pertains to the
*existing* versions that are already released with Java 8 support
(3.5.x and 3.6.x), especially since the runtime testing workflow is so
tightly coupled to the build. After all, you don't want to change user
expectations in a patch release for those versions.

However, I don't understand continuing to have the same expectations
for the *next*, currently not released, version of ZooKeeper (3.7.x)
that is still under development. Now (before the ".0" release) is the
perfect time to change the requirements and set new user expectations
(to either bump it for the build only, as in my proposal, or to also
bump it for the runtime requirement). Is there a good justification
for holding back 3.7 to Java 8 and maintaining the same user
expectations and requirements for 3.7 as 3.5/3.6?

My main concerns about holding back the build requirement to JDK 8 are:
  1) extra testing burden, which will get worse when JDK 17 is
released (which will probably happen while 3.7 is still being
supported)
  2) supporting build compatibility, runtime compatibility, and all
the testing across three LTS versions of Java (8, 11, and 17), not to
mention any intermediate non-LTS Java versions, is an enormous
commitment for a small, volunteer-driven, open source project; I would
generally expect the open source project to be more narrowly focused
on fewer commitments, and letting downstream vendors/packagers pick up
those additional commitments, and
  3) Maven plugin developers aren't going to support JDK 8 forever.
Many have already begun migrating to Java 11 as their minimum, which
means those plugins won't be usable by people building their projects
with JDK 8. It won't be long before a serious bug or security
vulnerability in a build plugin affects projects' ability to build
with JDK 8. In the interests of security and stability, it would be
good to keep the build tooling up to date.

If the ZK project isn't ready to move to Java 11 for 3.7.0, not even
for the build requirement, then when will it be ready (for either the
build or runtime)? Has that already been discussed and decided by this
project's PMC in a previous thread on the mailing lists? If so, where?

Regards,
Christopher

Re: [DISCUSS][PROPOSAL] Require JDK 11 to build for 3.7

[Enrico Olivelli <eo...@gmail.com>]

Christopher
I appreciate your idea and I also moved lots of my projects to work the way
you are suggesting.


We must run tests using real jdk8 to test the Zookeeper client. We must
ensure that Zookeeper works well, especially while dealing with security
stuff.
Currently the client is in the same module of the server and it will take a
good (huge) amount of work to separate them


Enrico

Il Ven 16 Ott 2020, 23:25 Christopher <ct...@apache.org> ha scritto:

> Hi ZK Devs,
>
> With recent advancements in Java (since Java 9), it is now generally
> no longer necessary to require that software be developed on an older
> JDK in order to have confidence that it will run on the older version
> of Java. This is because, as of Java 9, all JDK releases have better
> support for cross-compilation to older Java versions.
>
> What this means is that developers can confidently make the build
> requirements for a project higher than the Java version that will
> actually be supported at runtime.
>
> In fact, ZooKeeper already supports the necessary flags in its Maven
> build configuration to ensure that it uses JDK 8 compliance when
> building on a newer JDK (I added this way back in ZOOKEEPER-3739 /
> https://github.com/apache/zookeeper/pull/1269)
>
> So, I propose that we make JDK 11 the new minimum version to *build*
> ZooKeeper with. This would not change the runtime requirement, which
> would remain at JDK 8.
>
> The only necessary change to make this happen would be to add the
> minimum Java version to the maven-enforcer-plugin (like
>
> https://github.com/apache/accumulo/blob/438f0efd34ef9d200bc8c7ecdd11d5dedb146519/pom.xml#L1162-L1164
> )
>
> This would allow ZooKeeper to to streamline its development process a
> little bit by reducing the amount of CI testing that is done as part
> of the build. In other words, we can drop the CI builds for JDK 8,
> which saves on build resources and time. The return on investment is
> so low for the JDK 8 builds anyway, because of the improved
> cross-compilation in newer JDKs. So, there's not much value in
> building on JDK 8 anyway.
>
> Of course, I am only recommending this for *new* release lines,
> starting with ZooKeeper 3.7.0/master branch, because I would not want
> to change expectations for users who will build their own 3.5 and 3.6
> versions as they continue to have patch versions released.
>
> What do you think?
>
> Kind Regards,
> Christopher
>