You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Steve Loughran <st...@cloudera.com.INVALID> on 2023/03/28 11:54:22 UTC
[DISCUSS] hadoop branch-3.3+ going to java11 only
Now that hadoop 3.3.5 is out, i want to propose something new
we switch branch-3.3 and trunk to being java11 only
1. java 11 has been out for years
2. oracle java 8 is no longer available under "premier support"; you
can't really get upgrades
https://www.oracle.com/java/technologies/java-se-support-roadmap.html
3. openJDK 8 releases != oracle ones, and things you compile with them
don't always link to oracle java 8 (some classes in java.nio have added
more overrides)
4. more and more libraries we want to upgrade to/bundle are java 11 only
5. moving to java 11 would cut our yetus build workload in half, and
line up for adding java 17 builds instead.
I know there are some outstanding issues still in
https://issues.apache.org/jira/browse/HADOOP-16795 -but are they blockers?
Could we just move to java11 and enhance at our leisure, once java8 is no
longer a concern.
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Nick Dimiduk <nd...@apache.org>.
On Tue, Mar 28, 2023 at 1:56 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> Now that hadoop 3.3.5 is out, i want to propose something new
>
> we switch branch-3.3 and trunk to being java11 only
>
I agree that maintaining support for Java8 is becoming burdensome. Over in
HBase, we maintain support for both Java8 and Hadoop2 :(
We could roll-forward the code lines: branch from rel/3.3.5 as a new
branch-3.4 and trunk version bumps to 3.5 -- assuming the compatibility
guidelines permit dropping the JVM version on a minor release. If not,
rel/3.3.5 branches to branch-4 and trunk changes accordingly.
1. java 11 has been out for years
> 2. oracle java 8 is no longer available under "premier support"; you
> can't really get upgrades
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> 3. openJDK 8 releases != oracle ones, and things you compile with them
> don't always link to oracle java 8 (some classes in java.nio have added
> more overrides)
> 4. more and more libraries we want to upgrade to/bundle are java 11 only
> 5. moving to java 11 would cut our yetus build workload in half, and
> line up for adding java 17 builds instead.
>
>
> I know there are some outstanding issues still in
> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they blockers?
> Could we just move to java11 and enhance at our leisure, once java8 is no
> longer a concern.
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Iñigo Goiri <el...@gmail.com>.
I would also vote for targeting 3.4 and have a long term version of Java
there.
On Tue, Mar 28, 2023 at 11:52 AM Igor Dvorzhak <id...@google.com.invalid>
wrote:
> +1 to re-focusing on 3.4 branch and upgrading it to Java 11/17, instead of
> making potentially breaking changes to 3.3.
>
> On Tue, Mar 28, 2023 at 11:17 AM Chris Nauroth <cn...@gmail.com> wrote:
>
>> In theory, I like the idea of setting aside Java 8. Unfortunately, I don't
>> know that upgrading within the 3.3 line adheres to our binary
>> compatibility
>> policy [1]. I don't see specific discussion of the Java version there, but
>> it states that you should be able to drop in minor upgrades and have
>> existing apps keep working. Users might find it surprising if they try to
>> upgrade a cluster that has JDK 8.
>>
>> There is also the question of impact on downstream projects [2]. We'd have
>> to check plans with our consumers.
>>
>> What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)?
>> The downside is that we'd probably need to set boundaries on end of
>> life/limited support for 3.2 and 3.3 to keep the workload manageable.
>>
>> [1]
>>
>> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI
>> [2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109
>>
>> Chris Nauroth
>>
>>
>> On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena <ay...@gmail.com> wrote:
>>
>> > >
>> > > it's already hard to migrate from JDK8 why not retarget JDK17.
>> > >
>> >
>> > +1, makes sense to me, sounds like a win-win situation to me, though
>> there
>> > would be some additional issues to chase now :)
>> >
>> > -Ayush
>> >
>> >
>> > On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org>
>> wrote:
>> >
>> > > My random thoughts. Probably bad takes:
>> > >
>> > > There are projects experimenting with JDK17 now.
>> > > JDK11 active support will end in 6 months. If it's already hard to
>> > migrate
>> > > from JDK8 why not retarget JDK17.
>> > >
>> > > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com>
>> > wrote:
>> > >
>> > >> I know Jersey upgrade as a blocker. Some folks were chasing that last
>> > >> year during 3.3.4 time, I don’t know where it is now, didn’t see then
>> > >> what’s the problem there but I remember there was some intitial PR
>> which
>> > >> did it for HDFS atleast, so I never looked beyond that…
>> > >>
>> > >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
>> > >> java-11 only branch may be, but that is something later to decide,
>> once
>> > we
>> > >> get the code sorted…
>> > >>
>> > >> -Ayush
>> > >>
>> > >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran
>> > <st...@cloudera.com.invalid>
>> > >> wrote:
>> > >> >
>> > >> > well, how about we flip the switch and get on with it.
>> > >> >
>> > >> > slf4j seems happy on java11,
>> > >> >
>> > >> > side issue, anyone seen test failures on zulu1.8; somehow my test
>> run
>> > is
>> > >> > failing and i'm trying to work out whether its a mismatch in
>> command
>> > >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an
>> > openjdk
>> > >> > version which requires IntBuffer implements an overridden method
>> > >> IntBuffer
>> > >> > rewind().
>> > >> >
>> > >> > java.lang.NoSuchMethodError:
>> > >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
>> > >> >
>> > >> > at
>> > >>
>> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
>> > >> > at
>> > >> >
>> > >>
>> >
>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
>> > >> > at
>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
>> > >> > at
>> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
>> > >> > at java.io.DataInputStream.read(DataInputStream.java:149)
>> > >> >
>> > >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org>
>> > wrote:
>> > >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
>> > >> jersey 1
>> > >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
>> > >> stability.
>> > >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
>> > >> <stevel@cloudera.com.invalid
>> > >> >> wrote:
>> > >> >>> Now that hadoop 3.3.5 is out, i want to propose something new
>> > >> >>> we switch branch-3.3 and trunk to being java11 only
>> > >> >>> 1. java 11 has been out for years
>> > >> >>> 2. oracle java 8 is no longer available under "premier support";
>> you
>> > >> >>> can't really get upgrades
>> > >> >>>
>> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>> > >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with
>> > them
>> > >> >>> don't always link to oracle java 8 (some classes in java.nio have
>> > >> >> added
>> > >> >>> more overrides)
>> > >> >>> 4. more and more libraries we want to upgrade to/bundle are java
>> 11
>> > >> >> only
>> > >> >>> 5. moving to java 11 would cut our yetus build workload in half,
>> and
>> > >> >>> line up for adding java 17 builds instead.
>> > >> >>> I know there are some outstanding issues still in
>> > >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
>> > >> >> blockers?
>> > >> >>> Could we just move to java11 and enhance at our leisure, once
>> java8
>> > >> is no
>> > >> >>> longer a concern.
>> > >>
>> > >> ---------------------------------------------------------------------
>> > >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> > >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>> > >>
>> > >>
>> >
>>
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Iñigo Goiri <el...@gmail.com>.
I would also vote for targeting 3.4 and have a long term version of Java
there.
On Tue, Mar 28, 2023 at 11:52 AM Igor Dvorzhak <id...@google.com.invalid>
wrote:
> +1 to re-focusing on 3.4 branch and upgrading it to Java 11/17, instead of
> making potentially breaking changes to 3.3.
>
> On Tue, Mar 28, 2023 at 11:17 AM Chris Nauroth <cn...@gmail.com> wrote:
>
>> In theory, I like the idea of setting aside Java 8. Unfortunately, I don't
>> know that upgrading within the 3.3 line adheres to our binary
>> compatibility
>> policy [1]. I don't see specific discussion of the Java version there, but
>> it states that you should be able to drop in minor upgrades and have
>> existing apps keep working. Users might find it surprising if they try to
>> upgrade a cluster that has JDK 8.
>>
>> There is also the question of impact on downstream projects [2]. We'd have
>> to check plans with our consumers.
>>
>> What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)?
>> The downside is that we'd probably need to set boundaries on end of
>> life/limited support for 3.2 and 3.3 to keep the workload manageable.
>>
>> [1]
>>
>> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI
>> [2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109
>>
>> Chris Nauroth
>>
>>
>> On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena <ay...@gmail.com> wrote:
>>
>> > >
>> > > it's already hard to migrate from JDK8 why not retarget JDK17.
>> > >
>> >
>> > +1, makes sense to me, sounds like a win-win situation to me, though
>> there
>> > would be some additional issues to chase now :)
>> >
>> > -Ayush
>> >
>> >
>> > On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org>
>> wrote:
>> >
>> > > My random thoughts. Probably bad takes:
>> > >
>> > > There are projects experimenting with JDK17 now.
>> > > JDK11 active support will end in 6 months. If it's already hard to
>> > migrate
>> > > from JDK8 why not retarget JDK17.
>> > >
>> > > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com>
>> > wrote:
>> > >
>> > >> I know Jersey upgrade as a blocker. Some folks were chasing that last
>> > >> year during 3.3.4 time, I don’t know where it is now, didn’t see then
>> > >> what’s the problem there but I remember there was some intitial PR
>> which
>> > >> did it for HDFS atleast, so I never looked beyond that…
>> > >>
>> > >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
>> > >> java-11 only branch may be, but that is something later to decide,
>> once
>> > we
>> > >> get the code sorted…
>> > >>
>> > >> -Ayush
>> > >>
>> > >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran
>> > <st...@cloudera.com.invalid>
>> > >> wrote:
>> > >> >
>> > >> > well, how about we flip the switch and get on with it.
>> > >> >
>> > >> > slf4j seems happy on java11,
>> > >> >
>> > >> > side issue, anyone seen test failures on zulu1.8; somehow my test
>> run
>> > is
>> > >> > failing and i'm trying to work out whether its a mismatch in
>> command
>> > >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an
>> > openjdk
>> > >> > version which requires IntBuffer implements an overridden method
>> > >> IntBuffer
>> > >> > rewind().
>> > >> >
>> > >> > java.lang.NoSuchMethodError:
>> > >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
>> > >> >
>> > >> > at
>> > >>
>> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
>> > >> > at
>> > >> >
>> > >>
>> >
>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
>> > >> > at
>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
>> > >> > at
>> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
>> > >> > at java.io.DataInputStream.read(DataInputStream.java:149)
>> > >> >
>> > >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org>
>> > wrote:
>> > >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
>> > >> jersey 1
>> > >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
>> > >> stability.
>> > >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
>> > >> <stevel@cloudera.com.invalid
>> > >> >> wrote:
>> > >> >>> Now that hadoop 3.3.5 is out, i want to propose something new
>> > >> >>> we switch branch-3.3 and trunk to being java11 only
>> > >> >>> 1. java 11 has been out for years
>> > >> >>> 2. oracle java 8 is no longer available under "premier support";
>> you
>> > >> >>> can't really get upgrades
>> > >> >>>
>> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>> > >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with
>> > them
>> > >> >>> don't always link to oracle java 8 (some classes in java.nio have
>> > >> >> added
>> > >> >>> more overrides)
>> > >> >>> 4. more and more libraries we want to upgrade to/bundle are java
>> 11
>> > >> >> only
>> > >> >>> 5. moving to java 11 would cut our yetus build workload in half,
>> and
>> > >> >>> line up for adding java 17 builds instead.
>> > >> >>> I know there are some outstanding issues still in
>> > >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
>> > >> >> blockers?
>> > >> >>> Could we just move to java11 and enhance at our leisure, once
>> java8
>> > >> is no
>> > >> >>> longer a concern.
>> > >>
>> > >> ---------------------------------------------------------------------
>> > >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> > >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>> > >>
>> > >>
>> >
>>
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Iñigo Goiri <el...@gmail.com>.
I would also vote for targeting 3.4 and have a long term version of Java
there.
On Tue, Mar 28, 2023 at 11:52 AM Igor Dvorzhak <id...@google.com.invalid>
wrote:
> +1 to re-focusing on 3.4 branch and upgrading it to Java 11/17, instead of
> making potentially breaking changes to 3.3.
>
> On Tue, Mar 28, 2023 at 11:17 AM Chris Nauroth <cn...@gmail.com> wrote:
>
>> In theory, I like the idea of setting aside Java 8. Unfortunately, I don't
>> know that upgrading within the 3.3 line adheres to our binary
>> compatibility
>> policy [1]. I don't see specific discussion of the Java version there, but
>> it states that you should be able to drop in minor upgrades and have
>> existing apps keep working. Users might find it surprising if they try to
>> upgrade a cluster that has JDK 8.
>>
>> There is also the question of impact on downstream projects [2]. We'd have
>> to check plans with our consumers.
>>
>> What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)?
>> The downside is that we'd probably need to set boundaries on end of
>> life/limited support for 3.2 and 3.3 to keep the workload manageable.
>>
>> [1]
>>
>> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI
>> [2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109
>>
>> Chris Nauroth
>>
>>
>> On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena <ay...@gmail.com> wrote:
>>
>> > >
>> > > it's already hard to migrate from JDK8 why not retarget JDK17.
>> > >
>> >
>> > +1, makes sense to me, sounds like a win-win situation to me, though
>> there
>> > would be some additional issues to chase now :)
>> >
>> > -Ayush
>> >
>> >
>> > On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org>
>> wrote:
>> >
>> > > My random thoughts. Probably bad takes:
>> > >
>> > > There are projects experimenting with JDK17 now.
>> > > JDK11 active support will end in 6 months. If it's already hard to
>> > migrate
>> > > from JDK8 why not retarget JDK17.
>> > >
>> > > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com>
>> > wrote:
>> > >
>> > >> I know Jersey upgrade as a blocker. Some folks were chasing that last
>> > >> year during 3.3.4 time, I don’t know where it is now, didn’t see then
>> > >> what’s the problem there but I remember there was some intitial PR
>> which
>> > >> did it for HDFS atleast, so I never looked beyond that…
>> > >>
>> > >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
>> > >> java-11 only branch may be, but that is something later to decide,
>> once
>> > we
>> > >> get the code sorted…
>> > >>
>> > >> -Ayush
>> > >>
>> > >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran
>> > <st...@cloudera.com.invalid>
>> > >> wrote:
>> > >> >
>> > >> > well, how about we flip the switch and get on with it.
>> > >> >
>> > >> > slf4j seems happy on java11,
>> > >> >
>> > >> > side issue, anyone seen test failures on zulu1.8; somehow my test
>> run
>> > is
>> > >> > failing and i'm trying to work out whether its a mismatch in
>> command
>> > >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an
>> > openjdk
>> > >> > version which requires IntBuffer implements an overridden method
>> > >> IntBuffer
>> > >> > rewind().
>> > >> >
>> > >> > java.lang.NoSuchMethodError:
>> > >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
>> > >> >
>> > >> > at
>> > >>
>> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
>> > >> > at
>> > >> >
>> > >>
>> >
>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
>> > >> > at
>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
>> > >> > at
>> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
>> > >> > at java.io.DataInputStream.read(DataInputStream.java:149)
>> > >> >
>> > >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org>
>> > wrote:
>> > >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
>> > >> jersey 1
>> > >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
>> > >> stability.
>> > >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
>> > >> <stevel@cloudera.com.invalid
>> > >> >> wrote:
>> > >> >>> Now that hadoop 3.3.5 is out, i want to propose something new
>> > >> >>> we switch branch-3.3 and trunk to being java11 only
>> > >> >>> 1. java 11 has been out for years
>> > >> >>> 2. oracle java 8 is no longer available under "premier support";
>> you
>> > >> >>> can't really get upgrades
>> > >> >>>
>> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>> > >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with
>> > them
>> > >> >>> don't always link to oracle java 8 (some classes in java.nio have
>> > >> >> added
>> > >> >>> more overrides)
>> > >> >>> 4. more and more libraries we want to upgrade to/bundle are java
>> 11
>> > >> >> only
>> > >> >>> 5. moving to java 11 would cut our yetus build workload in half,
>> and
>> > >> >>> line up for adding java 17 builds instead.
>> > >> >>> I know there are some outstanding issues still in
>> > >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
>> > >> >> blockers?
>> > >> >>> Could we just move to java11 and enhance at our leisure, once
>> java8
>> > >> is no
>> > >> >>> longer a concern.
>> > >>
>> > >> ---------------------------------------------------------------------
>> > >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> > >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>> > >>
>> > >>
>> >
>>
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Iñigo Goiri <el...@gmail.com>.
I would also vote for targeting 3.4 and have a long term version of Java
there.
On Tue, Mar 28, 2023 at 11:52 AM Igor Dvorzhak <id...@google.com.invalid>
wrote:
> +1 to re-focusing on 3.4 branch and upgrading it to Java 11/17, instead of
> making potentially breaking changes to 3.3.
>
> On Tue, Mar 28, 2023 at 11:17 AM Chris Nauroth <cn...@gmail.com> wrote:
>
>> In theory, I like the idea of setting aside Java 8. Unfortunately, I don't
>> know that upgrading within the 3.3 line adheres to our binary
>> compatibility
>> policy [1]. I don't see specific discussion of the Java version there, but
>> it states that you should be able to drop in minor upgrades and have
>> existing apps keep working. Users might find it surprising if they try to
>> upgrade a cluster that has JDK 8.
>>
>> There is also the question of impact on downstream projects [2]. We'd have
>> to check plans with our consumers.
>>
>> What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)?
>> The downside is that we'd probably need to set boundaries on end of
>> life/limited support for 3.2 and 3.3 to keep the workload manageable.
>>
>> [1]
>>
>> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI
>> [2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109
>>
>> Chris Nauroth
>>
>>
>> On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena <ay...@gmail.com> wrote:
>>
>> > >
>> > > it's already hard to migrate from JDK8 why not retarget JDK17.
>> > >
>> >
>> > +1, makes sense to me, sounds like a win-win situation to me, though
>> there
>> > would be some additional issues to chase now :)
>> >
>> > -Ayush
>> >
>> >
>> > On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org>
>> wrote:
>> >
>> > > My random thoughts. Probably bad takes:
>> > >
>> > > There are projects experimenting with JDK17 now.
>> > > JDK11 active support will end in 6 months. If it's already hard to
>> > migrate
>> > > from JDK8 why not retarget JDK17.
>> > >
>> > > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com>
>> > wrote:
>> > >
>> > >> I know Jersey upgrade as a blocker. Some folks were chasing that last
>> > >> year during 3.3.4 time, I don’t know where it is now, didn’t see then
>> > >> what’s the problem there but I remember there was some intitial PR
>> which
>> > >> did it for HDFS atleast, so I never looked beyond that…
>> > >>
>> > >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
>> > >> java-11 only branch may be, but that is something later to decide,
>> once
>> > we
>> > >> get the code sorted…
>> > >>
>> > >> -Ayush
>> > >>
>> > >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran
>> > <st...@cloudera.com.invalid>
>> > >> wrote:
>> > >> >
>> > >> > well, how about we flip the switch and get on with it.
>> > >> >
>> > >> > slf4j seems happy on java11,
>> > >> >
>> > >> > side issue, anyone seen test failures on zulu1.8; somehow my test
>> run
>> > is
>> > >> > failing and i'm trying to work out whether its a mismatch in
>> command
>> > >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an
>> > openjdk
>> > >> > version which requires IntBuffer implements an overridden method
>> > >> IntBuffer
>> > >> > rewind().
>> > >> >
>> > >> > java.lang.NoSuchMethodError:
>> > >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
>> > >> >
>> > >> > at
>> > >>
>> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
>> > >> > at
>> > >> >
>> > >>
>> >
>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
>> > >> > at
>> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
>> > >> > at
>> org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
>> > >> > at java.io.DataInputStream.read(DataInputStream.java:149)
>> > >> >
>> > >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org>
>> > wrote:
>> > >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
>> > >> jersey 1
>> > >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
>> > >> stability.
>> > >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
>> > >> <stevel@cloudera.com.invalid
>> > >> >> wrote:
>> > >> >>> Now that hadoop 3.3.5 is out, i want to propose something new
>> > >> >>> we switch branch-3.3 and trunk to being java11 only
>> > >> >>> 1. java 11 has been out for years
>> > >> >>> 2. oracle java 8 is no longer available under "premier support";
>> you
>> > >> >>> can't really get upgrades
>> > >> >>>
>> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>> > >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with
>> > them
>> > >> >>> don't always link to oracle java 8 (some classes in java.nio have
>> > >> >> added
>> > >> >>> more overrides)
>> > >> >>> 4. more and more libraries we want to upgrade to/bundle are java
>> 11
>> > >> >> only
>> > >> >>> 5. moving to java 11 would cut our yetus build workload in half,
>> and
>> > >> >>> line up for adding java 17 builds instead.
>> > >> >>> I know there are some outstanding issues still in
>> > >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
>> > >> >> blockers?
>> > >> >>> Could we just move to java11 and enhance at our leisure, once
>> java8
>> > >> is no
>> > >> >>> longer a concern.
>> > >>
>> > >> ---------------------------------------------------------------------
>> > >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> > >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>> > >>
>> > >>
>> >
>>
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Igor Dvorzhak <id...@google.com.INVALID>.
+1 to re-focusing on 3.4 branch and upgrading it to Java 11/17, instead of
making potentially breaking changes to 3.3.
On Tue, Mar 28, 2023 at 11:17 AM Chris Nauroth <cn...@gmail.com> wrote:
> In theory, I like the idea of setting aside Java 8. Unfortunately, I don't
> know that upgrading within the 3.3 line adheres to our binary compatibility
> policy [1]. I don't see specific discussion of the Java version there, but
> it states that you should be able to drop in minor upgrades and have
> existing apps keep working. Users might find it surprising if they try to
> upgrade a cluster that has JDK 8.
>
> There is also the question of impact on downstream projects [2]. We'd have
> to check plans with our consumers.
>
> What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)?
> The downside is that we'd probably need to set boundaries on end of
> life/limited support for 3.2 and 3.3 to keep the workload manageable.
>
> [1]
>
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI
> [2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109
>
> Chris Nauroth
>
>
> On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena <ay...@gmail.com> wrote:
>
> > >
> > > it's already hard to migrate from JDK8 why not retarget JDK17.
> > >
> >
> > +1, makes sense to me, sounds like a win-win situation to me, though
> there
> > would be some additional issues to chase now :)
> >
> > -Ayush
> >
> >
> > On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org>
> wrote:
> >
> > > My random thoughts. Probably bad takes:
> > >
> > > There are projects experimenting with JDK17 now.
> > > JDK11 active support will end in 6 months. If it's already hard to
> > migrate
> > > from JDK8 why not retarget JDK17.
> > >
> > > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com>
> > wrote:
> > >
> > >> I know Jersey upgrade as a blocker. Some folks were chasing that last
> > >> year during 3.3.4 time, I don’t know where it is now, didn’t see then
> > >> what’s the problem there but I remember there was some intitial PR
> which
> > >> did it for HDFS atleast, so I never looked beyond that…
> > >>
> > >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
> > >> java-11 only branch may be, but that is something later to decide,
> once
> > we
> > >> get the code sorted…
> > >>
> > >> -Ayush
> > >>
> > >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran
> > <st...@cloudera.com.invalid>
> > >> wrote:
> > >> >
> > >> > well, how about we flip the switch and get on with it.
> > >> >
> > >> > slf4j seems happy on java11,
> > >> >
> > >> > side issue, anyone seen test failures on zulu1.8; somehow my test
> run
> > is
> > >> > failing and i'm trying to work out whether its a mismatch in command
> > >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an
> > openjdk
> > >> > version which requires IntBuffer implements an overridden method
> > >> IntBuffer
> > >> > rewind().
> > >> >
> > >> > java.lang.NoSuchMethodError:
> > >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
> > >> >
> > >> > at
> > >>
> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> > >> > at
> > >> >
> > >>
> >
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> > >> > at
> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> > >> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> > >> > at java.io.DataInputStream.read(DataInputStream.java:149)
> > >> >
> > >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org>
> > wrote:
> > >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
> > >> jersey 1
> > >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
> > >> stability.
> > >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
> > >> <stevel@cloudera.com.invalid
> > >> >> wrote:
> > >> >>> Now that hadoop 3.3.5 is out, i want to propose something new
> > >> >>> we switch branch-3.3 and trunk to being java11 only
> > >> >>> 1. java 11 has been out for years
> > >> >>> 2. oracle java 8 is no longer available under "premier support";
> you
> > >> >>> can't really get upgrades
> > >> >>>
> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> > >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with
> > them
> > >> >>> don't always link to oracle java 8 (some classes in java.nio have
> > >> >> added
> > >> >>> more overrides)
> > >> >>> 4. more and more libraries we want to upgrade to/bundle are java
> 11
> > >> >> only
> > >> >>> 5. moving to java 11 would cut our yetus build workload in half,
> and
> > >> >>> line up for adding java 17 builds instead.
> > >> >>> I know there are some outstanding issues still in
> > >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> > >> >> blockers?
> > >> >>> Could we just move to java11 and enhance at our leisure, once
> java8
> > >> is no
> > >> >>> longer a concern.
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >>
> > >>
> >
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Igor Dvorzhak <id...@google.com.INVALID>.
+1 to re-focusing on 3.4 branch and upgrading it to Java 11/17, instead of
making potentially breaking changes to 3.3.
On Tue, Mar 28, 2023 at 11:17 AM Chris Nauroth <cn...@gmail.com> wrote:
> In theory, I like the idea of setting aside Java 8. Unfortunately, I don't
> know that upgrading within the 3.3 line adheres to our binary compatibility
> policy [1]. I don't see specific discussion of the Java version there, but
> it states that you should be able to drop in minor upgrades and have
> existing apps keep working. Users might find it surprising if they try to
> upgrade a cluster that has JDK 8.
>
> There is also the question of impact on downstream projects [2]. We'd have
> to check plans with our consumers.
>
> What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)?
> The downside is that we'd probably need to set boundaries on end of
> life/limited support for 3.2 and 3.3 to keep the workload manageable.
>
> [1]
>
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI
> [2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109
>
> Chris Nauroth
>
>
> On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena <ay...@gmail.com> wrote:
>
> > >
> > > it's already hard to migrate from JDK8 why not retarget JDK17.
> > >
> >
> > +1, makes sense to me, sounds like a win-win situation to me, though
> there
> > would be some additional issues to chase now :)
> >
> > -Ayush
> >
> >
> > On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org>
> wrote:
> >
> > > My random thoughts. Probably bad takes:
> > >
> > > There are projects experimenting with JDK17 now.
> > > JDK11 active support will end in 6 months. If it's already hard to
> > migrate
> > > from JDK8 why not retarget JDK17.
> > >
> > > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com>
> > wrote:
> > >
> > >> I know Jersey upgrade as a blocker. Some folks were chasing that last
> > >> year during 3.3.4 time, I don’t know where it is now, didn’t see then
> > >> what’s the problem there but I remember there was some intitial PR
> which
> > >> did it for HDFS atleast, so I never looked beyond that…
> > >>
> > >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
> > >> java-11 only branch may be, but that is something later to decide,
> once
> > we
> > >> get the code sorted…
> > >>
> > >> -Ayush
> > >>
> > >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran
> > <st...@cloudera.com.invalid>
> > >> wrote:
> > >> >
> > >> > well, how about we flip the switch and get on with it.
> > >> >
> > >> > slf4j seems happy on java11,
> > >> >
> > >> > side issue, anyone seen test failures on zulu1.8; somehow my test
> run
> > is
> > >> > failing and i'm trying to work out whether its a mismatch in command
> > >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an
> > openjdk
> > >> > version which requires IntBuffer implements an overridden method
> > >> IntBuffer
> > >> > rewind().
> > >> >
> > >> > java.lang.NoSuchMethodError:
> > >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
> > >> >
> > >> > at
> > >>
> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> > >> > at
> > >> >
> > >>
> >
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> > >> > at
> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> > >> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> > >> > at java.io.DataInputStream.read(DataInputStream.java:149)
> > >> >
> > >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org>
> > wrote:
> > >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
> > >> jersey 1
> > >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
> > >> stability.
> > >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
> > >> <stevel@cloudera.com.invalid
> > >> >> wrote:
> > >> >>> Now that hadoop 3.3.5 is out, i want to propose something new
> > >> >>> we switch branch-3.3 and trunk to being java11 only
> > >> >>> 1. java 11 has been out for years
> > >> >>> 2. oracle java 8 is no longer available under "premier support";
> you
> > >> >>> can't really get upgrades
> > >> >>>
> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> > >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with
> > them
> > >> >>> don't always link to oracle java 8 (some classes in java.nio have
> > >> >> added
> > >> >>> more overrides)
> > >> >>> 4. more and more libraries we want to upgrade to/bundle are java
> 11
> > >> >> only
> > >> >>> 5. moving to java 11 would cut our yetus build workload in half,
> and
> > >> >>> line up for adding java 17 builds instead.
> > >> >>> I know there are some outstanding issues still in
> > >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> > >> >> blockers?
> > >> >>> Could we just move to java11 and enhance at our leisure, once
> java8
> > >> is no
> > >> >>> longer a concern.
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >>
> > >>
> >
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Igor Dvorzhak <id...@google.com.INVALID>.
+1 to re-focusing on 3.4 branch and upgrading it to Java 11/17, instead of
making potentially breaking changes to 3.3.
On Tue, Mar 28, 2023 at 11:17 AM Chris Nauroth <cn...@gmail.com> wrote:
> In theory, I like the idea of setting aside Java 8. Unfortunately, I don't
> know that upgrading within the 3.3 line adheres to our binary compatibility
> policy [1]. I don't see specific discussion of the Java version there, but
> it states that you should be able to drop in minor upgrades and have
> existing apps keep working. Users might find it surprising if they try to
> upgrade a cluster that has JDK 8.
>
> There is also the question of impact on downstream projects [2]. We'd have
> to check plans with our consumers.
>
> What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)?
> The downside is that we'd probably need to set boundaries on end of
> life/limited support for 3.2 and 3.3 to keep the workload manageable.
>
> [1]
>
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI
> [2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109
>
> Chris Nauroth
>
>
> On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena <ay...@gmail.com> wrote:
>
> > >
> > > it's already hard to migrate from JDK8 why not retarget JDK17.
> > >
> >
> > +1, makes sense to me, sounds like a win-win situation to me, though
> there
> > would be some additional issues to chase now :)
> >
> > -Ayush
> >
> >
> > On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org>
> wrote:
> >
> > > My random thoughts. Probably bad takes:
> > >
> > > There are projects experimenting with JDK17 now.
> > > JDK11 active support will end in 6 months. If it's already hard to
> > migrate
> > > from JDK8 why not retarget JDK17.
> > >
> > > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com>
> > wrote:
> > >
> > >> I know Jersey upgrade as a blocker. Some folks were chasing that last
> > >> year during 3.3.4 time, I don’t know where it is now, didn’t see then
> > >> what’s the problem there but I remember there was some intitial PR
> which
> > >> did it for HDFS atleast, so I never looked beyond that…
> > >>
> > >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
> > >> java-11 only branch may be, but that is something later to decide,
> once
> > we
> > >> get the code sorted…
> > >>
> > >> -Ayush
> > >>
> > >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran
> > <st...@cloudera.com.invalid>
> > >> wrote:
> > >> >
> > >> > well, how about we flip the switch and get on with it.
> > >> >
> > >> > slf4j seems happy on java11,
> > >> >
> > >> > side issue, anyone seen test failures on zulu1.8; somehow my test
> run
> > is
> > >> > failing and i'm trying to work out whether its a mismatch in command
> > >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an
> > openjdk
> > >> > version which requires IntBuffer implements an overridden method
> > >> IntBuffer
> > >> > rewind().
> > >> >
> > >> > java.lang.NoSuchMethodError:
> > >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
> > >> >
> > >> > at
> > >>
> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> > >> > at
> > >> >
> > >>
> >
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> > >> > at
> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> > >> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> > >> > at java.io.DataInputStream.read(DataInputStream.java:149)
> > >> >
> > >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org>
> > wrote:
> > >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
> > >> jersey 1
> > >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
> > >> stability.
> > >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
> > >> <stevel@cloudera.com.invalid
> > >> >> wrote:
> > >> >>> Now that hadoop 3.3.5 is out, i want to propose something new
> > >> >>> we switch branch-3.3 and trunk to being java11 only
> > >> >>> 1. java 11 has been out for years
> > >> >>> 2. oracle java 8 is no longer available under "premier support";
> you
> > >> >>> can't really get upgrades
> > >> >>>
> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> > >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with
> > them
> > >> >>> don't always link to oracle java 8 (some classes in java.nio have
> > >> >> added
> > >> >>> more overrides)
> > >> >>> 4. more and more libraries we want to upgrade to/bundle are java
> 11
> > >> >> only
> > >> >>> 5. moving to java 11 would cut our yetus build workload in half,
> and
> > >> >>> line up for adding java 17 builds instead.
> > >> >>> I know there are some outstanding issues still in
> > >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> > >> >> blockers?
> > >> >>> Could we just move to java11 and enhance at our leisure, once
> java8
> > >> is no
> > >> >>> longer a concern.
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >>
> > >>
> >
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Igor Dvorzhak <id...@google.com.INVALID>.
+1 to re-focusing on 3.4 branch and upgrading it to Java 11/17, instead of
making potentially breaking changes to 3.3.
On Tue, Mar 28, 2023 at 11:17 AM Chris Nauroth <cn...@gmail.com> wrote:
> In theory, I like the idea of setting aside Java 8. Unfortunately, I don't
> know that upgrading within the 3.3 line adheres to our binary compatibility
> policy [1]. I don't see specific discussion of the Java version there, but
> it states that you should be able to drop in minor upgrades and have
> existing apps keep working. Users might find it surprising if they try to
> upgrade a cluster that has JDK 8.
>
> There is also the question of impact on downstream projects [2]. We'd have
> to check plans with our consumers.
>
> What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)?
> The downside is that we'd probably need to set boundaries on end of
> life/limited support for 3.2 and 3.3 to keep the workload manageable.
>
> [1]
>
> https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI
> [2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109
>
> Chris Nauroth
>
>
> On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena <ay...@gmail.com> wrote:
>
> > >
> > > it's already hard to migrate from JDK8 why not retarget JDK17.
> > >
> >
> > +1, makes sense to me, sounds like a win-win situation to me, though
> there
> > would be some additional issues to chase now :)
> >
> > -Ayush
> >
> >
> > On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org>
> wrote:
> >
> > > My random thoughts. Probably bad takes:
> > >
> > > There are projects experimenting with JDK17 now.
> > > JDK11 active support will end in 6 months. If it's already hard to
> > migrate
> > > from JDK8 why not retarget JDK17.
> > >
> > > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com>
> > wrote:
> > >
> > >> I know Jersey upgrade as a blocker. Some folks were chasing that last
> > >> year during 3.3.4 time, I don’t know where it is now, didn’t see then
> > >> what’s the problem there but I remember there was some intitial PR
> which
> > >> did it for HDFS atleast, so I never looked beyond that…
> > >>
> > >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
> > >> java-11 only branch may be, but that is something later to decide,
> once
> > we
> > >> get the code sorted…
> > >>
> > >> -Ayush
> > >>
> > >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran
> > <st...@cloudera.com.invalid>
> > >> wrote:
> > >> >
> > >> > well, how about we flip the switch and get on with it.
> > >> >
> > >> > slf4j seems happy on java11,
> > >> >
> > >> > side issue, anyone seen test failures on zulu1.8; somehow my test
> run
> > is
> > >> > failing and i'm trying to work out whether its a mismatch in command
> > >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an
> > openjdk
> > >> > version which requires IntBuffer implements an overridden method
> > >> IntBuffer
> > >> > rewind().
> > >> >
> > >> > java.lang.NoSuchMethodError:
> > >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
> > >> >
> > >> > at
> > >>
> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> > >> > at
> > >> >
> > >>
> >
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> > >> > at
> org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> > >> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> > >> > at java.io.DataInputStream.read(DataInputStream.java:149)
> > >> >
> > >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org>
> > wrote:
> > >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
> > >> jersey 1
> > >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
> > >> stability.
> > >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
> > >> <stevel@cloudera.com.invalid
> > >> >> wrote:
> > >> >>> Now that hadoop 3.3.5 is out, i want to propose something new
> > >> >>> we switch branch-3.3 and trunk to being java11 only
> > >> >>> 1. java 11 has been out for years
> > >> >>> 2. oracle java 8 is no longer available under "premier support";
> you
> > >> >>> can't really get upgrades
> > >> >>>
> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> > >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with
> > them
> > >> >>> don't always link to oracle java 8 (some classes in java.nio have
> > >> >> added
> > >> >>> more overrides)
> > >> >>> 4. more and more libraries we want to upgrade to/bundle are java
> 11
> > >> >> only
> > >> >>> 5. moving to java 11 would cut our yetus build workload in half,
> and
> > >> >>> line up for adding java 17 builds instead.
> > >> >>> I know there are some outstanding issues still in
> > >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> > >> >> blockers?
> > >> >>> Could we just move to java11 and enhance at our leisure, once
> java8
> > >> is no
> > >> >>> longer a concern.
> > >>
> > >> ---------------------------------------------------------------------
> > >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> > >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> > >>
> > >>
> >
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Chris Nauroth <cn...@gmail.com>.
In theory, I like the idea of setting aside Java 8. Unfortunately, I don't
know that upgrading within the 3.3 line adheres to our binary compatibility
policy [1]. I don't see specific discussion of the Java version there, but
it states that you should be able to drop in minor upgrades and have
existing apps keep working. Users might find it surprising if they try to
upgrade a cluster that has JDK 8.
There is also the question of impact on downstream projects [2]. We'd have
to check plans with our consumers.
What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)?
The downside is that we'd probably need to set boundaries on end of
life/limited support for 3.2 and 3.3 to keep the workload manageable.
[1]
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI
[2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109
Chris Nauroth
On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena <ay...@gmail.com> wrote:
> >
> > it's already hard to migrate from JDK8 why not retarget JDK17.
> >
>
> +1, makes sense to me, sounds like a win-win situation to me, though there
> would be some additional issues to chase now :)
>
> -Ayush
>
>
> On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org> wrote:
>
> > My random thoughts. Probably bad takes:
> >
> > There are projects experimenting with JDK17 now.
> > JDK11 active support will end in 6 months. If it's already hard to
> migrate
> > from JDK8 why not retarget JDK17.
> >
> > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com>
> wrote:
> >
> >> I know Jersey upgrade as a blocker. Some folks were chasing that last
> >> year during 3.3.4 time, I don’t know where it is now, didn’t see then
> >> what’s the problem there but I remember there was some intitial PR which
> >> did it for HDFS atleast, so I never looked beyond that…
> >>
> >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
> >> java-11 only branch may be, but that is something later to decide, once
> we
> >> get the code sorted…
> >>
> >> -Ayush
> >>
> >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran
> <st...@cloudera.com.invalid>
> >> wrote:
> >> >
> >> > well, how about we flip the switch and get on with it.
> >> >
> >> > slf4j seems happy on java11,
> >> >
> >> > side issue, anyone seen test failures on zulu1.8; somehow my test run
> is
> >> > failing and i'm trying to work out whether its a mismatch in command
> >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an
> openjdk
> >> > version which requires IntBuffer implements an overridden method
> >> IntBuffer
> >> > rewind().
> >> >
> >> > java.lang.NoSuchMethodError:
> >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
> >> >
> >> > at
> >> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> >> > at
> >> >
> >>
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> >> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> >> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> >> > at java.io.DataInputStream.read(DataInputStream.java:149)
> >> >
> >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org>
> wrote:
> >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
> >> jersey 1
> >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
> >> stability.
> >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
> >> <stevel@cloudera.com.invalid
> >> >> wrote:
> >> >>> Now that hadoop 3.3.5 is out, i want to propose something new
> >> >>> we switch branch-3.3 and trunk to being java11 only
> >> >>> 1. java 11 has been out for years
> >> >>> 2. oracle java 8 is no longer available under "premier support"; you
> >> >>> can't really get upgrades
> >> >>>
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with
> them
> >> >>> don't always link to oracle java 8 (some classes in java.nio have
> >> >> added
> >> >>> more overrides)
> >> >>> 4. more and more libraries we want to upgrade to/bundle are java 11
> >> >> only
> >> >>> 5. moving to java 11 would cut our yetus build workload in half, and
> >> >>> line up for adding java 17 builds instead.
> >> >>> I know there are some outstanding issues still in
> >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> >> >> blockers?
> >> >>> Could we just move to java11 and enhance at our leisure, once java8
> >> is no
> >> >>> longer a concern.
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>
> >>
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Chris Nauroth <cn...@gmail.com>.
In theory, I like the idea of setting aside Java 8. Unfortunately, I don't
know that upgrading within the 3.3 line adheres to our binary compatibility
policy [1]. I don't see specific discussion of the Java version there, but
it states that you should be able to drop in minor upgrades and have
existing apps keep working. Users might find it surprising if they try to
upgrade a cluster that has JDK 8.
There is also the question of impact on downstream projects [2]. We'd have
to check plans with our consumers.
What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)?
The downside is that we'd probably need to set boundaries on end of
life/limited support for 3.2 and 3.3 to keep the workload manageable.
[1]
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI
[2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109
Chris Nauroth
On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena <ay...@gmail.com> wrote:
> >
> > it's already hard to migrate from JDK8 why not retarget JDK17.
> >
>
> +1, makes sense to me, sounds like a win-win situation to me, though there
> would be some additional issues to chase now :)
>
> -Ayush
>
>
> On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org> wrote:
>
> > My random thoughts. Probably bad takes:
> >
> > There are projects experimenting with JDK17 now.
> > JDK11 active support will end in 6 months. If it's already hard to
> migrate
> > from JDK8 why not retarget JDK17.
> >
> > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com>
> wrote:
> >
> >> I know Jersey upgrade as a blocker. Some folks were chasing that last
> >> year during 3.3.4 time, I don’t know where it is now, didn’t see then
> >> what’s the problem there but I remember there was some intitial PR which
> >> did it for HDFS atleast, so I never looked beyond that…
> >>
> >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
> >> java-11 only branch may be, but that is something later to decide, once
> we
> >> get the code sorted…
> >>
> >> -Ayush
> >>
> >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran
> <st...@cloudera.com.invalid>
> >> wrote:
> >> >
> >> > well, how about we flip the switch and get on with it.
> >> >
> >> > slf4j seems happy on java11,
> >> >
> >> > side issue, anyone seen test failures on zulu1.8; somehow my test run
> is
> >> > failing and i'm trying to work out whether its a mismatch in command
> >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an
> openjdk
> >> > version which requires IntBuffer implements an overridden method
> >> IntBuffer
> >> > rewind().
> >> >
> >> > java.lang.NoSuchMethodError:
> >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
> >> >
> >> > at
> >> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> >> > at
> >> >
> >>
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> >> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> >> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> >> > at java.io.DataInputStream.read(DataInputStream.java:149)
> >> >
> >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org>
> wrote:
> >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
> >> jersey 1
> >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
> >> stability.
> >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
> >> <stevel@cloudera.com.invalid
> >> >> wrote:
> >> >>> Now that hadoop 3.3.5 is out, i want to propose something new
> >> >>> we switch branch-3.3 and trunk to being java11 only
> >> >>> 1. java 11 has been out for years
> >> >>> 2. oracle java 8 is no longer available under "premier support"; you
> >> >>> can't really get upgrades
> >> >>>
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with
> them
> >> >>> don't always link to oracle java 8 (some classes in java.nio have
> >> >> added
> >> >>> more overrides)
> >> >>> 4. more and more libraries we want to upgrade to/bundle are java 11
> >> >> only
> >> >>> 5. moving to java 11 would cut our yetus build workload in half, and
> >> >>> line up for adding java 17 builds instead.
> >> >>> I know there are some outstanding issues still in
> >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> >> >> blockers?
> >> >>> Could we just move to java11 and enhance at our leisure, once java8
> >> is no
> >> >>> longer a concern.
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>
> >>
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Chris Nauroth <cn...@gmail.com>.
In theory, I like the idea of setting aside Java 8. Unfortunately, I don't
know that upgrading within the 3.3 line adheres to our binary compatibility
policy [1]. I don't see specific discussion of the Java version there, but
it states that you should be able to drop in minor upgrades and have
existing apps keep working. Users might find it surprising if they try to
upgrade a cluster that has JDK 8.
There is also the question of impact on downstream projects [2]. We'd have
to check plans with our consumers.
What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)?
The downside is that we'd probably need to set boundaries on end of
life/limited support for 3.2 and 3.3 to keep the workload manageable.
[1]
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI
[2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109
Chris Nauroth
On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena <ay...@gmail.com> wrote:
> >
> > it's already hard to migrate from JDK8 why not retarget JDK17.
> >
>
> +1, makes sense to me, sounds like a win-win situation to me, though there
> would be some additional issues to chase now :)
>
> -Ayush
>
>
> On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org> wrote:
>
> > My random thoughts. Probably bad takes:
> >
> > There are projects experimenting with JDK17 now.
> > JDK11 active support will end in 6 months. If it's already hard to
> migrate
> > from JDK8 why not retarget JDK17.
> >
> > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com>
> wrote:
> >
> >> I know Jersey upgrade as a blocker. Some folks were chasing that last
> >> year during 3.3.4 time, I don’t know where it is now, didn’t see then
> >> what’s the problem there but I remember there was some intitial PR which
> >> did it for HDFS atleast, so I never looked beyond that…
> >>
> >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
> >> java-11 only branch may be, but that is something later to decide, once
> we
> >> get the code sorted…
> >>
> >> -Ayush
> >>
> >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran
> <st...@cloudera.com.invalid>
> >> wrote:
> >> >
> >> > well, how about we flip the switch and get on with it.
> >> >
> >> > slf4j seems happy on java11,
> >> >
> >> > side issue, anyone seen test failures on zulu1.8; somehow my test run
> is
> >> > failing and i'm trying to work out whether its a mismatch in command
> >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an
> openjdk
> >> > version which requires IntBuffer implements an overridden method
> >> IntBuffer
> >> > rewind().
> >> >
> >> > java.lang.NoSuchMethodError:
> >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
> >> >
> >> > at
> >> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> >> > at
> >> >
> >>
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> >> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> >> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> >> > at java.io.DataInputStream.read(DataInputStream.java:149)
> >> >
> >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org>
> wrote:
> >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
> >> jersey 1
> >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
> >> stability.
> >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
> >> <stevel@cloudera.com.invalid
> >> >> wrote:
> >> >>> Now that hadoop 3.3.5 is out, i want to propose something new
> >> >>> we switch branch-3.3 and trunk to being java11 only
> >> >>> 1. java 11 has been out for years
> >> >>> 2. oracle java 8 is no longer available under "premier support"; you
> >> >>> can't really get upgrades
> >> >>>
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with
> them
> >> >>> don't always link to oracle java 8 (some classes in java.nio have
> >> >> added
> >> >>> more overrides)
> >> >>> 4. more and more libraries we want to upgrade to/bundle are java 11
> >> >> only
> >> >>> 5. moving to java 11 would cut our yetus build workload in half, and
> >> >>> line up for adding java 17 builds instead.
> >> >>> I know there are some outstanding issues still in
> >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> >> >> blockers?
> >> >>> Could we just move to java11 and enhance at our leisure, once java8
> >> is no
> >> >>> longer a concern.
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>
> >>
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Chris Nauroth <cn...@gmail.com>.
In theory, I like the idea of setting aside Java 8. Unfortunately, I don't
know that upgrading within the 3.3 line adheres to our binary compatibility
policy [1]. I don't see specific discussion of the Java version there, but
it states that you should be able to drop in minor upgrades and have
existing apps keep working. Users might find it surprising if they try to
upgrade a cluster that has JDK 8.
There is also the question of impact on downstream projects [2]. We'd have
to check plans with our consumers.
What about the idea of shooting for a 3.4 release on JDK 11 (or even 17)?
The downside is that we'd probably need to set boundaries on end of
life/limited support for 3.2 and 3.3 to keep the workload manageable.
[1]
https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/Compatibility.html#Java_Binary_compatibility_for_end-user_applications_i.e._Apache_Hadoop_ABI
[2] https://github.com/apache/spark/blob/v3.3.2/pom.xml#L109
Chris Nauroth
On Tue, Mar 28, 2023 at 11:10 AM Ayush Saxena <ay...@gmail.com> wrote:
> >
> > it's already hard to migrate from JDK8 why not retarget JDK17.
> >
>
> +1, makes sense to me, sounds like a win-win situation to me, though there
> would be some additional issues to chase now :)
>
> -Ayush
>
>
> On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org> wrote:
>
> > My random thoughts. Probably bad takes:
> >
> > There are projects experimenting with JDK17 now.
> > JDK11 active support will end in 6 months. If it's already hard to
> migrate
> > from JDK8 why not retarget JDK17.
> >
> > On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com>
> wrote:
> >
> >> I know Jersey upgrade as a blocker. Some folks were chasing that last
> >> year during 3.3.4 time, I don’t know where it is now, didn’t see then
> >> what’s the problem there but I remember there was some intitial PR which
> >> did it for HDFS atleast, so I never looked beyond that…
> >>
> >> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
> >> java-11 only branch may be, but that is something later to decide, once
> we
> >> get the code sorted…
> >>
> >> -Ayush
> >>
> >> > On 28-Mar-2023, at 9:16 PM, Steve Loughran
> <st...@cloudera.com.invalid>
> >> wrote:
> >> >
> >> > well, how about we flip the switch and get on with it.
> >> >
> >> > slf4j seems happy on java11,
> >> >
> >> > side issue, anyone seen test failures on zulu1.8; somehow my test run
> is
> >> > failing and i'm trying to work out whether its a mismatch in command
> >> > line/ide jvm versions, or the 3.3.5 JARs have been built with an
> openjdk
> >> > version which requires IntBuffer implements an overridden method
> >> IntBuffer
> >> > rewind().
> >> >
> >> > java.lang.NoSuchMethodError:
> >> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
> >> >
> >> > at
> >> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> >> > at
> >> >
> >>
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> >> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> >> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> >> > at java.io.DataInputStream.read(DataInputStream.java:149)
> >> >
> >> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org>
> wrote:
> >> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
> >> jersey 1
> >> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
> >> stability.
> >> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
> >> <stevel@cloudera.com.invalid
> >> >> wrote:
> >> >>> Now that hadoop 3.3.5 is out, i want to propose something new
> >> >>> we switch branch-3.3 and trunk to being java11 only
> >> >>> 1. java 11 has been out for years
> >> >>> 2. oracle java 8 is no longer available under "premier support"; you
> >> >>> can't really get upgrades
> >> >>>
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> >> >>> 3. openJDK 8 releases != oracle ones, and things you compile with
> them
> >> >>> don't always link to oracle java 8 (some classes in java.nio have
> >> >> added
> >> >>> more overrides)
> >> >>> 4. more and more libraries we want to upgrade to/bundle are java 11
> >> >> only
> >> >>> 5. moving to java 11 would cut our yetus build workload in half, and
> >> >>> line up for adding java 17 builds instead.
> >> >>> I know there are some outstanding issues still in
> >> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> >> >> blockers?
> >> >>> Could we just move to java11 and enhance at our leisure, once java8
> >> is no
> >> >>> longer a concern.
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> >> For additional commands, e-mail: common-dev-help@hadoop.apache.org
> >>
> >>
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Ayush Saxena <ay...@gmail.com>.
>
> it's already hard to migrate from JDK8 why not retarget JDK17.
>
+1, makes sense to me, sounds like a win-win situation to me, though there
would be some additional issues to chase now :)
-Ayush
On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org> wrote:
> My random thoughts. Probably bad takes:
>
> There are projects experimenting with JDK17 now.
> JDK11 active support will end in 6 months. If it's already hard to migrate
> from JDK8 why not retarget JDK17.
>
> On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com> wrote:
>
>> I know Jersey upgrade as a blocker. Some folks were chasing that last
>> year during 3.3.4 time, I don’t know where it is now, didn’t see then
>> what’s the problem there but I remember there was some intitial PR which
>> did it for HDFS atleast, so I never looked beyond that…
>>
>> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
>> java-11 only branch may be, but that is something later to decide, once we
>> get the code sorted…
>>
>> -Ayush
>>
>> > On 28-Mar-2023, at 9:16 PM, Steve Loughran <st...@cloudera.com.invalid>
>> wrote:
>> >
>> > well, how about we flip the switch and get on with it.
>> >
>> > slf4j seems happy on java11,
>> >
>> > side issue, anyone seen test failures on zulu1.8; somehow my test run is
>> > failing and i'm trying to work out whether its a mismatch in command
>> > line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
>> > version which requires IntBuffer implements an overridden method
>> IntBuffer
>> > rewind().
>> >
>> > java.lang.NoSuchMethodError:
>> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
>> >
>> > at
>> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
>> > at
>> >
>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
>> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
>> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
>> > at java.io.DataInputStream.read(DataInputStream.java:149)
>> >
>> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
>> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
>> jersey 1
>> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
>> stability.
>> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
>> <stevel@cloudera.com.invalid
>> >> wrote:
>> >>> Now that hadoop 3.3.5 is out, i want to propose something new
>> >>> we switch branch-3.3 and trunk to being java11 only
>> >>> 1. java 11 has been out for years
>> >>> 2. oracle java 8 is no longer available under "premier support"; you
>> >>> can't really get upgrades
>> >>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>> >>> 3. openJDK 8 releases != oracle ones, and things you compile with them
>> >>> don't always link to oracle java 8 (some classes in java.nio have
>> >> added
>> >>> more overrides)
>> >>> 4. more and more libraries we want to upgrade to/bundle are java 11
>> >> only
>> >>> 5. moving to java 11 would cut our yetus build workload in half, and
>> >>> line up for adding java 17 builds instead.
>> >>> I know there are some outstanding issues still in
>> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
>> >> blockers?
>> >>> Could we just move to java11 and enhance at our leisure, once java8
>> is no
>> >>> longer a concern.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Ayush Saxena <ay...@gmail.com>.
>
> it's already hard to migrate from JDK8 why not retarget JDK17.
>
+1, makes sense to me, sounds like a win-win situation to me, though there
would be some additional issues to chase now :)
-Ayush
On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org> wrote:
> My random thoughts. Probably bad takes:
>
> There are projects experimenting with JDK17 now.
> JDK11 active support will end in 6 months. If it's already hard to migrate
> from JDK8 why not retarget JDK17.
>
> On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com> wrote:
>
>> I know Jersey upgrade as a blocker. Some folks were chasing that last
>> year during 3.3.4 time, I don’t know where it is now, didn’t see then
>> what’s the problem there but I remember there was some intitial PR which
>> did it for HDFS atleast, so I never looked beyond that…
>>
>> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
>> java-11 only branch may be, but that is something later to decide, once we
>> get the code sorted…
>>
>> -Ayush
>>
>> > On 28-Mar-2023, at 9:16 PM, Steve Loughran <st...@cloudera.com.invalid>
>> wrote:
>> >
>> > well, how about we flip the switch and get on with it.
>> >
>> > slf4j seems happy on java11,
>> >
>> > side issue, anyone seen test failures on zulu1.8; somehow my test run is
>> > failing and i'm trying to work out whether its a mismatch in command
>> > line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
>> > version which requires IntBuffer implements an overridden method
>> IntBuffer
>> > rewind().
>> >
>> > java.lang.NoSuchMethodError:
>> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
>> >
>> > at
>> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
>> > at
>> >
>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
>> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
>> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
>> > at java.io.DataInputStream.read(DataInputStream.java:149)
>> >
>> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
>> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
>> jersey 1
>> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
>> stability.
>> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
>> <stevel@cloudera.com.invalid
>> >> wrote:
>> >>> Now that hadoop 3.3.5 is out, i want to propose something new
>> >>> we switch branch-3.3 and trunk to being java11 only
>> >>> 1. java 11 has been out for years
>> >>> 2. oracle java 8 is no longer available under "premier support"; you
>> >>> can't really get upgrades
>> >>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>> >>> 3. openJDK 8 releases != oracle ones, and things you compile with them
>> >>> don't always link to oracle java 8 (some classes in java.nio have
>> >> added
>> >>> more overrides)
>> >>> 4. more and more libraries we want to upgrade to/bundle are java 11
>> >> only
>> >>> 5. moving to java 11 would cut our yetus build workload in half, and
>> >>> line up for adding java 17 builds instead.
>> >>> I know there are some outstanding issues still in
>> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
>> >> blockers?
>> >>> Could we just move to java11 and enhance at our leisure, once java8
>> is no
>> >>> longer a concern.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Ayush Saxena <ay...@gmail.com>.
>
> it's already hard to migrate from JDK8 why not retarget JDK17.
>
+1, makes sense to me, sounds like a win-win situation to me, though there
would be some additional issues to chase now :)
-Ayush
On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org> wrote:
> My random thoughts. Probably bad takes:
>
> There are projects experimenting with JDK17 now.
> JDK11 active support will end in 6 months. If it's already hard to migrate
> from JDK8 why not retarget JDK17.
>
> On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com> wrote:
>
>> I know Jersey upgrade as a blocker. Some folks were chasing that last
>> year during 3.3.4 time, I don’t know where it is now, didn’t see then
>> what’s the problem there but I remember there was some intitial PR which
>> did it for HDFS atleast, so I never looked beyond that…
>>
>> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
>> java-11 only branch may be, but that is something later to decide, once we
>> get the code sorted…
>>
>> -Ayush
>>
>> > On 28-Mar-2023, at 9:16 PM, Steve Loughran <st...@cloudera.com.invalid>
>> wrote:
>> >
>> > well, how about we flip the switch and get on with it.
>> >
>> > slf4j seems happy on java11,
>> >
>> > side issue, anyone seen test failures on zulu1.8; somehow my test run is
>> > failing and i'm trying to work out whether its a mismatch in command
>> > line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
>> > version which requires IntBuffer implements an overridden method
>> IntBuffer
>> > rewind().
>> >
>> > java.lang.NoSuchMethodError:
>> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
>> >
>> > at
>> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
>> > at
>> >
>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
>> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
>> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
>> > at java.io.DataInputStream.read(DataInputStream.java:149)
>> >
>> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
>> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
>> jersey 1
>> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
>> stability.
>> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
>> <stevel@cloudera.com.invalid
>> >> wrote:
>> >>> Now that hadoop 3.3.5 is out, i want to propose something new
>> >>> we switch branch-3.3 and trunk to being java11 only
>> >>> 1. java 11 has been out for years
>> >>> 2. oracle java 8 is no longer available under "premier support"; you
>> >>> can't really get upgrades
>> >>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>> >>> 3. openJDK 8 releases != oracle ones, and things you compile with them
>> >>> don't always link to oracle java 8 (some classes in java.nio have
>> >> added
>> >>> more overrides)
>> >>> 4. more and more libraries we want to upgrade to/bundle are java 11
>> >> only
>> >>> 5. moving to java 11 would cut our yetus build workload in half, and
>> >>> line up for adding java 17 builds instead.
>> >>> I know there are some outstanding issues still in
>> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
>> >> blockers?
>> >>> Could we just move to java11 and enhance at our leisure, once java8
>> is no
>> >>> longer a concern.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Ayush Saxena <ay...@gmail.com>.
>
> it's already hard to migrate from JDK8 why not retarget JDK17.
>
+1, makes sense to me, sounds like a win-win situation to me, though there
would be some additional issues to chase now :)
-Ayush
On Tue, 28 Mar 2023 at 23:29, Wei-Chiu Chuang <we...@apache.org> wrote:
> My random thoughts. Probably bad takes:
>
> There are projects experimenting with JDK17 now.
> JDK11 active support will end in 6 months. If it's already hard to migrate
> from JDK8 why not retarget JDK17.
>
> On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com> wrote:
>
>> I know Jersey upgrade as a blocker. Some folks were chasing that last
>> year during 3.3.4 time, I don’t know where it is now, didn’t see then
>> what’s the problem there but I remember there was some intitial PR which
>> did it for HDFS atleast, so I never looked beyond that…
>>
>> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as
>> java-11 only branch may be, but that is something later to decide, once we
>> get the code sorted…
>>
>> -Ayush
>>
>> > On 28-Mar-2023, at 9:16 PM, Steve Loughran <st...@cloudera.com.invalid>
>> wrote:
>> >
>> > well, how about we flip the switch and get on with it.
>> >
>> > slf4j seems happy on java11,
>> >
>> > side issue, anyone seen test failures on zulu1.8; somehow my test run is
>> > failing and i'm trying to work out whether its a mismatch in command
>> > line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
>> > version which requires IntBuffer implements an overridden method
>> IntBuffer
>> > rewind().
>> >
>> > java.lang.NoSuchMethodError:
>> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
>> >
>> > at
>> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
>> > at
>> >
>> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
>> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
>> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
>> > at java.io.DataInputStream.read(DataInputStream.java:149)
>> >
>> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
>> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
>> jersey 1
>> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
>> stability.
>> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
>> <stevel@cloudera.com.invalid
>> >> wrote:
>> >>> Now that hadoop 3.3.5 is out, i want to propose something new
>> >>> we switch branch-3.3 and trunk to being java11 only
>> >>> 1. java 11 has been out for years
>> >>> 2. oracle java 8 is no longer available under "premier support"; you
>> >>> can't really get upgrades
>> >>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>> >>> 3. openJDK 8 releases != oracle ones, and things you compile with them
>> >>> don't always link to oracle java 8 (some classes in java.nio have
>> >> added
>> >>> more overrides)
>> >>> 4. more and more libraries we want to upgrade to/bundle are java 11
>> >> only
>> >>> 5. moving to java 11 would cut our yetus build workload in half, and
>> >>> line up for adding java 17 builds instead.
>> >>> I know there are some outstanding issues still in
>> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
>> >> blockers?
>> >>> Could we just move to java11 and enhance at our leisure, once java8
>> is no
>> >>> longer a concern.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
>> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>>
>>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Wei-Chiu Chuang <we...@apache.org>.
My random thoughts. Probably bad takes:
There are projects experimenting with JDK17 now.
JDK11 active support will end in 6 months. If it's already hard to migrate
from JDK8 why not retarget JDK17.
On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com> wrote:
> I know Jersey upgrade as a blocker. Some folks were chasing that last year
> during 3.3.4 time, I don’t know where it is now, didn’t see then what’s the
> problem there but I remember there was some intitial PR which did it for
> HDFS atleast, so I never looked beyond that…
>
> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as java-11
> only branch may be, but that is something later to decide, once we get the
> code sorted…
>
> -Ayush
>
> > On 28-Mar-2023, at 9:16 PM, Steve Loughran <st...@cloudera.com.invalid>
> wrote:
> >
> > well, how about we flip the switch and get on with it.
> >
> > slf4j seems happy on java11,
> >
> > side issue, anyone seen test failures on zulu1.8; somehow my test run is
> > failing and i'm trying to work out whether its a mismatch in command
> > line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
> > version which requires IntBuffer implements an overridden method
> IntBuffer
> > rewind().
> >
> > java.lang.NoSuchMethodError:
> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
> >
> > at
> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> > at
> >
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> > at java.io.DataInputStream.read(DataInputStream.java:149)
> >
> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
> jersey 1
> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
> stability.
> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
> <stevel@cloudera.com.invalid
> >> wrote:
> >>> Now that hadoop 3.3.5 is out, i want to propose something new
> >>> we switch branch-3.3 and trunk to being java11 only
> >>> 1. java 11 has been out for years
> >>> 2. oracle java 8 is no longer available under "premier support"; you
> >>> can't really get upgrades
> >>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> >>> 3. openJDK 8 releases != oracle ones, and things you compile with them
> >>> don't always link to oracle java 8 (some classes in java.nio have
> >> added
> >>> more overrides)
> >>> 4. more and more libraries we want to upgrade to/bundle are java 11
> >> only
> >>> 5. moving to java 11 would cut our yetus build workload in half, and
> >>> line up for adding java 17 builds instead.
> >>> I know there are some outstanding issues still in
> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> >> blockers?
> >>> Could we just move to java11 and enhance at our leisure, once java8 is
> no
> >>> longer a concern.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Wei-Chiu Chuang <we...@apache.org>.
My random thoughts. Probably bad takes:
There are projects experimenting with JDK17 now.
JDK11 active support will end in 6 months. If it's already hard to migrate
from JDK8 why not retarget JDK17.
On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com> wrote:
> I know Jersey upgrade as a blocker. Some folks were chasing that last year
> during 3.3.4 time, I don’t know where it is now, didn’t see then what’s the
> problem there but I remember there was some intitial PR which did it for
> HDFS atleast, so I never looked beyond that…
>
> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as java-11
> only branch may be, but that is something later to decide, once we get the
> code sorted…
>
> -Ayush
>
> > On 28-Mar-2023, at 9:16 PM, Steve Loughran <st...@cloudera.com.invalid>
> wrote:
> >
> > well, how about we flip the switch and get on with it.
> >
> > slf4j seems happy on java11,
> >
> > side issue, anyone seen test failures on zulu1.8; somehow my test run is
> > failing and i'm trying to work out whether its a mismatch in command
> > line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
> > version which requires IntBuffer implements an overridden method
> IntBuffer
> > rewind().
> >
> > java.lang.NoSuchMethodError:
> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
> >
> > at
> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> > at
> >
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> > at java.io.DataInputStream.read(DataInputStream.java:149)
> >
> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
> jersey 1
> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
> stability.
> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
> <stevel@cloudera.com.invalid
> >> wrote:
> >>> Now that hadoop 3.3.5 is out, i want to propose something new
> >>> we switch branch-3.3 and trunk to being java11 only
> >>> 1. java 11 has been out for years
> >>> 2. oracle java 8 is no longer available under "premier support"; you
> >>> can't really get upgrades
> >>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> >>> 3. openJDK 8 releases != oracle ones, and things you compile with them
> >>> don't always link to oracle java 8 (some classes in java.nio have
> >> added
> >>> more overrides)
> >>> 4. more and more libraries we want to upgrade to/bundle are java 11
> >> only
> >>> 5. moving to java 11 would cut our yetus build workload in half, and
> >>> line up for adding java 17 builds instead.
> >>> I know there are some outstanding issues still in
> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> >> blockers?
> >>> Could we just move to java11 and enhance at our leisure, once java8 is
> no
> >>> longer a concern.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Wei-Chiu Chuang <we...@apache.org>.
My random thoughts. Probably bad takes:
There are projects experimenting with JDK17 now.
JDK11 active support will end in 6 months. If it's already hard to migrate
from JDK8 why not retarget JDK17.
On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com> wrote:
> I know Jersey upgrade as a blocker. Some folks were chasing that last year
> during 3.3.4 time, I don’t know where it is now, didn’t see then what’s the
> problem there but I remember there was some intitial PR which did it for
> HDFS atleast, so I never looked beyond that…
>
> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as java-11
> only branch may be, but that is something later to decide, once we get the
> code sorted…
>
> -Ayush
>
> > On 28-Mar-2023, at 9:16 PM, Steve Loughran <st...@cloudera.com.invalid>
> wrote:
> >
> > well, how about we flip the switch and get on with it.
> >
> > slf4j seems happy on java11,
> >
> > side issue, anyone seen test failures on zulu1.8; somehow my test run is
> > failing and i'm trying to work out whether its a mismatch in command
> > line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
> > version which requires IntBuffer implements an overridden method
> IntBuffer
> > rewind().
> >
> > java.lang.NoSuchMethodError:
> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
> >
> > at
> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> > at
> >
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> > at java.io.DataInputStream.read(DataInputStream.java:149)
> >
> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
> jersey 1
> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
> stability.
> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
> <stevel@cloudera.com.invalid
> >> wrote:
> >>> Now that hadoop 3.3.5 is out, i want to propose something new
> >>> we switch branch-3.3 and trunk to being java11 only
> >>> 1. java 11 has been out for years
> >>> 2. oracle java 8 is no longer available under "premier support"; you
> >>> can't really get upgrades
> >>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> >>> 3. openJDK 8 releases != oracle ones, and things you compile with them
> >>> don't always link to oracle java 8 (some classes in java.nio have
> >> added
> >>> more overrides)
> >>> 4. more and more libraries we want to upgrade to/bundle are java 11
> >> only
> >>> 5. moving to java 11 would cut our yetus build workload in half, and
> >>> line up for adding java 17 builds instead.
> >>> I know there are some outstanding issues still in
> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> >> blockers?
> >>> Could we just move to java11 and enhance at our leisure, once java8 is
> no
> >>> longer a concern.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Wei-Chiu Chuang <we...@apache.org>.
My random thoughts. Probably bad takes:
There are projects experimenting with JDK17 now.
JDK11 active support will end in 6 months. If it's already hard to migrate
from JDK8 why not retarget JDK17.
On Tue, Mar 28, 2023 at 10:30 AM Ayush Saxena <ay...@gmail.com> wrote:
> I know Jersey upgrade as a blocker. Some folks were chasing that last year
> during 3.3.4 time, I don’t know where it is now, didn’t see then what’s the
> problem there but I remember there was some intitial PR which did it for
> HDFS atleast, so I never looked beyond that…
>
> I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as java-11
> only branch may be, but that is something later to decide, once we get the
> code sorted…
>
> -Ayush
>
> > On 28-Mar-2023, at 9:16 PM, Steve Loughran <st...@cloudera.com.invalid>
> wrote:
> >
> > well, how about we flip the switch and get on with it.
> >
> > slf4j seems happy on java11,
> >
> > side issue, anyone seen test failures on zulu1.8; somehow my test run is
> > failing and i'm trying to work out whether its a mismatch in command
> > line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
> > version which requires IntBuffer implements an overridden method
> IntBuffer
> > rewind().
> >
> > java.lang.NoSuchMethodError:
> java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
> >
> > at
> org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> > at
> >
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> > at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> > at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> > at java.io.DataInputStream.read(DataInputStream.java:149)
> >
> >> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
> >> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2,
> jersey 1
> >> to 2 and junit 4 to 5) are blockers for java 11 compile + test
> stability.
> >> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran
> <stevel@cloudera.com.invalid
> >> wrote:
> >>> Now that hadoop 3.3.5 is out, i want to propose something new
> >>> we switch branch-3.3 and trunk to being java11 only
> >>> 1. java 11 has been out for years
> >>> 2. oracle java 8 is no longer available under "premier support"; you
> >>> can't really get upgrades
> >>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> >>> 3. openJDK 8 releases != oracle ones, and things you compile with them
> >>> don't always link to oracle java 8 (some classes in java.nio have
> >> added
> >>> more overrides)
> >>> 4. more and more libraries we want to upgrade to/bundle are java 11
> >> only
> >>> 5. moving to java 11 would cut our yetus build workload in half, and
> >>> line up for adding java 17 builds instead.
> >>> I know there are some outstanding issues still in
> >>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> >> blockers?
> >>> Could we just move to java11 and enhance at our leisure, once java8 is
> no
> >>> longer a concern.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
> For additional commands, e-mail: common-dev-help@hadoop.apache.org
>
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Ayush Saxena <ay...@gmail.com>.
I know Jersey upgrade as a blocker. Some folks were chasing that last year during 3.3.4 time, I don’t know where it is now, didn’t see then what’s the problem there but I remember there was some intitial PR which did it for HDFS atleast, so I never looked beyond that…
I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as java-11 only branch may be, but that is something later to decide, once we get the code sorted…
-Ayush
> On 28-Mar-2023, at 9:16 PM, Steve Loughran <st...@cloudera.com.invalid> wrote:
>
> well, how about we flip the switch and get on with it.
>
> slf4j seems happy on java11,
>
> side issue, anyone seen test failures on zulu1.8; somehow my test run is
> failing and i'm trying to work out whether its a mismatch in command
> line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
> version which requires IntBuffer implements an overridden method IntBuffer
> rewind().
>
> java.lang.NoSuchMethodError: java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
>
> at org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> at
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> at java.io.DataInputStream.read(DataInputStream.java:149)
>
>> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
>> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1
>> to 2 and junit 4 to 5) are blockers for java 11 compile + test stability.
>> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran <stevel@cloudera.com.invalid
>> wrote:
>>> Now that hadoop 3.3.5 is out, i want to propose something new
>>> we switch branch-3.3 and trunk to being java11 only
>>> 1. java 11 has been out for years
>>> 2. oracle java 8 is no longer available under "premier support"; you
>>> can't really get upgrades
>>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>>> 3. openJDK 8 releases != oracle ones, and things you compile with them
>>> don't always link to oracle java 8 (some classes in java.nio have
>> added
>>> more overrides)
>>> 4. more and more libraries we want to upgrade to/bundle are java 11
>> only
>>> 5. moving to java 11 would cut our yetus build workload in half, and
>>> line up for adding java 17 builds instead.
>>> I know there are some outstanding issues still in
>>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
>> blockers?
>>> Could we just move to java11 and enhance at our leisure, once java8 is no
>>> longer a concern.
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Ayush Saxena <ay...@gmail.com>.
I know Jersey upgrade as a blocker. Some folks were chasing that last year during 3.3.4 time, I don’t know where it is now, didn’t see then what’s the problem there but I remember there was some intitial PR which did it for HDFS atleast, so I never looked beyond that…
I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as java-11 only branch may be, but that is something later to decide, once we get the code sorted…
-Ayush
> On 28-Mar-2023, at 9:16 PM, Steve Loughran <st...@cloudera.com.invalid> wrote:
>
> well, how about we flip the switch and get on with it.
>
> slf4j seems happy on java11,
>
> side issue, anyone seen test failures on zulu1.8; somehow my test run is
> failing and i'm trying to work out whether its a mismatch in command
> line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
> version which requires IntBuffer implements an overridden method IntBuffer
> rewind().
>
> java.lang.NoSuchMethodError: java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
>
> at org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> at
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> at java.io.DataInputStream.read(DataInputStream.java:149)
>
>> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
>> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1
>> to 2 and junit 4 to 5) are blockers for java 11 compile + test stability.
>> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran <stevel@cloudera.com.invalid
>> wrote:
>>> Now that hadoop 3.3.5 is out, i want to propose something new
>>> we switch branch-3.3 and trunk to being java11 only
>>> 1. java 11 has been out for years
>>> 2. oracle java 8 is no longer available under "premier support"; you
>>> can't really get upgrades
>>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>>> 3. openJDK 8 releases != oracle ones, and things you compile with them
>>> don't always link to oracle java 8 (some classes in java.nio have
>> added
>>> more overrides)
>>> 4. more and more libraries we want to upgrade to/bundle are java 11
>> only
>>> 5. moving to java 11 would cut our yetus build workload in half, and
>>> line up for adding java 17 builds instead.
>>> I know there are some outstanding issues still in
>>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
>> blockers?
>>> Could we just move to java11 and enhance at our leisure, once java8 is no
>>> longer a concern.
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Ayush Saxena <ay...@gmail.com>.
I know Jersey upgrade as a blocker. Some folks were chasing that last year during 3.3.4 time, I don’t know where it is now, didn’t see then what’s the problem there but I remember there was some intitial PR which did it for HDFS atleast, so I never looked beyond that…
I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as java-11 only branch may be, but that is something later to decide, once we get the code sorted…
-Ayush
> On 28-Mar-2023, at 9:16 PM, Steve Loughran <st...@cloudera.com.invalid> wrote:
>
> well, how about we flip the switch and get on with it.
>
> slf4j seems happy on java11,
>
> side issue, anyone seen test failures on zulu1.8; somehow my test run is
> failing and i'm trying to work out whether its a mismatch in command
> line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
> version which requires IntBuffer implements an overridden method IntBuffer
> rewind().
>
> java.lang.NoSuchMethodError: java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
>
> at org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> at
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> at java.io.DataInputStream.read(DataInputStream.java:149)
>
>> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
>> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1
>> to 2 and junit 4 to 5) are blockers for java 11 compile + test stability.
>> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran <stevel@cloudera.com.invalid
>> wrote:
>>> Now that hadoop 3.3.5 is out, i want to propose something new
>>> we switch branch-3.3 and trunk to being java11 only
>>> 1. java 11 has been out for years
>>> 2. oracle java 8 is no longer available under "premier support"; you
>>> can't really get upgrades
>>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>>> 3. openJDK 8 releases != oracle ones, and things you compile with them
>>> don't always link to oracle java 8 (some classes in java.nio have
>> added
>>> more overrides)
>>> 4. more and more libraries we want to upgrade to/bundle are java 11
>> only
>>> 5. moving to java 11 would cut our yetus build workload in half, and
>>> line up for adding java 17 builds instead.
>>> I know there are some outstanding issues still in
>>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
>> blockers?
>>> Could we just move to java11 and enhance at our leisure, once java8 is no
>>> longer a concern.
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-help@hadoop.apache.org
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Ayush Saxena <ay...@gmail.com>.
I know Jersey upgrade as a blocker. Some folks were chasing that last year during 3.3.4 time, I don’t know where it is now, didn’t see then what’s the problem there but I remember there was some intitial PR which did it for HDFS atleast, so I never looked beyond that…
I too had jdk-11 in my mind, but only for trunk. 3.4.x can stay as java-11 only branch may be, but that is something later to decide, once we get the code sorted…
-Ayush
> On 28-Mar-2023, at 9:16 PM, Steve Loughran <st...@cloudera.com.invalid> wrote:
>
> well, how about we flip the switch and get on with it.
>
> slf4j seems happy on java11,
>
> side issue, anyone seen test failures on zulu1.8; somehow my test run is
> failing and i'm trying to work out whether its a mismatch in command
> line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
> version which requires IntBuffer implements an overridden method IntBuffer
> rewind().
>
> java.lang.NoSuchMethodError: java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
>
> at org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
> at
> org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
> at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
> at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
> at java.io.DataInputStream.read(DataInputStream.java:149)
>
>> On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
>> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1
>> to 2 and junit 4 to 5) are blockers for java 11 compile + test stability.
>> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran <stevel@cloudera.com.invalid
>> wrote:
>>> Now that hadoop 3.3.5 is out, i want to propose something new
>>> we switch branch-3.3 and trunk to being java11 only
>>> 1. java 11 has been out for years
>>> 2. oracle java 8 is no longer available under "premier support"; you
>>> can't really get upgrades
>>> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
>>> 3. openJDK 8 releases != oracle ones, and things you compile with them
>>> don't always link to oracle java 8 (some classes in java.nio have
>> added
>>> more overrides)
>>> 4. more and more libraries we want to upgrade to/bundle are java 11
>> only
>>> 5. moving to java 11 would cut our yetus build workload in half, and
>>> line up for adding java 17 builds instead.
>>> I know there are some outstanding issues still in
>>> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
>> blockers?
>>> Could we just move to java11 and enhance at our leisure, once java8 is no
>>> longer a concern.
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
well, how about we flip the switch and get on with it.
slf4j seems happy on java11,
side issue, anyone seen test failures on zulu1.8; somehow my test run is
failing and i'm trying to work out whether its a mismatch in command
line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
version which requires IntBuffer implements an overridden method IntBuffer
rewind().
java.lang.NoSuchMethodError: java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
at org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
at
org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
at java.io.DataInputStream.read(DataInputStream.java:149)
On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1
> to 2 and junit 4 to 5) are blockers for java 11 compile + test stability.
>
>
> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > Now that hadoop 3.3.5 is out, i want to propose something new
> >
> > we switch branch-3.3 and trunk to being java11 only
> >
> >
> > 1. java 11 has been out for years
> > 2. oracle java 8 is no longer available under "premier support"; you
> > can't really get upgrades
> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> > 3. openJDK 8 releases != oracle ones, and things you compile with them
> > don't always link to oracle java 8 (some classes in java.nio have
> added
> > more overrides)
> > 4. more and more libraries we want to upgrade to/bundle are java 11
> only
> > 5. moving to java 11 would cut our yetus build workload in half, and
> > line up for adding java 17 builds instead.
> >
> >
> > I know there are some outstanding issues still in
> > https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> blockers?
> > Could we just move to java11 and enhance at our leisure, once java8 is no
> > longer a concern.
> >
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
well, how about we flip the switch and get on with it.
slf4j seems happy on java11,
side issue, anyone seen test failures on zulu1.8; somehow my test run is
failing and i'm trying to work out whether its a mismatch in command
line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
version which requires IntBuffer implements an overridden method IntBuffer
rewind().
java.lang.NoSuchMethodError: java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
at org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
at
org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
at java.io.DataInputStream.read(DataInputStream.java:149)
On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1
> to 2 and junit 4 to 5) are blockers for java 11 compile + test stability.
>
>
> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > Now that hadoop 3.3.5 is out, i want to propose something new
> >
> > we switch branch-3.3 and trunk to being java11 only
> >
> >
> > 1. java 11 has been out for years
> > 2. oracle java 8 is no longer available under "premier support"; you
> > can't really get upgrades
> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> > 3. openJDK 8 releases != oracle ones, and things you compile with them
> > don't always link to oracle java 8 (some classes in java.nio have
> added
> > more overrides)
> > 4. more and more libraries we want to upgrade to/bundle are java 11
> only
> > 5. moving to java 11 would cut our yetus build workload in half, and
> > line up for adding java 17 builds instead.
> >
> >
> > I know there are some outstanding issues still in
> > https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> blockers?
> > Could we just move to java11 and enhance at our leisure, once java8 is no
> > longer a concern.
> >
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
well, how about we flip the switch and get on with it.
slf4j seems happy on java11,
side issue, anyone seen test failures on zulu1.8; somehow my test run is
failing and i'm trying to work out whether its a mismatch in command
line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
version which requires IntBuffer implements an overridden method IntBuffer
rewind().
java.lang.NoSuchMethodError: java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
at org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
at
org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
at java.io.DataInputStream.read(DataInputStream.java:149)
On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1
> to 2 and junit 4 to 5) are blockers for java 11 compile + test stability.
>
>
> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > Now that hadoop 3.3.5 is out, i want to propose something new
> >
> > we switch branch-3.3 and trunk to being java11 only
> >
> >
> > 1. java 11 has been out for years
> > 2. oracle java 8 is no longer available under "premier support"; you
> > can't really get upgrades
> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> > 3. openJDK 8 releases != oracle ones, and things you compile with them
> > don't always link to oracle java 8 (some classes in java.nio have
> added
> > more overrides)
> > 4. more and more libraries we want to upgrade to/bundle are java 11
> only
> > 5. moving to java 11 would cut our yetus build workload in half, and
> > line up for adding java 17 builds instead.
> >
> >
> > I know there are some outstanding issues still in
> > https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> blockers?
> > Could we just move to java11 and enhance at our leisure, once java8 is no
> > longer a concern.
> >
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Steve Loughran <st...@cloudera.com.INVALID>.
well, how about we flip the switch and get on with it.
slf4j seems happy on java11,
side issue, anyone seen test failures on zulu1.8; somehow my test run is
failing and i'm trying to work out whether its a mismatch in command
line/ide jvm versions, or the 3.3.5 JARs have been built with an openjdk
version which requires IntBuffer implements an overridden method IntBuffer
rewind().
java.lang.NoSuchMethodError: java.nio.IntBuffer.rewind()Ljava/nio/IntBuffer;
at org.apache.hadoop.fs.FSInputChecker.verifySums(FSInputChecker.java:341)
at
org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:308)
at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:257)
at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:202)
at java.io.DataInputStream.read(DataInputStream.java:149)
On Tue, 28 Mar 2023 at 15:52, Viraj Jasani <vj...@apache.org> wrote:
> IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1
> to 2 and junit 4 to 5) are blockers for java 11 compile + test stability.
>
>
> On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran <stevel@cloudera.com.invalid
> >
> wrote:
>
> > Now that hadoop 3.3.5 is out, i want to propose something new
> >
> > we switch branch-3.3 and trunk to being java11 only
> >
> >
> > 1. java 11 has been out for years
> > 2. oracle java 8 is no longer available under "premier support"; you
> > can't really get upgrades
> > https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> > 3. openJDK 8 releases != oracle ones, and things you compile with them
> > don't always link to oracle java 8 (some classes in java.nio have
> added
> > more overrides)
> > 4. more and more libraries we want to upgrade to/bundle are java 11
> only
> > 5. moving to java 11 would cut our yetus build workload in half, and
> > line up for adding java 17 builds instead.
> >
> >
> > I know there are some outstanding issues still in
> > https://issues.apache.org/jira/browse/HADOOP-16795 -but are they
> blockers?
> > Could we just move to java11 and enhance at our leisure, once java8 is no
> > longer a concern.
> >
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Viraj Jasani <vj...@apache.org>.
IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1
to 2 and junit 4 to 5) are blockers for java 11 compile + test stability.
On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> Now that hadoop 3.3.5 is out, i want to propose something new
>
> we switch branch-3.3 and trunk to being java11 only
>
>
> 1. java 11 has been out for years
> 2. oracle java 8 is no longer available under "premier support"; you
> can't really get upgrades
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> 3. openJDK 8 releases != oracle ones, and things you compile with them
> don't always link to oracle java 8 (some classes in java.nio have added
> more overrides)
> 4. more and more libraries we want to upgrade to/bundle are java 11 only
> 5. moving to java 11 would cut our yetus build workload in half, and
> line up for adding java 17 builds instead.
>
>
> I know there are some outstanding issues still in
> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they blockers?
> Could we just move to java11 and enhance at our leisure, once java8 is no
> longer a concern.
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Viraj Jasani <vj...@apache.org>.
IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1
to 2 and junit 4 to 5) are blockers for java 11 compile + test stability.
On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> Now that hadoop 3.3.5 is out, i want to propose something new
>
> we switch branch-3.3 and trunk to being java11 only
>
>
> 1. java 11 has been out for years
> 2. oracle java 8 is no longer available under "premier support"; you
> can't really get upgrades
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> 3. openJDK 8 releases != oracle ones, and things you compile with them
> don't always link to oracle java 8 (some classes in java.nio have added
> more overrides)
> 4. more and more libraries we want to upgrade to/bundle are java 11 only
> 5. moving to java 11 would cut our yetus build workload in half, and
> line up for adding java 17 builds instead.
>
>
> I know there are some outstanding issues still in
> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they blockers?
> Could we just move to java11 and enhance at our leisure, once java8 is no
> longer a concern.
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Nick Dimiduk <nd...@apache.org>.
On Tue, Mar 28, 2023 at 1:56 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> Now that hadoop 3.3.5 is out, i want to propose something new
>
> we switch branch-3.3 and trunk to being java11 only
>
I agree that maintaining support for Java8 is becoming burdensome. Over in
HBase, we maintain support for both Java8 and Hadoop2 :(
We could roll-forward the code lines: branch from rel/3.3.5 as a new
branch-3.4 and trunk version bumps to 3.5 -- assuming the compatibility
guidelines permit dropping the JVM version on a minor release. If not,
rel/3.3.5 branches to branch-4 and trunk changes accordingly.
1. java 11 has been out for years
> 2. oracle java 8 is no longer available under "premier support"; you
> can't really get upgrades
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> 3. openJDK 8 releases != oracle ones, and things you compile with them
> don't always link to oracle java 8 (some classes in java.nio have added
> more overrides)
> 4. more and more libraries we want to upgrade to/bundle are java 11 only
> 5. moving to java 11 would cut our yetus build workload in half, and
> line up for adding java 17 builds instead.
>
>
> I know there are some outstanding issues still in
> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they blockers?
> Could we just move to java11 and enhance at our leisure, once java8 is no
> longer a concern.
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Nick Dimiduk <nd...@apache.org>.
On Tue, Mar 28, 2023 at 1:56 PM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> Now that hadoop 3.3.5 is out, i want to propose something new
>
> we switch branch-3.3 and trunk to being java11 only
>
I agree that maintaining support for Java8 is becoming burdensome. Over in
HBase, we maintain support for both Java8 and Hadoop2 :(
We could roll-forward the code lines: branch from rel/3.3.5 as a new
branch-3.4 and trunk version bumps to 3.5 -- assuming the compatibility
guidelines permit dropping the JVM version on a minor release. If not,
rel/3.3.5 branches to branch-4 and trunk changes accordingly.
1. java 11 has been out for years
> 2. oracle java 8 is no longer available under "premier support"; you
> can't really get upgrades
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> 3. openJDK 8 releases != oracle ones, and things you compile with them
> don't always link to oracle java 8 (some classes in java.nio have added
> more overrides)
> 4. more and more libraries we want to upgrade to/bundle are java 11 only
> 5. moving to java 11 would cut our yetus build workload in half, and
> line up for adding java 17 builds instead.
>
>
> I know there are some outstanding issues still in
> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they blockers?
> Could we just move to java11 and enhance at our leisure, once java8 is no
> longer a concern.
>
Re: [DISCUSS] hadoop branch-3.3+ going to java11 only
Posted by Viraj Jasani <vj...@apache.org>.
IIRC some of the ongoing major dependency upgrades (log4j 1 to 2, jersey 1
to 2 and junit 4 to 5) are blockers for java 11 compile + test stability.
On Tue, Mar 28, 2023 at 4:55 AM Steve Loughran <st...@cloudera.com.invalid>
wrote:
> Now that hadoop 3.3.5 is out, i want to propose something new
>
> we switch branch-3.3 and trunk to being java11 only
>
>
> 1. java 11 has been out for years
> 2. oracle java 8 is no longer available under "premier support"; you
> can't really get upgrades
> https://www.oracle.com/java/technologies/java-se-support-roadmap.html
> 3. openJDK 8 releases != oracle ones, and things you compile with them
> don't always link to oracle java 8 (some classes in java.nio have added
> more overrides)
> 4. more and more libraries we want to upgrade to/bundle are java 11 only
> 5. moving to java 11 would cut our yetus build workload in half, and
> line up for adding java 17 builds instead.
>
>
> I know there are some outstanding issues still in
> https://issues.apache.org/jira/browse/HADOOP-16795 -but are they blockers?
> Could we just move to java11 and enhance at our leisure, once java8 is no
> longer a concern.
>