Re: [DISCUSS] Switch to JDK 11 for releases?

[Laurent Goujon <la...@dremio.com.INVALID>]

I can give it a try if the community is okay with it. I have some
experience using it, it actually works quite well as long as you are not
using things like `sun.*` classes

Laurent

On Wed, Apr 26, 2023 at 8:28 AM Anton Okolnychyi
<ao...@apple.com.invalid> wrote:

> Is anyone interested to give it a try?
>
> - Anton
>
> On Apr 25, 2023, at 4:13 PM, Ryan Blue <bl...@tabular.io> wrote:
>
> Yeah, I do like the idea of using --release. We'll need to test it with
> those platforms though.
>
> On Tue, Apr 25, 2023 at 12:04 PM Anton Okolnychyi <
> aokolnychyi@apple.com.invalid> wrote:
>
>> It would be interesting to hear more from Hive/Impala folks on this topic.
>>
>> I’d avoid supporting both JDK8 and JDK11 given the complexity we need to
>> add. Our build is already complicated.
>>
>> Jack’s idea of using JDK11 with --release flag may be worth exploring.
>>
>> - Anton
>>
>> On Apr 24, 2023, at 10:11 AM, Jack Ye <ye...@gmail.com> wrote:
>>
>> I agree, it wouldn't help given the fact that we won't be able to use the
>> features in newer JDKs.
>>
>> However, I think there is still a difference in the artifact compiled by
>> JDK8, vs compiled by JDK11 with --release=8, that might be useful. For
>> example, I came across this try with resource introduces unreachable
>> byte code issue
>> <https://stackoverflow.com/questions/25615417/try-with-resources-introduce-unreachable-bytecode/25746587#25746587>
>> which is fixed in JDK11, and some optimizations in JDK9 related to string
>> concatenation <https://openjdk.org/jeps/280>. Although Javac is not an
>> optimizing compiler and there should not be much difference in performance
>> of the jars produced by different compilers, these changes might be worth
>> for the project to declare a newer compile-time JDK across all modules, and
>> just always enforce --release flag to 8 to ensure people do not use newer
>> JDK features. But these are quite minor changes so I am not very
>> opinionated either way.
>>
>> Any thoughts?
>>
>> -Jack
>>
>> On Mon, Apr 24, 2023 at 2:22 AM Zoltán Borók-Nagy <bo...@apache.org>
>> wrote:
>>
>>> Besides Hive, neither Impala is compatible with Java11 right now. This
>>> work is in-progress: https://issues.apache.org/jira/browse/IMPALA-11360
>>>
>>> - Zoltan
>>>
>>> On Mon, Apr 24, 2023 at 11:07 AM Mass Dosage <ma...@gmail.com>
>>> wrote:
>>>
>>>> I agree with Ryan, unless you can change the source version there's not
>>>> that much point.
>>>>
>>>> On the Hive front, as you can see from that ticket it's been open for
>>>> 4(!) years and hasn't received much action recently. I think it's one of
>>>> the reasons AWS EMR still defaults to Java 8. It would be really great if
>>>> they could finally push that one over the finish line.
>>>>
>>>> On Sat, 22 Apr 2023 at 20:43, Ryan Blue <bl...@tabular.io> wrote:
>>>>
>>>>> I think in order to publish multiple versions we would need to have
>>>>> different artifact names, like Scala uses (e.g. _2.12).
>>>>>
>>>>> It probably also wouldn't help. If we have to remain compatible with
>>>>> JDK 8, then publishing some artifacts for JDK 11 would still mean only
>>>>> using JDK 8 features. The source version is what we care about more, so if
>>>>> we can't change it then we can't really do anything else.
>>>>>
>>>>> On Sat, Apr 22, 2023 at 10:12 AM Jack Ye <ye...@gmail.com> wrote:
>>>>>
>>>>>> Would it be an option to use --release flag to control the release
>>>>>> target version, and publish 2 versions of the library to Maven, 1 for JDK8
>>>>>> and 1 for JDK11?
>>>>>>
>>>>>> Jack
>>>>>>
>>>>>> On Fri, Apr 21, 2023 at 5:17 PM Ryan Blue <bl...@tabular.io> wrote:
>>>>>>
>>>>>>> Looks like Hive isn't quite done migrating to Java 11:
>>>>>>> https://issues.apache.org/jira/browse/HIVE-22415
>>>>>>>
>>>>>>> I'm not sure whether that's still a problem, but we currently don't
>>>>>>> build Hive 3 support unless we're using Java 8. That makes me think that
>>>>>>> dropping JDK 8 support would probably also make it a lot more difficult for
>>>>>>> Hive to do releases based on Iceberg. Even with some of the integration
>>>>>>> moving into the Hive project, if we started shipping JDK 11 Jars then Hive
>>>>>>> would no longer be able to update.
>>>>>>>
>>>>>>> Ryan
>>>>>>>
>>>>>>> On Fri, Apr 21, 2023 at 5:02 PM Anton Okolnychyi <
>>>>>>> aokolnychyi@apple.com.invalid> wrote:
>>>>>>>
>>>>>>>> Sorry, I wasn’t clear that I also imply dropping JDK 8 (unless
>>>>>>>> there is a good reason to keep it?).
>>>>>>>>
>>>>>>>> - Anton
>>>>>>>>
>>>>>>>> On Apr 21, 2023, at 4:59 PM, Ryan Blue <bl...@tabular.io> wrote:
>>>>>>>>
>>>>>>>> Would we also drop support for JDK 8?
>>>>>>>>
>>>>>>>> On Fri, Apr 21, 2023 at 4:58 PM Anton Okolnychyi <
>>>>>>>> aokolnychyi@apple.com.invalid> wrote:
>>>>>>>>
>>>>>>>>> Following up on the discussion in the Spark 2.4 thread, shall we
>>>>>>>>> move to JDK 11 for releases as Spark 2.4 support has been dropped?
>>>>>>>>>
>>>>>>>>> - Anton
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Ryan Blue
>>>>>>>> Tabular
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Ryan Blue
>>>>>>> Tabular
>>>>>>>
>>>>>>
>>>>>
>>>>> --
>>>>> Ryan Blue
>>>>> Tabular
>>>>>
>>>>
>>
>
> --
> Ryan Blue
> Tabular
>
>
>

Re: [DISCUSS] Switch to JDK 11 for releases?

[Anton Okolnychyi <ao...@apple.com.INVALID>]

That would be great, I don’t think anyone picked it up.

We don't use sun.* classes directly anywhere in the code, as far as I remember.

- Anton

> On May 25, 2023, at 11:57 AM, Laurent Goujon <laurent@dremio.com.INVALID <ma...@dremio.com.INVALID>> wrote:
> 
> I can give it a try if the community is okay with it. I have some experience using it, it actually works quite well as long as you are not using things like `sun.*` classes
> 
> Laurent
> 
> On Wed, Apr 26, 2023 at 8:28 AM Anton Okolnychyi <aokolnychyi@apple.com.invalid <ma...@apple.com.invalid>> wrote:
> Is anyone interested to give it a try?
> 
> - Anton
> 
>> On Apr 25, 2023, at 4:13 PM, Ryan Blue <blue@tabular.io <ma...@tabular.io>> wrote:
>> 
>> Yeah, I do like the idea of using --release. We'll need to test it with those platforms though.
>> 
>> On Tue, Apr 25, 2023 at 12:04 PM Anton Okolnychyi <aokolnychyi@apple.com.invalid <ma...@apple.com.invalid>> wrote:
>> It would be interesting to hear more from Hive/Impala folks on this topic.
>> 
>> I’d avoid supporting both JDK8 and JDK11 given the complexity we need to add. Our build is already complicated.
>> 
>> Jack’s idea of using JDK11 with --release flag may be worth exploring.
>> 
>> - Anton
>> 
>>> On Apr 24, 2023, at 10:11 AM, Jack Ye <yezhaoqin@gmail.com <ma...@gmail.com>> wrote:
>>> 
>>> I agree, it wouldn't help given the fact that we won't be able to use the features in newer JDKs.
>>> 
>>> However, I think there is still a difference in the artifact compiled by JDK8, vs compiled by JDK11 with --release=8, that might be useful. For example, I came across this try with resource introduces unreachable byte code issue <https://stackoverflow.com/questions/25615417/try-with-resources-introduce-unreachable-bytecode/25746587#25746587> which is fixed in JDK11, and some optimizations in JDK9 related to string concatenation <https://openjdk.org/jeps/280>. Although Javac is not an optimizing compiler and there should not be much difference in performance of the jars produced by different compilers, these changes might be worth for the project to declare a newer compile-time JDK across all modules, and just always enforce --release flag to 8 to ensure people do not use newer JDK features. But these are quite minor changes so I am not very opinionated either way.
>>> 
>>> Any thoughts?
>>> 
>>> -Jack
>>> 
>>> On Mon, Apr 24, 2023 at 2:22 AM Zoltán Borók-Nagy <boroknagyz@apache.org <ma...@apache.org>> wrote:
>>> Besides Hive, neither Impala is compatible with Java11 right now. This work is in-progress: https://issues.apache.org/jira/browse/IMPALA-11360 <https://issues.apache.org/jira/browse/IMPALA-11360>
>>> 
>>> - Zoltan
>>> 
>>> On Mon, Apr 24, 2023 at 11:07 AM Mass Dosage <massdosage@gmail.com <ma...@gmail.com>> wrote:
>>> I agree with Ryan, unless you can change the source version there's not that much point. 
>>> 
>>> On the Hive front, as you can see from that ticket it's been open for 4(!) years and hasn't received much action recently. I think it's one of the reasons AWS EMR still defaults to Java 8. It would be really great if they could finally push that one over the finish line.
>>> 
>>> On Sat, 22 Apr 2023 at 20:43, Ryan Blue <blue@tabular.io <ma...@tabular.io>> wrote:
>>> I think in order to publish multiple versions we would need to have different artifact names, like Scala uses (e.g. _2.12).
>>> 
>>> It probably also wouldn't help. If we have to remain compatible with JDK 8, then publishing some artifacts for JDK 11 would still mean only using JDK 8 features. The source version is what we care about more, so if we can't change it then we can't really do anything else.
>>> 
>>> On Sat, Apr 22, 2023 at 10:12 AM Jack Ye <yezhaoqin@gmail.com <ma...@gmail.com>> wrote:
>>> Would it be an option to use --release flag to control the release target version, and publish 2 versions of the library to Maven, 1 for JDK8 and 1 for JDK11?
>>> 
>>> Jack
>>> 
>>> On Fri, Apr 21, 2023 at 5:17 PM Ryan Blue <blue@tabular.io <ma...@tabular.io>> wrote:
>>> Looks like Hive isn't quite done migrating to Java 11: https://issues.apache.org/jira/browse/HIVE-22415 <https://issues.apache.org/jira/browse/HIVE-22415>
>>> 
>>> I'm not sure whether that's still a problem, but we currently don't build Hive 3 support unless we're using Java 8. That makes me think that dropping JDK 8 support would probably also make it a lot more difficult for Hive to do releases based on Iceberg. Even with some of the integration moving into the Hive project, if we started shipping JDK 11 Jars then Hive would no longer be able to update.
>>> 
>>> Ryan
>>> 
>>> On Fri, Apr 21, 2023 at 5:02 PM Anton Okolnychyi <aokolnychyi@apple.com.invalid <ma...@apple.com.invalid>> wrote:
>>> Sorry, I wasn’t clear that I also imply dropping JDK 8 (unless there is a good reason to keep it?).
>>> 
>>> - Anton
>>> 
>>>> On Apr 21, 2023, at 4:59 PM, Ryan Blue <blue@tabular.io <ma...@tabular.io>> wrote:
>>>> 
>>>> Would we also drop support for JDK 8?
>>>> 
>>>> On Fri, Apr 21, 2023 at 4:58 PM Anton Okolnychyi <aokolnychyi@apple.com.invalid <ma...@apple.com.invalid>> wrote:
>>>> Following up on the discussion in the Spark 2.4 thread, shall we move to JDK 11 for releases as Spark 2.4 support has been dropped?
>>>> 
>>>> - Anton
>>>> 
>>>> 
>>>> -- 
>>>> Ryan Blue
>>>> Tabular
>>> 
>>> 
>>> 
>>> -- 
>>> Ryan Blue
>>> Tabular
>>> 
>>> 
>>> -- 
>>> Ryan Blue
>>> Tabular
>> 
>> 
>> 
>> -- 
>> Ryan Blue
>> Tabular
>