You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Rainer Jung <ra...@kippdata.de> on 2017/09/23 18:01:47 UTC
Multi-release Jars [Was: Java 9 TC shutdown warnings (reflection in
WebappClassLoaderBase)]
Am 23.09.2017 um 17:22 schrieb Mark Thomas:
> On 23/09/17 16:10, Rainer Jung wrote:
>> Thanks Mark!
>>
>> You might also want to look at r1809434 as a candidate and check whether
>> that workaround looks OK for you.
>
> Looks good to me. I've added to the list of J9 back-port candidates. It
> does beg the question whether or not we want to look at switching to
> multi-release JARs for all of the compatibility code. WDYT?
Indeed, I ran my first multi-release Jar experiment (successfully)
today. For those who haven't yet seen it: starting with Java 9 one can
include instances of a class in a jar file that will only get used when
a specific JVM (major) version is detected at runtime. You can e.g. have
a default one as usual plus additional ones for specific Java runtime
versions. For Java 9 one would add a class at
META-INF/versions/9/org/apache/juli/... which would then be picked when
the runtime Java version is 9.
The enable the feature, one simply has to add the attribute
Multi-Release: true
to the Manifest file of the jar.
Thinks that come to my mind:
- you can have multiple versions of the same class. If we would use it
like that, we would have to slightly reorganize our svn layout to
reflect that, e.g. java/org/... plus new java9/org/....
- accordingly we would need e.g. a directory output/classes9 or similar.
- If we need Java 9 for compilation of the Java 9 classes, we would
mimic the Java 7 lines from the TC 7 build.xml.
- To add the Java 9 classes to one of our jars it would suffice in
build.xml to do something like
<jar jarfile="${somefile.jar}" update="true">
<manifest>
<attribute name="Multi-Release" value="true"/>
</manifest>
<zipfileset prefix="META-INF/versions/9/" dir="${tomcat.classes}9"/>
</jar>
- to reduce redundant maintenance it would be good to factor out JVM
dependent code, so that the classes we have to maintain multiply mostly
contain the code that's really version-dependent.
For instance when using the feature for my Java 9 change in Juli
http://svn.apache.org/viewvc?rev=1809434&view=rev one could use e.g. a
small but version dependent Constants.java which would provide the
correct directory name ("lib" versus "conf"). Or more easily a one line
properties file available in two versions providing the directory name.
I have not made any check on performance implications for the slightly
more complex class loading. What I did see, is that -verbose:class does
not indicate, which of the versions was chosen.
I currently do not have a real insight on how our most important JVM
version dependencies look. I guess how useful multi-release is, might
depend on the type of code dependencies we have.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Multi-release Jars
Posted by Mark Thomas <ma...@apache.org>.
On 11/10/17 17:12, Konstantin Kolinko wrote:
> 2017-10-11 14:26 GMT+03:00 Mark Thomas <ma...@apache.org>:
>>
>> Having looked at a couple of options I settled on that one too.
>>
>> I haven't yet found an IDE with a GA release that handles this. Support
>> in the tools is fairly embryonic as well.
>
> Several thoughts:
>
> 1. I think Eclipse does not support having different version of Java
> for different subsets of files in a project.
> (I am on 4.6 Neon now, haven't tried 4.7 Oxygen yet)
Correct. I'm working with the latest (4.7.1a) and this is still the case.
(Aside: My initial impressions are 4.7.1a is a lot faster then 4.6.x.
You might want to consider an upgrade)
> As such, we will have to use two separate projects,
> similar to the two projects in Tomcat 7.0 ("tomcat-7.0.x" and
> "tomcat-7.0.x-java7").
That is one way and the only one I'm aware of that allows working in
both versions at the same time. There are other options if you don't
mind changing config to switch between versions. Whatever the approach,
none of them are ideal.
> There are two separate goals in build.xml to generate the two
> projects. Documented in
> http://tomcat.apache.org/tomcat-7.0-doc/building.html#Building_with_Eclipse
>
> It was easier for Tomcat 7 with clear separation of WebSocket APIs.
> I wonder how it will go with Tomcat 9, and with further backport of
> Java 9 support to Tomcat 8.5 and 7.0
It would be worth testing. There might be a conflict between the
multiple JreCompat versions.
> 2. I think of changing JDK configuration in build files to use
> explicit paths to JDK versions.
>
> In Tomcat 7 we have "java.7.home" property to specify location of JDK 7.
> My idea is to have explicit
> java.6.home=
> java.7.home=
> java.8.home=
>
> If either of them is missing, ${env.JAVA_HOME} provides the value -
> like it is done in Tomcat 7 build.xml.
>
> The goal is to be able to run build.xml with a later version of Java.
I agree that should always be possible. I thought it was at the moment.
However, my preference would always be to use the minimum for a release
build as a safety check that features from a later version hadn't
accidentally crept in.
> Advantages of this approach:
>
> 1) It resolves problem with download of some of dependent libraries.
> (Those libraries require https connection and cannot be downloaded
> with an old version of Java (Java 6), but can be downloaded with Java
> 8).
>
> 2) It allows us to upgrade to a never version of Checkstyle.
>
> As 1) and 2) are not much of a problem, I have not pursued this idea yet.
Makes sense.
>>>> - accordingly we would need e.g. a directory output/classes9 or similar.
>>
>> I went for classes/META-INF/versions/9 so it mirrored the JAR structure
>> but that probably isn't essential. One advantage was that - with my
>> other build.xml changes - supporting additional versions was minimal
>> effort.
>>
>>>> - If we need Java 9 for compilation of the Java 9 classes, we would
>>>> mimic the Java 7 lines from the TC 7 build.xml.
>>
>> For the JreCompat code, we would need this. This is where most of the
>> complexity was added for this approach.
>>
>>>> - To add the Java 9 classes to one of our jars it would suffice in
>>>> build.xml to do something like
>>>>
>>>> <jar jarfile="${somefile.jar}" update="true">
>>>> <manifest>
>>>> <attribute name="Multi-Release" value="true"/>
>>>> </manifest>
>>>> <zipfileset prefix="META-INF/versions/9/" dir="${tomcat.classes}9"/>
>>>> </jar>
>>
>> I took a slightly different approach. I extended the existing jarIt
>> macro and provided a default.mr.manifest. We'd also need to decide how
>> to handle the source JARs in this case. I didn't implement it but my
>> conclusion was we should mirror the structure of the binary JARs.
>>
>>>> - to reduce redundant maintenance it would be good to factor out JVM
>>>> dependent code, so that the classes we have to maintain multiply
>>>> mostly contain the code that's really version-dependent.
>>
>> Big +1. I looked at removing JreCompat entirely but that would have
>> required a lot of duplication. We have some large classes that only need
>> one or two lines of Java 9 code.
>>
>> [...]
>>
>> MR means release builds will required Java 8 and Java 9 (or just Java 9
>> with a risk we add a hard Java 9 dependency without realising).
>>
>> MR is not handled well by IDEs requiring manual inclusion / exclusion of
>> source files (or multiple projects) to switch between the different
>> versions.
>>
>> I do like the reduction in code and the build script complexity doesn't
>> concern me overly - it is mostly adding volume rather than an complex
>> logic. The IDE issues are more annoying.
>
> I agree with the above. And the IDE issues are annoying.
>
>> Overall, I'm torn. I guess I'm +0 on switching at this point. What does
>> everyone else think?
>>
>> Mark
>> [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=61597
>
> Best regards,
> Konstantin Kolinko
Thanks,
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Multi-release Jars
Posted by Konstantin Kolinko <kn...@gmail.com>.
2017-10-11 14:26 GMT+03:00 Mark Thomas <ma...@apache.org>:
>
> Having looked at a couple of options I settled on that one too.
>
> I haven't yet found an IDE with a GA release that handles this. Support
> in the tools is fairly embryonic as well.
Several thoughts:
1. I think Eclipse does not support having different version of Java
for different subsets of files in a project.
(I am on 4.6 Neon now, haven't tried 4.7 Oxygen yet)
As such, we will have to use two separate projects,
similar to the two projects in Tomcat 7.0 ("tomcat-7.0.x" and
"tomcat-7.0.x-java7").
There are two separate goals in build.xml to generate the two
projects. Documented in
http://tomcat.apache.org/tomcat-7.0-doc/building.html#Building_with_Eclipse
It was easier for Tomcat 7 with clear separation of WebSocket APIs.
I wonder how it will go with Tomcat 9, and with further backport of
Java 9 support to Tomcat 8.5 and 7.0
2. I think of changing JDK configuration in build files to use
explicit paths to JDK versions.
In Tomcat 7 we have "java.7.home" property to specify location of JDK 7.
My idea is to have explicit
java.6.home=
java.7.home=
java.8.home=
If either of them is missing, ${env.JAVA_HOME} provides the value -
like it is done in Tomcat 7 build.xml.
The goal is to be able to run build.xml with a later version of Java.
Advantages of this approach:
1) It resolves problem with download of some of dependent libraries.
(Those libraries require https connection and cannot be downloaded
with an old version of Java (Java 6), but can be downloaded with Java
8).
2) It allows us to upgrade to a never version of Checkstyle.
As 1) and 2) are not much of a problem, I have not pursued this idea yet.
>>> - accordingly we would need e.g. a directory output/classes9 or similar.
>
> I went for classes/META-INF/versions/9 so it mirrored the JAR structure
> but that probably isn't essential. One advantage was that - with my
> other build.xml changes - supporting additional versions was minimal
> effort.
>
>>> - If we need Java 9 for compilation of the Java 9 classes, we would
>>> mimic the Java 7 lines from the TC 7 build.xml.
>
> For the JreCompat code, we would need this. This is where most of the
> complexity was added for this approach.
>
>>> - To add the Java 9 classes to one of our jars it would suffice in
>>> build.xml to do something like
>>>
>>> <jar jarfile="${somefile.jar}" update="true">
>>> <manifest>
>>> <attribute name="Multi-Release" value="true"/>
>>> </manifest>
>>> <zipfileset prefix="META-INF/versions/9/" dir="${tomcat.classes}9"/>
>>> </jar>
>
> I took a slightly different approach. I extended the existing jarIt
> macro and provided a default.mr.manifest. We'd also need to decide how
> to handle the source JARs in this case. I didn't implement it but my
> conclusion was we should mirror the structure of the binary JARs.
>
>>> - to reduce redundant maintenance it would be good to factor out JVM
>>> dependent code, so that the classes we have to maintain multiply
>>> mostly contain the code that's really version-dependent.
>
> Big +1. I looked at removing JreCompat entirely but that would have
> required a lot of duplication. We have some large classes that only need
> one or two lines of Java 9 code.
>
> [...]
>
> MR means release builds will required Java 8 and Java 9 (or just Java 9
> with a risk we add a hard Java 9 dependency without realising).
>
> MR is not handled well by IDEs requiring manual inclusion / exclusion of
> source files (or multiple projects) to switch between the different
> versions.
>
> I do like the reduction in code and the build script complexity doesn't
> concern me overly - it is mostly adding volume rather than an complex
> logic. The IDE issues are more annoying.
I agree with the above. And the IDE issues are annoying.
> Overall, I'm torn. I guess I'm +0 on switching at this point. What does
> everyone else think?
>
> Mark
> [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=61597
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Multi-release Jars
Posted by Mark Thomas <ma...@apache.org>.
Hi,
Following up on this because I have been looking at a multi-release JAR
approach for bug 61597 [1] and the current JreCompat implementation.
On 23/09/17 19:02, Rainer Jung wrote:
> Forgot the reference: http://openjdk.java.net/jeps/238
>
> Am 23.09.2017 um 20:01 schrieb Rainer Jung:
>> Am 23.09.2017 um 17:22 schrieb Mark Thomas:
>>> On 23/09/17 16:10, Rainer Jung wrote:
>>>> Thanks Mark!
>>>>
>>>> You might also want to look at r1809434 as a candidate and check
>>>> whether
>>>> that workaround looks OK for you.
>>>
>>> Looks good to me. I've added to the list of J9 back-port candidates. It
>>> does beg the question whether or not we want to look at switching to
>>> multi-release JARs for all of the compatibility code. WDYT?
>>
>> Indeed, I ran my first multi-release Jar experiment (successfully)
>> today. For those who haven't yet seen it: starting with Java 9 one can
>> include instances of a class in a jar file that will only get used
>> when a specific JVM (major) version is detected at runtime. You can
>> e.g. have a default one as usual plus additional ones for specific
>> Java runtime versions. For Java 9 one would add a class at
>> META-INF/versions/9/org/apache/juli/... which would then be picked
>> when the runtime Java version is 9.
>>
>> The enable the feature, one simply has to add the attribute
>>
>> Multi-Release: true
>>
>> to the Manifest file of the jar.
>>
>> Thinks that come to my mind:
>>
>> - you can have multiple versions of the same class. If we would use it
>> like that, we would have to slightly reorganize our svn layout to
>> reflect that, e.g. java/org/... plus new java9/org/....
Having looked at a couple of options I settled on that one too.
I haven't yet found an IDE with a GA release that handles this. Support
in the tools is fairly embryonic as well.
>> - accordingly we would need e.g. a directory output/classes9 or similar.
I went for classes/META-INF/versions/9 so it mirrored the JAR structure
but that probably isn't essential. One advantage was that - with my
other build.xml changes - supporting additional versions was minimal
effort.
>> - If we need Java 9 for compilation of the Java 9 classes, we would
>> mimic the Java 7 lines from the TC 7 build.xml.
For the JreCompat code, we would need this. This is where most of the
complexity was added for this approach.
>> - To add the Java 9 classes to one of our jars it would suffice in
>> build.xml to do something like
>>
>> <jar jarfile="${somefile.jar}" update="true">
>> <manifest>
>> <attribute name="Multi-Release" value="true"/>
>> </manifest>
>> <zipfileset prefix="META-INF/versions/9/" dir="${tomcat.classes}9"/>
>> </jar>
I took a slightly different approach. I extended the existing jarIt
macro and provided a default.mr.manifest. We'd also need to decide how
to handle the source JARs in this case. I didn't implement it but my
conclusion was we should mirror the structure of the binary JARs.
>> - to reduce redundant maintenance it would be good to factor out JVM
>> dependent code, so that the classes we have to maintain multiply
>> mostly contain the code that's really version-dependent.
Big +1. I looked at removing JreCompat entirely but that would have
required a lot of duplication. We have some large classes that only need
one or two lines of Java 9 code.
>> For instance when using the feature for my Java 9 change in Juli
>> http://svn.apache.org/viewvc?rev=1809434&view=rev one could use e.g. a
>> small but version dependent Constants.java which would provide the
>> correct directory name ("lib" versus "conf"). Or more easily a one
>> line properties file available in two versions providing the directory
>> name.
>>
>> I have not made any check on performance implications for the slightly
>> more complex class loading. What I did see, is that -verbose:class
>> does not indicate, which of the versions was chosen.
I haven't checked performance either. My working assumption is that if
it is an issue, the JVM vendors will have to address it somehow.
>> I currently do not have a real insight on how our most important JVM
>> version dependencies look. I guess how useful multi-release is, might
>> depend on the type of code dependencies we have.
With the fix for bug 61597 applied the MR JAR approach looks to use
slightly less code (maybe ~20 lines overall including comments).
So, comparing the MR approach vs continuing with the reflection approach:
MR uses less code but requires more complexity in the build script.
Overall, MR requires slightly less code.
MR means release builds will required Java 8 and Java 9 (or just Java 9
with a risk we add a hard Java 9 dependency without realising).
MR is not handled well by IDEs requiring manual inclusion / exclusion of
source files (or multiple projects) to switch between the different
versions.
I do like the reduction in code and the build script complexity doesn't
concern me overly - it is mostly adding volume rather than an complex
logic. The IDE issues are more annoying.
Overall, I'm torn. I guess I'm +0 on switching at this point. What does
everyone else think?
Mark
[1] https://bz.apache.org/bugzilla/show_bug.cgi?id=61597
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Multi-release Jars [Was: Java 9 TC shutdown warnings (reflection
in WebappClassLoaderBase)]
Posted by Rainer Jung <ra...@kippdata.de>.
Forgot the reference: http://openjdk.java.net/jeps/238
Am 23.09.2017 um 20:01 schrieb Rainer Jung:
> Am 23.09.2017 um 17:22 schrieb Mark Thomas:
>> On 23/09/17 16:10, Rainer Jung wrote:
>>> Thanks Mark!
>>>
>>> You might also want to look at r1809434 as a candidate and check whether
>>> that workaround looks OK for you.
>>
>> Looks good to me. I've added to the list of J9 back-port candidates. It
>> does beg the question whether or not we want to look at switching to
>> multi-release JARs for all of the compatibility code. WDYT?
>
> Indeed, I ran my first multi-release Jar experiment (successfully)
> today. For those who haven't yet seen it: starting with Java 9 one can
> include instances of a class in a jar file that will only get used when
> a specific JVM (major) version is detected at runtime. You can e.g. have
> a default one as usual plus additional ones for specific Java runtime
> versions. For Java 9 one would add a class at
> META-INF/versions/9/org/apache/juli/... which would then be picked when
> the runtime Java version is 9.
>
> The enable the feature, one simply has to add the attribute
>
> Multi-Release: true
>
> to the Manifest file of the jar.
>
> Thinks that come to my mind:
>
> - you can have multiple versions of the same class. If we would use it
> like that, we would have to slightly reorganize our svn layout to
> reflect that, e.g. java/org/... plus new java9/org/....
>
> - accordingly we would need e.g. a directory output/classes9 or similar.
>
> - If we need Java 9 for compilation of the Java 9 classes, we would
> mimic the Java 7 lines from the TC 7 build.xml.
>
> - To add the Java 9 classes to one of our jars it would suffice in
> build.xml to do something like
>
> <jar jarfile="${somefile.jar}" update="true">
> <manifest>
> <attribute name="Multi-Release" value="true"/>
> </manifest>
> <zipfileset prefix="META-INF/versions/9/" dir="${tomcat.classes}9"/>
> </jar>
>
> - to reduce redundant maintenance it would be good to factor out JVM
> dependent code, so that the classes we have to maintain multiply mostly
> contain the code that's really version-dependent.
>
> For instance when using the feature for my Java 9 change in Juli
> http://svn.apache.org/viewvc?rev=1809434&view=rev one could use e.g. a
> small but version dependent Constants.java which would provide the
> correct directory name ("lib" versus "conf"). Or more easily a one line
> properties file available in two versions providing the directory name.
>
> I have not made any check on performance implications for the slightly
> more complex class loading. What I did see, is that -verbose:class does
> not indicate, which of the versions was chosen.
>
> I currently do not have a real insight on how our most important JVM
> version dependencies look. I guess how useful multi-release is, might
> depend on the type of code dependencies we have.
>
> Regards,
>
> Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org