You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Filip Hanik <fh...@vmware.com> on 2020/07/13 21:59:46 UTC
Native Image - Reflectionless Concept
for discussion, all feedback and questions welcome:
I've created a concept of having Apache Tomcat, embedded, run without reflection in a native image.
This concept creates a jar, tomcat-embedded-programmatic.jar, that can be fine tuned to only include what is needed in a default configuration when an embedded tomcat instance is used and configured programatically.
Steps to run Apache Tomcat using Java 8 without reflection
1. Make sure you have native-image (from the graal installation) on your path
2. git clone -b feature/embed-minimal-programmatic-jar-file-master git@github.com:fhanik/tomcat.git
3. cd tomcat/res/graal/
4. ./build-tomcat-native-image.sh && ./graal-measure.sh
Should yield an output similar to (Graal 20.1):
SUCCESS: the servlet is working
RSS memory: 20.7M
Image size: 20.5M
or using an older graal, 19.2
SUCCESS: the servlet is working
RSS memory: 18.0M
Image size: 16.7M
This also leaves a file named ${java.io.tmpdir}/XReflectionIntrospectionUtils.java so that you can review the solution to IntrospectionUtils.java
Goals of this concept
1. Do not break anything
2. Create a new and optimized for size artifact, tomcat-embedded-programmatic
3. Remove reflection by introspecting classes that are currently passed into IntrospectionUtils.set/getProperty by generating setters/getters at build time
How it's done
1. I've build out a small introspection tool in the package org.apache.tomcat.util.xreflect
2. During build time, it analyses a set of known classes that are used with IntrospectionUtils.java, and generates XReflectionIntrospectionUtils.java
3. When it packages tomcat-embed-programmatic.jar it uses the generated code when calling setProperty and getProperty
A PR would look like this:
https://github.com/apache/tomcat/compare/master...fhanik:feature/embed-minimal-programmatic-jar-file-master?expand=1
Re: Native Image - Reflectionless Concept
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Filip,
On 7/13/20 17:59, Filip Hanik wrote:
> for discussion, all feedback and questions welcome:
>
>
> I've created a concept of having Apache Tomcat, embedded, run
> without reflection in a native image. This concept creates a jar,
> tomcat-embedded-programmatic.jar, that can be fine tuned to only
> include what is needed in a default configuration when an embedded
> tomcat instance is used and configured programatically.
>
> Steps to run Apache Tomcat using Java 8 without reflection
>
> 1. Make sure you have native-image (from the graal installation)
> on your path 2. git clone -b
> feature/embed-minimal-programmatic-jar-file-master
> git@github.com:fhanik/tomcat.git 3. cd tomcat/res/graal/ 4.
> ./build-tomcat-native-image.sh && ./graal-measure.sh
>
> Should yield an output similar to (Graal 20.1): SUCCESS: the
> servlet is working RSS memory: 20.7M Image size: 20.5M
>
>
> or using an older graal, 19.2 SUCCESS: the servlet is working RSS
> memory: 18.0M Image size: 16.7M
>
>
> This also leaves a file named
> ${java.io.tmpdir}/XReflectionIntrospectionUtils.java so that you
> can review the solution to IntrospectionUtils.java
>
> Goals of this concept
>
> 1. Do not break anything 2. Create a new and optimized for size
> artifact, tomcat-embedded-programmatic 3. Remove reflection by
> introspecting classes that are currently passed into
> IntrospectionUtils.set/getProperty by generating setters/getters at
> build time
>
> How it's done
>
> 1. I've build out a small introspection tool in the package
> org.apache.tomcat.util.xreflect 2. During build time, it analyses a
> set of known classes that are used with IntrospectionUtils.java,
> and generates XReflectionIntrospectionUtils.java 3. When it
> packages tomcat-embed-programmatic.jar it uses the generated code
> when calling setProperty and getProperty
>
> A PR would look like this:
> https://github.com/apache/tomcat/compare/master...fhanik:feature/embed
- -minimal-programmatic-jar-file-master?expand=1
Sounds
>
like a really interesting ApacheCon presentation ;)
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAl8M5KkACgkQHPApP6U8
pFhhthAAiJjZwWzNNHwx3YQA1J/9sKn2KX50/QrcB+IOkjL8FBbM0bvRCRL1I0ZB
17nQEg5fb7Y+EHOtceYjmJTDrqdj6WIvF60JS5y0E4FUOt0wrqd1v7oTZnKGqgqs
84J8oPW71vpzR/2EL8tzCjn/Oa7rLU58yGtKObBxbgdZcCDa4j0iuklLy6dyktAb
JmxTeHWKmvkezbUKoflydSLb9Dxq7dL2XK2JDGw6iQi4FBiAXleooVw/MpZ1fOAm
jxPjOnf8NhjNoFJooYs/i7H8Oa0MF2BtdrC1qS7gJBJkAp7bOr0HIO2QZLshkSKe
AsC5wp6QfzMWLQsTc25Z5SKGZYNrP6aWkcALYgETjJywDZGmrJ/UO16+zOayqYGq
TvSbE+6+cVzlmj4xMf1UPJf/D9gdpqqWkC/YJpfyZgvqgXkBywbHrSUVyC0AbBKU
REQ6WFNO2idUrZNbmbC0OaLt4BdikjjazhaCoOWQHij83TElQqXCYbH+JcOrc3JE
+9w6Y7fdSwaR+1v2XUUk2McBQtJf9kP7G2iKBYgwrn1JTSrNSB2jvdi5f/HbtGUe
txV6AFJQeFEH/grfHbGeLh21B4UyOa2CkAbAD+dikIJqWBMxvm4DMG1cnJW/x/Rs
vVX3EbVDc6kEBgLQ1blfC7Yl2QdmUap7wP/X399bvH7bmtWJPPI=
=7nG7
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Native Image - Reflectionless Concept
Posted by Raymond Auge <ra...@liferay.com.INVALID>.
Xml-less Tomcat would also be very useful for OSGi http whiteboard
implementation which I'd like to work on in the coming months.
So I +1 more programmatic API.
Just let's not forget about tear down. So many APIs completely ignore a
tear down lifecycle.
- Ray
On Mon, Jul 20, 2020, 12:16 Romain Manni-Bucau, <rm...@gmail.com>
wrote:
> I think a xml-less tomcat is awaited since servlet 3 - graal or not - but
> agree risk is a bit higher.
> That said path is different so wonder if skipping a temp solution can not
> be worth after all for the community.
>
> Le lun. 20 juil. 2020 à 17:58, Filip Hanik <fi...@hanik.com> a écrit :
>
>>
>> On 7/20/20 8:47 AM, Romain Manni-Bucau wrote:
>>
>>
>>
>> Le lun. 20 juil. 2020 à 17:41, Filip Hanik <fi...@hanik.com> a écrit :
>>
>>> Thanks for chiming in:
>>> On 7/16/20 6:46 AM, Romain Manni-Bucau wrote:
>>>
>>> Hi everyone,
>>>
>>> I think the generation is the sanest option since code stay clean but it
>>> shouldn't be done in tomcat IMHO but in user code and with a nice wrapper
>>> (mvn tomcat:dump/gradle tomcatDump etc, or whatever name you like ;)).
>>>
>>> That's always an option, but it would become an external artifact and
>>> easily end up out of sync.
>>>
>>
>> Was thinking to keep the dumper in tomcat code base and the plugin to
>> consume it so it would stay in sync, but agree it is a small risk.
>>
>>
>>> This build phase would dump the descriptors in plain java and would load
>>> them with an unique - ie single for the whole webapp - plain SPI -
>>> ServiceLoader - maybe?
>>>
>>> The goal of this artifact was to reduce the size and classes from a full
>>> tomcat (already available in tomcat-embed-core), down to a code base where
>>> XML/digester/descriptors aren't used, hence tomcat-embed-programmatic
>>>
>>> This kind of build tool assumes you have all the runtime state in the
>>> build - which is typically the case for graalvm - so you can completely
>>> dump StandardContext state after start phase there and just reload it from
>>> the precomputed model.
>>> Only trick is about file paths which must either be recomputed or be
>>> configurable to another base but it does not sound crazy.
>>>
>>> The less tool-ed option would be to extract all "reflectionfull" code in
>>> methods and use graalvm substitutions to drop them and use plain java
>>> instead (with a good naming convention, it can be generated as well).
>>> Keeps the duplication but at least the main code stays clean and
>>> optimizations stays together.
>>>
>>> That's pretty much what we're doing right now. Many of these feel like
>>> hacks simply to mitigate how GraalVM/AOT does code initialization (all code
>>> loaded initialized at startup)
>>>
>>
>> Reviewing the hacks, all can be done cleanly if extracted in methods.
>> Pushing the logic next step - I don't know if worth it but trying to use
>> this picture to explain:
>>
>> 1. A noxml module can be done with protected methods/extension-points for
>> XML loading - even usable in java standalone mode
>> 2. Current tomcat can extend noxml module
>> 3. Graal can be based on 1
>>
>> This would benefit both jvm and graal users at the end.
>> Today 1 is possible with some hacks on tomcat embedded but it is highly
>> not natural so this can be an opportunity to make it a feature maybe?
>>
>> I believe that tomcat-embed-programmatic is a viable interim solution,
>> it's a low risk. The question for you, and the rest of the community, is
>> the reward itself enough? ie, is it worth it?
>>
>> There is some talk about making "native-ness" part of the Java itself,
>> and that could change a lot of assumptions. Making it a feature,
>> refactoring code to satisfy 1, is a bit more intrusive at this point in
>> time. I believe it introduces more risk than reward.
>>
>>
>> Filip
>>
>>
>>
>>
>>> Filip
>>>
>>>
>>>
>>> Romain Manni-Bucau
>>> @rmannibucau <https://twitter.com/rmannibucau> | Blog
>>> <https://rmannibucau.metawerx.net/> | Old Blog
>>> <http://rmannibucau.wordpress.com> | Github
>>> <https://github.com/rmannibucau> | LinkedIn
>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>>>
>>>
>>> Le jeu. 16 juil. 2020 à 14:31, Rémy Maucherat <re...@apache.org> a
>>> écrit :
>>>
>>>> On Mon, Jul 13, 2020 at 11:59 PM Filip Hanik <fh...@vmware.com> wrote:
>>>>
>>>>> for discussion, all feedback and questions welcome:
>>>>>
>>>>>
>>>>> I've created a concept of having Apache Tomcat, embedded, run without
>>>>> reflection in a native image.
>>>>> This concept creates a jar, tomcat-embedded-programmatic.jar, that can
>>>>> be fine tuned to only include what is needed in a default configuration
>>>>> when an embedded tomcat instance is used and configured programatically.
>>>>>
>>>>> Steps to run Apache Tomcat using Java 8 without reflection
>>>>>
>>>>> 1. Make sure you have native-image (from the graal installation)
>>>>> on your path
>>>>> 2. git clone -b feature/embed-minimal-programmatic-jar-file-master
>>>>> git@github.com:fhanik/tomcat.git
>>>>> 3. cd tomcat/res/graal/
>>>>> 4. ./build-tomcat-native-image.sh && ./graal-measure.sh
>>>>>
>>>>> Should yield an output similar to (Graal 20.1):
>>>>> SUCCESS: the servlet is working
>>>>> RSS memory: 20.7M
>>>>> Image size: 20.5M
>>>>>
>>>>>
>>>>> or using an older graal, 19.2
>>>>> SUCCESS: the servlet is working
>>>>> RSS memory: 18.0M
>>>>> Image size: 16.7M
>>>>>
>>>>>
>>>>> This also leaves a file named ${java.io.tmpdir}/
>>>>> XReflectionIntrospectionUtils.java so that you can review the
>>>>> solution to IntrospectionUtils.java
>>>>>
>>>>> Goals of this concept
>>>>>
>>>>> 1. Do not break anything
>>>>> 2. Create a new and optimized for size artifact,
>>>>> tomcat-embedded-programmatic
>>>>> 3. Remove reflection by introspecting classes that are currently
>>>>> passed into IntrospectionUtils.set/getProperty by generating
>>>>> setters/getters at build time
>>>>>
>>>>> How it's done
>>>>>
>>>>> 1. I've build out a small introspection tool in the package
>>>>> org.apache.tomcat.util.xreflect
>>>>> 2. During build time, it analyses a set of known classes that are
>>>>> used with IntrospectionUtils.java, and generates
>>>>> XReflectionIntrospectionUtils.java
>>>>> 3. When it packages tomcat-embed-programmatic.jar it uses the
>>>>> generated code when calling setProperty and getProperty
>>>>>
>>>>> A PR would look like this:
>>>>>
>>>>> https://github.com/apache/tomcat/compare/master...fhanik:feature/embed-minimal-programmatic-jar-file-master?expand=1
>>>>>
>>>>>
>>>> Well, this is a bit complex and hard to maintain (like, for example,
>>>> storeconfig), so that's a downside.
>>>>
>>>> So starting with Tomcat and its initial server.xml, the process would
>>>> be:
>>>> server.xml -> equivalent Tomcat embedded code -> equivalent Tomcat
>>>> embedded code with custom IntrospectionUtils code
>>>> The concrete benefits may be limited though.
>>>>
>>>> I looked at more code generation for web.xml since the digester is nice
>>>> for that, but the benefit becomes even more questionable. It is harder to
>>>> manage, and the generated classes have to be loaded dynamically [unless
>>>> even more code is generated]. If there are tons of fragments, there is a
>>>> good intuitive reason why it becomes useless. so I didn't want to do it. I
>>>> prefer if things remain a bit EE-ish, ultimately.
>>>>
>>>> Rémy
>>>>
>>>>
Re: Native Image - Reflectionless Concept
Posted by Romain Manni-Bucau <rm...@gmail.com>.
I think a xml-less tomcat is awaited since servlet 3 - graal or not - but
agree risk is a bit higher.
That said path is different so wonder if skipping a temp solution can not
be worth after all for the community.
Le lun. 20 juil. 2020 à 17:58, Filip Hanik <fi...@hanik.com> a écrit :
>
> On 7/20/20 8:47 AM, Romain Manni-Bucau wrote:
>
>
>
> Le lun. 20 juil. 2020 à 17:41, Filip Hanik <fi...@hanik.com> a écrit :
>
>> Thanks for chiming in:
>> On 7/16/20 6:46 AM, Romain Manni-Bucau wrote:
>>
>> Hi everyone,
>>
>> I think the generation is the sanest option since code stay clean but it
>> shouldn't be done in tomcat IMHO but in user code and with a nice wrapper
>> (mvn tomcat:dump/gradle tomcatDump etc, or whatever name you like ;)).
>>
>> That's always an option, but it would become an external artifact and
>> easily end up out of sync.
>>
>
> Was thinking to keep the dumper in tomcat code base and the plugin to
> consume it so it would stay in sync, but agree it is a small risk.
>
>
>> This build phase would dump the descriptors in plain java and would load
>> them with an unique - ie single for the whole webapp - plain SPI -
>> ServiceLoader - maybe?
>>
>> The goal of this artifact was to reduce the size and classes from a full
>> tomcat (already available in tomcat-embed-core), down to a code base where
>> XML/digester/descriptors aren't used, hence tomcat-embed-programmatic
>>
>> This kind of build tool assumes you have all the runtime state in the
>> build - which is typically the case for graalvm - so you can completely
>> dump StandardContext state after start phase there and just reload it from
>> the precomputed model.
>> Only trick is about file paths which must either be recomputed or be
>> configurable to another base but it does not sound crazy.
>>
>> The less tool-ed option would be to extract all "reflectionfull" code in
>> methods and use graalvm substitutions to drop them and use plain java
>> instead (with a good naming convention, it can be generated as well).
>> Keeps the duplication but at least the main code stays clean and
>> optimizations stays together.
>>
>> That's pretty much what we're doing right now. Many of these feel like
>> hacks simply to mitigate how GraalVM/AOT does code initialization (all code
>> loaded initialized at startup)
>>
>
> Reviewing the hacks, all can be done cleanly if extracted in methods.
> Pushing the logic next step - I don't know if worth it but trying to use
> this picture to explain:
>
> 1. A noxml module can be done with protected methods/extension-points for
> XML loading - even usable in java standalone mode
> 2. Current tomcat can extend noxml module
> 3. Graal can be based on 1
>
> This would benefit both jvm and graal users at the end.
> Today 1 is possible with some hacks on tomcat embedded but it is highly
> not natural so this can be an opportunity to make it a feature maybe?
>
> I believe that tomcat-embed-programmatic is a viable interim solution,
> it's a low risk. The question for you, and the rest of the community, is
> the reward itself enough? ie, is it worth it?
>
> There is some talk about making "native-ness" part of the Java itself, and
> that could change a lot of assumptions. Making it a feature, refactoring
> code to satisfy 1, is a bit more intrusive at this point in time. I believe
> it introduces more risk than reward.
>
>
> Filip
>
>
>
>
>> Filip
>>
>>
>>
>> Romain Manni-Bucau
>> @rmannibucau <https://twitter.com/rmannibucau> | Blog
>> <https://rmannibucau.metawerx.net/> | Old Blog
>> <http://rmannibucau.wordpress.com> | Github
>> <https://github.com/rmannibucau> | LinkedIn
>> <https://www.linkedin.com/in/rmannibucau> | Book
>> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>>
>>
>> Le jeu. 16 juil. 2020 à 14:31, Rémy Maucherat <re...@apache.org> a écrit :
>>
>>> On Mon, Jul 13, 2020 at 11:59 PM Filip Hanik <fh...@vmware.com> wrote:
>>>
>>>> for discussion, all feedback and questions welcome:
>>>>
>>>>
>>>> I've created a concept of having Apache Tomcat, embedded, run without
>>>> reflection in a native image.
>>>> This concept creates a jar, tomcat-embedded-programmatic.jar, that can
>>>> be fine tuned to only include what is needed in a default configuration
>>>> when an embedded tomcat instance is used and configured programatically.
>>>>
>>>> Steps to run Apache Tomcat using Java 8 without reflection
>>>>
>>>> 1. Make sure you have native-image (from the graal installation) on
>>>> your path
>>>> 2. git clone -b feature/embed-minimal-programmatic-jar-file-master
>>>> git@github.com:fhanik/tomcat.git
>>>> 3. cd tomcat/res/graal/
>>>> 4. ./build-tomcat-native-image.sh && ./graal-measure.sh
>>>>
>>>> Should yield an output similar to (Graal 20.1):
>>>> SUCCESS: the servlet is working
>>>> RSS memory: 20.7M
>>>> Image size: 20.5M
>>>>
>>>>
>>>> or using an older graal, 19.2
>>>> SUCCESS: the servlet is working
>>>> RSS memory: 18.0M
>>>> Image size: 16.7M
>>>>
>>>>
>>>> This also leaves a file named ${java.io.tmpdir}/
>>>> XReflectionIntrospectionUtils.java so that you can review the solution
>>>> to IntrospectionUtils.java
>>>>
>>>> Goals of this concept
>>>>
>>>> 1. Do not break anything
>>>> 2. Create a new and optimized for size artifact,
>>>> tomcat-embedded-programmatic
>>>> 3. Remove reflection by introspecting classes that are currently
>>>> passed into IntrospectionUtils.set/getProperty by generating
>>>> setters/getters at build time
>>>>
>>>> How it's done
>>>>
>>>> 1. I've build out a small introspection tool in the package
>>>> org.apache.tomcat.util.xreflect
>>>> 2. During build time, it analyses a set of known classes that are
>>>> used with IntrospectionUtils.java, and generates
>>>> XReflectionIntrospectionUtils.java
>>>> 3. When it packages tomcat-embed-programmatic.jar it uses the
>>>> generated code when calling setProperty and getProperty
>>>>
>>>> A PR would look like this:
>>>>
>>>> https://github.com/apache/tomcat/compare/master...fhanik:feature/embed-minimal-programmatic-jar-file-master?expand=1
>>>>
>>>>
>>> Well, this is a bit complex and hard to maintain (like, for example,
>>> storeconfig), so that's a downside.
>>>
>>> So starting with Tomcat and its initial server.xml, the process would be:
>>> server.xml -> equivalent Tomcat embedded code -> equivalent Tomcat
>>> embedded code with custom IntrospectionUtils code
>>> The concrete benefits may be limited though.
>>>
>>> I looked at more code generation for web.xml since the digester is nice
>>> for that, but the benefit becomes even more questionable. It is harder to
>>> manage, and the generated classes have to be loaded dynamically [unless
>>> even more code is generated]. If there are tons of fragments, there is a
>>> good intuitive reason why it becomes useless. so I didn't want to do it. I
>>> prefer if things remain a bit EE-ish, ultimately.
>>>
>>> Rémy
>>>
>>>
Re: Native Image - Reflectionless Concept
Posted by Filip Hanik <fi...@hanik.com>.
On 7/20/20 8:47 AM, Romain Manni-Bucau wrote:
>
>
> Le lun. 20 juil. 2020 à 17:41, Filip Hanik <filip@hanik.com
> <ma...@hanik.com>> a écrit :
>
> Thanks for chiming in:
>
> On 7/16/20 6:46 AM, Romain Manni-Bucau wrote:
>> Hi everyone,
>>
>> I think the generation is the sanest option since code stay clean
>> but it shouldn't be done in tomcat IMHO but in user code and with
>> a nice wrapper (mvn tomcat:dump/gradle tomcatDump etc, or
>> whatever name you like ;)).
> That's always an option, but it would become an external artifact
> and easily end up out of sync.
>
>
> Was thinking to keep the dumper in tomcat code base and the plugin to
> consume it so it would stay in sync, but agree it is a small risk.
>
>> This build phase would dump the descriptors in plain java and
>> would load them with an unique - ie single for the whole webapp -
>> plain SPI - ServiceLoader - maybe?
> The goal of this artifact was to reduce the size and classes from
> a full tomcat (already available in tomcat-embed-core), down to a
> code base where XML/digester/descriptors aren't used, hence
> tomcat-embed-programmatic
>> This kind of build tool assumes you have all the runtime state in
>> the build - which is typically the case for graalvm - so you can
>> completely dump StandardContext state after start phase there and
>> just reload it from the precomputed model.
>> Only trick is about file paths which must either be recomputed or
>> be configurable to another base but it does not sound crazy.
>>
>> The less tool-ed option would be to extract all
>> "reflectionfull" code in methods and use graalvm substitutions to
>> drop them and use plain java instead (with a good naming
>> convention, it can be generated as well).
>> Keeps the duplication but at least the main code stays clean and
>> optimizations stays together.
>
> That's pretty much what we're doing right now. Many of these feel
> like hacks simply to mitigate how GraalVM/AOT does code
> initialization (all code loaded initialized at startup)
>
>
> Reviewing the hacks, all can be done cleanly if extracted in methods.
> Pushing the logic next step - I don't know if worth it but trying to
> use this picture to explain:
>
> 1. A noxml module can be done with protected methods/extension-points
> for XML loading - even usable in java standalone mode
> 2. Current tomcat can extend noxml module
> 3. Graal can be based on 1
>
> This would benefit both jvm and graal users at the end.
> Today 1 is possible with some hacks on tomcat embedded but it is
> highly not natural so this can be an opportunity to make it a feature
> maybe?
I believe that tomcat-embed-programmatic is a viable interim solution,
it's a low risk. The question for you, and the rest of the community, is
the reward itself enough? ie, is it worth it?
There is some talk about making "native-ness" part of the Java itself,
and that could change a lot of assumptions. Making it a feature,
refactoring code to satisfy 1, is a bit more intrusive at this point in
time. I believe it introduces more risk than reward.
Filip
> Filip
>
>
>>
>> Romain Manni-Bucau
>> @rmannibucau <https://twitter.com/rmannibucau> | Blog
>> <https://rmannibucau.metawerx.net/> | Old Blog
>> <http://rmannibucau.wordpress.com> | Github
>> <https://github.com/rmannibucau> | LinkedIn
>> <https://www.linkedin.com/in/rmannibucau> | Book
>> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>>
>>
>> Le jeu. 16 juil. 2020 à 14:31, Rémy Maucherat <remm@apache.org
>> <ma...@apache.org>> a écrit :
>>
>> On Mon, Jul 13, 2020 at 11:59 PM Filip Hanik
>> <fhanik@vmware.com <ma...@vmware.com>> wrote:
>>
>> for discussion, all feedback and questions welcome:
>>
>>
>> I've created a concept of having Apache Tomcat, embedded,
>> run without reflection in a native image.
>> This concept creates a jar,
>> tomcat-embedded-programmatic.jar, that can be fine tuned
>> to only include what is needed in a default configuration
>> when an embedded tomcat instance is used and configured
>> programatically.
>>
>> Steps to run Apache Tomcat using Java 8 without reflection
>>
>> 1. Make sure you have native-image (from the graal
>> installation) on your path
>> 2. git clone -b
>> feature/embed-minimal-programmatic-jar-file-master
>> git@github.com:fhanik/tomcat.git
>> <mailto:git@github.com:fhanik/tomcat.git>
>> 3. cd tomcat/res/graal/
>> 4. ./build-tomcat-native-image.sh && ./graal-measure.sh
>>
>> Should yield an output similar to (Graal 20.1):
>> SUCCESS: the servlet is working
>> RSS memory: 20.7M
>> Image size: 20.5M
>>
>>
>> or using an older graal, 19.2
>> SUCCESS: the servlet is working
>> RSS memory: 18.0M
>> Image size: 16.7M
>>
>>
>> This also leaves a file named
>> ${java.io.tmpdir}/XReflectionIntrospectionUtils.java so
>> that you can review the solution to IntrospectionUtils.java
>>
>> Goals of this concept
>>
>> 1. Do not break anything
>> 2. Create a new and optimized for size artifact,
>> tomcat-embedded-programmatic
>> 3. Remove reflection by introspecting classes that are
>> currently passed into
>> IntrospectionUtils.set/getProperty by generating
>> setters/getters at build time
>>
>> How it's done
>>
>> 1. I've build out a small introspection tool in the
>> package org.apache.tomcat.util.xreflect
>> 2. During build time, it analyses a set of known classes
>> that are used with IntrospectionUtils.java, and
>> generates XReflectionIntrospectionUtils.java
>> 3. When it packages tomcat-embed-programmatic.jar it
>> uses the generated code when calling setProperty and
>> getProperty
>>
>> A PR would look like this:
>> https://github.com/apache/tomcat/compare/master...fhanik:feature/embed-minimal-programmatic-jar-file-master?expand=1
>>
>>
>> Well, this is a bit complex and hard to maintain (like, for
>> example, storeconfig), so that's a downside.
>>
>> So starting with Tomcat and its initial server.xml, the
>> process would be:
>> server.xml -> equivalent Tomcat embedded code -> equivalent
>> Tomcat embedded code with custom IntrospectionUtils code
>> The concrete benefits may be limited though.
>>
>> I looked at more code generation for web.xml since the
>> digester is nice for that, but the benefit becomes even more
>> questionable. It is harder to manage, and the generated
>> classes have to be loaded dynamically [unless even more code
>> is generated]. If there are tons of fragments, there is a
>> good intuitive reason why it becomes useless. so I didn't
>> want to do it. I prefer if things remain a bit EE-ish,
>> ultimately.
>>
>> Rémy
>>
Re: Native Image - Reflectionless Concept
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Le lun. 20 juil. 2020 à 17:41, Filip Hanik <fi...@hanik.com> a écrit :
> Thanks for chiming in:
> On 7/16/20 6:46 AM, Romain Manni-Bucau wrote:
>
> Hi everyone,
>
> I think the generation is the sanest option since code stay clean but it
> shouldn't be done in tomcat IMHO but in user code and with a nice wrapper
> (mvn tomcat:dump/gradle tomcatDump etc, or whatever name you like ;)).
>
> That's always an option, but it would become an external artifact and
> easily end up out of sync.
>
Was thinking to keep the dumper in tomcat code base and the plugin to
consume it so it would stay in sync, but agree it is a small risk.
> This build phase would dump the descriptors in plain java and would load
> them with an unique - ie single for the whole webapp - plain SPI -
> ServiceLoader - maybe?
>
> The goal of this artifact was to reduce the size and classes from a full
> tomcat (already available in tomcat-embed-core), down to a code base where
> XML/digester/descriptors aren't used, hence tomcat-embed-programmatic
>
> This kind of build tool assumes you have all the runtime state in the
> build - which is typically the case for graalvm - so you can completely
> dump StandardContext state after start phase there and just reload it from
> the precomputed model.
> Only trick is about file paths which must either be recomputed or be
> configurable to another base but it does not sound crazy.
>
> The less tool-ed option would be to extract all "reflectionfull" code in
> methods and use graalvm substitutions to drop them and use plain java
> instead (with a good naming convention, it can be generated as well).
> Keeps the duplication but at least the main code stays clean and
> optimizations stays together.
>
> That's pretty much what we're doing right now. Many of these feel like
> hacks simply to mitigate how GraalVM/AOT does code initialization (all code
> loaded initialized at startup)
>
Reviewing the hacks, all can be done cleanly if extracted in methods.
Pushing the logic next step - I don't know if worth it but trying to use
this picture to explain:
1. A noxml module can be done with protected methods/extension-points for
XML loading - even usable in java standalone mode
2. Current tomcat can extend noxml module
3. Graal can be based on 1
This would benefit both jvm and graal users at the end.
Today 1 is possible with some hacks on tomcat embedded but it is highly not
natural so this can be an opportunity to make it a feature maybe?
> Filip
>
>
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github
> <https://github.com/rmannibucau> | LinkedIn
> <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>
>
> Le jeu. 16 juil. 2020 à 14:31, Rémy Maucherat <re...@apache.org> a écrit :
>
>> On Mon, Jul 13, 2020 at 11:59 PM Filip Hanik <fh...@vmware.com> wrote:
>>
>>> for discussion, all feedback and questions welcome:
>>>
>>>
>>> I've created a concept of having Apache Tomcat, embedded, run without
>>> reflection in a native image.
>>> This concept creates a jar, tomcat-embedded-programmatic.jar, that can
>>> be fine tuned to only include what is needed in a default configuration
>>> when an embedded tomcat instance is used and configured programatically.
>>>
>>> Steps to run Apache Tomcat using Java 8 without reflection
>>>
>>> 1. Make sure you have native-image (from the graal installation) on
>>> your path
>>> 2. git clone -b feature/embed-minimal-programmatic-jar-file-master
>>> git@github.com:fhanik/tomcat.git
>>> 3. cd tomcat/res/graal/
>>> 4. ./build-tomcat-native-image.sh && ./graal-measure.sh
>>>
>>> Should yield an output similar to (Graal 20.1):
>>> SUCCESS: the servlet is working
>>> RSS memory: 20.7M
>>> Image size: 20.5M
>>>
>>>
>>> or using an older graal, 19.2
>>> SUCCESS: the servlet is working
>>> RSS memory: 18.0M
>>> Image size: 16.7M
>>>
>>>
>>> This also leaves a file named ${java.io.tmpdir}/
>>> XReflectionIntrospectionUtils.java so that you can review the solution
>>> to IntrospectionUtils.java
>>>
>>> Goals of this concept
>>>
>>> 1. Do not break anything
>>> 2. Create a new and optimized for size artifact,
>>> tomcat-embedded-programmatic
>>> 3. Remove reflection by introspecting classes that are currently
>>> passed into IntrospectionUtils.set/getProperty by generating
>>> setters/getters at build time
>>>
>>> How it's done
>>>
>>> 1. I've build out a small introspection tool in the package
>>> org.apache.tomcat.util.xreflect
>>> 2. During build time, it analyses a set of known classes that are
>>> used with IntrospectionUtils.java, and generates
>>> XReflectionIntrospectionUtils.java
>>> 3. When it packages tomcat-embed-programmatic.jar it uses the
>>> generated code when calling setProperty and getProperty
>>>
>>> A PR would look like this:
>>>
>>> https://github.com/apache/tomcat/compare/master...fhanik:feature/embed-minimal-programmatic-jar-file-master?expand=1
>>>
>>>
>> Well, this is a bit complex and hard to maintain (like, for example,
>> storeconfig), so that's a downside.
>>
>> So starting with Tomcat and its initial server.xml, the process would be:
>> server.xml -> equivalent Tomcat embedded code -> equivalent Tomcat
>> embedded code with custom IntrospectionUtils code
>> The concrete benefits may be limited though.
>>
>> I looked at more code generation for web.xml since the digester is nice
>> for that, but the benefit becomes even more questionable. It is harder to
>> manage, and the generated classes have to be loaded dynamically [unless
>> even more code is generated]. If there are tons of fragments, there is a
>> good intuitive reason why it becomes useless. so I didn't want to do it. I
>> prefer if things remain a bit EE-ish, ultimately.
>>
>> Rémy
>>
>>
Re: Native Image - Reflectionless Concept
Posted by Filip Hanik <fi...@hanik.com>.
Thanks for chiming in:
On 7/16/20 6:46 AM, Romain Manni-Bucau wrote:
> Hi everyone,
>
> I think the generation is the sanest option since code stay clean but
> it shouldn't be done in tomcat IMHO but in user code and with a nice
> wrapper (mvn tomcat:dump/gradle tomcatDump etc, or whatever name you
> like ;)).
That's always an option, but it would become an external artifact and
easily end up out of sync.
> This build phase would dump the descriptors in plain java and would
> load them with an unique - ie single for the whole webapp - plain SPI
> - ServiceLoader - maybe?
The goal of this artifact was to reduce the size and classes from a full
tomcat (already available in tomcat-embed-core), down to a code base
where XML/digester/descriptors aren't used, hence tomcat-embed-programmatic
> This kind of build tool assumes you have all the runtime state in the
> build - which is typically the case for graalvm - so you can
> completely dump StandardContext state after start phase there and just
> reload it from the precomputed model.
> Only trick is about file paths which must either be recomputed or be
> configurable to another base but it does not sound crazy.
>
> The less tool-ed option would be to extract all "reflectionfull" code
> in methods and use graalvm substitutions to drop them and use plain
> java instead (with a good naming convention, it can be generated as well).
> Keeps the duplication but at least the main code stays clean and
> optimizations stays together.
That's pretty much what we're doing right now. Many of these feel like
hacks simply to mitigate how GraalVM/AOT does code initialization (all
code loaded initialized at startup)
Filip
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github
> <https://github.com/rmannibucau> | LinkedIn
> <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>
>
> Le jeu. 16 juil. 2020 à 14:31, Rémy Maucherat <remm@apache.org
> <ma...@apache.org>> a écrit :
>
> On Mon, Jul 13, 2020 at 11:59 PM Filip Hanik <fhanik@vmware.com
> <ma...@vmware.com>> wrote:
>
> for discussion, all feedback and questions welcome:
>
>
> I've created a concept of having Apache Tomcat, embedded, run
> without reflection in a native image.
> This concept creates a jar, tomcat-embedded-programmatic.jar,
> that can be fine tuned to only include what is needed in a
> default configuration when an embedded tomcat instance is used
> and configured programatically.
>
> Steps to run Apache Tomcat using Java 8 without reflection
>
> 1. Make sure you have native-image (from the graal
> installation) on your path
> 2. git clone -b
> feature/embed-minimal-programmatic-jar-file-master
> git@github.com:fhanik/tomcat.git
> 3. cd tomcat/res/graal/
> 4. ./build-tomcat-native-image.sh && ./graal-measure.sh
>
> Should yield an output similar to (Graal 20.1):
> SUCCESS: the servlet is working
> RSS memory: 20.7M
> Image size: 20.5M
>
>
> or using an older graal, 19.2
> SUCCESS: the servlet is working
> RSS memory: 18.0M
> Image size: 16.7M
>
>
> This also leaves a file named
> ${java.io.tmpdir}/XReflectionIntrospectionUtils.java so that
> you can review the solution to IntrospectionUtils.java
>
> Goals of this concept
>
> 1. Do not break anything
> 2. Create a new and optimized for size artifact,
> tomcat-embedded-programmatic
> 3. Remove reflection by introspecting classes that are
> currently passed into IntrospectionUtils.set/getProperty
> by generating setters/getters at build time
>
> How it's done
>
> 1. I've build out a small introspection tool in the package
> org.apache.tomcat.util.xreflect
> 2. During build time, it analyses a set of known classes that
> are used with IntrospectionUtils.java, and generates
> XReflectionIntrospectionUtils.java
> 3. When it packages tomcat-embed-programmatic.jar it uses the
> generated code when calling setProperty and getProperty
>
> A PR would look like this:
> https://github.com/apache/tomcat/compare/master...fhanik:feature/embed-minimal-programmatic-jar-file-master?expand=1
>
>
> Well, this is a bit complex and hard to maintain (like, for
> example, storeconfig), so that's a downside.
>
> So starting with Tomcat and its initial server.xml, the process
> would be:
> server.xml -> equivalent Tomcat embedded code -> equivalent Tomcat
> embedded code with custom IntrospectionUtils code
> The concrete benefits may be limited though.
>
> I looked at more code generation for web.xml since the digester is
> nice for that, but the benefit becomes even more questionable. It
> is harder to manage, and the generated classes have to be loaded
> dynamically [unless even more code is generated]. If there are
> tons of fragments, there is a good intuitive reason why it becomes
> useless. so I didn't want to do it. I prefer if things remain a
> bit EE-ish, ultimately.
>
> Rémy
>
Re: Native Image - Reflectionless Concept
Posted by Rémy Maucherat <re...@apache.org>.
On Thu, Jul 16, 2020 at 3:46 PM Romain Manni-Bucau <rm...@gmail.com>
wrote:
> Hi everyone,
>
> I think the generation is the sanest option since code stay clean but it
> shouldn't be done in tomcat IMHO but in user code and with a nice wrapper
> (mvn tomcat:dump/gradle tomcatDump etc, or whatever name you like ;)).
> This build phase would dump the descriptors in plain java and would load
> them with an unique - ie single for the whole webapp - plain SPI -
> ServiceLoader - maybe?
> This kind of build tool assumes you have all the runtime state in the
> build - which is typically the case for graalvm - so you can completely
> dump StandardContext state after start phase there and just reload it from
> the precomputed model.
>
Yes, well, I examined that, and I don't think it is possible unfortunately.
More specifically, not with the StandardContext, but going straight to the
final WebXml [and then figuring out caching for the SCI], which is similar.
[Why did I do that ? Because I had added a code generator to the digester,
and the digester parses all web.xml files and fragments]
This will work just great from Tomcat perspective, the webapp will be just
fine however I think there's a major problem: webapps and frameworks get a
myriad of callbacks during the deployment process. So right there, going
straight for the deployed webapp for sure won't be compliant :(
Another factor to consider is if the user (interested in AOT) did his
homework and tries to reduce the deployment cost of his webapps. It is
possible to avoid SCIs (well, the handles types part), use metadata
complete and have a big web.xml without tons of fragments. Then, is web.xml
a super inefficient serialization for web.xml ? I'm not that convinced.
Since I notice there's interest (well, curiosity mostly), I will try to add
code generation to all the digester rules. This way it will generate code
equivalent to web.xml files and we can see what it looks like.
Rémy
> Only trick is about file paths which must either be recomputed or be
> configurable to another base but it does not sound crazy.
>
> The less tool-ed option would be to extract all "reflectionfull" code in
> methods and use graalvm substitutions to drop them and use plain java
> instead (with a good naming convention, it can be generated as well).
> Keeps the duplication but at least the main code stays clean and
> optimizations stays together.
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> | Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github
> <https://github.com/rmannibucau> | LinkedIn
> <https://www.linkedin.com/in/rmannibucau> | Book
> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>
>
> Le jeu. 16 juil. 2020 à 14:31, Rémy Maucherat <re...@apache.org> a écrit :
>
>> On Mon, Jul 13, 2020 at 11:59 PM Filip Hanik <fh...@vmware.com> wrote:
>>
>>> for discussion, all feedback and questions welcome:
>>>
>>>
>>> I've created a concept of having Apache Tomcat, embedded, run without
>>> reflection in a native image.
>>> This concept creates a jar, tomcat-embedded-programmatic.jar, that can
>>> be fine tuned to only include what is needed in a default configuration
>>> when an embedded tomcat instance is used and configured programatically.
>>>
>>> Steps to run Apache Tomcat using Java 8 without reflection
>>>
>>> 1. Make sure you have native-image (from the graal installation) on
>>> your path
>>> 2. git clone -b feature/embed-minimal-programmatic-jar-file-master
>>> git@github.com:fhanik/tomcat.git
>>> 3. cd tomcat/res/graal/
>>> 4. ./build-tomcat-native-image.sh && ./graal-measure.sh
>>>
>>> Should yield an output similar to (Graal 20.1):
>>> SUCCESS: the servlet is working
>>> RSS memory: 20.7M
>>> Image size: 20.5M
>>>
>>>
>>> or using an older graal, 19.2
>>> SUCCESS: the servlet is working
>>> RSS memory: 18.0M
>>> Image size: 16.7M
>>>
>>>
>>> This also leaves a file named ${java.io.tmpdir}/
>>> XReflectionIntrospectionUtils.java so that you can review the solution
>>> to IntrospectionUtils.java
>>>
>>> Goals of this concept
>>>
>>> 1. Do not break anything
>>> 2. Create a new and optimized for size artifact,
>>> tomcat-embedded-programmatic
>>> 3. Remove reflection by introspecting classes that are currently
>>> passed into IntrospectionUtils.set/getProperty by generating
>>> setters/getters at build time
>>>
>>> How it's done
>>>
>>> 1. I've build out a small introspection tool in the package
>>> org.apache.tomcat.util.xreflect
>>> 2. During build time, it analyses a set of known classes that are
>>> used with IntrospectionUtils.java, and generates
>>> XReflectionIntrospectionUtils.java
>>> 3. When it packages tomcat-embed-programmatic.jar it uses the
>>> generated code when calling setProperty and getProperty
>>>
>>> A PR would look like this:
>>>
>>> https://github.com/apache/tomcat/compare/master...fhanik:feature/embed-minimal-programmatic-jar-file-master?expand=1
>>>
>>>
>> Well, this is a bit complex and hard to maintain (like, for example,
>> storeconfig), so that's a downside.
>>
>> So starting with Tomcat and its initial server.xml, the process would be:
>> server.xml -> equivalent Tomcat embedded code -> equivalent Tomcat
>> embedded code with custom IntrospectionUtils code
>> The concrete benefits may be limited though.
>>
>> I looked at more code generation for web.xml since the digester is nice
>> for that, but the benefit becomes even more questionable. It is harder to
>> manage, and the generated classes have to be loaded dynamically [unless
>> even more code is generated]. If there are tons of fragments, there is a
>> good intuitive reason why it becomes useless. so I didn't want to do it. I
>> prefer if things remain a bit EE-ish, ultimately.
>>
>> Rémy
>>
>>
Re: Native Image - Reflectionless Concept
Posted by Romain Manni-Bucau <rm...@gmail.com>.
Hi everyone,
I think the generation is the sanest option since code stay clean but it
shouldn't be done in tomcat IMHO but in user code and with a nice wrapper
(mvn tomcat:dump/gradle tomcatDump etc, or whatever name you like ;)).
This build phase would dump the descriptors in plain java and would load
them with an unique - ie single for the whole webapp - plain SPI -
ServiceLoader - maybe?
This kind of build tool assumes you have all the runtime state in the build
- which is typically the case for graalvm - so you can completely dump
StandardContext state after start phase there and just reload it from the
precomputed model.
Only trick is about file paths which must either be recomputed or be
configurable to another base but it does not sound crazy.
The less tool-ed option would be to extract all "reflectionfull" code in
methods and use graalvm substitutions to drop them and use plain java
instead (with a good naming convention, it can be generated as well).
Keeps the duplication but at least the main code stays clean and
optimizations stays together.
Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> | Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>
Le jeu. 16 juil. 2020 à 14:31, Rémy Maucherat <re...@apache.org> a écrit :
> On Mon, Jul 13, 2020 at 11:59 PM Filip Hanik <fh...@vmware.com> wrote:
>
>> for discussion, all feedback and questions welcome:
>>
>>
>> I've created a concept of having Apache Tomcat, embedded, run without
>> reflection in a native image.
>> This concept creates a jar, tomcat-embedded-programmatic.jar, that can be
>> fine tuned to only include what is needed in a default configuration when
>> an embedded tomcat instance is used and configured programatically.
>>
>> Steps to run Apache Tomcat using Java 8 without reflection
>>
>> 1. Make sure you have native-image (from the graal installation) on
>> your path
>> 2. git clone -b feature/embed-minimal-programmatic-jar-file-master
>> git@github.com:fhanik/tomcat.git
>> 3. cd tomcat/res/graal/
>> 4. ./build-tomcat-native-image.sh && ./graal-measure.sh
>>
>> Should yield an output similar to (Graal 20.1):
>> SUCCESS: the servlet is working
>> RSS memory: 20.7M
>> Image size: 20.5M
>>
>>
>> or using an older graal, 19.2
>> SUCCESS: the servlet is working
>> RSS memory: 18.0M
>> Image size: 16.7M
>>
>>
>> This also leaves a file named ${java.io.tmpdir}/
>> XReflectionIntrospectionUtils.java so that you can review the solution
>> to IntrospectionUtils.java
>>
>> Goals of this concept
>>
>> 1. Do not break anything
>> 2. Create a new and optimized for size artifact,
>> tomcat-embedded-programmatic
>> 3. Remove reflection by introspecting classes that are currently
>> passed into IntrospectionUtils.set/getProperty by generating
>> setters/getters at build time
>>
>> How it's done
>>
>> 1. I've build out a small introspection tool in the package
>> org.apache.tomcat.util.xreflect
>> 2. During build time, it analyses a set of known classes that are
>> used with IntrospectionUtils.java, and generates
>> XReflectionIntrospectionUtils.java
>> 3. When it packages tomcat-embed-programmatic.jar it uses the
>> generated code when calling setProperty and getProperty
>>
>> A PR would look like this:
>>
>> https://github.com/apache/tomcat/compare/master...fhanik:feature/embed-minimal-programmatic-jar-file-master?expand=1
>>
>>
> Well, this is a bit complex and hard to maintain (like, for example,
> storeconfig), so that's a downside.
>
> So starting with Tomcat and its initial server.xml, the process would be:
> server.xml -> equivalent Tomcat embedded code -> equivalent Tomcat
> embedded code with custom IntrospectionUtils code
> The concrete benefits may be limited though.
>
> I looked at more code generation for web.xml since the digester is nice
> for that, but the benefit becomes even more questionable. It is harder to
> manage, and the generated classes have to be loaded dynamically [unless
> even more code is generated]. If there are tons of fragments, there is a
> good intuitive reason why it becomes useless. so I didn't want to do it. I
> prefer if things remain a bit EE-ish, ultimately.
>
> Rémy
>
>
Re: Native Image - Reflectionless Concept
Posted by Filip Hanik <fi...@hanik.com>.
Thanks for reviewing and engaging.
On 7/16/20 5:31 AM, Rémy Maucherat wrote:
> On Mon, Jul 13, 2020 at 11:59 PM Filip Hanik <fhanik@vmware.com
> <ma...@vmware.com>> wrote:
>
> for discussion, all feedback and questions welcome:
>
>
> I've created a concept of having Apache Tomcat, embedded, run
> without reflection in a native image.
> This concept creates a jar, tomcat-embedded-programmatic.jar, that
> can be fine tuned to only include what is needed in a default
> configuration when an embedded tomcat instance is used and
> configured programatically.
>
> Steps to run Apache Tomcat using Java 8 without reflection
>
> 1. Make sure you have native-image (from the graal installation)
> on your path
> 2. git clone -b
> feature/embed-minimal-programmatic-jar-file-master
> git@github.com:fhanik/tomcat.git
> 3. cd tomcat/res/graal/
> 4. ./build-tomcat-native-image.sh && ./graal-measure.sh
>
> Should yield an output similar to (Graal 20.1):
> SUCCESS: the servlet is working
> RSS memory: 20.7M
> Image size: 20.5M
>
>
> or using an older graal, 19.2
> SUCCESS: the servlet is working
> RSS memory: 18.0M
> Image size: 16.7M
>
>
> This also leaves a file named
> ${java.io.tmpdir}/XReflectionIntrospectionUtils.java so that you
> can review the solution to IntrospectionUtils.java
>
> Goals of this concept
>
> 1. Do not break anything
> 2. Create a new and optimized for size artifact,
> tomcat-embedded-programmatic
> 3. Remove reflection by introspecting classes that are currently
> passed into IntrospectionUtils.set/getProperty by generating
> setters/getters at build time
>
> How it's done
>
> 1. I've build out a small introspection tool in the package
> org.apache.tomcat.util.xreflect
> 2. During build time, it analyses a set of known classes that are
> used with IntrospectionUtils.java, and generates
> XReflectionIntrospectionUtils.java
> 3. When it packages tomcat-embed-programmatic.jar it uses the
> generated code when calling setProperty and getProperty
>
> A PR would look like this:
> https://github.com/apache/tomcat/compare/master...fhanik:feature/embed-minimal-programmatic-jar-file-master?expand=1
>
>
> Well, this is a bit complex and hard to maintain (like, for example,
> storeconfig), so that's a downside.
It is keeping track of one more artifact, tomcat-embed-programmatic.
Right now, it is manually maintaining the list of classes that it
generates setProperty/getProperty for. I derived this list by running
the test suite. There is of course an option to fully automate this.
>
> So starting with Tomcat and its initial server.xml, the process would be:
> server.xml -> equivalent Tomcat embedded code -> equivalent Tomcat
> embedded code with custom IntrospectionUtils code
> The concrete benefits may be limited though.
The goal of this artifact is to actually strip all non programmatic code
and bring down the code base to minimum bare essentials.
That means, it's not for applications that want to deploy a WAR file, or
do XML based configuration. My primary use case, is a Spring Boot
application where the framework servlets and filters are configured
programmatically, easy for the AOT compiler to work with, and everything
else is then handled by the framework.
There wouldn't be support by this artifact to handle EE style XML
configurations, including fragments. That is already done by
tomcat-embed-core, which is almost 50% larger generated native image
size in order to support all use cases and features.
>
> I looked at more code generation for web.xml since the digester is
> nice for that, but the benefit becomes even more questionable. It is
> harder to manage, and the generated classes have to be loaded
> dynamically [unless even more code is generated]. If there are tons of
> fragments, there is a good intuitive reason why it becomes useless. so
> I didn't want to do it. I prefer if things remain a bit EE-ish,
> ultimately.
I believe that tomcat-embed-core.jar with its META-INF GraalVM metadata
files does support the EE feature set quite well, even though, those are
manually maintained as well.
My primary goal was to create a leaner alternative, targeting images
that specifically are looking to optimize image size and memory usage.
Thanks for chiming in,
Filip
Re: Native Image - Reflectionless Concept
Posted by Rémy Maucherat <re...@apache.org>.
On Mon, Jul 13, 2020 at 11:59 PM Filip Hanik <fh...@vmware.com> wrote:
> for discussion, all feedback and questions welcome:
>
>
> I've created a concept of having Apache Tomcat, embedded, run without
> reflection in a native image.
> This concept creates a jar, tomcat-embedded-programmatic.jar, that can be
> fine tuned to only include what is needed in a default configuration when
> an embedded tomcat instance is used and configured programatically.
>
> Steps to run Apache Tomcat using Java 8 without reflection
>
> 1. Make sure you have native-image (from the graal installation) on
> your path
> 2. git clone -b feature/embed-minimal-programmatic-jar-file-master
> git@github.com:fhanik/tomcat.git
> 3. cd tomcat/res/graal/
> 4. ./build-tomcat-native-image.sh && ./graal-measure.sh
>
> Should yield an output similar to (Graal 20.1):
> SUCCESS: the servlet is working
> RSS memory: 20.7M
> Image size: 20.5M
>
>
> or using an older graal, 19.2
> SUCCESS: the servlet is working
> RSS memory: 18.0M
> Image size: 16.7M
>
>
> This also leaves a file named ${java.io.tmpdir}/
> XReflectionIntrospectionUtils.java so that you can review the solution to
> IntrospectionUtils.java
>
> Goals of this concept
>
> 1. Do not break anything
> 2. Create a new and optimized for size artifact,
> tomcat-embedded-programmatic
> 3. Remove reflection by introspecting classes that are currently
> passed into IntrospectionUtils.set/getProperty by generating
> setters/getters at build time
>
> How it's done
>
> 1. I've build out a small introspection tool in the package
> org.apache.tomcat.util.xreflect
> 2. During build time, it analyses a set of known classes that are used
> with IntrospectionUtils.java, and generates
> XReflectionIntrospectionUtils.java
> 3. When it packages tomcat-embed-programmatic.jar it uses the
> generated code when calling setProperty and getProperty
>
> A PR would look like this:
>
> https://github.com/apache/tomcat/compare/master...fhanik:feature/embed-minimal-programmatic-jar-file-master?expand=1
>
>
Well, this is a bit complex and hard to maintain (like, for example,
storeconfig), so that's a downside.
So starting with Tomcat and its initial server.xml, the process would be:
server.xml -> equivalent Tomcat embedded code -> equivalent Tomcat embedded
code with custom IntrospectionUtils code
The concrete benefits may be limited though.
I looked at more code generation for web.xml since the digester is nice for
that, but the benefit becomes even more questionable. It is harder to
manage, and the generated classes have to be loaded dynamically [unless
even more code is generated]. If there are tons of fragments, there is a
good intuitive reason why it becomes useless. so I didn't want to do it. I
prefer if things remain a bit EE-ish, ultimately.
Rémy
Re: Native Image - Reflectionless Concept
Posted by Filip Hanik <fi...@hanik.com>.
We've had some discussions around this, mostly high level about where
this could go.
At this point, anyone strongly opposed to ship a light weight jar?
Do we feel the value is or is not there compared to the maintenance of it?
Filip
On 7/13/20 2:59 PM, Filip Hanik wrote:
> for discussion, all feedback and questions welcome:
>
>
> I've created a concept of having Apache Tomcat, embedded, run without
> reflection in a native image.
> This concept creates a jar, tomcat-embedded-programmatic.jar, that can
> be fine tuned to only include what is needed in a default
> configuration when an embedded tomcat instance is used and configured
> programatically.
>
> Steps to run Apache Tomcat using Java 8 without reflection
>
> 1. Make sure you have native-image (from the graal installation) on
> your path
> 2. git clone -b feature/embed-minimal-programmatic-jar-file-master
> git@github.com:fhanik/tomcat.git
> 3. cd tomcat/res/graal/
> 4. ./build-tomcat-native-image.sh && ./graal-measure.sh
>
> Should yield an output similar to (Graal 20.1):
> SUCCESS: the servlet is working
> RSS memory: 20.7M
> Image size: 20.5M
>
>
> or using an older graal, 19.2
> SUCCESS: the servlet is working
> RSS memory: 18.0M
> Image size: 16.7M
>
>
> This also leaves a file named
> ${java.io.tmpdir}/XReflectionIntrospectionUtils.java so that you can
> review the solution to IntrospectionUtils.java
>
> Goals of this concept
>
> 1. Do not break anything
> 2. Create a new and optimized for size artifact,
> tomcat-embedded-programmatic
> 3. Remove reflection by introspecting classes that are currently
> passed into IntrospectionUtils.set/getProperty by generating
> setters/getters at build time
>
> How it's done
>
> 1. I've build out a small introspection tool in the package
> org.apache.tomcat.util.xreflect
> 2. During build time, it analyses a set of known classes that are
> used with IntrospectionUtils.java, and generates
> XReflectionIntrospectionUtils.java
> 3. When it packages tomcat-embed-programmatic.jar it uses the
> generated code when calling setProperty and getProperty
>
> A PR would look like this:
> https://github.com/apache/tomcat/compare/master...fhanik:feature/embed-minimal-programmatic-jar-file-master?expand=1
>