You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@maven.apache.org by Ralph Goers <ra...@dslextreme.com> on 2021/06/22 04:44:33 UTC
JPMS, Maven and test jars
I have recently had quite an adventure modifying several of Log4j’s Maven modules to
implement JPMS on our master branch. It was quite an adventure due to a few issues:
1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826.
This bug has been fixed in Java 17 but Log4j uses Java 11 to build.
2. Log4j-plugins builds an annotation processor, packages it with the annotations
and classes necessary to build and run plugins, creates and test jar and runs unit tests.
3. It is not possible to compile an annotation processor with a module-info.java present.
The compile will fail because it can’t find the annotation processor “service” when
compiling module-info.java.
4. It is very difficult to compile an annotation processor and then use it in the same Maven
module. JPMS expects the annotation processor to either be on the classpath or specified
with the processorpath option. When a module-info.java is present, Maven automatically
moves everything to the module pathMaven only supports using coordinates to specify the
processor path which don’t exist when the processor is in the same module. The only way
to solve this is to compile all the classes with proc=only without a module-info.java present.
5. Although number 4 might seem bad, it really doesn’t matter because javac will fail if a
module-info.java is present because module-info.java will have a reference to the service
class being generated by the annotation processor and for some reason module-info.java
is resolved before the annotation processor runs.
6. If the main set of classes are compiled with a module-info.java every other compile
in the Maven module must also be modularized. Likewise, if the main module does
not contain a module-info.java no other compile can either.
7. JPMS requires that every module have its own package space with no overlap
with any other JPMS module.
So while generating the log4j-plugins module is quite painful, generating log4j-core isn’t
much better. That is actually the primary focus for this list.
Log4j-core consists of the main classes packaged in the jar, a test jar that is used by
downstream Maven modules, and the unit tests. Prior to JPMS one would just create
the main jar and then package all the test classes and unit tests in a test jar. This can
no longer be done with JPMS.
When a project publishes a test jar along with the main jar, just like any other JPMS
module. the test jar cannot use the package space of the main classes. Log4j core
uses org.apache.logging.log4j.core so I placed all the test utility classes under
org.apache.logging.log4j.core.test. However, the unit tests all need to be packaged
in the main package space since its module-info.java “extends” the main module and
several unit tests are required to be in the same main package so that they can access
package public methods specifically provided for testing.
In order to get this to work I had to perform the following steps:
• Compile all the main classes allowing the
• Compile the main module-info.java.
• Compile the test classes used by other modules with module-info.java and
using the plugin preprocessor.
• Package these test classes in a test jar.
• Delete the module-info and generated source for the test classes.
• Move the main module-info to a temp location.
• Compile the unit test classes without module-info.java.
• Move the main module-info back to the classes directory.
• Compile module-info.java for unit tests.
• Run the unit tests.
• Create the main jar if the unit tests pass.
Were it not for JDK-8265826 I believe this could have been simplified somewhat to:
• Compile all the main classes except module-info.java with the Plugin preprocessor.
• Compile the main module-info.java.
• Compile the test classes used by other modules with its module-info.java and
using the plugin preprocessor.
• Package these test classes in a test jar.
• Delete the module-info and generated source for the test classes.
• Compile the unit test classes with its module-info.java.
• Compile module-info.java for unit tests.
• Run the unit tests.
• Create the main jar if the unit tests pass.
So the gist of this entire email is pretty simple. Is there a way Maven could be modified
to better support creating test jars with JPMS?
Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Andres Almiray <aa...@gmail.com>.
Ralph,
Apologies. I did not link to Jipsy as a suggestion to be used as a dependency but rather as an example of how the ModiTect plugin may be applied to a project that produces and tests an APT.
I believe Jipsy has a 2nd Maven module that produces a “test” jar without it being an actual test jar, that is, with the -tests classifier. This is the approach mentioned by Robert.
ModiTect helps you keep a codebase compatible with Java 8 while also supporting JPMS with full module descriptors added to the final jar as an MRJAR.
Hope this helps.
Cheers
Andres
Sent from my primitive tricorder
> On 22 Jun 2021, at 22:42, Ralph Goers <ra...@dslextreme.com> wrote:
>
> Andres,
>
> I looked at moditect when I first started doing the work. I’m afraid I am not understanding how
> it could help solve the problem with the test jar and unit tests. Could you please explain that?
>
> I am also not sure I understand how Jipsy will help. Log4j generates the file in META-INF/services
> when it generates the service class. It does not add the provides statement to module-info.java.
> Perhaps if it also did that it would bypass the compiler error saying the service can’t be found.
>
> That is certainly worth looking into.
>
> Ralph
>
>
>
>> On Jun 22, 2021, at 12:41 AM, Andres Almiray <aa...@gmail.com> wrote:
>>
>> Have you tried using the ModiTect Maven plugin?
>>
>> <goog_1688275269>
>> https://github.com/moditect/moditect
>>
>> There are many ways to use it:
>> - with explicit module-info.java
>> - with an embedded DSL in the pom.xml
>> - with an embedded module descriptor in the pom.xml
>>
>> Ii use the last option on Jipsy [https://github.com/kordamp/jipsy/], which
>> happens to define an APT as well.
>>
>> https://github.com/kordamp/jipsy/blob/master/jipsy-processor/pom.xml
>>
>> Cheers,
>> Andres
>>
>> -------------------------------------------
>> Java Champion; Groovy Enthusiast
>> http://andresalmiray.com
>> http://www.linkedin.com/in/aalmiray
>> --
>> What goes up, must come down. Ask any system administrator.
>> There are 10 types of people in the world: Those who understand binary, and
>> those who don't.
>> To understand recursion, we must first understand recursion.
>>
>>
>>> On Tue, Jun 22, 2021 at 7:02 AM Ralph Goers <ra...@dslextreme.com>
>>> wrote:
>>>
>>> Sorry for posting again. I really need to proof-read better. Please ignore
>>> the prior email.
>>>
>>> I have recently had quite an adventure modifying several of Log4j’s Maven
>>> modules to
>>> implement JPMS on our master branch. It was an adventure due to a few
>>> issues:
>>>
>>> 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 <
>>> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826>.
>>> This bug has been fixed in Java 17 but Log4j uses Java 11 to build.
>>> 2. Log4j-plugins builds an annotation processor, packages it with the
>>> annotations
>>> and classes necessary to build and run plugins, creates and test jar
>>> and runs unit tests.
>>> 3. It is not possible to compile an annotation processor with a
>>> module-info.java present.
>>> The compile will fail because it can’t find the annotation processor
>>> “service” when
>>> compiling module-info.java.
>>> 4. It is very difficult to compile an annotation processor and then use it
>>> in the same Maven
>>> module. JPMS expects the annotation processor to either be on the
>>> classpath or specified
>>> with the processorpath option. When a module-info.java is present,
>>> Maven automatically
>>> moves everything to the module path. Maven only supports using
>>> coordinates to specify the
>>> processor path, which don’t exist when the processor is in the same
>>> module. The only way
>>> to solve this is to compile all the classes with proc=only without a
>>> module-info.java present.
>>> 5. Although number 4 might seem bad, it really doesn’t matter because
>>> javac will fail if a
>>> module-info.java is present because module-info.java will have a
>>> reference to the service
>>> class being generated by the annotation processor and for some reason
>>> module-info.java
>>> is resolved before the annotation processor runs.
>>> 6. If the main set of classes are compiled with a module-info.java every
>>> other compile
>>> in the Maven module must also be modularized. Likewise, if the main
>>> module does
>>> not contain a module-info.java no other compile can either.
>>> 7. JPMS requires that every module have its own package space with no
>>> overlap
>>> with any other JPMS module.
>>>
>>> So while generating the log4j-plugins module is quite painful, generating
>>> log4j-core isn’t
>>> much better. That is actually the primary focus for this list.
>>>
>>> Log4j-core consists of the main classes packaged in the jar, a test jar
>>> that is used by
>>> downstream Maven modules, and the unit tests. Prior to JPMS one would just
>>> create
>>> the main jar and then package all the test classes and unit tests in a
>>> test jar. This can
>>> no longer be done with JPMS.
>>>
>>> When a project publishes a test jar along with the main jar, just like any
>>> other JPMS
>>> module. the test jar cannot use the package space of the main classes.
>>> Log4j core
>>> uses org.apache.logging.log4j.core so I placed all the test utility
>>> classes under
>>> org.apache.logging.log4j.core.test. However, the unit tests all need to be
>>> packaged
>>> in the main package space since its module-info.java “extends” the main
>>> module and
>>> several unit tests are required to be in the same main package so that
>>> they can access
>>> package private methods specifically provided for testing.
>>>
>>> In order to get this to work I had to perform the following steps:
>>>
>>> • Compile all the main classes except module-info.java with the
>>> Plugin annotation processor.
>>> • Compile the main module-info.java.
>>> • Compile the test classes used by other modules with
>>> module-info.java and
>>> using the plugin preprocessor.
>>> • Package these test classes in a test jar.
>>> • Delete the module-info and generated source for the test classes.
>>> • Move the main module-info to a temp location.
>>> • Compile the unit test classes without module-info.java.
>>> • Move the main module-info back to the classes directory.
>>> • Compile module-info.java for unit tests.
>>> • Run the unit tests.
>>> • Create the main jar if the unit tests pass.
>>>
>>> Were it not for JDK-8265826 I believe this could have been simplified
>>> somewhat to:
>>>
>>> • Compile all the main classes except module-info.java with the
>>> Plugin annotation processor.
>>> • Compile the main module-info.java.
>>> • Compile the test classes used by other modules with its
>>> module-info.java and
>>> using the plugin preprocessor.
>>> • Package these test classes in a test jar.
>>> • Delete the module-info and generated source for the test classes.
>>> • Compile the unit test classes with its module-info.java.
>>> • Compile module-info.java for unit tests.
>>> • Run the unit tests.
>>> • Create the main jar if the unit tests pass.
>>>
>>> So the gist of this entire email is pretty simple. Is there a way Maven
>>> could be modified
>>> to better support creating test jars with JPMS?
>>>
>>> Ralph
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Ralph Goers <ra...@dslextreme.com>.
Andres,
I looked at moditect when I first started doing the work. I’m afraid I am not understanding how
it could help solve the problem with the test jar and unit tests. Could you please explain that?
I am also not sure I understand how Jipsy will help. Log4j generates the file in META-INF/services
when it generates the service class. It does not add the provides statement to module-info.java.
Perhaps if it also did that it would bypass the compiler error saying the service can’t be found.
That is certainly worth looking into.
Ralph
> On Jun 22, 2021, at 12:41 AM, Andres Almiray <aa...@gmail.com> wrote:
>
> Have you tried using the ModiTect Maven plugin?
>
> <goog_1688275269>
> https://github.com/moditect/moditect
>
> There are many ways to use it:
> - with explicit module-info.java
> - with an embedded DSL in the pom.xml
> - with an embedded module descriptor in the pom.xml
>
> Ii use the last option on Jipsy [https://github.com/kordamp/jipsy/], which
> happens to define an APT as well.
>
> https://github.com/kordamp/jipsy/blob/master/jipsy-processor/pom.xml
>
> Cheers,
> Andres
>
> -------------------------------------------
> Java Champion; Groovy Enthusiast
> http://andresalmiray.com
> http://www.linkedin.com/in/aalmiray
> --
> What goes up, must come down. Ask any system administrator.
> There are 10 types of people in the world: Those who understand binary, and
> those who don't.
> To understand recursion, we must first understand recursion.
>
>
> On Tue, Jun 22, 2021 at 7:02 AM Ralph Goers <ra...@dslextreme.com>
> wrote:
>
>> Sorry for posting again. I really need to proof-read better. Please ignore
>> the prior email.
>>
>> I have recently had quite an adventure modifying several of Log4j’s Maven
>> modules to
>> implement JPMS on our master branch. It was an adventure due to a few
>> issues:
>>
>> 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 <
>> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826>.
>> This bug has been fixed in Java 17 but Log4j uses Java 11 to build.
>> 2. Log4j-plugins builds an annotation processor, packages it with the
>> annotations
>> and classes necessary to build and run plugins, creates and test jar
>> and runs unit tests.
>> 3. It is not possible to compile an annotation processor with a
>> module-info.java present.
>> The compile will fail because it can’t find the annotation processor
>> “service” when
>> compiling module-info.java.
>> 4. It is very difficult to compile an annotation processor and then use it
>> in the same Maven
>> module. JPMS expects the annotation processor to either be on the
>> classpath or specified
>> with the processorpath option. When a module-info.java is present,
>> Maven automatically
>> moves everything to the module path. Maven only supports using
>> coordinates to specify the
>> processor path, which don’t exist when the processor is in the same
>> module. The only way
>> to solve this is to compile all the classes with proc=only without a
>> module-info.java present.
>> 5. Although number 4 might seem bad, it really doesn’t matter because
>> javac will fail if a
>> module-info.java is present because module-info.java will have a
>> reference to the service
>> class being generated by the annotation processor and for some reason
>> module-info.java
>> is resolved before the annotation processor runs.
>> 6. If the main set of classes are compiled with a module-info.java every
>> other compile
>> in the Maven module must also be modularized. Likewise, if the main
>> module does
>> not contain a module-info.java no other compile can either.
>> 7. JPMS requires that every module have its own package space with no
>> overlap
>> with any other JPMS module.
>>
>> So while generating the log4j-plugins module is quite painful, generating
>> log4j-core isn’t
>> much better. That is actually the primary focus for this list.
>>
>> Log4j-core consists of the main classes packaged in the jar, a test jar
>> that is used by
>> downstream Maven modules, and the unit tests. Prior to JPMS one would just
>> create
>> the main jar and then package all the test classes and unit tests in a
>> test jar. This can
>> no longer be done with JPMS.
>>
>> When a project publishes a test jar along with the main jar, just like any
>> other JPMS
>> module. the test jar cannot use the package space of the main classes.
>> Log4j core
>> uses org.apache.logging.log4j.core so I placed all the test utility
>> classes under
>> org.apache.logging.log4j.core.test. However, the unit tests all need to be
>> packaged
>> in the main package space since its module-info.java “extends” the main
>> module and
>> several unit tests are required to be in the same main package so that
>> they can access
>> package private methods specifically provided for testing.
>>
>> In order to get this to work I had to perform the following steps:
>>
>> • Compile all the main classes except module-info.java with the
>> Plugin annotation processor.
>> • Compile the main module-info.java.
>> • Compile the test classes used by other modules with
>> module-info.java and
>> using the plugin preprocessor.
>> • Package these test classes in a test jar.
>> • Delete the module-info and generated source for the test classes.
>> • Move the main module-info to a temp location.
>> • Compile the unit test classes without module-info.java.
>> • Move the main module-info back to the classes directory.
>> • Compile module-info.java for unit tests.
>> • Run the unit tests.
>> • Create the main jar if the unit tests pass.
>>
>> Were it not for JDK-8265826 I believe this could have been simplified
>> somewhat to:
>>
>> • Compile all the main classes except module-info.java with the
>> Plugin annotation processor.
>> • Compile the main module-info.java.
>> • Compile the test classes used by other modules with its
>> module-info.java and
>> using the plugin preprocessor.
>> • Package these test classes in a test jar.
>> • Delete the module-info and generated source for the test classes.
>> • Compile the unit test classes with its module-info.java.
>> • Compile module-info.java for unit tests.
>> • Run the unit tests.
>> • Create the main jar if the unit tests pass.
>>
>> So the gist of this entire email is pretty simple. Is there a way Maven
>> could be modified
>> to better support creating test jars with JPMS?
>>
>> Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Andres Almiray <aa...@gmail.com>.
Have you tried using the ModiTect Maven plugin?
<goog_1688275269>
https://github.com/moditect/moditect
There are many ways to use it:
- with explicit module-info.java
- with an embedded DSL in the pom.xml
- with an embedded module descriptor in the pom.xml
Ii use the last option on Jipsy [https://github.com/kordamp/jipsy/], which
happens to define an APT as well.
https://github.com/kordamp/jipsy/blob/master/jipsy-processor/pom.xml
Cheers,
Andres
-------------------------------------------
Java Champion; Groovy Enthusiast
http://andresalmiray.com
http://www.linkedin.com/in/aalmiray
--
What goes up, must come down. Ask any system administrator.
There are 10 types of people in the world: Those who understand binary, and
those who don't.
To understand recursion, we must first understand recursion.
On Tue, Jun 22, 2021 at 7:02 AM Ralph Goers <ra...@dslextreme.com>
wrote:
> Sorry for posting again. I really need to proof-read better. Please ignore
> the prior email.
>
> I have recently had quite an adventure modifying several of Log4j’s Maven
> modules to
> implement JPMS on our master branch. It was an adventure due to a few
> issues:
>
> 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 <
> https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826>.
> This bug has been fixed in Java 17 but Log4j uses Java 11 to build.
> 2. Log4j-plugins builds an annotation processor, packages it with the
> annotations
> and classes necessary to build and run plugins, creates and test jar
> and runs unit tests.
> 3. It is not possible to compile an annotation processor with a
> module-info.java present.
> The compile will fail because it can’t find the annotation processor
> “service” when
> compiling module-info.java.
> 4. It is very difficult to compile an annotation processor and then use it
> in the same Maven
> module. JPMS expects the annotation processor to either be on the
> classpath or specified
> with the processorpath option. When a module-info.java is present,
> Maven automatically
> moves everything to the module path. Maven only supports using
> coordinates to specify the
> processor path, which don’t exist when the processor is in the same
> module. The only way
> to solve this is to compile all the classes with proc=only without a
> module-info.java present.
> 5. Although number 4 might seem bad, it really doesn’t matter because
> javac will fail if a
> module-info.java is present because module-info.java will have a
> reference to the service
> class being generated by the annotation processor and for some reason
> module-info.java
> is resolved before the annotation processor runs.
> 6. If the main set of classes are compiled with a module-info.java every
> other compile
> in the Maven module must also be modularized. Likewise, if the main
> module does
> not contain a module-info.java no other compile can either.
> 7. JPMS requires that every module have its own package space with no
> overlap
> with any other JPMS module.
>
> So while generating the log4j-plugins module is quite painful, generating
> log4j-core isn’t
> much better. That is actually the primary focus for this list.
>
> Log4j-core consists of the main classes packaged in the jar, a test jar
> that is used by
> downstream Maven modules, and the unit tests. Prior to JPMS one would just
> create
> the main jar and then package all the test classes and unit tests in a
> test jar. This can
> no longer be done with JPMS.
>
> When a project publishes a test jar along with the main jar, just like any
> other JPMS
> module. the test jar cannot use the package space of the main classes.
> Log4j core
> uses org.apache.logging.log4j.core so I placed all the test utility
> classes under
> org.apache.logging.log4j.core.test. However, the unit tests all need to be
> packaged
> in the main package space since its module-info.java “extends” the main
> module and
> several unit tests are required to be in the same main package so that
> they can access
> package private methods specifically provided for testing.
>
> In order to get this to work I had to perform the following steps:
>
> • Compile all the main classes except module-info.java with the
> Plugin annotation processor.
> • Compile the main module-info.java.
> • Compile the test classes used by other modules with
> module-info.java and
> using the plugin preprocessor.
> • Package these test classes in a test jar.
> • Delete the module-info and generated source for the test classes.
> • Move the main module-info to a temp location.
> • Compile the unit test classes without module-info.java.
> • Move the main module-info back to the classes directory.
> • Compile module-info.java for unit tests.
> • Run the unit tests.
> • Create the main jar if the unit tests pass.
>
> Were it not for JDK-8265826 I believe this could have been simplified
> somewhat to:
>
> • Compile all the main classes except module-info.java with the
> Plugin annotation processor.
> • Compile the main module-info.java.
> • Compile the test classes used by other modules with its
> module-info.java and
> using the plugin preprocessor.
> • Package these test classes in a test jar.
> • Delete the module-info and generated source for the test classes.
> • Compile the unit test classes with its module-info.java.
> • Compile module-info.java for unit tests.
> • Run the unit tests.
> • Create the main jar if the unit tests pass.
>
> So the gist of this entire email is pretty simple. Is there a way Maven
> could be modified
> to better support creating test jars with JPMS?
>
> Ralph
Re: JPMS, Maven and test jars
Posted by Ralph Goers <ra...@dslextreme.com>.
> On Jun 22, 2021, at 2:29 PM, Andres Almiray <aa...@gmail.com> wrote:
>
> Ralph,
>
> Once you move test code to their own Maven module they become a top level module, that is, they have a life of their own as any other top level module.
>
> In contrast -tests jars are not top level, but a “variant” (that’s why they have a classifier) of their owning top level module.
>
> Test jars must be published to a repository in order to be consumed, just like any other jar, variant or not. The only difference (at least that I can see) is that test jars may be downloaded using the coordinates of the owning module (plus test classifier) and their release cycle is tightly coupled with their owning module.
>
> But if you take away the owning relationship you’re left with, well, just another regular Maven module.
>
> Personally I prefer the use of explicit dependencies and modules, barely make use of test jar support, but I do recognize others feel differently about them and prefer the workflow enables by such jars and conventions
Well, I see that Maven now recommends not generating jars with the tests classifier - https://maven.apache.org/plugins/maven-jar-plugin/examples/create-test-jar.html.
I am not sure when that was changed to be the recommended approach but given that it is I can at least give it a shot. I’m still concerned that the Maven compiler plugin
won’t understand what is happening but I can only determine that by testing it.
Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Andres Almiray <aa...@gmail.com>.
Ralph,
Once you move test code to their own Maven module they become a top level module, that is, they have a life of their own as any other top level module.
In contrast -tests jars are not top level, but a “variant” (that’s why they have a classifier) of their owning top level module.
Test jars must be published to a repository in order to be consumed, just like any other jar, variant or not. The only difference (at least that I can see) is that test jars may be downloaded using the coordinates of the owning module (plus test classifier) and their release cycle is tightly coupled with their owning module.
But if you take away the owning relationship you’re left with, well, just another regular Maven module.
Personally I prefer the use of explicit dependencies and modules, barely make use of test jar support, but I do recognize others feel differently about them and prefer the workflow enables by such jars and conventions
Cheers
Andres
Sent from my primitive tricorder
> On 22 Jun 2021, at 23:20, Ralph Goers <ra...@dslextreme.com> wrote:
>
>
>
>> On Jun 22, 2021, at 11:16 AM, Robert Scholte <rf...@apache.org> wrote:
>>
>> If I understand correctly you want both log4j-core main classes and test classes be distributed as modularized jars.
>> To me with JPMS you should move the modularized test-classes to a separate Maven module as a first citizen artifact.
>> That should make it a lot easier.
>
> Robert,
>
> I have a question on the above. I am generating log4j-core-3.0.0-SNAPSHOT.jar and log4j-core-3.0.0-SNAPSHOT-tests.jar.
> If the source is left in log4j-core and I create a log4j-core-tests module what would I specify in the pom to get it to create the
> test jar as log4j-core-3.0.0-SNAPSHOT-tests.jar? I would not want it to be log4j-core-tests-3.0.0-SNAPSHOT.jar as that
> would not be understood to be a test jar.
>
> Ralph
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Robert Scholte <rf...@apache.org>.
>> I would not want it to be log4j-core-tests-3.0.0-SNAPSHOT.jar as that
would not be understood to be a test jar.
If this artifact is intended for log4j only, consider skipping install/deploy. But then I wonder why there's a need for a module descriptor.
If it is intended to be used by users of log4j for testing, this name does make sense to me, similar to spring-test.
Robert
On 22-6-2021 23:20:11, Ralph Goers <ra...@dslextreme.com> wrote:
> On Jun 22, 2021, at 11:16 AM, Robert Scholte wrote:
>
> If I understand correctly you want both log4j-core main classes and test classes be distributed as modularized jars.
> To me with JPMS you should move the modularized test-classes to a separate Maven module as a first citizen artifact.
> That should make it a lot easier.
Robert,
I have a question on the above. I am generating log4j-core-3.0.0-SNAPSHOT.jar and log4j-core-3.0.0-SNAPSHOT-tests.jar.
If the source is left in log4j-core and I create a log4j-core-tests module what would I specify in the pom to get it to create the
test jar as log4j-core-3.0.0-SNAPSHOT-tests.jar? I would not want it to be log4j-core-tests-3.0.0-SNAPSHOT.jar as that
would not be understood to be a test jar.
Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Ralph Goers <ra...@dslextreme.com>.
> On Jun 22, 2021, at 11:16 AM, Robert Scholte <rf...@apache.org> wrote:
>
> If I understand correctly you want both log4j-core main classes and test classes be distributed as modularized jars.
> To me with JPMS you should move the modularized test-classes to a separate Maven module as a first citizen artifact.
> That should make it a lot easier.
Robert,
I have a question on the above. I am generating log4j-core-3.0.0-SNAPSHOT.jar and log4j-core-3.0.0-SNAPSHOT-tests.jar.
If the source is left in log4j-core and I create a log4j-core-tests module what would I specify in the pom to get it to create the
test jar as log4j-core-3.0.0-SNAPSHOT-tests.jar? I would not want it to be log4j-core-tests-3.0.0-SNAPSHOT.jar as that
would not be understood to be a test jar.
Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Ralph Goers <ra...@dslextreme.com>.
Gary, the test jar doesn’t perform tests. It provides test utility classes like LoggerContextRule.
> On Jun 22, 2021, at 12:31 PM, Gary Gregory <ga...@gmail.com> wrote:
>
> It feels to me like JPMS just plainly breaks the informal industry standard
> of the Maven project layout we have all been using for a million years.
>
> In addition, how is one supposed to test package private code if a test jar
> tests a main jar where all packages are now different? This seems like a
> problem whether the test code is in the same Maven module or not. Or am
> I missing something? Am I the only one feeling like JPMS what developed in
> a vaccum?
>
> Gary
>
Gary, testing package private stuff is fine and easy if you don’t generate a test jar. In that case unit tests
behave as they always have except that you have to create a module-info.java that gives you access
to the internals of the module. The problem occurs simply because the test jar being created cannot
contain classes in the same package space as the main module, so you can no longer just publish
your test utility classes combined with the unit tests as the test jar.
Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Gary Gregory <ga...@gmail.com>.
That's just broken, I can have any number of package private classes (or
methods) that perform lower level or non-customer appropriate tasks but
that are large enough or complex enough to merrit unit testing on their own.
Only testing public APIs is saying that blackboxing is the only way to test
and makes whitebox testing impossible. Less testing is not more brittle.
Bummer.
Gary
On Tue, Jun 22, 2021, 16:20 Elliotte Rusty Harold <el...@ibiblio.org>
wrote:
> On Tue, Jun 22, 2021 at 7:32 PM Gary Gregory <ga...@gmail.com>
> wrote:
>
> > In addition, how is one supposed to test package private code if a test
> jar
> > tests a main jar where all packages are now different?
>
> Some projects adhere to strictly testing through the public API only.
> It definitely makes tests more robust and less brittle.
>
>
>
> --
> Elliotte Rusty Harold
> elharo@ibiblio.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: JPMS, Maven and test jars
Posted by Ralph Goers <ra...@dslextreme.com>.
> On Jun 22, 2021, at 1:19 PM, Elliotte Rusty Harold <el...@ibiblio.org> wrote:
>
> On Tue, Jun 22, 2021 at 7:32 PM Gary Gregory <ga...@gmail.com> wrote:
>
>> In addition, how is one supposed to test package private code if a test jar
>> tests a main jar where all packages are now different?
>
> Some projects adhere to strictly testing through the public API only.
> It definitely makes tests more robust and less brittle.
I completely disagree with this. There are tons of things I would never be able to test in
Log4j without being able to access the internals. You simply can’t get high enough code
coverage if you are limited to the public API.
Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Elliotte Rusty Harold <el...@ibiblio.org>.
On Tue, Jun 22, 2021 at 7:32 PM Gary Gregory <ga...@gmail.com> wrote:
> In addition, how is one supposed to test package private code if a test jar
> tests a main jar where all packages are now different?
Some projects adhere to strictly testing through the public API only.
It definitely makes tests more robust and less brittle.
--
Elliotte Rusty Harold
elharo@ibiblio.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Gary Gregory <ga...@gmail.com>.
Some of the code is shared on the test side just like you shared code on
the main side, there is nothing unusual abour that. For example test
infrastructure, utilities, JUnit rules, and do on.
Gary
On Wed, Jun 23, 2021, 02:54 Jochen Wiedmann <jo...@gmail.com>
wrote:
> On Tue, Jun 22, 2021 at 9:32 PM Gary Gregory <ga...@gmail.com>
> wrote:
> >
> > It feels to me like JPMS just plainly breaks the informal industry
> standard
> > of the Maven project layout we have all been using for a million years.
>
> I don't think so. Reusing test classes downstream sounds to me highly
> unusual.
>
> Jochen
>
>
> --
>
> Look, that's why there's rules, understand? So that you think before
> you break 'em.
>
> -- (Terry Pratchett, Thief of Time)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
Re: JPMS, Maven and test jars
Posted by Jochen Wiedmann <jo...@gmail.com>.
On Tue, Jun 22, 2021 at 9:32 PM Gary Gregory <ga...@gmail.com> wrote:
>
> It feels to me like JPMS just plainly breaks the informal industry standard
> of the Maven project layout we have all been using for a million years.
I don't think so. Reusing test classes downstream sounds to me highly unusual.
Jochen
--
Look, that's why there's rules, understand? So that you think before
you break 'em.
-- (Terry Pratchett, Thief of Time)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Gary Gregory <ga...@gmail.com>.
It feels to me like JPMS just plainly breaks the informal industry standard
of the Maven project layout we have all been using for a million years.
In addition, how is one supposed to test package private code if a test jar
tests a main jar where all packages are now different? This seems like a
problem whether the test code is in the same Maven module or not. Or am
I missing something? Am I the only one feeling like JPMS what developed in
a vaccum?
Gary
On Tue, Jun 22, 2021, 14:16 Robert Scholte <rf...@apache.org> wrote:
> If I understand correctly you want both log4j-core main classes and test
> classes be distributed as modularized jars.
> To me with JPMS you should move the modularized test-classes to a separate
> Maven module as a first citizen artifact.
> That should make it a lot easier.
> It should also make the separation more clear: classes under
> target/test-classes are there to unittest its main classes.
> Whereas this new module contains reusable (main) code for testing (which
> should be tested too ;) )
> I understand that this will change the GA(-classifier) for the testing
> module, but to me for a good reason.
>
> Even with your very small reduction of steps to simplify the process, it
> doesn't fit in the default lifecycle unless you write you introduce a new
> lifecycle-binding or write a new extension.
>
> Robert
>
> On 22-6-2021 07:02:59, Ralph Goers <ra...@dslextreme.com> wrote:
> Sorry for posting again. I really need to proof-read better. Please ignore
> the prior email.
>
> I have recently had quite an adventure modifying several of Log4j’s Maven
> modules to
> implement JPMS on our master branch. It was an adventure due to a few
> issues:
>
> 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 .
> This bug has been fixed in Java 17 but Log4j uses Java 11 to build.
> 2. Log4j-plugins builds an annotation processor, packages it with the
> annotations
> and classes necessary to build and run plugins, creates and test jar and
> runs unit tests.
> 3. It is not possible to compile an annotation processor with a
> module-info.java present.
> The compile will fail because it can’t find the annotation processor
> “service” when
> compiling module-info.java.
> 4. It is very difficult to compile an annotation processor and then use it
> in the same Maven
> module. JPMS expects the annotation processor to either be on the
> classpath or specified
> with the processorpath option. When a module-info.java is present, Maven
> automatically
> moves everything to the module path. Maven only supports using coordinates
> to specify the
> processor path, which don’t exist when the processor is in the same
> module. The only way
> to solve this is to compile all the classes with proc=only without a
> module-info.java present.
> 5. Although number 4 might seem bad, it really doesn’t matter because
> javac will fail if a
> module-info.java is present because module-info.java will have a reference
> to the service
> class being generated by the annotation processor and for some reason
> module-info.java
> is resolved before the annotation processor runs.
> 6. If the main set of classes are compiled with a module-info.java every
> other compile
> in the Maven module must also be modularized. Likewise, if the main module
> does
> not contain a module-info.java no other compile can either.
> 7. JPMS requires that every module have its own package space with no
> overlap
> with any other JPMS module.
>
> So while generating the log4j-plugins module is quite painful, generating
> log4j-core isn’t
> much better. That is actually the primary focus for this list.
>
> Log4j-core consists of the main classes packaged in the jar, a test jar
> that is used by
> downstream Maven modules, and the unit tests. Prior to JPMS one would just
> create
> the main jar and then package all the test classes and unit tests in a
> test jar. This can
> no longer be done with JPMS.
>
> When a project publishes a test jar along with the main jar, just like any
> other JPMS
> module. the test jar cannot use the package space of the main classes.
> Log4j core
> uses org.apache.logging.log4j.core so I placed all the test utility
> classes under
> org.apache.logging.log4j.core.test. However, the unit tests all need to be
> packaged
> in the main package space since its module-info.java “extends” the main
> module and
> several unit tests are required to be in the same main package so that
> they can access
> package private methods specifically provided for testing.
>
> In order to get this to work I had to perform the following steps:
>
> • Compile all the main classes except module-info.java with the Plugin
> annotation processor.
> • Compile the main module-info.java.
> • Compile the test classes used by other modules with module-info.java and
> using the plugin preprocessor.
> • Package these test classes in a test jar.
> • Delete the module-info and generated source for the test classes.
> • Move the main module-info to a temp location.
> • Compile the unit test classes without module-info.java.
> • Move the main module-info back to the classes directory.
> • Compile module-info.java for unit tests.
> • Run the unit tests.
> • Create the main jar if the unit tests pass.
>
> Were it not for JDK-8265826 I believe this could have been simplified
> somewhat to:
>
> • Compile all the main classes except module-info.java with the Plugin
> annotation processor.
> • Compile the main module-info.java.
> • Compile the test classes used by other modules with its module-info.java
> and
> using the plugin preprocessor.
> • Package these test classes in a test jar.
> • Delete the module-info and generated source for the test classes.
> • Compile the unit test classes with its module-info.java.
> • Compile module-info.java for unit tests.
> • Run the unit tests.
> • Create the main jar if the unit tests pass.
>
> So the gist of this entire email is pretty simple. Is there a way Maven
> could be modified
> to better support creating test jars with JPMS?
>
> Ralph
Re: JPMS, Maven and test jars
Posted by Ralph Goers <ra...@dslextreme.com>.
> On Jun 22, 2021, at 12:01 PM, Benjamin Marwell <bm...@apache.org> wrote:
>
>> So now you would have a maven module with only the main source and no tests,
>> and a second module
>> where the “main” source is actually the test classes
>> and the unit tests test the code in a different Maven
>> module with a module info that opens the module-info of the main source jar.
>> Seems downright weird to me.
>
> Well, it is not. First of all, you can test your main code nicely
> using standard non-modularized tests.
> No problem here.
Yes, there is a problem here. First, I am pretty sure that if I didn’t have a module-info.java file in my
unit tests the build failed. Even if I could do that I wouldn’t be verifying that the module was constructed correctly.
>
> Then you can use your module containing the abstract/common test
> classes with the scope <scope>test</scope>.
> This is actually what junit etc. is – you are using their main classes
> in YOUR test scope.
> So, nothing weird here, too.
>
> The third project pulls them in altogether and is modularized.
> Call it "IT" or "integration test", and everything feels normal again.
So you would propose that any time someone wants to publish a test jar that they must have 3 Maven modules.
Obviously I wouldn’t call it IT since it doesn’t contain integration tests, it contains unit tests. If I also had integration
tests then I would need a fourth Maven module. That feels very “un-normal” to me. I don’t recall ever seeing a
project that has the unit tests in a module separate from the main source.
It would make more sense to me to follow Robert’s suggestion of having the test jar and unit tests in a second
module, but it feels strange that the unit tests in that module test the source from a different module. I haven’t tried,
but I am also not sure if Maven will behave the same way when operating on a packaged jar as it does when the
module-info.java for the unit tests extends the module-info for the main classes. I’d have to test that. I have a concern
that it will think it is extending the test classes JPMS module.
I’ve also not tested a Maven module that has only unit tests. I suspect in that case it will not be possible to test the
main source as a JPMS module since Maven seems to require a module-info.java in the main source before it
will allow the unit tests to include one.
>
> This will also help your IDE recognize the code.
> And as this is a common pattern, it will also help other coders to
> more easily recognize the structure of your project.
Yes, this will obviously make the IDE happier. I’m not sure about the developers though.
Ralph
>
> Am Di., 22. Juni 2021 um 20:46 Uhr schrieb Ralph Goers
> <ra...@dslextreme.com>:
>>
>> Yes, you understand correctly. And yes, the test classes could be moved to a separate maven module, but
>> then the unit tests would also have to move as well as they are dependent on those test classes. Of course,
>> the test classes are dependent on the main source. As an example, log4j-core has a LoggerContext class,
>> the test jar has a LoggerContextRule class that uses the LoggerContext and the LoggerContextRule is used
>> by many unit tests.
>>
>> So now you would have a maven module with only the main source and no tests, and a second module
>> where the “main” source is actually the test classes and the unit tests test the code in a different Maven
>> module with a module info that opens the module-info of the main source jar.
>>
>> Seems downright weird to me.
>>
>> Yes, I realize this doesn’t fit in the normal Maven lifecycle. I brought it up because if and/or when more
>> stuff starts becoming JPMS modularized this issue is going to become more and more common.
>>
>> To be honest, I’d be OK living with the build the way it is except people are complaining that their IDE
>> has no idea what to do with the project.
>>
>> Ralph
>>
>>> On Jun 22, 2021, at 11:16 AM, Robert Scholte <rf...@apache.org> wrote:
>>>
>>> If I understand correctly you want both log4j-core main classes and test classes be distributed as modularized jars.
>>> To me with JPMS you should move the modularized test-classes to a separate Maven module as a first citizen artifact.
>>> That should make it a lot easier.
>>> It should also make the separation more clear: classes under target/test-classes are there to unittest its main classes.
>>> Whereas this new module contains reusable (main) code for testing (which should be tested too ;) )
>>> I understand that this will change the GA(-classifier) for the testing module, but to me for a good reason.
>>>
>>> Even with your very small reduction of steps to simplify the process, it doesn't fit in the default lifecycle unless you write you introduce a new lifecycle-binding or write a new extension.
>>>
>>> Robert
>>>
>>> On 22-6-2021 07:02:59, Ralph Goers <ra...@dslextreme.com> wrote:
>>> Sorry for posting again. I really need to proof-read better. Please ignore the prior email.
>>>
>>> I have recently had quite an adventure modifying several of Log4j’s Maven modules to
>>> implement JPMS on our master branch. It was an adventure due to a few issues:
>>>
>>> 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 .
>>> This bug has been fixed in Java 17 but Log4j uses Java 11 to build.
>>> 2. Log4j-plugins builds an annotation processor, packages it with the annotations
>>> and classes necessary to build and run plugins, creates and test jar and runs unit tests.
>>> 3. It is not possible to compile an annotation processor with a module-info.java present.
>>> The compile will fail because it can’t find the annotation processor “service” when
>>> compiling module-info.java.
>>> 4. It is very difficult to compile an annotation processor and then use it in the same Maven
>>> module. JPMS expects the annotation processor to either be on the classpath or specified
>>> with the processorpath option. When a module-info.java is present, Maven automatically
>>> moves everything to the module path. Maven only supports using coordinates to specify the
>>> processor path, which don’t exist when the processor is in the same module. The only way
>>> to solve this is to compile all the classes with proc=only without a module-info.java present.
>>> 5. Although number 4 might seem bad, it really doesn’t matter because javac will fail if a
>>> module-info.java is present because module-info.java will have a reference to the service
>>> class being generated by the annotation processor and for some reason module-info.java
>>> is resolved before the annotation processor runs.
>>> 6. If the main set of classes are compiled with a module-info.java every other compile
>>> in the Maven module must also be modularized. Likewise, if the main module does
>>> not contain a module-info.java no other compile can either.
>>> 7. JPMS requires that every module have its own package space with no overlap
>>> with any other JPMS module.
>>>
>>> So while generating the log4j-plugins module is quite painful, generating log4j-core isn’t
>>> much better. That is actually the primary focus for this list.
>>>
>>> Log4j-core consists of the main classes packaged in the jar, a test jar that is used by
>>> downstream Maven modules, and the unit tests. Prior to JPMS one would just create
>>> the main jar and then package all the test classes and unit tests in a test jar. This can
>>> no longer be done with JPMS.
>>>
>>> When a project publishes a test jar along with the main jar, just like any other JPMS
>>> module. the test jar cannot use the package space of the main classes. Log4j core
>>> uses org.apache.logging.log4j.core so I placed all the test utility classes under
>>> org.apache.logging.log4j.core.test. However, the unit tests all need to be packaged
>>> in the main package space since its module-info.java “extends” the main module and
>>> several unit tests are required to be in the same main package so that they can access
>>> package private methods specifically provided for testing.
>>>
>>> In order to get this to work I had to perform the following steps:
>>>
>>> • Compile all the main classes except module-info.java with the Plugin annotation processor.
>>> • Compile the main module-info.java.
>>> • Compile the test classes used by other modules with module-info.java and
>>> using the plugin preprocessor.
>>> • Package these test classes in a test jar.
>>> • Delete the module-info and generated source for the test classes.
>>> • Move the main module-info to a temp location.
>>> • Compile the unit test classes without module-info.java.
>>> • Move the main module-info back to the classes directory.
>>> • Compile module-info.java for unit tests.
>>> • Run the unit tests.
>>> • Create the main jar if the unit tests pass.
>>>
>>> Were it not for JDK-8265826 I believe this could have been simplified somewhat to:
>>>
>>> • Compile all the main classes except module-info.java with the Plugin annotation processor.
>>> • Compile the main module-info.java.
>>> • Compile the test classes used by other modules with its module-info.java and
>>> using the plugin preprocessor.
>>> • Package these test classes in a test jar.
>>> • Delete the module-info and generated source for the test classes.
>>> • Compile the unit test classes with its module-info.java.
>>> • Compile module-info.java for unit tests.
>>> • Run the unit tests.
>>> • Create the main jar if the unit tests pass.
>>>
>>> So the gist of this entire email is pretty simple. Is there a way Maven could be modified
>>> to better support creating test jars with JPMS?
>>>
>>> Ralph
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
>> For additional commands, e-mail: dev-help@maven.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Benjamin Marwell <bm...@apache.org>.
> So now you would have a maven module with only the main source and no tests,
> and a second module
> where the “main” source is actually the test classes
> and the unit tests test the code in a different Maven
> module with a module info that opens the module-info of the main source jar.
> Seems downright weird to me.
Well, it is not. First of all, you can test your main code nicely
using standard non-modularized tests.
No problem here.
Then you can use your module containing the abstract/common test
classes with the scope <scope>test</scope>.
This is actually what junit etc. is – you are using their main classes
in YOUR test scope.
So, nothing weird here, too.
The third project pulls them in altogether and is modularized.
Call it "IT" or "integration test", and everything feels normal again.
This will also help your IDE recognize the code.
And as this is a common pattern, it will also help other coders to
more easily recognize the structure of your project.
Am Di., 22. Juni 2021 um 20:46 Uhr schrieb Ralph Goers
<ra...@dslextreme.com>:
>
> Yes, you understand correctly. And yes, the test classes could be moved to a separate maven module, but
> then the unit tests would also have to move as well as they are dependent on those test classes. Of course,
> the test classes are dependent on the main source. As an example, log4j-core has a LoggerContext class,
> the test jar has a LoggerContextRule class that uses the LoggerContext and the LoggerContextRule is used
> by many unit tests.
>
> So now you would have a maven module with only the main source and no tests, and a second module
> where the “main” source is actually the test classes and the unit tests test the code in a different Maven
> module with a module info that opens the module-info of the main source jar.
>
> Seems downright weird to me.
>
> Yes, I realize this doesn’t fit in the normal Maven lifecycle. I brought it up because if and/or when more
> stuff starts becoming JPMS modularized this issue is going to become more and more common.
>
> To be honest, I’d be OK living with the build the way it is except people are complaining that their IDE
> has no idea what to do with the project.
>
> Ralph
>
> > On Jun 22, 2021, at 11:16 AM, Robert Scholte <rf...@apache.org> wrote:
> >
> > If I understand correctly you want both log4j-core main classes and test classes be distributed as modularized jars.
> > To me with JPMS you should move the modularized test-classes to a separate Maven module as a first citizen artifact.
> > That should make it a lot easier.
> > It should also make the separation more clear: classes under target/test-classes are there to unittest its main classes.
> > Whereas this new module contains reusable (main) code for testing (which should be tested too ;) )
> > I understand that this will change the GA(-classifier) for the testing module, but to me for a good reason.
> >
> > Even with your very small reduction of steps to simplify the process, it doesn't fit in the default lifecycle unless you write you introduce a new lifecycle-binding or write a new extension.
> >
> > Robert
> >
> > On 22-6-2021 07:02:59, Ralph Goers <ra...@dslextreme.com> wrote:
> > Sorry for posting again. I really need to proof-read better. Please ignore the prior email.
> >
> > I have recently had quite an adventure modifying several of Log4j’s Maven modules to
> > implement JPMS on our master branch. It was an adventure due to a few issues:
> >
> > 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 .
> > This bug has been fixed in Java 17 but Log4j uses Java 11 to build.
> > 2. Log4j-plugins builds an annotation processor, packages it with the annotations
> > and classes necessary to build and run plugins, creates and test jar and runs unit tests.
> > 3. It is not possible to compile an annotation processor with a module-info.java present.
> > The compile will fail because it can’t find the annotation processor “service” when
> > compiling module-info.java.
> > 4. It is very difficult to compile an annotation processor and then use it in the same Maven
> > module. JPMS expects the annotation processor to either be on the classpath or specified
> > with the processorpath option. When a module-info.java is present, Maven automatically
> > moves everything to the module path. Maven only supports using coordinates to specify the
> > processor path, which don’t exist when the processor is in the same module. The only way
> > to solve this is to compile all the classes with proc=only without a module-info.java present.
> > 5. Although number 4 might seem bad, it really doesn’t matter because javac will fail if a
> > module-info.java is present because module-info.java will have a reference to the service
> > class being generated by the annotation processor and for some reason module-info.java
> > is resolved before the annotation processor runs.
> > 6. If the main set of classes are compiled with a module-info.java every other compile
> > in the Maven module must also be modularized. Likewise, if the main module does
> > not contain a module-info.java no other compile can either.
> > 7. JPMS requires that every module have its own package space with no overlap
> > with any other JPMS module.
> >
> > So while generating the log4j-plugins module is quite painful, generating log4j-core isn’t
> > much better. That is actually the primary focus for this list.
> >
> > Log4j-core consists of the main classes packaged in the jar, a test jar that is used by
> > downstream Maven modules, and the unit tests. Prior to JPMS one would just create
> > the main jar and then package all the test classes and unit tests in a test jar. This can
> > no longer be done with JPMS.
> >
> > When a project publishes a test jar along with the main jar, just like any other JPMS
> > module. the test jar cannot use the package space of the main classes. Log4j core
> > uses org.apache.logging.log4j.core so I placed all the test utility classes under
> > org.apache.logging.log4j.core.test. However, the unit tests all need to be packaged
> > in the main package space since its module-info.java “extends” the main module and
> > several unit tests are required to be in the same main package so that they can access
> > package private methods specifically provided for testing.
> >
> > In order to get this to work I had to perform the following steps:
> >
> > • Compile all the main classes except module-info.java with the Plugin annotation processor.
> > • Compile the main module-info.java.
> > • Compile the test classes used by other modules with module-info.java and
> > using the plugin preprocessor.
> > • Package these test classes in a test jar.
> > • Delete the module-info and generated source for the test classes.
> > • Move the main module-info to a temp location.
> > • Compile the unit test classes without module-info.java.
> > • Move the main module-info back to the classes directory.
> > • Compile module-info.java for unit tests.
> > • Run the unit tests.
> > • Create the main jar if the unit tests pass.
> >
> > Were it not for JDK-8265826 I believe this could have been simplified somewhat to:
> >
> > • Compile all the main classes except module-info.java with the Plugin annotation processor.
> > • Compile the main module-info.java.
> > • Compile the test classes used by other modules with its module-info.java and
> > using the plugin preprocessor.
> > • Package these test classes in a test jar.
> > • Delete the module-info and generated source for the test classes.
> > • Compile the unit test classes with its module-info.java.
> > • Compile module-info.java for unit tests.
> > • Run the unit tests.
> > • Create the main jar if the unit tests pass.
> >
> > So the gist of this entire email is pretty simple. Is there a way Maven could be modified
> > to better support creating test jars with JPMS?
> >
> > Ralph
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
> For additional commands, e-mail: dev-help@maven.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Ralph Goers <ra...@dslextreme.com>.
Yes, you understand correctly. And yes, the test classes could be moved to a separate maven module, but
then the unit tests would also have to move as well as they are dependent on those test classes. Of course,
the test classes are dependent on the main source. As an example, log4j-core has a LoggerContext class,
the test jar has a LoggerContextRule class that uses the LoggerContext and the LoggerContextRule is used
by many unit tests.
So now you would have a maven module with only the main source and no tests, and a second module
where the “main” source is actually the test classes and the unit tests test the code in a different Maven
module with a module info that opens the module-info of the main source jar.
Seems downright weird to me.
Yes, I realize this doesn’t fit in the normal Maven lifecycle. I brought it up because if and/or when more
stuff starts becoming JPMS modularized this issue is going to become more and more common.
To be honest, I’d be OK living with the build the way it is except people are complaining that their IDE
has no idea what to do with the project.
Ralph
> On Jun 22, 2021, at 11:16 AM, Robert Scholte <rf...@apache.org> wrote:
>
> If I understand correctly you want both log4j-core main classes and test classes be distributed as modularized jars.
> To me with JPMS you should move the modularized test-classes to a separate Maven module as a first citizen artifact.
> That should make it a lot easier.
> It should also make the separation more clear: classes under target/test-classes are there to unittest its main classes.
> Whereas this new module contains reusable (main) code for testing (which should be tested too ;) )
> I understand that this will change the GA(-classifier) for the testing module, but to me for a good reason.
>
> Even with your very small reduction of steps to simplify the process, it doesn't fit in the default lifecycle unless you write you introduce a new lifecycle-binding or write a new extension.
>
> Robert
>
> On 22-6-2021 07:02:59, Ralph Goers <ra...@dslextreme.com> wrote:
> Sorry for posting again. I really need to proof-read better. Please ignore the prior email.
>
> I have recently had quite an adventure modifying several of Log4j’s Maven modules to
> implement JPMS on our master branch. It was an adventure due to a few issues:
>
> 1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 .
> This bug has been fixed in Java 17 but Log4j uses Java 11 to build.
> 2. Log4j-plugins builds an annotation processor, packages it with the annotations
> and classes necessary to build and run plugins, creates and test jar and runs unit tests.
> 3. It is not possible to compile an annotation processor with a module-info.java present.
> The compile will fail because it can’t find the annotation processor “service” when
> compiling module-info.java.
> 4. It is very difficult to compile an annotation processor and then use it in the same Maven
> module. JPMS expects the annotation processor to either be on the classpath or specified
> with the processorpath option. When a module-info.java is present, Maven automatically
> moves everything to the module path. Maven only supports using coordinates to specify the
> processor path, which don’t exist when the processor is in the same module. The only way
> to solve this is to compile all the classes with proc=only without a module-info.java present.
> 5. Although number 4 might seem bad, it really doesn’t matter because javac will fail if a
> module-info.java is present because module-info.java will have a reference to the service
> class being generated by the annotation processor and for some reason module-info.java
> is resolved before the annotation processor runs.
> 6. If the main set of classes are compiled with a module-info.java every other compile
> in the Maven module must also be modularized. Likewise, if the main module does
> not contain a module-info.java no other compile can either.
> 7. JPMS requires that every module have its own package space with no overlap
> with any other JPMS module.
>
> So while generating the log4j-plugins module is quite painful, generating log4j-core isn’t
> much better. That is actually the primary focus for this list.
>
> Log4j-core consists of the main classes packaged in the jar, a test jar that is used by
> downstream Maven modules, and the unit tests. Prior to JPMS one would just create
> the main jar and then package all the test classes and unit tests in a test jar. This can
> no longer be done with JPMS.
>
> When a project publishes a test jar along with the main jar, just like any other JPMS
> module. the test jar cannot use the package space of the main classes. Log4j core
> uses org.apache.logging.log4j.core so I placed all the test utility classes under
> org.apache.logging.log4j.core.test. However, the unit tests all need to be packaged
> in the main package space since its module-info.java “extends” the main module and
> several unit tests are required to be in the same main package so that they can access
> package private methods specifically provided for testing.
>
> In order to get this to work I had to perform the following steps:
>
> • Compile all the main classes except module-info.java with the Plugin annotation processor.
> • Compile the main module-info.java.
> • Compile the test classes used by other modules with module-info.java and
> using the plugin preprocessor.
> • Package these test classes in a test jar.
> • Delete the module-info and generated source for the test classes.
> • Move the main module-info to a temp location.
> • Compile the unit test classes without module-info.java.
> • Move the main module-info back to the classes directory.
> • Compile module-info.java for unit tests.
> • Run the unit tests.
> • Create the main jar if the unit tests pass.
>
> Were it not for JDK-8265826 I believe this could have been simplified somewhat to:
>
> • Compile all the main classes except module-info.java with the Plugin annotation processor.
> • Compile the main module-info.java.
> • Compile the test classes used by other modules with its module-info.java and
> using the plugin preprocessor.
> • Package these test classes in a test jar.
> • Delete the module-info and generated source for the test classes.
> • Compile the unit test classes with its module-info.java.
> • Compile module-info.java for unit tests.
> • Run the unit tests.
> • Create the main jar if the unit tests pass.
>
> So the gist of this entire email is pretty simple. Is there a way Maven could be modified
> to better support creating test jars with JPMS?
>
> Ralph
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@maven.apache.org
For additional commands, e-mail: dev-help@maven.apache.org
Re: JPMS, Maven and test jars
Posted by Robert Scholte <rf...@apache.org>.
If I understand correctly you want both log4j-core main classes and test classes be distributed as modularized jars.
To me with JPMS you should move the modularized test-classes to a separate Maven module as a first citizen artifact.
That should make it a lot easier.
It should also make the separation more clear: classes under target/test-classes are there to unittest its main classes.
Whereas this new module contains reusable (main) code for testing (which should be tested too ;) )
I understand that this will change the GA(-classifier) for the testing module, but to me for a good reason.
Even with your very small reduction of steps to simplify the process, it doesn't fit in the default lifecycle unless you write you introduce a new lifecycle-binding or write a new extension.
Robert
On 22-6-2021 07:02:59, Ralph Goers <ra...@dslextreme.com> wrote:
Sorry for posting again. I really need to proof-read better. Please ignore the prior email.
I have recently had quite an adventure modifying several of Log4j’s Maven modules to
implement JPMS on our master branch. It was an adventure due to a few issues:
1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 .
This bug has been fixed in Java 17 but Log4j uses Java 11 to build.
2. Log4j-plugins builds an annotation processor, packages it with the annotations
and classes necessary to build and run plugins, creates and test jar and runs unit tests.
3. It is not possible to compile an annotation processor with a module-info.java present.
The compile will fail because it can’t find the annotation processor “service” when
compiling module-info.java.
4. It is very difficult to compile an annotation processor and then use it in the same Maven
module. JPMS expects the annotation processor to either be on the classpath or specified
with the processorpath option. When a module-info.java is present, Maven automatically
moves everything to the module path. Maven only supports using coordinates to specify the
processor path, which don’t exist when the processor is in the same module. The only way
to solve this is to compile all the classes with proc=only without a module-info.java present.
5. Although number 4 might seem bad, it really doesn’t matter because javac will fail if a
module-info.java is present because module-info.java will have a reference to the service
class being generated by the annotation processor and for some reason module-info.java
is resolved before the annotation processor runs.
6. If the main set of classes are compiled with a module-info.java every other compile
in the Maven module must also be modularized. Likewise, if the main module does
not contain a module-info.java no other compile can either.
7. JPMS requires that every module have its own package space with no overlap
with any other JPMS module.
So while generating the log4j-plugins module is quite painful, generating log4j-core isn’t
much better. That is actually the primary focus for this list.
Log4j-core consists of the main classes packaged in the jar, a test jar that is used by
downstream Maven modules, and the unit tests. Prior to JPMS one would just create
the main jar and then package all the test classes and unit tests in a test jar. This can
no longer be done with JPMS.
When a project publishes a test jar along with the main jar, just like any other JPMS
module. the test jar cannot use the package space of the main classes. Log4j core
uses org.apache.logging.log4j.core so I placed all the test utility classes under
org.apache.logging.log4j.core.test. However, the unit tests all need to be packaged
in the main package space since its module-info.java “extends” the main module and
several unit tests are required to be in the same main package so that they can access
package private methods specifically provided for testing.
In order to get this to work I had to perform the following steps:
• Compile all the main classes except module-info.java with the Plugin annotation processor.
• Compile the main module-info.java.
• Compile the test classes used by other modules with module-info.java and
using the plugin preprocessor.
• Package these test classes in a test jar.
• Delete the module-info and generated source for the test classes.
• Move the main module-info to a temp location.
• Compile the unit test classes without module-info.java.
• Move the main module-info back to the classes directory.
• Compile module-info.java for unit tests.
• Run the unit tests.
• Create the main jar if the unit tests pass.
Were it not for JDK-8265826 I believe this could have been simplified somewhat to:
• Compile all the main classes except module-info.java with the Plugin annotation processor.
• Compile the main module-info.java.
• Compile the test classes used by other modules with its module-info.java and
using the plugin preprocessor.
• Package these test classes in a test jar.
• Delete the module-info and generated source for the test classes.
• Compile the unit test classes with its module-info.java.
• Compile module-info.java for unit tests.
• Run the unit tests.
• Create the main jar if the unit tests pass.
So the gist of this entire email is pretty simple. Is there a way Maven could be modified
to better support creating test jars with JPMS?
Ralph
Re: JPMS, Maven and test jars
Posted by Ralph Goers <ra...@dslextreme.com>.
Sorry for posting again. I really need to proof-read better. Please ignore the prior email.
I have recently had quite an adventure modifying several of Log4j’s Maven modules to
implement JPMS on our master branch. It was an adventure due to a few issues:
1. https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826 <https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8265826>.
This bug has been fixed in Java 17 but Log4j uses Java 11 to build.
2. Log4j-plugins builds an annotation processor, packages it with the annotations
and classes necessary to build and run plugins, creates and test jar and runs unit tests.
3. It is not possible to compile an annotation processor with a module-info.java present.
The compile will fail because it can’t find the annotation processor “service” when
compiling module-info.java.
4. It is very difficult to compile an annotation processor and then use it in the same Maven
module. JPMS expects the annotation processor to either be on the classpath or specified
with the processorpath option. When a module-info.java is present, Maven automatically
moves everything to the module path. Maven only supports using coordinates to specify the
processor path, which don’t exist when the processor is in the same module. The only way
to solve this is to compile all the classes with proc=only without a module-info.java present.
5. Although number 4 might seem bad, it really doesn’t matter because javac will fail if a
module-info.java is present because module-info.java will have a reference to the service
class being generated by the annotation processor and for some reason module-info.java
is resolved before the annotation processor runs.
6. If the main set of classes are compiled with a module-info.java every other compile
in the Maven module must also be modularized. Likewise, if the main module does
not contain a module-info.java no other compile can either.
7. JPMS requires that every module have its own package space with no overlap
with any other JPMS module.
So while generating the log4j-plugins module is quite painful, generating log4j-core isn’t
much better. That is actually the primary focus for this list.
Log4j-core consists of the main classes packaged in the jar, a test jar that is used by
downstream Maven modules, and the unit tests. Prior to JPMS one would just create
the main jar and then package all the test classes and unit tests in a test jar. This can
no longer be done with JPMS.
When a project publishes a test jar along with the main jar, just like any other JPMS
module. the test jar cannot use the package space of the main classes. Log4j core
uses org.apache.logging.log4j.core so I placed all the test utility classes under
org.apache.logging.log4j.core.test. However, the unit tests all need to be packaged
in the main package space since its module-info.java “extends” the main module and
several unit tests are required to be in the same main package so that they can access
package private methods specifically provided for testing.
In order to get this to work I had to perform the following steps:
• Compile all the main classes except module-info.java with the Plugin annotation processor.
• Compile the main module-info.java.
• Compile the test classes used by other modules with module-info.java and
using the plugin preprocessor.
• Package these test classes in a test jar.
• Delete the module-info and generated source for the test classes.
• Move the main module-info to a temp location.
• Compile the unit test classes without module-info.java.
• Move the main module-info back to the classes directory.
• Compile module-info.java for unit tests.
• Run the unit tests.
• Create the main jar if the unit tests pass.
Were it not for JDK-8265826 I believe this could have been simplified somewhat to:
• Compile all the main classes except module-info.java with the Plugin annotation processor.
• Compile the main module-info.java.
• Compile the test classes used by other modules with its module-info.java and
using the plugin preprocessor.
• Package these test classes in a test jar.
• Delete the module-info and generated source for the test classes.
• Compile the unit test classes with its module-info.java.
• Compile module-info.java for unit tests.
• Run the unit tests.
• Create the main jar if the unit tests pass.
So the gist of this entire email is pretty simple. Is there a way Maven could be modified
to better support creating test jars with JPMS?
Ralph