You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@maven.apache.org by Garret Wilson <ga...@globalmentor.com> on 2023/08/01 16:32:32 UTC
Re: How to force Maven to put a dependency on the module-path?
On 7/26/2023 1:42 PM, Martin Desruisseaux wrote:
> … If a dependency is on the classpath, then the dependency is loaded
> as an unnamed module, its "module-info" file is ignored and the
> services that it contains are not discovered.
Can you elaborate on the last point a little more? I haven't modularized
my core libraries yet, and want to know how they would work with
non-modularized applications.
You say that if the modularized library is put on the classpath, its
services are not discovered. But wouldn't normal pre-module-era
classpath-based service discovery still work via the `META-INF/services`
configurations in the library JAR files? Please clarify what won't work
if I modularize my core libraries, which contain services to be
discovered, and I try to use that with a non-modularized application.
Thanks,
Garret
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: How to force Maven to put a dependency on the module-path?
Posted by Martin Desruisseaux <ma...@geomatys.com>.
Hello Thorsten
Le 02/08/2023 à 11:47, Thorsten Heit a écrit :
> I see your point, but we're not living in a fully modularized Java
> world: If you insist on having a Jar reside on the module path only,
> this prevents others from using it on applications that are not fully
> modularized or that cannot be modularized (yet).
No, an application does not need to be modularized for putting
dependencies on the module-path. This is what I tried to explain in my
yesterday's email. All applications, modularized or not, are already
using dependencies on the module path: the JDK itself. We can test that
with the following code:
System.out.println(String.class.getModule().getName());
The idea that a non-modularized application must put all its
dependencies on the class-path seems to be a widely-spread
misunderstanding of how JPMS works. Putting a library on the module-path
does not force an application to be modularized. Admittedly there is
some cases where a developer way want to use the class-path anyway, but
it should be under developer's control.
> Furthermore I don't really see a problem of duplication. Normally you
> don't have tons of service provider implementations in one Jar (IMO)
> so this shouldn't be too much hassle to have a few lines in your
> module descriptor AND in the provider configuration file under
> META-INF/services...
In Apache SIS, there are 76 providers for coordinate operations (e.g.
map projections) alone, not counting other services. This is not huge,
but the risk of inconsistency exists. Furthermore, as explained
yesterday, it blocks us from providing singletons, so we have to
workaround with wrappers. Finally, contrarily to module-info,
META-INF/services/ files get no compile-time verification. JUnit tests
does not always help in discovering for example that a
META-INF/services/ file was in the wrong module (a mistake that we
discovered precisely thanks to the migration to module-info).
> To be compatible with fully-modular applications and those that aren't
> I'd extract the necessary parts from the provider implementation into
> a singleton class and use that one instead. In this case you still
> could have multiple instances of your provider implementation, and all
> of them internally use a singleton for configuration data or whatever...
Yes this is the above-mentioned wrapper pattern. But things become more
complicated when some providers may implement more than one public
interface and we don't want to block advanced users from accessing those
additional API if they need to.
> Can you show an example that depends on such methods? I personally
> don't see a reason why a library should behave different if it is on
> the module path or class path...
There is 167 occurrences of the @CallerSensitive annotation in OpenJDK
source code as of today. Not all of them are module-related, but many
are, especially in tasks involving reflections. When a modularized
library is put on the module-path, encapsulation is enforced. So for
example a call to the following code:
InputStream in = Foo.class.getResourceAsStream("Something.properties");
returns a non-null value (assuming the file exists) when the library
invoking above code is on the class-path, but a null value if the same
library is on the module-path and Foo is a class declared in a
non-exported package of another module. The same restriction applies to
all variants such as ClassLoader.getResource(String), there is no way to
bypass the encapsulation. It may look likes a regression, but the key
point is that the behavior is not the same and a library could very well
depend on encapsulation being enforced. Other methods annotated with
@CallerSensitive are some logging methods. I didn't tested but I see
that the source code uses module information, so I would not be
surprised if logging messages are formatted differently.
> IMHO a library is buggy if it's *behavior* depends on whether it is
> being used on the module path or on the classpath.
With 167 OpenJDK methods annotated with @CallerSensitive, there is
plenty of room for behavioral differences even if the library developers
where very careful. I think it is okay for a library to said "I have
been designed for working as a module, please put me on the
module-path". As said above, it does not force applications to be
themselves modularized. There is possibly different opinions about what
is best (and it may depend on the library), but the current problem is
that Maven imposes its black magic on everyone with no way I can see to
specify what we need.
Martin
Re: How to force Maven to put a dependency on the module-path?
Posted by Thorsten Heit <th...@gmx.de>.
Hi Martin,
> Even if a library decide to do the duplication, it may still not work.
> Starting with Java 9, a service provider can be instantiated by invoking
> a public static method named "provider" in the provider declared in
> "module-info.class". This is an alternative to invoking the default
> public constructor. This alternative is very convenient when we want the
> provider to be a singleton for example. However the public static
> "provider" method is invoked only for services declared in
> "module-info.class", not for services declared in the old way. So if a
> service provider relies on that, it will not work on the class-path.
I see your point, but we're not living in a fully modularized Java world:
If you insist on having a Jar reside on the module path only, this
prevents others from using it on applications that are not fully
modularized or that cannot be modularized (yet).
Furthermore I don't really see a problem of duplication. Normally you
don't have tons of service provider implementations in one Jar (IMO) so
this shouldn't be too much hassle to have a few lines in your module
descriptor AND in the provider configuration file under META-INF/services...
To the singleton example you mentioned:
I don't know if that really makes sense. To be compatible with
fully-modular applications and those that aren't I'd extract the
necessary parts from the provider implementation into a singleton class
and use that one instead. In this case you still could have multiple
instances of your provider implementation, and all of them internally
use a singleton for configuration data or whatever...
> Finally the consequence on putting a dependency on the class-path rather
> than the module-path is much larger than only "java.util.ServiceLoader".
> The behavior of any methods annotated @CallerSensitive in OpenJDK source
> code may be impacted. It includes for example
> "ClassLoader.getResource(String)". If a library depends on those methods
> behaving the way the behave when the file is loaded at a module, that
> library may be broken if put on the class-path.
Out of curiosity:
Can you show an example that depends on such methods? I personally don't
see a reason why a library should behave different if it is on the
module path or class path...
> So to summarize: putting a JAR file on the module-path or class-path has
> deep consequences. A developer way want either ways on intend (including
> putting a modularized JAR file on the class-path or a non-modularized
> JAR file on the module-path, yes the opposite of the "sane" way, there
> is use cases for that). Maven should not keep developers prisoners of
> black magic. We need control on that.
IMHO a library is buggy if it's *behavior* depends on whether it is
being used on the module path or on the classpath.
Just my thoughts...
Kind regards
Thorsten
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org
Re: How to force Maven to put a dependency on the module-path?
Posted by Martin Desruisseaux <ma...@geomatys.com>.
Hello Garret
Le 01/08/2023 à 18:32, Garret Wilson a écrit :
> On 7/26/2023 1:42 PM, Martin Desruisseaux wrote:
>> … If a dependency is on the classpath, then the dependency is loaded
>> as an unnamed module, its "module-info" file is ignored and the
>> services that it contains are not discovered.
> Can you elaborate on the last point a little more? I haven't
> modularized my core libraries yet, and want to know how they would
> work with non-modularized applications.
There is a very simple test case here, for both Maven and Gradle, with a
README that tries to explain the problem. I tried to make the most
trivial "hello world" reproducing the problem. The test case can also be
run by invoking java directly on the command-line, which makes easy to
experiment different Java options for understanding the problem:
https://github.com/Geomatys/MavenModulepathBug
The problem can be reproduced with the following command-line (omitting
paths and version number for simplicity). This command-line reproduces
what Maven and Gradle do:
java --class-path service.jar:client.jar test.client.Main
In this example:
* "client" is the main application and is not modularized.
* "service" is a modularized dependency. In this JAR, I put four services:
o 2 services in "module-info.class"
o 2 services in "META-INF/services/" but with different texts, so
we can see which services is loaded.
If you run above command-line, you will see the following text:
|Start searching for services... Provider B declared in META-INF.
Provider C declared in META-INF. Done. The dependency has been
loaded as an unnamed module. Consequently its `module-info` file has
been ignored, and the `META-INF/services` directory is used instead.|
The call to "java.util.ServiceLoader" was done from the "service" JAR
file, which was supposed to be modularized. Despite that fact, we see
that "module-info.class" has been ignored and "META-INF/services/" used
instead. In other words, the modularized "service.jar" is unable to
access to its own "module-info.class". We are not even talking about the
behavior of the non-modularized "client.jar" file here.
The correct behavior can be reproduced with the following command-line
(again omitting paths and version numbers). The main difference is that
the modularized "service.jar" file is put on the module-path instead
than the class-path. I will skip discussion about the "-add-modules"
option for this email.
java --module-path service.jar --class-path client.jar --add-modules ALL-MODULE-PATH test.client.Main
The output is then:
|Start searching for services... Provider A declared in module-info.
Provider D declared in module-info. Done. The dependency has been
loaded as named module. Great! This is what we need for the
`module-info` to be used.|
This time "module-info.class" has been used and "META-INF/services/"
ignored (as it should be). I claim that it should be the Maven and
Gradle behavior. If this claim is disputed (e.g. for compatibility
reason), then at least it should be configurable. The fact that there is
no way I could find for overriding the (in my opinion broken) Maven
behavior is what I call the bug. Note that this is not a JPMS issue,
this is a Maven/Gradle issue. Maven and Gradle uses the following
heuristic rules for deciding if a dependency should be put on the
module-path:
1. the dependency is modularized (i.e. contains a module-info.class
file or an Automatic-Module-Name attribute in MANIFEST.MF), and
2. the project using the dependency is itself modularized.
I claim that condition 2 should be removed, i.e. a modularized
dependency should be put on the module-path regardless if the project
using it is modularized or not. Or if condition 2 is not removed, then
at least the developer should have some way to control that.
> You say that if the modularized library is put on the classpath, its
> services are not discovered. But wouldn't normal pre-module-era
> classpath-based service discovery still work via the
> `META-INF/services` configurations in the library JAR files? Please
> clarify what won't work if I modularize my core libraries, which
> contain services to be discovered, and I try to use that with a
> non-modularized application.
"META-INF/services/" is not necessary for compatibility with
non-modularized applications. It is necessary only for environments that
decide to put the modules on the class-path rather than the module-path
(reminder: putting dependencies on the module-path works as well with
non-modules client application). Maintaining both "module-info.class"
and "META-INF/services/" with identical content is duplication, with
risk of inconsistent behavior if their content accidentally diverge or
declare services in different order. If a library chose to not do this
duplication, its services will not be discovered if the JAR file is not
used in the proper way.
Even if a library decide to do the duplication, it may still not work.
Starting with Java 9, a service provider can be instantiated by invoking
a public static method named "provider" in the provider declared in
"module-info.class". This is an alternative to invoking the default
public constructor. This alternative is very convenient when we want the
provider to be a singleton for example. However the public static
"provider" method is invoked only for services declared in
"module-info.class", not for services declared in the old way. So if a
service provider relies on that, it will not work on the class-path.
Finally the consequence on putting a dependency on the class-path rather
than the module-path is much larger than only "java.util.ServiceLoader".
The behavior of any methods annotated @CallerSensitive in OpenJDK source
code may be impacted. It includes for example
"ClassLoader.getResource(String)". If a library depends on those methods
behaving the way the behave when the file is loaded at a module, that
library may be broken if put on the class-path.
Whether a library has been put on the class-path or module-path can be
observed by invoking the following Java code, where "Foo" is any class
of the library:
boolean b = Foo.class.getModule().isNamed()
If true, "Foo" is in a modularized JAR file which was on the
module-path. If "b" is false despite the fact that the JAR file
containing "Foo" was modularized, than it is because that JAR file was
on the class-path.
So to summarize: putting a JAR file on the module-path or class-path has
deep consequences. A developer way want either ways on intend (including
putting a modularized JAR file on the class-path or a non-modularized
JAR file on the module-path, yes the opposite of the "sane" way, there
is use cases for that). Maven should not keep developers prisoners of
black magic. We need control on that.
Martin