You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Rainer Jung <ra...@kippdata.de> on 2011/06/23 16:34:10 UTC
Supporting more general globs in loader definition?
At the moment the file globs we support in the loader definitions in
catalina.properties are hard coded to DIRECTORY/*.jar.
Sometimes it would be helpful to allow a bit more flexible globs, like
e.g. log4j-1.2.*.jar or similar.
Would there be any interest if I would investigate whether we can use
e.g. the DirectoryScanner from ant to support more general globs?
Since we want to define the loader path, we only want to end up with
directories and jar files. So one idea would be to filter the result of
the DirectoryScanner to only include dirs and jars that match. If we
find that more secure, we could also restrict to jars only, because I
think that's the most probable use case for globs and picking up any
dirs e.g. when using a recursive pattern might not be what the user
expected.
The result should be compatible to what the users got when using the
strict *.jar pattern.
Since we scan the directories only on startup and the ant code is
mature, I don't expect any problematic performance impact.
I don't have a patch yet, but wanted to check for general feedback
before investigating more deeply.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Supporting more general globs in loader definition?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Rainer,
On 6/23/2011 4:02 PM, Rainer Jung wrote:
> But supporting a glob in the file name would also work.
I'm always irritated when something that looks like it should work
doesn't work. The fact that .../*.jar works but .../foo*.jar does not
work is surprising, so I'm in support of adding filename globbing.
Ignoring path globbing until someone demands it is perfectly okay with
me, too, especially since the implementation is probably more involved.
-chris
Re: Supporting more general globs in loader definition?
Posted by Rainer Jung <ra...@kippdata.de>.
On 23.06.2011 19:46, Konstantin Kolinko wrote:
> 2011/6/23 Rainer Jung <ra...@kippdata.de>:
>> At the moment the file globs we support in the loader definitions in
>> catalina.properties are hard coded to DIRECTORY/*.jar.
>>
>> Sometimes it would be helpful to allow a bit more flexible globs, like
>> e.g. log4j-1.2.*.jar or similar.
>>
>> Would there be any interest if I would investigate whether we can use
>> e.g. the DirectoryScanner from ant to support more general globs?
>>
>> Since we want to define the loader path, we only want to end up with
>> directories and jar files. So one idea would be to filter the result of
>> the DirectoryScanner to only include dirs and jars that match. If we
>> find that more secure, we could also restrict to jars only, because I
>> think that's the most probable use case for globs and picking up any
>> dirs e.g. when using a recursive pattern might not be what the user
>> expected.
>>
>> The result should be compatible to what the users got when using the
>> strict *.jar pattern.
>>
>> Since we scan the directories only on startup and the ant code is
>> mature, I don't expect any problematic performance impact.
>>
>> I don't have a patch yet, but wanted to check for general feedback
>> before investigating more deeply.
>>
>> Regards,
>> Rainer
>
> So, you are talking about common.loader, shared.loader, and
> server.loader values in catalina.properties file.
Yes.
> (I somehow thought it was about jarsToSkip).
I added globs there for TC 7.0.3.
> It is hard to find a use case for this feature you are asking for.
I will explain below.
> The only use case that comes to my mind is to prioritize some jars
> over the others.
> E.g. to put ${catalina.base}/lib/patch-*.jar ahead of the other
> *.jars in the same directory. Though it does not differ much from
> using ${catalina.base}/lib/patches/*.jar
Yeah, I didn't think about that one.
> So, technically I do not mind allowing any name prefix before *.jar. I
> do not think there should be more wide glob support, like allowing '?'
> or '*' in any position in the name. Do you have a use case for globs?
>
>
> Politically, I do not like to have much emphasis on the
> "common.loader" property.
>
> I expect the list of jars to be put in "common.loader" to be rather
> short. You put there the jars used by Tomcat, the jars used by
> components declared in server.xml and context files, database drivers.
>
> People should really put everything else into WEB-INF/lib, or use a
> VirtualWebappLoader.
Agreed.
> As an example of a mess: using the pattern "log4j-1.2.*.jar" from
> your example it is easy to select several versions of Log4J, which
> might result in a disaster. If there is only one version of log4j,
> why not to copy or symlink it into designated directory? Thus all the
> libraries will be in one place and will be easier managed.
See below, it is about updating.
> One more note:
> I recently added support for arbitrary system properties expansion in
> the value of common.loader and other *.loader properties. As a side
> effect it is now possible to use whatever Unix command (e.g. find) in
> setenv.sh to build a list of jars and define it as a system property.
I saw that, a great improvement.
> Summarizing: I do not mind to add support for name*.jar, but will
> think twice about anything else.
The only concrete example I have in mind is about adding log4j to Tomcat
to let it log via Log4J.
Here the situation I'm thinking about is a split CATALINA_HOME vs.
CATALINA_BASE installation. Log4J is provided in some extra directory of
CATALINA_HOME using the normal jar file name, e.g. log4j-1.2.16.jar.
The jar will be referenced from catalina.properties in CATALINA_BASE not
using a *.jar pattern, because in the extra dir there is a couple of
jars, some you want to include, some not.
Now when CATALINA_HOME is updated, there might be a new version of log4j
(not: multiple versions) and the entry in catalina.properties does no
longer match, leading to Tomcat using java.util.logging instead.
Of course there are several solutions:
- installing log4j in a directory not used for other jars, so one can
use *.jar
- installing the log4j jar under a name, that does not include the version
- Fixing the entry in catalina.properties as part of the update
- Ignoring the problem, because there might never be another version of
log4j after 1.2.16
But supporting a glob in the file name would also work.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Supporting more general globs in loader definition?
Posted by Konstantin Kolinko <kn...@gmail.com>.
2011/6/23 Rainer Jung <ra...@kippdata.de>:
> At the moment the file globs we support in the loader definitions in
> catalina.properties are hard coded to DIRECTORY/*.jar.
>
> Sometimes it would be helpful to allow a bit more flexible globs, like
> e.g. log4j-1.2.*.jar or similar.
>
> Would there be any interest if I would investigate whether we can use
> e.g. the DirectoryScanner from ant to support more general globs?
>
> Since we want to define the loader path, we only want to end up with
> directories and jar files. So one idea would be to filter the result of
> the DirectoryScanner to only include dirs and jars that match. If we
> find that more secure, we could also restrict to jars only, because I
> think that's the most probable use case for globs and picking up any
> dirs e.g. when using a recursive pattern might not be what the user
> expected.
>
> The result should be compatible to what the users got when using the
> strict *.jar pattern.
>
> Since we scan the directories only on startup and the ant code is
> mature, I don't expect any problematic performance impact.
>
> I don't have a patch yet, but wanted to check for general feedback
> before investigating more deeply.
>
> Regards,
> Rainer
So, you are talking about common.loader, shared.loader, and
server.loader values in catalina.properties file.
(I somehow thought it was about jarsToSkip).
It is hard to find a use case for this feature you are asking for.
The only use case that comes to my mind is to prioritize some jars
over the others.
E.g. to put ${catalina.base}/lib/patch-*.jar ahead of the other
*.jars in the same directory. Though it does not differ much from
using ${catalina.base}/lib/patches/*.jar
So, technically I do not mind allowing any name prefix before *.jar. I
do not think there should be more wide glob support, like allowing '?'
or '*' in any position in the name. Do you have a use case for globs?
Politically, I do not like to have much emphasis on the
"common.loader" property.
I expect the list of jars to be put in "common.loader" to be rather
short. You put there the jars used by Tomcat, the jars used by
components declared in server.xml and context files, database drivers.
People should really put everything else into WEB-INF/lib, or use a
VirtualWebappLoader.
As an example of a mess: using the pattern "log4j-1.2.*.jar" from
your example it is easy to select several versions of Log4J, which
might result in a disaster. If there is only one version of log4j,
why not to copy or symlink it into designated directory? Thus all the
libraries will be in one place and will be easier managed.
One more note:
I recently added support for arbitrary system properties expansion in
the value of common.loader and other *.loader properties. As a side
effect it is now possible to use whatever Unix command (e.g. find) in
setenv.sh to build a list of jars and define it as a system property.
Summarizing: I do not mind to add support for name*.jar, but will
think twice about anything else.
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Supporting more general globs in loader definition?
Posted by Rainer Jung <ra...@kippdata.de>.
On 23.06.2011 16:51, Christopher Schultz wrote:
> Rainer,
>
> On 6/23/2011 10:39 AM, Rainer Jung wrote:
>> Globs in the file name and not
>> the directory part is the most important use case and easy to understand
>> in its consequences. So I lean towards this type of improvement right now.
>
> So:
>
> /path/to/*.jar (allowed today)
> /path/to/foo*.jar (proposed)
> /path/*/foo.jar (not proposed)
> /path/**/foo.jar (ant-style "**" for "any depth", not proposed)
>
> ?
>
> Any reason not to allow globs in the non-filename path portion? I can't
> really see a use case for actually wanting a path-glob but am interested
> in your reasoning.
I think there are no very strong arguments for or against supporting
globs in directory names and supporting **. I just finished the
implementation for only supporting globs in the file name part and
that's actually very simple using the util class we already have today.
I'd say when defining the class path for Tomcat you want some kind of
strictness to avoid unexpected stuff to get loaded. We allow to shortcut
the jar list at the moment by loading all Jars from a directory (the
*.jar pattern), but I feel a bit uncomfortable with searching recursive
via ** and also about whether people understand what it means to include
matched directories.
The use case I have in mind, is some directoyr with extensions, where
you install all of them, but want to choose which ones to load. At the
moment you do it by listing all Jars explicitely.
Now when upgrading the dependencies and the Jars have versions in their
name, you have to update all your CATALINA_BASE/catalina.properties,
because you had to include the explicit file names. Using globs like
log4j-1.2.*.jar you can keep the config stable for a longer time.
Will post a patch to inspect once my testing is done.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Supporting more general globs in loader definition?
Posted by Christopher Schultz <ch...@christopherschultz.net>.
Rainer,
On 6/23/2011 10:39 AM, Rainer Jung wrote:
> Globs in the file name and not
> the directory part is the most important use case and easy to understand
> in its consequences. So I lean towards this type of improvement right now.
So:
/path/to/*.jar (allowed today)
/path/to/foo*.jar (proposed)
/path/*/foo.jar (not proposed)
/path/**/foo.jar (ant-style "**" for "any depth", not proposed)
?
Any reason not to allow globs in the non-filename path portion? I can't
really see a use case for actually wanting a path-glob but am interested
in your reasoning.
-chris
Re: Supporting more general globs in loader definition?
Posted by Rainer Jung <ra...@kippdata.de>.
And this would be the patch:
http://people.apache.org/~rjung/patches/tc7-loader-glob.patch
Regards,
Rainer
On 23.06.2011 16:39, Rainer Jung wrote:
> On 23.06.2011 16:34, Rainer Jung wrote:
>> At the moment the file globs we support in the loader definitions in
>> catalina.properties are hard coded to DIRECTORY/*.jar.
>>
>> Sometimes it would be helpful to allow a bit more flexible globs, like
>> e.g. log4j-1.2.*.jar or similar.
>>
>> Would there be any interest if I would investigate whether we can use
>> e.g. the DirectoryScanner from ant to support more general globs?
>>
>> Since we want to define the loader path, we only want to end up with
>> directories and jar files. So one idea would be to filter the result of
>> the DirectoryScanner to only include dirs and jars that match. If we
>> find that more secure, we could also restrict to jars only, because I
>> think that's the most probable use case for globs and picking up any
>> dirs e.g. when using a recursive pattern might not be what the user
>> expected.
>>
>> The result should be compatible to what the users got when using the
>> strict *.jar pattern.
>>
>> Since we scan the directories only on startup and the ant code is
>> mature, I don't expect any problematic performance impact.
>>
>> I don't have a patch yet, but wanted to check for general feedback
>> before investigating more deeply.
>
> Sent a bit to quickly:
>
> A much simplr solution would be using the Matcher util class we already
> imported from ant to support globs in the jar scanner skip list. If we
> only allow the globs to be part of the file name, then the impl would be
> pretty straightforward using the Matcher. Globs in the file name and not
> the directory part is the most important use case and easy to understand
> in its consequences. So I lean towards this type of improvement right now.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: Supporting more general globs in loader definition?
Posted by Rainer Jung <ra...@kippdata.de>.
On 23.06.2011 16:34, Rainer Jung wrote:
> At the moment the file globs we support in the loader definitions in
> catalina.properties are hard coded to DIRECTORY/*.jar.
>
> Sometimes it would be helpful to allow a bit more flexible globs, like
> e.g. log4j-1.2.*.jar or similar.
>
> Would there be any interest if I would investigate whether we can use
> e.g. the DirectoryScanner from ant to support more general globs?
>
> Since we want to define the loader path, we only want to end up with
> directories and jar files. So one idea would be to filter the result of
> the DirectoryScanner to only include dirs and jars that match. If we
> find that more secure, we could also restrict to jars only, because I
> think that's the most probable use case for globs and picking up any
> dirs e.g. when using a recursive pattern might not be what the user
> expected.
>
> The result should be compatible to what the users got when using the
> strict *.jar pattern.
>
> Since we scan the directories only on startup and the ant code is
> mature, I don't expect any problematic performance impact.
>
> I don't have a patch yet, but wanted to check for general feedback
> before investigating more deeply.
Sent a bit to quickly:
A much simplr solution would be using the Matcher util class we already
imported from ant to support globs in the jar scanner skip list. If we
only allow the globs to be part of the file name, then the impl would be
pretty straightforward using the Matcher. Globs in the file name and not
the directory part is the most important use case and easy to understand
in its consequences. So I lean towards this type of improvement right now.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org