You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tomcat.apache.org by Rainer Jung <ra...@kippdata.de> on 2016/01/23 13:17:49 UTC
TC WebappClassLoaderBase bottleneck in filter()
I observed a bottleneck in WebappClassLoaderBase.filter() during a
stress test. The reason is, that the method is synchronized. It looks to
me, that the reason for the synchronization is only the access to the
non-thread safe Matchers packageTriggersPermit and packageTriggersDeny.
Since they are not used anywhere else, IMHO one can get away with using
a separate Lock object for access to the two. E.g.
Index: WebappClassLoaderBase.java
===================================================================
--- WebappClassLoaderBase.java (revision 1726388)
+++ WebappClassLoaderBase.java (working copy)
@@ -191,6 +191,12 @@
// ------------------------------------------------------- Static
Variables
/**
+ * Monitor Object used to synchronize calling
+ * packageTriggersDeny and packageTriggersPermit.
+ */
+ protected final Object packageTriggersLock = new Object();
+
+ /**
* Regular expression of package names which are not allowed to be
loaded
* from a webapp class loader without delegating first.
*/
@@ -2781,7 +2787,7 @@
* @param name class name
* @return <code>true</code> if the class should be filtered
*/
- protected synchronized boolean filter(String name) {
+ protected boolean filter(String name) {
if (name == null)
return false;
@@ -2795,16 +2801,17 @@
else
return false;
- packageTriggersPermit.reset(packageName);
- if (packageTriggersPermit.lookingAt()) {
- return false;
+ synchronized(packageTriggersLock) {
+ packageTriggersPermit.reset(packageName);
+ if (packageTriggersPermit.lookingAt()) {
+ return false;
+ }
+ packageTriggersDeny.reset(packageName);
+ if (packageTriggersDeny.lookingAt()) {
+ return true;
+ }
}
- packageTriggersDeny.reset(packageName);
- if (packageTriggersDeny.lookingAt()) {
- return true;
- }
-
return false;
}
Comments on the patch?
In addition:
Since the calls to filter() are in a hot code path, I wonder whether
using a more complex code instead of a single regexp could be better.
The code should be fast in the common case, which IMHO is the case when
the resource name neither starts with javax nor with org.apache. That
case could be handled by a simple prefix comparison without regexp.
Only when the resource name starts with javax or org.apache (or
org/apache) the various cases to check may make using a regexp be the
better decision. Of course such code is a bit harder to maintain, than a
single regexp.
I think I'll run a simple micro benchmark to get an idea.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC WebappClassLoaderBase bottleneck in filter()
Posted by Rainer Jung <ra...@kippdata.de>.
Am 24.01.2016 um 19:50 schrieb Felix Schumacher:
> Am 23.01.2016 um 17:05 schrieb Rainer Jung:
>> Am 23.01.2016 um 13:17 schrieb Rainer Jung:
>>> Since the calls to filter() are in a hot code path, I wonder whether
>>> using a more complex code instead of a single regexp could be better.
>>> The code should be fast in the common case, which IMHO is the case when
>>> the resource name neither starts with javax nor with org.apache. That
>>> case could be handled by a simple prefix comparison without regexp.
>>>
>>> Only when the resource name starts with javax or org.apache (or
>>> org/apache) the various cases to check may make using a regexp be the
>>> better decision. Of course such code is a bit harder to maintain, than a
>>> single regexp.
>>>
>>> I think I'll run a simple micro benchmark to get an idea.
>>
>> The below code nees about 5% of the execution time of the regexp based
>> one. In some cases 15%. I found no case worse than 15%.
>
> The regex can be sped up, when "[./]" will be used instead of "(\.|/)"
> and by combining the conditions like
>
> "^(?:javax[./](?el|security[./]auth[./]message|servlet|websocket)|org[./]apache[./](?:catalina|coyote|el|jasper|juli|naming|tomcat))[./]"
>
>
> That one is a bit nearer to the hand written logic below and performs
> better in my micro tests with jmh on the strings "nomatch" and
> "org.apache.catalina.shouldmatch". (3535124,947 ops/s vs. 5800147,877
> ops/s).
Yes, using that regexp (with ?el -> ?:el) and a similar one for the
Permit pattern the numbers change slightly, but still in most cases, the
hand written matcher needs 5-10% of the regexp executions, worst case
20%. At least after warmup.
>>
>> /**
>> * Filter classes.
>> *
>> * @param name class name
>> * @return <code>true</code> if the class should be filtered
>> */
>> protected boolean filter(String name) {
>>
>> if (name == null)
>> return false;
>>
>> char sep;
>> boolean javax_or_org_apache;
>> boolean dot_or_slash;
> I found the names of the variables ..._or_... a bit confusing, as I read
> a boolean or into it and thought it would be true, if name started with
> javax. or org.apache.
Yes, agreed. I ran out of ideas for a better name. I did another test
and Enums are fast enough, so I would replace the two "_or_" booleans
with a Separator and a Prefix enum.
Thanks for having a look.
Rainer
> Regards,
> Felix
>
>>
>> if (name.startsWith("javax")) {
>> /* 5 == length("javax") */
>> sep = name.charAt(5);
>> if (sep == '.') {
>> javax_or_org_apache = true;
>> dot_or_slash = true;
>> } else if (sep == '/') {
>> javax_or_org_apache = true;
>> dot_or_slash = false;
>> } else {
>> return false;
>> }
>> } else if (name.startsWith("org")) {
>> /* 3 == length("org") */
>> sep = name.charAt(3);
>> /* 4 == length("org.") */
>> if (sep == '.' && name.startsWith("apache.", 4)) {
>> javax_or_org_apache = false;
>> dot_or_slash = true;
>> /* 4 == length("org/") */
>> } else if (sep == '/' && name.startsWith("apache/", 4)) {
>> javax_or_org_apache = false;
>> dot_or_slash = false;
>> } else {
>> return false;
>> }
>> } else {
>> return false;
>> }
>>
>> if (javax_or_org_apache) {
>> if (dot_or_slash) {
>> /* 6 == length("javax.") */
>> if (name.startsWith("servlet.jsp.jstl.", 6)) {
>> return false;
>> }
>> if (name.startsWith("el.", 6) ||
>> name.startsWith("servlet.", 6) ||
>> name.startsWith("websocket.", 6) ||
>> name.startsWith("security.auth.message.", 6)) {
>> return true;
>> }
>> } else {
>> /* 6 == length("javax/") */
>> if (name.startsWith("servlet/jsp/jstl/", 6)) {
>> return false;
>> }
>> if (name.startsWith("el/", 6) ||
>> name.startsWith("servlet/", 6) ||
>> name.startsWith("websocket/", 6) ||
>> name.startsWith("security/auth/message/", 6)) {
>> return true;
>> }
>> }
>> } else {
>> if (dot_or_slash) {
>> /* 11 == length("org.apache.") */
>> if (name.startsWith("tomcat.jdbc.", 11)) {
>> return false;
>> }
>> if (name.startsWith("el.", 11) ||
>> name.startsWith("catalina.", 11) ||
>> name.startsWith("jasper.", 11) ||
>> name.startsWith("juli.", 11) ||
>> name.startsWith("tomcat.", 11) ||
>> name.startsWith("naming.", 11) ||
>> name.startsWith("coyote.", 11)) {
>> return true;
>> }
>> } else {
>> /* 11 == length("org/apache/") */
>> if (name.startsWith("tomcat/jdbc/", 11)) {
>> return false;
>> }
>> if (name.startsWith("el/", 11) ||
>> name.startsWith("catalina/", 11) ||
>> name.startsWith("jasper/", 11) ||
>> name.startsWith("juli/", 11) ||
>> name.startsWith("tomcat/", 11) ||
>> name.startsWith("naming/", 11) ||
>> name.startsWith("coyote/", 11)) {
>> return true;
>> }
>> }
>> }
>> return false;
>> }
>>
>>
>> Regards,
>>
>> Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC WebappClassLoaderBase bottleneck in filter()
Posted by Felix Schumacher <fe...@internetallee.de>.
Am 23.01.2016 um 17:05 schrieb Rainer Jung:
> Am 23.01.2016 um 13:17 schrieb Rainer Jung:
>> Since the calls to filter() are in a hot code path, I wonder whether
>> using a more complex code instead of a single regexp could be better.
>> The code should be fast in the common case, which IMHO is the case when
>> the resource name neither starts with javax nor with org.apache. That
>> case could be handled by a simple prefix comparison without regexp.
>>
>> Only when the resource name starts with javax or org.apache (or
>> org/apache) the various cases to check may make using a regexp be the
>> better decision. Of course such code is a bit harder to maintain, than a
>> single regexp.
>>
>> I think I'll run a simple micro benchmark to get an idea.
>
> The below code nees about 5% of the execution time of the regexp based
> one. In some cases 15%. I found no case worse than 15%.
The regex can be sped up, when "[./]" will be used instead of "(\.|/)"
and by combining the conditions like
"^(?:javax[./](?el|security[./]auth[./]message|servlet|websocket)|org[./]apache[./](?:catalina|coyote|el|jasper|juli|naming|tomcat))[./]"
That one is a bit nearer to the hand written logic below and performs
better in my micro tests with jmh on the strings "nomatch" and
"org.apache.catalina.shouldmatch". (3535124,947 ops/s vs. 5800147,877
ops/s).
>
> /**
> * Filter classes.
> *
> * @param name class name
> * @return <code>true</code> if the class should be filtered
> */
> protected boolean filter(String name) {
>
> if (name == null)
> return false;
>
> char sep;
> boolean javax_or_org_apache;
> boolean dot_or_slash;
I found the names of the variables ..._or_... a bit confusing, as I read
a boolean or into it and thought it would be true, if name started with
javax. or org.apache.
Regards,
Felix
>
> if (name.startsWith("javax")) {
> /* 5 == length("javax") */
> sep = name.charAt(5);
> if (sep == '.') {
> javax_or_org_apache = true;
> dot_or_slash = true;
> } else if (sep == '/') {
> javax_or_org_apache = true;
> dot_or_slash = false;
> } else {
> return false;
> }
> } else if (name.startsWith("org")) {
> /* 3 == length("org") */
> sep = name.charAt(3);
> /* 4 == length("org.") */
> if (sep == '.' && name.startsWith("apache.", 4)) {
> javax_or_org_apache = false;
> dot_or_slash = true;
> /* 4 == length("org/") */
> } else if (sep == '/' && name.startsWith("apache/", 4)) {
> javax_or_org_apache = false;
> dot_or_slash = false;
> } else {
> return false;
> }
> } else {
> return false;
> }
>
> if (javax_or_org_apache) {
> if (dot_or_slash) {
> /* 6 == length("javax.") */
> if (name.startsWith("servlet.jsp.jstl.", 6)) {
> return false;
> }
> if (name.startsWith("el.", 6) ||
> name.startsWith("servlet.", 6) ||
> name.startsWith("websocket.", 6) ||
> name.startsWith("security.auth.message.", 6)) {
> return true;
> }
> } else {
> /* 6 == length("javax/") */
> if (name.startsWith("servlet/jsp/jstl/", 6)) {
> return false;
> }
> if (name.startsWith("el/", 6) ||
> name.startsWith("servlet/", 6) ||
> name.startsWith("websocket/", 6) ||
> name.startsWith("security/auth/message/", 6)) {
> return true;
> }
> }
> } else {
> if (dot_or_slash) {
> /* 11 == length("org.apache.") */
> if (name.startsWith("tomcat.jdbc.", 11)) {
> return false;
> }
> if (name.startsWith("el.", 11) ||
> name.startsWith("catalina.", 11) ||
> name.startsWith("jasper.", 11) ||
> name.startsWith("juli.", 11) ||
> name.startsWith("tomcat.", 11) ||
> name.startsWith("naming.", 11) ||
> name.startsWith("coyote.", 11)) {
> return true;
> }
> } else {
> /* 11 == length("org/apache/") */
> if (name.startsWith("tomcat/jdbc/", 11)) {
> return false;
> }
> if (name.startsWith("el/", 11) ||
> name.startsWith("catalina/", 11) ||
> name.startsWith("jasper/", 11) ||
> name.startsWith("juli/", 11) ||
> name.startsWith("tomcat/", 11) ||
> name.startsWith("naming/", 11) ||
> name.startsWith("coyote/", 11)) {
> return true;
> }
> }
> }
> return false;
> }
>
>
> Regards,
>
> Rainer
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: dev-help@tomcat.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC WebappClassLoaderBase bottleneck in filter()
Posted by Rainer Jung <ra...@kippdata.de>.
Am 23.01.2016 um 13:17 schrieb Rainer Jung:
> Since the calls to filter() are in a hot code path, I wonder whether
> using a more complex code instead of a single regexp could be better.
> The code should be fast in the common case, which IMHO is the case when
> the resource name neither starts with javax nor with org.apache. That
> case could be handled by a simple prefix comparison without regexp.
>
> Only when the resource name starts with javax or org.apache (or
> org/apache) the various cases to check may make using a regexp be the
> better decision. Of course such code is a bit harder to maintain, than a
> single regexp.
>
> I think I'll run a simple micro benchmark to get an idea.
The below code nees about 5% of the execution time of the regexp based
one. In some cases 15%. I found no case worse than 15%.
/**
* Filter classes.
*
* @param name class name
* @return <code>true</code> if the class should be filtered
*/
protected boolean filter(String name) {
if (name == null)
return false;
char sep;
boolean javax_or_org_apache;
boolean dot_or_slash;
if (name.startsWith("javax")) {
/* 5 == length("javax") */
sep = name.charAt(5);
if (sep == '.') {
javax_or_org_apache = true;
dot_or_slash = true;
} else if (sep == '/') {
javax_or_org_apache = true;
dot_or_slash = false;
} else {
return false;
}
} else if (name.startsWith("org")) {
/* 3 == length("org") */
sep = name.charAt(3);
/* 4 == length("org.") */
if (sep == '.' && name.startsWith("apache.", 4)) {
javax_or_org_apache = false;
dot_or_slash = true;
/* 4 == length("org/") */
} else if (sep == '/' && name.startsWith("apache/", 4)) {
javax_or_org_apache = false;
dot_or_slash = false;
} else {
return false;
}
} else {
return false;
}
if (javax_or_org_apache) {
if (dot_or_slash) {
/* 6 == length("javax.") */
if (name.startsWith("servlet.jsp.jstl.", 6)) {
return false;
}
if (name.startsWith("el.", 6) ||
name.startsWith("servlet.", 6) ||
name.startsWith("websocket.", 6) ||
name.startsWith("security.auth.message.", 6)) {
return true;
}
} else {
/* 6 == length("javax/") */
if (name.startsWith("servlet/jsp/jstl/", 6)) {
return false;
}
if (name.startsWith("el/", 6) ||
name.startsWith("servlet/", 6) ||
name.startsWith("websocket/", 6) ||
name.startsWith("security/auth/message/", 6)) {
return true;
}
}
} else {
if (dot_or_slash) {
/* 11 == length("org.apache.") */
if (name.startsWith("tomcat.jdbc.", 11)) {
return false;
}
if (name.startsWith("el.", 11) ||
name.startsWith("catalina.", 11) ||
name.startsWith("jasper.", 11) ||
name.startsWith("juli.", 11) ||
name.startsWith("tomcat.", 11) ||
name.startsWith("naming.", 11) ||
name.startsWith("coyote.", 11)) {
return true;
}
} else {
/* 11 == length("org/apache/") */
if (name.startsWith("tomcat/jdbc/", 11)) {
return false;
}
if (name.startsWith("el/", 11) ||
name.startsWith("catalina/", 11) ||
name.startsWith("jasper/", 11) ||
name.startsWith("juli/", 11) ||
name.startsWith("tomcat/", 11) ||
name.startsWith("naming/", 11) ||
name.startsWith("coyote/", 11)) {
return true;
}
}
}
return false;
}
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC WebappClassLoaderBase bottleneck in filter()
Posted by Rainer Jung <ra...@kippdata.de>.
Am 25.01.2016 um 11:23 schrieb Mark Thomas:
> On 25/01/2016 09:32, Rainer Jung wrote:
>> Hi Konstantin,
>>
>> thanks for the feedback. More inline.
>>
>> Am 25.01.2016 um 03:51 schrieb Konstantin Kolinko:
>>> 2016-01-23 15:17 GMT+03:00 Rainer Jung <ra...@kippdata.de>:
>>>> I observed a bottleneck in WebappClassLoaderBase.filter() during a
>>>> stress
>>>> test. The reason is, that the method is synchronized. It looks to me,
>>>> that
>>>> the reason for the synchronization is only the access to the
>>>> non-thread safe
>>>> Matchers packageTriggersPermit and packageTriggersDeny. Since they
>>>> are not
>>>> used anywhere else, IMHO one can get away with using a separate Lock
>>>> object
>>>> for access to the two. E.g.
>>>>
>>>
>>> 1. Instead of a lock it is possible to create a new Matcher object on
>>> each invocation.
>>
>> For performance reasons - this is a hot code path - I would actually
>> prefer the variant I sent later. The hand written code that doesn't use
>> regexp (in a variant that uses enums instead of the "*_or_*" boolean
>> variables. Probably also adjusted due to your remark about the use of
>> "." and "/".
>
> Given the performance improvements, that works for me.
>
>>> 2. There is no need for those Matcher / new Lock object to be
>>> "protected". An interested party can overwrite the whole filter()
>>> method.
>>
>> Right. With the hand written code they would be completely gone.
>>
>>> 3. Filtering resources and classes with the same regexp looks wrong.
>>>
>>> It is safe to use that regexp on class names (those do not have '/'
>>> character in them), but it is wrong to use it on resources ('.' is a
>>> valid character in a file name, not a package separator).
>>
>> The hand written code no longer supports filtering names with mixed "."
>> and "/". I guess those were never intended to get filtered.
>>
>> If we want to make it even more correct, namely using "." when filtering
>> class names and "/" when filtering resources (I hope that would be
>> correct), the filter() method would need a flag to indicate which case
>> we are in. That would mean changing the signature of a protected method.
>> I guess we need to keep the old signature additionally for compatibility.
>>
>>> 4. Generally (if I remember correctly) this filter
>>> a) saves people that do a stupid thing of trying to bundle API classes
>>> or Tomcat classes in their war (e.g. due to misconfiguration of Maven
>>> dependencies)
>>>
>>> b) allows a performance improvement, as the search for those classes
>>> in web application classloader can be skipped
>>
>> It doesn't skip, but it delegates to the parent first for the filtered
>> names, where it most likely will find the searched items.
>>
>> I forgot whether there were also spec reasons. You indicate, that the
>> reasons are not spec compliance but performance.
>
> The specification requirements are in 10.7.2. The filter method is
> implementing the part that states:
>
> <quote>
> The container should not allow applications to override or access the
> container’s implementation classes.
> </quote>
>
> The JavaSE parts of that section are implemented elsewhere (the svn
> history for WebappClassLoaderBase has the details).
I committed first some regexp minor enhancements, then the improved
locking and finally the hand tuned code that doesn't use any regexp or
locking. The intermediate commits are just in case if we want to go back
to the regexp based solution. I plan to backport all changes unless I
hear objections.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC WebappClassLoaderBase bottleneck in filter()
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
All,
On 1/25/16 5:23 AM, Mark Thomas wrote:
> On 25/01/2016 09:32, Rainer Jung wrote:
>> Hi Konstantin,
>>
>> thanks for the feedback. More inline.
>>
>> Am 25.01.2016 um 03:51 schrieb Konstantin Kolinko:
>>> 2016-01-23 15:17 GMT+03:00 Rainer Jung
>>> <ra...@kippdata.de>:
>>>> I observed a bottleneck in WebappClassLoaderBase.filter()
>>>> during a stress test. The reason is, that the method is
>>>> synchronized. It looks to me, that the reason for the
>>>> synchronization is only the access to the non-thread safe
>>>> Matchers packageTriggersPermit and packageTriggersDeny. Since
>>>> they are not used anywhere else, IMHO one can get away with
>>>> using a separate Lock object for access to the two. E.g.
>>>>
>>>
>>> 1. Instead of a lock it is possible to create a new Matcher
>>> object on each invocation.
>>
>> For performance reasons - this is a hot code path - I would
>> actually prefer the variant I sent later. The hand written code
>> that doesn't use regexp (in a variant that uses enums instead of
>> the "*_or_*" boolean variables. Probably also adjusted due to
>> your remark about the use of "." and "/".
>
> Given the performance improvements, that works for me.
>
>>> 2. There is no need for those Matcher / new Lock object to be
>>> "protected". An interested party can overwrite the whole
>>> filter() method.
>>
>> Right. With the hand written code they would be completely gone.
>>
>>> 3. Filtering resources and classes with the same regexp looks
>>> wrong.
>>>
>>> It is safe to use that regexp on class names (those do not have
>>> '/' character in them), but it is wrong to use it on resources
>>> ('.' is a valid character in a file name, not a package
>>> separator).
>>
>> The hand written code no longer supports filtering names with
>> mixed "." and "/". I guess those were never intended to get
>> filtered.
>>
>> If we want to make it even more correct, namely using "." when
>> filtering class names and "/" when filtering resources (I hope
>> that would be correct), the filter() method would need a flag to
>> indicate which case we are in. That would mean changing the
>> signature of a protected method. I guess we need to keep the old
>> signature additionally for compatibility.
>>
>>> 4. Generally (if I remember correctly) this filter a) saves
>>> people that do a stupid thing of trying to bundle API classes
>>> or Tomcat classes in their war (e.g. due to misconfiguration of
>>> Maven dependencies)
>>>
>>> b) allows a performance improvement, as the search for those
>>> classes in web application classloader can be skipped
>>
>> It doesn't skip, but it delegates to the parent first for the
>> filtered names, where it most likely will find the searched
>> items.
>>
>> I forgot whether there were also spec reasons. You indicate, that
>> the reasons are not spec compliance but performance.
>
> The specification requirements are in 10.7.2. The filter method is
> implementing the part that states:
>
> <quote> The container should not allow applications to override or
> access the container’s implementation classes. </quote>
>
> The JavaSE parts of that section are implemented elsewhere (the
> svn history for WebappClassLoaderBase has the details).
If there were a chain of filters with a common interface, the "easy"
ones (e.g. className.startsWith("org.apache")) ones could be written
with a very fast matching algorithm and the complicated ones could be
written using e.g. regular expressions.
There's more infrastructure to build, but better performance, plus we
don't have to hand-write our own hybrid matching algorithms.
This suggestion has been unpopular in the past. I still think it's a
good idea.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iEYEARECAAYFAlan454ACgkQ9CaO5/Lv0PAYIwCfRWBGbIMM6FKePcdIpZmtReIE
vYkAnAi7UcdxNzbVqkGOFhz2VfQwQ6Al
=P1AM
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC WebappClassLoaderBase bottleneck in filter()
Posted by Mark Thomas <ma...@apache.org>.
On 25/01/2016 09:32, Rainer Jung wrote:
> Hi Konstantin,
>
> thanks for the feedback. More inline.
>
> Am 25.01.2016 um 03:51 schrieb Konstantin Kolinko:
>> 2016-01-23 15:17 GMT+03:00 Rainer Jung <ra...@kippdata.de>:
>>> I observed a bottleneck in WebappClassLoaderBase.filter() during a
>>> stress
>>> test. The reason is, that the method is synchronized. It looks to me,
>>> that
>>> the reason for the synchronization is only the access to the
>>> non-thread safe
>>> Matchers packageTriggersPermit and packageTriggersDeny. Since they
>>> are not
>>> used anywhere else, IMHO one can get away with using a separate Lock
>>> object
>>> for access to the two. E.g.
>>>
>>
>> 1. Instead of a lock it is possible to create a new Matcher object on
>> each invocation.
>
> For performance reasons - this is a hot code path - I would actually
> prefer the variant I sent later. The hand written code that doesn't use
> regexp (in a variant that uses enums instead of the "*_or_*" boolean
> variables. Probably also adjusted due to your remark about the use of
> "." and "/".
Given the performance improvements, that works for me.
>> 2. There is no need for those Matcher / new Lock object to be
>> "protected". An interested party can overwrite the whole filter()
>> method.
>
> Right. With the hand written code they would be completely gone.
>
>> 3. Filtering resources and classes with the same regexp looks wrong.
>>
>> It is safe to use that regexp on class names (those do not have '/'
>> character in them), but it is wrong to use it on resources ('.' is a
>> valid character in a file name, not a package separator).
>
> The hand written code no longer supports filtering names with mixed "."
> and "/". I guess those were never intended to get filtered.
>
> If we want to make it even more correct, namely using "." when filtering
> class names and "/" when filtering resources (I hope that would be
> correct), the filter() method would need a flag to indicate which case
> we are in. That would mean changing the signature of a protected method.
> I guess we need to keep the old signature additionally for compatibility.
>
>> 4. Generally (if I remember correctly) this filter
>> a) saves people that do a stupid thing of trying to bundle API classes
>> or Tomcat classes in their war (e.g. due to misconfiguration of Maven
>> dependencies)
>>
>> b) allows a performance improvement, as the search for those classes
>> in web application classloader can be skipped
>
> It doesn't skip, but it delegates to the parent first for the filtered
> names, where it most likely will find the searched items.
>
> I forgot whether there were also spec reasons. You indicate, that the
> reasons are not spec compliance but performance.
The specification requirements are in 10.7.2. The filter method is
implementing the part that states:
<quote>
The container should not allow applications to override or access the
container’s implementation classes.
</quote>
The JavaSE parts of that section are implemented elsewhere (the svn
history for WebappClassLoaderBase has the details).
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC WebappClassLoaderBase bottleneck in filter()
Posted by Rainer Jung <ra...@kippdata.de>.
Hi Konstantin,
thanks for the feedback. More inline.
Am 25.01.2016 um 03:51 schrieb Konstantin Kolinko:
> 2016-01-23 15:17 GMT+03:00 Rainer Jung <ra...@kippdata.de>:
>> I observed a bottleneck in WebappClassLoaderBase.filter() during a stress
>> test. The reason is, that the method is synchronized. It looks to me, that
>> the reason for the synchronization is only the access to the non-thread safe
>> Matchers packageTriggersPermit and packageTriggersDeny. Since they are not
>> used anywhere else, IMHO one can get away with using a separate Lock object
>> for access to the two. E.g.
>>
>
> 1. Instead of a lock it is possible to create a new Matcher object on
> each invocation.
For performance reasons - this is a hot code path - I would actually
prefer the variant I sent later. The hand written code that doesn't use
regexp (in a variant that uses enums instead of the "*_or_*" boolean
variables. Probably also adjusted due to your remark about the use of
"." and "/".
> 2. There is no need for those Matcher / new Lock object to be
> "protected". An interested party can overwrite the whole filter()
> method.
Right. With the hand written code they would be completely gone.
> 3. Filtering resources and classes with the same regexp looks wrong.
>
> It is safe to use that regexp on class names (those do not have '/'
> character in them), but it is wrong to use it on resources ('.' is a
> valid character in a file name, not a package separator).
The hand written code no longer supports filtering names with mixed "."
and "/". I guess those were never intended to get filtered.
If we want to make it even more correct, namely using "." when filtering
class names and "/" when filtering resources (I hope that would be
correct), the filter() method would need a flag to indicate which case
we are in. That would mean changing the signature of a protected method.
I guess we need to keep the old signature additionally for compatibility.
> 4. Generally (if I remember correctly) this filter
> a) saves people that do a stupid thing of trying to bundle API classes
> or Tomcat classes in their war (e.g. due to misconfiguration of Maven
> dependencies)
>
> b) allows a performance improvement, as the search for those classes
> in web application classloader can be skipped
It doesn't skip, but it delegates to the parent first for the filtered
names, where it most likely will find the searched items.
I forgot whether there were also spec reasons. You indicate, that the
reasons are not spec compliance but performance.
Regards,
Rainer
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org
Re: TC WebappClassLoaderBase bottleneck in filter()
Posted by Konstantin Kolinko <kn...@gmail.com>.
2016-01-23 15:17 GMT+03:00 Rainer Jung <ra...@kippdata.de>:
> I observed a bottleneck in WebappClassLoaderBase.filter() during a stress
> test. The reason is, that the method is synchronized. It looks to me, that
> the reason for the synchronization is only the access to the non-thread safe
> Matchers packageTriggersPermit and packageTriggersDeny. Since they are not
> used anywhere else, IMHO one can get away with using a separate Lock object
> for access to the two. E.g.
>
1. Instead of a lock it is possible to create a new Matcher object on
each invocation.
2. There is no need for those Matcher / new Lock object to be
"protected". An interested party can overwrite the whole filter()
method.
3. Filtering resources and classes with the same regexp looks wrong.
It is safe to use that regexp on class names (those do not have '/'
character in them), but it is wrong to use it on resources ('.' is a
valid character in a file name, not a package separator).
4. Generally (if I remember correctly) this filter
a) saves people that do a stupid thing of trying to bundle API classes
or Tomcat classes in their war (e.g. due to misconfiguration of Maven
dependencies)
b) allows a performance improvement, as the search for those classes
in web application classloader can be skipped
Best regards,
Konstantin Kolinko
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org