Severe 7.0.47 startup performance regression

[David Rees <dr...@gmail.com>]

Since upgrading to 7.0.47 from 7.0.42, I've seen a very severe
performance hit during startup.

One Tomcat instance with a lot of webapps slowed down from ~33 seconds
to ~180 seconds. It appears to be related to jar-scanning - If I
change catalina.properties'
tomcat.util.scan.DefaultJarScanner.jarsToSkip to *.jar, startup
performance is actually about 10% better than before, but with the
expected app-breakage. If I go and modify jarsToSkip to manually
exclude only the jars I know should be excluded, performance is near
normal - about 10-15% worse than 7.0.42 which is acceptable but
obviously could still be better.

I've reviewed the changelog between 7.0.42 and 7.0.47, but can't find
anything that might cause this.

All the deployed apps are using servlet api 2.5.

I've reviewed the FasterStartUp startup wiki and the apps already use
metadata-complete="true" in the web.xml.

Any other suggestions? Update the apps to servlet api 3.0 and put in
an empty <absolute-ordering /> element?

Is there a way to get the scanners to log whether or not a scan found
anything so they can be added to the appropriate jarsToSkip parameter?
I've only found that the TLD scanner does this unless I'm missing
something.

Any other ideas?

-Dave

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Severe 7.0.47 startup performance regression

[David Rees <dr...@gmail.com>]

On Thu, Dec 5, 2013 at 12:18 AM, Konstantin Kolinko
<kn...@gmail.com> wrote:
> Remove the tomcat7-websocket.jar library if you do not need it.  This
> library (added in 7.0.47, a JSR-356 Java WebSocket 1.0 implementation)
> has a ServletContainerInitializer and thus forces Tomcat to perform
> annotation scanning.

That seems to be another work around. Updating the app's web.xml to
3.0 and specifying an empty <absolute-ordering/> also works.

Kind of a pain though, really!

> If you read the "FasterStartUp" page carefully, it is explained there.

Yeah - I found that page a bit difficult to parse. I must have read it
at least 10 times now!

>> Is there a way to get the scanners to log whether or not a scan found
>> anything so they can be added to the appropriate jarsToSkip parameter?
>> I've only found that the TLD scanner does this unless I'm missing
>> something.

Any thoughts on this? If I'm going to go through the trouble of
excluding jars, it would be very nice to be able to determine what
jars were scanned, especially if nothing was found like the TLD
scanner does.

-Dave

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Severe 7.0.47 startup performance regression

[Rossen Stoyanchev <rs...@gopivotal.com>]

On Thu, Dec 5, 2013 at 10:57 AM, Mark Thomas <ma...@apache.org> wrote:

> On 05/12/2013 14:22, Rossen Stoyanchev wrote:
> > On Thu, Dec 5, 2013 at 7:05 AM, Mark Thomas <ma...@apache.org> wrote:
> >
> >>
> >> There is also the programmatic interface to WebSockets so once there is
> >> a way to disable the SCI, you can use the programmatic interface to set
> >> everything up.
> >
> >
> > Yes, the very existence of a programmatic interface alternative more or
> > less implies there should be a way to disable the SCI and still be able
> to
> > use WebSocket. Is it not possible to use an <absolute-ordering> element
> in
> > web.xml?
>
> That provides a way to exclude JARs from scanning (which should be most
> of the problem) but not classes.


Okay, I wasn't aware of that. It would have been useful to somehow exclude
the WAR itself too, so it's all one mechanism.


> > I was under the impression that's the mechanism for applications
> > to control which web fragments to include or exclude. My understanding
> from
> > the EG discussions was that SCI was chosen precisely because it is an
> > existing mechanism that is understood, so I don't understand the reason
> why
> > a new mechanism is needed to control SCI scanning.
>
> I think there is still a use for being able to disable container
> provided SCIs on a per context basis outside of what is available in the
> spec. That said, if no-one agrees with me then I have plenty of other
> things to be working on.
>

I agree there should be a way to disable container provided SCI. My
surprise was that there isn't already a way to do that.

Re: Severe 7.0.47 startup performance regression

[Mark Thomas <ma...@apache.org>]

On 05/12/2013 14:22, Rossen Stoyanchev wrote:
> On Thu, Dec 5, 2013 at 7:05 AM, Mark Thomas <ma...@apache.org> wrote:
> 
>>
>> There is also the programmatic interface to WebSockets so once there is
>> a way to disable the SCI, you can use the programmatic interface to set
>> everything up.
> 
> 
> Yes, the very existence of a programmatic interface alternative more or
> less implies there should be a way to disable the SCI and still be able to
> use WebSocket. Is it not possible to use an <absolute-ordering> element in
> web.xml?

That provides a way to exclude JARs from scanning (which should be most
of the problem) but not classes.

> I was under the impression that's the mechanism for applications
> to control which web fragments to include or exclude. My understanding from
> the EG discussions was that SCI was chosen precisely because it is an
> existing mechanism that is understood, so I don't understand the reason why
> a new mechanism is needed to control SCI scanning.

I think there is still a use for being able to disable container
provided SCIs on a per context basis outside of what is available in the
spec. That said, if no-one agrees with me then I have plenty of other
things to be working on.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Severe 7.0.47 startup performance regression

[Rossen Stoyanchev <rs...@gopivotal.com>]

On Thu, Dec 5, 2013 at 7:05 AM, Mark Thomas <ma...@apache.org> wrote:

>
> There is also the programmatic interface to WebSockets so once there is
> a way to disable the SCI, you can use the programmatic interface to set
> everything up.


Yes, the very existence of a programmatic interface alternative more or
less implies there should be a way to disable the SCI and still be able to
use WebSocket. Is it not possible to use an <absolute-ordering> element in
web.xml? I was under the impression that's the mechanism for applications
to control which web fragments to include or exclude. My understanding from
the EG discussions was that SCI was chosen precisely because it is an
existing mechanism that is understood, so I don't understand the reason why
a new mechanism is needed to control SCI scanning.

Rossen

Re: Severe 7.0.47 startup performance regression

[Mark Thomas <ma...@apache.org>]

On 05/12/2013 11:50, Johan Compagner wrote:
>>
>>> Does anyone know why this wasn't done by using services? (SPI)
>>>  So that we can directly point to a class that is the websocket or point
>> to
>>> a class that registers the websockets?
>>
>> The Java WebSocket 1.0 specification requires this behaviour.
>>
>> As has been pointed out previously, there really should be a way for a
>> web application to disable a container provided SCI if it knows it
>> doesn't need it. The specification doesn't currently allow this. A
>> Tomcat specific feature to do this is on my TODO list. I'm thinking
>> something like a regular expression of SCI implementation classes to
>> exclude configured on the Context. As always, patches welcome.
>>
>>
>>
> I was just wondering what the reasons where for depending on a full
> jar/class scan.
> I didn't follow the spec discussion at all (i am just about to start using
> websockets)
> I know the spec is build like that that they are not depending at all of a
> web container or something like that
>  (i think you explained this before)
> 
> but doing full jar/class scans instead is for me just a weird choice, thats
> bad for performance for huge apps.
> 
> Also having some kind of setting so that tomcat doesn't scan if you as a
> developer know that it doesn't need it at all.
> That only fixes stuff that really don't use it. But now i have a big
> project and i do use 1 or 2 websocket here and there.
> Then it must be scanned. What i rather would have that it works like a SPI
> or some other kind of (osgi) services
> so that websocket scanner just ask for a SPI instances and then gets the
> websocket classes from that.
> then only 1 file is looked for in a jar (manifest/services/xxxx) instead of
> all the classes

There is also the programmatic interface to WebSockets so once there is
a way to disable the SCI, you can use the programmatic interface to set
everything up.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Severe 7.0.47 startup performance regression

[Johan Compagner <jc...@servoy.com>]

>
> > Does anyone know why this wasn't done by using services? (SPI)
> >  So that we can directly point to a class that is the websocket or point
> to
> > a class that registers the websockets?
>
> The Java WebSocket 1.0 specification requires this behaviour.
>
> As has been pointed out previously, there really should be a way for a
> web application to disable a container provided SCI if it knows it
> doesn't need it. The specification doesn't currently allow this. A
> Tomcat specific feature to do this is on my TODO list. I'm thinking
> something like a regular expression of SCI implementation classes to
> exclude configured on the Context. As always, patches welcome.
>
>
>
I was just wondering what the reasons where for depending on a full
jar/class scan.
I didn't follow the spec discussion at all (i am just about to start using
websockets)
I know the spec is build like that that they are not depending at all of a
web container or something like that
 (i think you explained this before)

but doing full jar/class scans instead is for me just a weird choice, thats
bad for performance for huge apps.

Also having some kind of setting so that tomcat doesn't scan if you as a
developer know that it doesn't need it at all.
That only fixes stuff that really don't use it. But now i have a big
project and i do use 1 or 2 websocket here and there.
Then it must be scanned. What i rather would have that it works like a SPI
or some other kind of (osgi) services
so that websocket scanner just ask for a SPI instances and then gets the
websocket classes from that.
then only 1 file is looked for in a jar (manifest/services/xxxx) instead of
all the classes

johan

Re: Severe 7.0.47 startup performance regression

[Mark Thomas <ma...@apache.org>]

On 05/12/2013 08:33, Johan Compagner wrote:
>>
>>
>> Good.
>>
>> Remove the tomcat7-websocket.jar library if you do not need it.  This
>> library (added in 7.0.47, a JSR-356 Java WebSocket 1.0 implementation)
>> has a ServletContainerInitializer and thus forces Tomcat to perform
>> annotation scanning.
>>
>>
> Does anyone know why this wasn't done by using services? (SPI)
>  So that we can directly point to a class that is the websocket or point to
> a class that registers the websockets?

The Java WebSocket 1.0 specification requires this behaviour.

As has been pointed out previously, there really should be a way for a
web application to disable a container provided SCI if it knows it
doesn't need it. The specification doesn't currently allow this. A
Tomcat specific feature to do this is on my TODO list. I'm thinking
something like a regular expression of SCI implementation classes to
exclude configured on the Context. As always, patches welcome.

Mark


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org

Re: Severe 7.0.47 startup performance regression

[Johan Compagner <jc...@servoy.com>]

>
>
> Good.
>
> Remove the tomcat7-websocket.jar library if you do not need it.  This
> library (added in 7.0.47, a JSR-356 Java WebSocket 1.0 implementation)
> has a ServletContainerInitializer and thus forces Tomcat to perform
> annotation scanning.
>
>
Does anyone know why this wasn't done by using services? (SPI)
 So that we can directly point to a class that is the websocket or point to
a class that registers the websockets?

-- 
Johan Compagner
Servoy

Re: Severe 7.0.47 startup performance regression

[Konstantin Kolinko <kn...@gmail.com>]

2013/12/5 David Rees <dr...@gmail.com>:
> Since upgrading to 7.0.47 from 7.0.42, I've seen a very severe
> performance hit during startup.
>
> One Tomcat instance with a lot of webapps slowed down from ~33 seconds
> to ~180 seconds. It appears to be related to jar-scanning - If I
> change catalina.properties'
> tomcat.util.scan.DefaultJarScanner.jarsToSkip to *.jar, startup
> performance is actually about 10% better than before, but with the
> expected app-breakage. If I go and modify jarsToSkip to manually
> exclude only the jars I know should be excluded, performance is near
> normal - about 10-15% worse than 7.0.42 which is acceptable but
> obviously could still be better.
>
> I've reviewed the changelog between 7.0.42 and 7.0.47, but can't find
> anything that might cause this.
>
> All the deployed apps are using servlet api 2.5.
>
> I've reviewed the FasterStartUp startup wiki and the apps already use
> metadata-complete="true" in the web.xml.

Good.

Remove the tomcat7-websocket.jar library if you do not need it.  This
library (added in 7.0.47, a JSR-356 Java WebSocket 1.0 implementation)
has a ServletContainerInitializer and thus forces Tomcat to perform
annotation scanning.

If you read the "FasterStartUp" page carefully, it is explained there.

> Any other suggestions? Update the apps to servlet api 3.0 and put in
> an empty <absolute-ordering /> element?
>
> Is there a way to get the scanners to log whether or not a scan found
> anything so they can be added to the appropriate jarsToSkip parameter?
> I've only found that the TLD scanner does this unless I'm missing
> something.
>
> Any other ideas?
>

Best regards,
Konstantin Kolinko

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org