You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cocoon.apache.org by Berin Loritsch <bl...@apache.org> on 2002/02/13 21:12:42 UTC
Possible points of scalibility issues
I have been performing some performance analysis of the Avalon Excalibur code, and I discovered
some serious points of thread contention. In a web environment, there can be as many as 150
threads (or more) if the web server uses simple thread per connection technology
(most common aproach for Java based servers like Tomcat). I expanded the Profile tests to work
using 100 threads. The default pooling implementation has some serious slowdown due to thread
contention.
A new Avalon committer named Leif Mortenson has created a new Pool implementation called
ResourceLimitingPool. This pool implementation is very flexible, and many orders of magnitude
more efficient than the current pooling implementations. I have posted a VOTE to promote it
from scratchpad to production. When that happens, there will be less thread contention on
Pooled components.
Another, more serious point of contention is the ExcaliburComponentManager implementation itself.
The reason is that there are some *potentially* long stretches of code that are blocking. This
is particularly true if the ECM needs to initialize a PoolableComponent. I have tried to make
the blocking portions as small as possible, but we still have some work to do.
I am also in the process of the new ContainerManager/Container abstraction. The current CVS
version is functional (not in Cocoon's CVS, but Avalon's), but the API needs some sprucing up.
I also need to put in some Profiling tests. In the new ContainerManager/Container abstraction
we make use of a new ManagedPool. The pooling implementation for the managed pool is even more
efficient than Leif's excellent ResourceLimitingPool. It is less deterministic as to the
precise number of instances available, which the ResourceLimitingPool is better suited for.
The alternative FixedSizePool is both ThreadSafe and the fastest implementation.
The new ContainerManager/Container abstraction does have a new Role configuration format, and
does not pay attention to lifestyle interfaces. It is an experiment to see if we can live without
them--and seems to be working well. The RoleManager will match the ComponentHandler implementation
to the Component implementation. This is much more flexible, and I believe it will work better
in the long run. The default ComponentHandler has been changed from Factory to PerThread (Using
a ThreadLocal to manage component instances).
I am very interested in making Cocoon much better, and the best way I can do that is to work
on the core Avalon stuff. The future will require changes, but the payoff will be worth it.
I wanted to point out the hotspots that Cocoon is facing, and why it is having a hard time scaling.
In the interim, I would suggest limiting the number of threads that your Servlet container will
allow to be used on Cocoon to around 40--but when the core is made better, we won't need to have
those limitations.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Possible points of scalibility issues
Posted by Berin Loritsch <bl...@apache.org>.
Berin Loritsch wrote:
> Berin Loritsch wrote:
>
> We have a new ThreadSafe BucketMap class that helped out tremendously. The
> BucketMap synchronizes on a "bucket" or group of keys that fit the same
> hash
> value.
BTW, I highly recommend upgrading to the BucketMap for any place you would
normally use a synchronized map. This is a "most excellent" improvement.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Possible points of scalibility issues
Posted by Berin Loritsch <bl...@apache.org>.
Berin Loritsch wrote:
> Stefano Mazzocchi wrote:
>
>> Berin Loritsch wrote:
>>
>>
>>> The question remains about the ECM itself yet. I haven't had time to
>>> write
>>> the profiling tests yet.
>>>
>>
>> I fully trust your judgement on this.
>
>
>
> :)
>
> I found out that the ECM implementation has been changed, and the
> hashmaps backing
> the ECM are now Container.synchronizedMap() wrapped. :(
>
> When I took off the synchronization, the time spent for 50,000 requests
> with 20
> concurrent threads went from ~80,000ms to ~9,000ms. However this is not a
> threadsafe solution--I will add back in the external locking mechanism
> (the ReadWriteLock)
> and make this work safely and a bit better.
Ok, The current version of ExcaliburComponentManager/ExcaliburComponentSelector
is as efficient as we can make it.
We have a new ThreadSafe BucketMap class that helped out tremendously. The
BucketMap synchronizes on a "bucket" or group of keys that fit the same hash
value.
Since the whole class is not synchronized, we can have acesses on the map using
different keys without running into thread contention.
Granted, the hash algorithm is a simple modulous:
hash = object.hashCode() % m_buckets.length;
If anyone has a cheap algorithm that distributes the hashes more evenly, please
let me know.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Possible points of scalibility issues
Posted by Berin Loritsch <bl...@apache.org>.
Stefano Mazzocchi wrote:
> Berin Loritsch wrote:
>
>
>>The question remains about the ECM itself yet. I haven't had time to write
>>the profiling tests yet.
>>
>
> I fully trust your judgement on this.
:)
I found out that the ECM implementation has been changed, and the hashmaps backing
the ECM are now Container.synchronizedMap() wrapped. :(
When I took off the synchronization, the time spent for 50,000 requests with 20
concurrent threads went from ~80,000ms to ~9,000ms. However this is not a
threadsafe solution--I will add back in the external locking mechanism (the ReadWriteLock)
and make this work safely and a bit better.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Possible points of scalibility issues
Posted by Stefano Mazzocchi <st...@apache.org>.
Berin Loritsch wrote:
> > Berin, I spent several *months* finetuning the thread-pool code that was
> > used in Apache JServ 1.0 (and is still there, used in both IBM WebSphere
> > and Oracle Application Server). I remember that I got a report from
> > Alcatel that they were using it with loads up to 12000 req/sec.
>
> :) And how many *threads* running simultaneously?
If I'm not mistaken, they were more than several hundreds concurrent
requests.
> Remember, Requests/Second != cuncurrent threads
Of course.
> Apache HTTPD in it's default configuration will run up to 150 processess,
> and when there are more connections than processess, it will make further
> requests wait.
>
> 12000 req/sec translates to 12.5 ms/request with 150 processes/threads.
> It also translates to 41.6ms/request with 500 processes/threads.
>
> (In another mail I will post my formula for scaling requirements)
>
> > Guess that if you need something clean and scalable, that's the way to
> > go :)
> >
> > [I proposed this a while back but you and Peter tought it was too
> > complex.... but I still like mine more :) but I'm biased]
>
> ( Aren't we all ;P )
>
> I think the implementations we have now fit that bill--clean and scalable.
kool.
> The question remains about the ECM itself yet. I haven't had time X-Mozilla-Status: 0009ing tests yet.
I fully trust your judgement on this.
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Possible points of scalibility issues
Posted by Stefano Mazzocchi <st...@apache.org>.
Berin Loritsch wrote:
> > Berin, I spent several *months* finetuning the thread-pool code that was
> > used in Apache JServ 1.0 (and is still there, used in both IBM WebSphere
> > and Oracle Application Server). I remember that I got a report from
> > Alcatel that they were using it with loads up to 12000 req/sec.
>
> :) And how many *threads* running simultaneously?
If I'm not mistaken, they were more than several hundreds concurrent
requests.
> Remember, Requests/Second != cuncurrent threads
Of course.
> Apache HTTPD in it's default configuration will run up to 150 processess,
> and when there are more connections than processess, it will make further
> requests wait.
>
> 12000 req/sec translates to 12.5 ms/request with 150 processes/threads.
> It also translates to 41.6ms/request with 500 processes/threads.
>
> (In another mail I will post my formula for scaling requirements)
>
> > Guess that if you need something clean and scalable, that's the way to
> > go :)
> >
> > [I proposed this a while back but you and Peter tought it was too
> > complex.... but I still like mine more :) but I'm biased]
>
> ( Aren't we all ;P )
>
> I think the implementations we have now fit that bill--clean and scalable.
kool.
> The question remains about the ECM itself yet. I haven't had time to write
> the profiling tests yet.
I fully trust your judgement on this.
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Possible points of scalibility issues
Posted by Berin Loritsch <bl...@apache.org>.
Stefano Mazzocchi wrote:
> Berin Loritsch wrote:
>
>>I have been performing some performance analysis of the Avalon Excalibur code, and I discovered
>>some serious points of thread contention. In a web environment, there can be as many as 150
>>threads (or more) if the web server uses simple thread per connection technology
>>(most common aproach for Java based servers like Tomcat). I expanded the Profile tests to work
>>using 100 threads. The default pooling implementation has some serious slowdown due to thread
>>contention.
>>
>
> Berin, I spent several *months* finetuning the thread-pool code that was
> used in Apache JServ 1.0 (and is still there, used in both IBM WebSphere
> and Oracle Application Server). I remember that I got a report from
> Alcatel that they were using it with loads up to 12000 req/sec.
:) And how many *threads* running simultaneously?
Remember, Requests/Second != cuncurrent threads
Apache HTTPD in it's default configuration will run up to 150 processess,
and when there are more connections than processess, it will make further
requests wait.
12000 req/sec translates to 12.5 ms/request with 150 processes/threads.
It also translates to 41.6ms/request with 500 processes/threads.
(In another mail I will post my formula for scaling requirements)
> Guess that if you need something clean and scalable, that's the way to
> go :)
>
> [I proposed this a while back but you and Peter tought it was too
> complex.... but I still like mine more :) but I'm biased]
( Aren't we all ;P )
I think the implementations we have now fit that bill--clean and scalable.
The question remains about the ECM itself yet. I haven't had time to write
the profiling tests yet.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Possible points of scalibility issues
Posted by Stefano Mazzocchi <st...@apache.org>.
Berin Loritsch wrote:
>
> I have been performing some performance analysis of the Avalon Excalibur code, and I discovered
> some serious points of thread contention. In a web environment, there can be as many as 150
> threads (or more) if the web server uses simple thread per connection technology
> (most common aproach for Java based servers like Tomcat). I expanded the Profile tests to work
> using 100 threads. The default pooling implementation has some serious slowdown due to thread
> contention.
Berin, I spent several *months* finetuning the thread-pool code that was
used in Apache JServ 1.0 (and is still there, used in both IBM WebSphere
and Oracle Application Server). I remember that I got a report from
Alcatel that they were using it with loads up to 12000 req/sec.
Guess that if you need something clean and scalable, that's the way to
go :)
[I proposed this a while back but you and Peter tought it was too
complex.... but I still like mine more :) but I'm biased]
--
Stefano Mazzocchi One must still have chaos in oneself to be
able to give birth to a dancing star.
<st...@apache.org> Friedrich Nietzsche
--------------------------------------------------------------------
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Possible points of scalibility issues
Posted by Berin Loritsch <bl...@apache.org>.
Berin Loritsch wrote:
> giacomo wrote:
>
>> On Wed, 13 Feb 2002, Berin Loritsch wrote:
>>
>> Berin,
>>
>> Thanks giving us background information from Avalon land. It sounds very
>> interesting and I am very curious to see Cocoon performance number with
>> those new or rewritten classes.
>
>
Just as I expected--the ECM causes some serious thread contention issues.
When run with 10 simultaneous threads, the ECM completes in 6.8 seconds.
When I double the number of threads, the ECM completes in over 80 seconds.
2x the number of threads = 12x longer to complete. This isn't good. The
new ContainerManager stuff is experiencing bugs with pooled implementations,
but I will fix it as soon as I can.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Possible points of scalibility issues
Posted by Berin Loritsch <bl...@apache.org>.
giacomo wrote:
> On Wed, 13 Feb 2002, Berin Loritsch wrote:
>
> Berin,
>
> Thanks giving us background information from Avalon land. It sounds very
> interesting and I am very curious to see Cocoon performance number with
> those new or rewritten classes.
I am in the process of writing a Profile test that compares ContainerManager code and
ECM code. One of the coolest things is that the ContainerManager and AbstractContainer
implementations have a constant startup time (i.e. initialization is done asynchronously).
This allows expensive operations like initializing the DataSourceComponent to not affect
startup times.
With only three components in an Xconf file, I am seeing the ECM consistently in the 1.5
second range and ContainerManager in the 600-700 ms range. I haven't gotten to the point
where I pound the ECM and ContainerManager's ComponentManager with 100 threads simultaneously,
performing lookup and release on all three components. Currently, just startup times are
pretty cool.
There are usability issues, so I need to simplify the maintenance of ContainerManager (it
is somewhat of a bear right now), but it absolutely rocks from the client perspective.
>
> Giacomo
>
>
>>I have been performing some performance analysis of the Avalon Excalibur code, and I discovered
>>some serious points of thread contention. In a web environment, there can be as many as 150
>>threads (or more) if the web server uses simple thread per connection technology
>>(most common aproach for Java based servers like Tomcat). I expanded the Profile tests to work
>>using 100 threads. The default pooling implementation has some serious slowdown due to thread
>>contention.
>>
>>A new Avalon committer named Leif Mortenson has created a new Pool implementation called
>>ResourceLimitingPool. This pool implementation is very flexible, and many orders of magnitude
>>more efficient than the current pooling implementations. I have posted a VOTE to promote it
>>from scratchpad to production. When that happens, there will be less thread contention on
>>Pooled components.
>>
>>Another, more serious point of contention is the ExcaliburComponentManager implementation itself.
>>The reason is that there are some *potentially* long stretches of code that are blocking. This
>>is particularly true if the ECM needs to initialize a PoolableComponent. I have tried to make
>>the blocking portions as small as possible, but we still have some work to do.
>>
>>I am also in the process of the new ContainerManager/Container abstraction. The current CVS
>>version is functional (not in Cocoon's CVS, but Avalon's), but the API needs some sprucing up.
>>I also need to put in some Profiling tests. In the new ContainerManager/Container abstraction
>>we make use of a new ManagedPool. The pooling implementation for the managed pool is even more
>>efficient than Leif's excellent ResourceLimitingPool. It is less deterministic as to the
>>precise number of instances available, which the ResourceLimitingPool is better suited for.
>>The alternative FixedSizePool is both ThreadSafe and the fastest implementation.
>>
>>The new ContainerManager/Container abstraction does have a new Role configuration format, and
>>does not pay attention to lifestyle interfaces. It is an experiment to see if we can live without
>>them--and seems to be working well. The RoleManager will match the ComponentHandler implementation
>>to the Component implementation. This is much more flexible, and I believe it will work better
>>in the long run. The default ComponentHandler has been changed from Factory to PerThread (Using
>>a ThreadLocal to manage component instances).
>>
>>I am very interested in making Cocoon much better, and the best way I can do that is to work
>>on the core Avalon stuff. The future will require changes, but the payoff will be worth it.
>>I wanted to point out the hotspots that Cocoon is facing, and why it is having a hard time scaling.
>>
>>In the interim, I would suggest limiting the number of threads that your Servlet container will
>>allow to be used on Cocoon to around 40--but when the core is made better, we won't need to have
>>those limitations.
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
> For additional commands, email: cocoon-dev-help@xml.apache.org
>
> .
>
>
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Possible points of scalibility issues
Posted by giacomo <gi...@apache.org>.
On Wed, 13 Feb 2002, Berin Loritsch wrote:
Berin,
Thanks giving us background information from Avalon land. It sounds very
interesting and I am very curious to see Cocoon performance number with
those new or rewritten classes.
Giacomo
>
> I have been performing some performance analysis of the Avalon Excalibur code, and I discovered
> some serious points of thread contention. In a web environment, there can be as many as 150
> threads (or more) if the web server uses simple thread per connection technology
> (most common aproach for Java based servers like Tomcat). I expanded the Profile tests to work
> using 100 threads. The default pooling implementation has some serious slowdown due to thread
> contention.
>
> A new Avalon committer named Leif Mortenson has created a new Pool implementation called
> ResourceLimitingPool. This pool implementation is very flexible, and many orders of magnitude
> more efficient than the current pooling implementations. I have posted a VOTE to promote it
> from scratchpad to production. When that happens, there will be less thread contention on
> Pooled components.
>
> Another, more serious point of contention is the ExcaliburComponentManager implementation itself.
> The reason is that there are some *potentially* long stretches of code that are blocking. This
> is particularly true if the ECM needs to initialize a PoolableComponent. I have tried to make
> the blocking portions as small as possible, but we still have some work to do.
>
> I am also in the process of the new ContainerManager/Container abstraction. The current CVS
> version is functional (not in Cocoon's CVS, but Avalon's), but the API needs some sprucing up.
> I also need to put in some Profiling tests. In the new ContainerManager/Container abstraction
> we make use of a new ManagedPool. The pooling implementation for the managed pool is even more
> efficient than Leif's excellent ResourceLimitingPool. It is less deterministic as to the
> precise number of instances available, which the ResourceLimitingPool is better suited for.
> The alternative FixedSizePool is both ThreadSafe and the fastest implementation.
>
> The new ContainerManager/Container abstraction does have a new Role configuration format, and
> does not pay attention to lifestyle interfaces. It is an experiment to see if we can live without
> them--and seems to be working well. The RoleManager will match the ComponentHandler implementation
> to the Component implementation. This is much more flexible, and I believe it will work better
> in the long run. The default ComponentHandler has been changed from Factory to PerThread (Using
> a ThreadLocal to manage component instances).
>
> I am very interested in making Cocoon much better, and the best way I can do that is to work
> on the core Avalon stuff. The future will require changes, but the payoff will be worth it.
> I wanted to point out the hotspots that Cocoon is facing, and why it is having a hard time scaling.
>
> In the interim, I would suggest limiting the number of threads that your Servlet container will
> allow to be used on Cocoon to around 40--but when the core is made better, we won't need to have
> those limitations.
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org
Re: Possible points of scalibility issues
Posted by Berin Loritsch <bl...@apache.org>.
Vadim Gritsenko wrote:
> A little bit on another (but connected) topic...
>
>
>>From: Berin Loritsch [mailto:bloritsch@apache.org]
>>
>>I have been performing some performance analysis of the Avalon
>>Excalibur code, and I discovered
>>some serious points of thread contention. In a web environment,
>>there can be as many as 150
>>threads (or more) if the web server uses simple thread per
>>connection technology
>>(most common aproach for Java based servers like Tomcat).
>>I expanded the Profile tests to work
>>using 100 threads. The default pooling implementation has
>>some serious slowdown due to thread contention.
>>
>
> <snip summary="Avalon enhancements"/>
>
>
>>In the interim, I would suggest limiting the number of threads that
>>
> your
>
>>Servlet container will
>>allow to be used on Cocoon to around 40--but when the core is made
>>
> better, we
>
>>won't need to have those limitations.
>>
>
> I see one weak link in this: After you spend lots of time eliminating
> synchronization bottlenecks (which is good!!!), you will get into
> another issue: thread switching is actually done by the operating
> system, and increasing number of threads will result in larger
> performance penalties caused by thread switching in the operating
> system. Next level will be the context switch in the CPU hardware which
> is costly operation.
>
> Better (IMHO) thing to do is to switch to the asynchronous I/O API and
> to use limited number of worker threads, which should give better
> performance then increasing number of threads. Am I correct here?
Absoposolutely!
However, as of right now, there is no Servlet Container that affords this.
I have tested the new pooled objects with 100 threads, and seen some big
improvements. Attached is the log dumps from the profile tests. You can
see the performance increases you can look forward to with just the pools.
--
"They that give up essential liberty to obtain a little temporary safety
deserve neither liberty nor safety."
- Benjamin Franklin
RE: Possible points of scalibility issues
Posted by Vadim Gritsenko <va...@verizon.net>.
A little bit on another (but connected) topic...
> From: Berin Loritsch [mailto:bloritsch@apache.org]
>
> I have been performing some performance analysis of the Avalon
> Excalibur code, and I discovered
> some serious points of thread contention. In a web environment,
> there can be as many as 150
> threads (or more) if the web server uses simple thread per
> connection technology
> (most common aproach for Java based servers like Tomcat).
> I expanded the Profile tests to work
> using 100 threads. The default pooling implementation has
> some serious slowdown due to thread contention.
<snip summary="Avalon enhancements"/>
> In the interim, I would suggest limiting the number of threads that
your
> Servlet container will
> allow to be used on Cocoon to around 40--but when the core is made
better, we
> won't need to have those limitations.
I see one weak link in this: After you spend lots of time eliminating
synchronization bottlenecks (which is good!!!), you will get into
another issue: thread switching is actually done by the operating
system, and increasing number of threads will result in larger
performance penalties caused by thread switching in the operating
system. Next level will be the context switch in the CPU hardware which
is costly operation.
Better (IMHO) thing to do is to switch to the asynchronous I/O API and
to use limited number of worker threads, which should give better
performance then increasing number of threads. Am I correct here?
Vadim
---------------------------------------------------------------------
To unsubscribe, e-mail: cocoon-dev-unsubscribe@xml.apache.org
For additional commands, email: cocoon-dev-help@xml.apache.org