You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Danny Angus <da...@apache.org> on 2002/10/13 19:27:30 UTC
Peters patch doesn't actually work...
there shouldn't be thread manager refrences in pop3server.xinfo or remotemanager.xinfo
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Peters patch doesn't actually work...
Posted by "Noel J. Bergman" <no...@devtech.com>.
> This change was already committed. Please consult the CVS.
The only change I'm noticing with respect to a thread configuration is this:
Old: <max-threads>40</max-threads>
New: <max-threads>100</max-threads>
As I understand it, in the old code, all threads were available for any
service that needed any thread. In the new code, the default pool is
available, but the connection manager can be told to limit its use:
<connections>
<idle-timeout>300000</idle-timeout>
<max-connections>30</max-connections>
</connections>
The default is 30 connections, so 60 threads, leaving 40 for everything
else.
One thing I'm seeing is that the Excalibur ThreadPool code is broken as far
as I can read. The chain of events is this:
ThreadManager reads max value from XML
Threadmanager passes max value to DefaultThreadPool
DefaultThreadPool creates SoftResourceLimitingPool, passing max as MIN
SoftResourceLimitingPool uses 2*min as the new max value
The new code is able to use per service thread pools, and perhaps Peter
should implement that change in the configuration. That way we can tune
each service by telling it which thread pool to use. If we don't want a
dedicated pool, we just allow that service to use the default. If a service
is configured to use a specific pool, and told that it is dedicated, it can
do further optimization.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Peters patch doesn't actually work...
Posted by Danny Angus <da...@apache.org>.
> This change was already committed. Please consult the CVS.
I made a full,clean checkout of CVS before I applied your patches.
If it is commited it was used.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Peters patch doesn't actually work...
Posted by "Peter M. Goldstein" <pe...@yahoo.com>.
Danny,
> Ok, fine, but..
> I'd expect any change that purports to address this issue to perform
> better than the existing code under the default configuration, I did a
> completely clean checkout and build, ran the tests, applied your
patch,
> ran the test.
> If I have to make changes to the config to see the improvement perhaps
> your patch should include them.
This change was already committed. Please consult the CVS. The default
configuration uses Avalon poorly in a number of ways, this being the
most obvious. In reality, it would be best if each service used its own
thread pool, but I didn't want to make overly radical changes, so I just
boosted the already too low thread pool count.
--Peter
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Peters patch doesn't actually work...
Posted by Danny Angus <da...@apache.org>.
> I understand your comparison. And that's one of the reasons I asked
> what your thread pool maximum was. Because I've got a different set of
> results, and you clearly didn't grab all the changes at once (witness
> the assembly.xml change).
I applied your patch to a clean checkout of the HEAD.
> So I'm asking you to confirm that you've got
> a valid thread pool maximum. Obviously the Watchdog approach needs more
> threads that the other system - it was designed that way and the
> configuration modified. If it's still at the earlier default of 40,
> then the Watchdog method won't allow 20 connections. Period.
Ok, fine, but..
I'd expect any change that purports to address this issue to perform better than the existing code under the default configuration, I did a completely clean checkout and build, ran the tests, applied your patch, ran the test.
If I have to make changes to the config to see the improvement perhaps your patch should include them.
d.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Peters patch doesn't actually work...
Posted by "Noel J. Bergman" <no...@devtech.com>.
Danny,
There were some coding defects in Peter's earlier submission. He's running
revised code now. I suppose that he could post his revisions, but he's
still analyzing 12 hours worth of logs.
--- Noel
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Peters patch doesn't actually work...
Posted by "Peter M. Goldstein" <pe...@yahoo.com>.
Danny,
> Peter, I've got two sets of results, one from James before I applied
your
> patch, the other after it all other factors remian the same, thats the
> only comparison I'm making.
>
> in previous tests I consistently ran 10,000 1k mails through the
spooler
> at slightly over 14 mails per second. (using a single connection per
100
> mails)
Ok.
I understand your comparison. And that's one of the reasons I asked
what your thread pool maximum was. Because I've got a different set of
results, and you clearly didn't grab all the changes at once (witness
the assembly.xml change). So I'm asking you to confirm that you've got
a valid thread pool maximum. Obviously the Watchdog approach needs more
threads that the other system - it was designed that way and the
configuration modified. If it's still at the earlier default of 40,
then the Watchdog method won't allow 20 connections. Period.
--Peter
>
> d.
>
> > -----Original Message-----
> > From: Peter M. Goldstein [mailto:peter_m_goldstein@yahoo.com]
> > Sent: 13 October 2002 19:37
> > To: 'James Developers List'
> > Subject: RE: Peters patch doesn't actually work...
> >
> >
> >
> > Danny,
> >
> > > sorry, thats the wrong way round its assmebly.xml that needs
twiddled.
> > >
> > > However, applying the same SMTP test, this time 2000 1k mails from
20
> > > threads, Peter's changes resulted in more refused connections, 1st
> > after
> > > only 16 connections have been attempted and with no more at all
after
> > 361
> > > mails have been delivered.
> > >
> > > I haven't looked at the code really, but my empirical view is that
> > SMTP
> > > suffers from these changes.
> >
> > I've been running these tests all night with a slightly tweaked
version
> > of the patch I submitted (there was an erroneous notify() call in
that
> > patch). It runs fine under load, although there do seem to be some
> > problems with the spooler if its put under consistent load. It's
not
> > clear what the relationship is to the handler changes. We've never
> > driven the spooler under serious load, so we have no idea what load
it
> > can take. I'm looking into this.
> >
> > Might I suggest that you check your configuration. My first guess
would
> > be that you didn't correctly bump up the maximum number of threads
in
> > your server's thread pool. The specified number of 40 would be too
low
> > for your test, since 20 connections would consume 40 threads alone,
not
> > to mention that all the other blocks use at least one thread apiece.
> >
> > --Peter
> >
> >
> >
> >
> >
> >
> >
> > --
> > To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail: <mailto:james-dev-
> help@jakarta.apache.org>
>
>
> --
> To unsubscribe, e-mail: <mailto:james-dev-
> unsubscribe@jakarta.apache.org>
> For additional commands, e-mail: <mailto:james-dev-
> help@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Peters patch doesn't actually work...
Posted by Danny Angus <da...@apache.org>.
Peter, I've got two sets of results, one from James before I applied your patch, the other after it all other factors remian the same, thats the only comparison I'm making.
in previous tests I consistently ran 10,000 1k mails through the spooler at slightly over 14 mails per second. (using a single connection per 100 mails)
d.
> -----Original Message-----
> From: Peter M. Goldstein [mailto:peter_m_goldstein@yahoo.com]
> Sent: 13 October 2002 19:37
> To: 'James Developers List'
> Subject: RE: Peters patch doesn't actually work...
>
>
>
> Danny,
>
> > sorry, thats the wrong way round its assmebly.xml that needs twiddled.
> >
> > However, applying the same SMTP test, this time 2000 1k mails from 20
> > threads, Peter's changes resulted in more refused connections, 1st
> after
> > only 16 connections have been attempted and with no more at all after
> 361
> > mails have been delivered.
> >
> > I haven't looked at the code really, but my empirical view is that
> SMTP
> > suffers from these changes.
>
> I've been running these tests all night with a slightly tweaked version
> of the patch I submitted (there was an erroneous notify() call in that
> patch). It runs fine under load, although there do seem to be some
> problems with the spooler if its put under consistent load. It's not
> clear what the relationship is to the handler changes. We've never
> driven the spooler under serious load, so we have no idea what load it
> can take. I'm looking into this.
>
> Might I suggest that you check your configuration. My first guess would
> be that you didn't correctly bump up the maximum number of threads in
> your server's thread pool. The specified number of 40 would be too low
> for your test, since 20 connections would consume 40 threads alone, not
> to mention that all the other blocks use at least one thread apiece.
>
> --Peter
>
>
>
>
>
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Peters patch doesn't actually work...
Posted by "Peter M. Goldstein" <pe...@yahoo.com>.
Danny,
> sorry, thats the wrong way round its assmebly.xml that needs twiddled.
>
> However, applying the same SMTP test, this time 2000 1k mails from 20
> threads, Peter's changes resulted in more refused connections, 1st
after
> only 16 connections have been attempted and with no more at all after
361
> mails have been delivered.
>
> I haven't looked at the code really, but my empirical view is that
SMTP
> suffers from these changes.
I've been running these tests all night with a slightly tweaked version
of the patch I submitted (there was an erroneous notify() call in that
patch). It runs fine under load, although there do seem to be some
problems with the spooler if its put under consistent load. It's not
clear what the relationship is to the handler changes. We've never
driven the spooler under serious load, so we have no idea what load it
can take. I'm looking into this.
Might I suggest that you check your configuration. My first guess would
be that you didn't correctly bump up the maximum number of threads in
your server's thread pool. The specified number of 40 would be too low
for your test, since 20 connections would consume 40 threads alone, not
to mention that all the other blocks use at least one thread apiece.
--Peter
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Peters patch doesn't actually work...
Posted by Danny Angus <da...@apache.org>.
sorry, thats the wrong way round its assmebly.xml that needs twiddled.
However, applying the same SMTP test, this time 2000 1k mails from 20 threads, Peter's changes resulted in more refused connections, 1st after only 16 connections have been attempted and with no more at all after 361 mails have been delivered.
I haven't looked at the code really, but my empirical view is that SMTP suffers from these changes.
d.
> -----Original Message-----
> From: Danny Angus [mailto:danny@apache.org]
> Sent: 13 October 2002 18:28
> To: James Developers List
> Subject: Peters patch doesn't actually work...
>
>
> there shouldn't be thread manager refrences in pop3server.xinfo
> or remotemanager.xinfo
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>