You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Danny Angus <da...@thought.co.uk> on 2002/03/01 10:44:05 UTC
RE: How about update to lastest avalon?
Sounds like a good idea.
I haven't looked at DBCP, could you wrap it and put it into the build so
that a config change would use it instead of mordered for comparative (read
subjective) testing?
d.
> -----Original Message-----
> From: Serge Knystautas [mailto:sergek@lokitech.com]
> Sent: Wednesday, February 27, 2002 2:27 PM
> To: James Developers List
> Subject: Re: How about update to lastest avalon?
>
>
> As JDBC pools go, what about using the DBCP package in commons?
> Seems like
> it's been around quite a while, it's an Apache project, and I
> could quickly
> throw an avalon conf wrapper around it. If I had known about it a few
> months ago, I wouldn't have added in my db pooler code
> (mordred)... and the
> avalon db code isn't as mature as DBCP. Thoughts?
>
> Serge Knystautas
> Loki Technologies - Unstoppable Websites
> http://www.lokitech.com/
> ----- Original Message -----
> From: "Danny Angus" <da...@thought.co.uk>
> To: "James Developers List" <ja...@jakarta.apache.org>
> Sent: Wednesday, February 27, 2002 3:24 AM
> Subject: RE: How about update to lastest avalon?
>
>
> > if thats the version in cvs I'd be happy to make a "nightly" release
> today.
> > is it?
> > d.
> >
> > > -----Original Message-----
> > > From: Eung-ju Park [mailto:colus@apache.org]
> > > Sent: Wednesday, February 27, 2002 4:06 AM
> > > To: james-dev@jakarta.apache.org
> > > Subject: How about update to lastest avalon?
> > >
> > >
> > > Hi.
> > >
> > > Many people want james as daemon(or nt service).
> > > Lastest phoenix support 'wrapper'.
> > > 'wrapper' is daemon wrapper for java program.
> > >
> > > And current excalibur datasource support getConnection with timeout.
> > > see
> org.apache.avalon.excalibur.datasource.ResourceLimitingJdbcDataSource.
> > >
> > > How about update to lastest avalon before next alpha release?
> > >
> > >
> > > --
> > > To unsubscribe, e-mail:
> > > <ma...@jakarta.apache.org>
> > > For additional commands, e-mail:
> > > <ma...@jakarta.apache.org>
> > >
> >
> >
> > --
> > To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> > For additional commands, e-mail:
> <ma...@jakarta.apache.org>
> >
> >
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: How about update to lastest avalon?
Posted by Serge Knystautas <se...@lokitech.com>.
I should be able to... the configuration for the drivers will probably be
different, but perhaps I can write the DBCP wrapper to use most of the same
conf names to avoid confusion. I'll see what I can do.
Serge Knystautas
Loki Technologies - Unstoppable Websites
http://www.lokitech.com/
----- Original Message -----
From: "Danny Angus" <da...@thought.co.uk>
To: "James Developers List" <ja...@jakarta.apache.org>
Sent: Friday, March 01, 2002 4:44 AM
Subject: RE: How about update to lastest avalon?
> Sounds like a good idea.
> I haven't looked at DBCP, could you wrap it and put it into the build so
> that a config change would use it instead of mordered for comparative
(read
> subjective) testing?
> d.
>
> > -----Original Message-----
> > From: Serge Knystautas [mailto:sergek@lokitech.com]
> > Sent: Wednesday, February 27, 2002 2:27 PM
> > To: James Developers List
> > Subject: Re: How about update to lastest avalon?
> >
> >
> > As JDBC pools go, what about using the DBCP package in commons?
> > Seems like
> > it's been around quite a while, it's an Apache project, and I
> > could quickly
> > throw an avalon conf wrapper around it. If I had known about it a few
> > months ago, I wouldn't have added in my db pooler code
> > (mordred)... and the
> > avalon db code isn't as mature as DBCP. Thoughts?
> >
> > Serge Knystautas
> > Loki Technologies - Unstoppable Websites
> > http://www.lokitech.com/
> > ----- Original Message -----
> > From: "Danny Angus" <da...@thought.co.uk>
> > To: "James Developers List" <ja...@jakarta.apache.org>
> > Sent: Wednesday, February 27, 2002 3:24 AM
> > Subject: RE: How about update to lastest avalon?
> >
> >
> > > if thats the version in cvs I'd be happy to make a "nightly" release
> > today.
> > > is it?
> > > d.
> > >
> > > > -----Original Message-----
> > > > From: Eung-ju Park [mailto:colus@apache.org]
> > > > Sent: Wednesday, February 27, 2002 4:06 AM
> > > > To: james-dev@jakarta.apache.org
> > > > Subject: How about update to lastest avalon?
> > > >
> > > >
> > > > Hi.
> > > >
> > > > Many people want james as daemon(or nt service).
> > > > Lastest phoenix support 'wrapper'.
> > > > 'wrapper' is daemon wrapper for java program.
> > > >
> > > > And current excalibur datasource support getConnection with timeout.
> > > > see
> > org.apache.avalon.excalibur.datasource.ResourceLimitingJdbcDataSource.
> > > >
> > > > How about update to lastest avalon before next alpha release?
> > > >
> > > >
> > > > --
> > > > To unsubscribe, e-mail:
> > > > <ma...@jakarta.apache.org>
> > > > For additional commands, e-mail:
> > > > <ma...@jakarta.apache.org>
> > > >
> > >
> > >
> > > --
> > > To unsubscribe, e-mail:
> > <ma...@jakarta.apache.org>
> > > For additional commands, e-mail:
> > <ma...@jakarta.apache.org>
> > >
> > >
> >
> >
> > --
> > To unsubscribe, e-mail:
> > <ma...@jakarta.apache.org>
> > For additional commands, e-mail:
> > <ma...@jakarta.apache.org>
> >
>
>
> --
> To unsubscribe, e-mail:
<ma...@jakarta.apache.org>
> For additional commands, e-mail:
<ma...@jakarta.apache.org>
>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Spool processing (was 'Spool manager on M$ SQL' at James-User)
Posted by Serge Knystautas <se...@lokitech.com>.
Have you tried boosting the number of connections the pool has? Someone
spotted that the default was 2 and you can set a <max> parameter to the pool
to something higher. This might get you much better throughput.
The spool manager configuration shouldn't affect incoming throughput. The
SMTP Handler will grab a db connection and puts a message in the top of the
spool, and separately the spool manager grabs and processes it.
Maybe what you could do is define 2 db pools, have the SMTP manager use one
that has a higher connection limit, and the spool manager use a similarly
configured but different pool with relatively few connections.
I don't know... the problem is these two thread groups (SMTP handler and
spool manager) are fighting over connections in the db pool, and I don't
know any pooling code that supports prioritization of requests for db
connections.
Serge Knystautas
Loki Technologies - Unstoppable Websites
http://www.lokitech.com/
----- Original Message -----
From: "David Shpil" <da...@harmeron.com>
To: "James Developers List" <ja...@jakarta.apache.org>
Sent: Thursday, March 14, 2002 5:16 AM
Subject: Spool processing (was 'Spool manager on M$ SQL' at James-User)
> Guys,
>
> I'm still trying to get spool manager working better under some load.
> As I mentioned before, message rate of more then 4 per sec overloads
> the spool manager and dramaticly increases local delivery time. If the
> message rate remains the same for some time, a local delivery time
> continues to grow.
>
> IMHO, the problem is that Spool Manager grabs all messages available from
> the spool and then processes them. In my case, message rate is relatively
> high
> and it causes the manager to grab more and more messages every time.
>
> So, first thing I would like to achieve is to provide kind of "quality of
> service"
> for local delivery. In other words: I want to have minimal "local delivery
> message rate",
> regardless of "incoming message rate". This might be done by making a
spool
> manager
> to grab limited number of messages ...
>
> Am I going in right direction? If I am, what changes should be done at
> sqlResouces.xml file?
>
> TIA
>
> David.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Re: Spool processing (was 'Spool manager on M$ SQL' at James-User)
Posted by Serge Knystautas <se...@lokitech.com>.
In theory the JDBC spool repository is already doing it as I'm telling the
JDBC driver to only return a certain number of records. Probably not all
JDBC drivers implement this and still return the entire result... what you
should be able to do is override this for your particular database, if there
are known failings in JDBC drivers, e.g.,
<sql name="listMessagesSQL" db="mysql"> what you said with the limit part at
the end</sql>
I'm realizing it's probably confusing to have the mail use listMessagesSQL
and the spool to have a query of the same name... the spool one should be
renamed because it only needs a partial listing, while the mail one needs
all messages.
Serge Knystautas
Loki Technologies - Unstoppable Websites
http://www.lokitech.com/
----- Original Message -----
From: "Danny Angus" <da...@thought.co.uk>
To: "James Developers List" <ja...@jakarta.apache.org>
Sent: Thursday, March 14, 2002 9:21 AM
Subject: RE: Spool processing (was 'Spool manager on M$ SQL' at James-User)
> I think you are on to something here, you could hide messages from the
spool
> manager, try to use a LIMIT in the SQL ..
>
> in this block:
> <sqlDefs name="org.apache.james.mailrepository.JDBCSpoolRepository">
> change this:
> <sql name="listMessagesSQL">SELECT message_name, message_state,
> last_updated FROM ${table} WHERE repository_name = ? ORDER BY last_updated
> ASC</sql>
>
> to this:
> <sql name="listMessagesSQL">SELECT message_name, message_state,
last_updated
> FROM ${table} WHERE repository_name = ? ORDER BY last_updated ASC LIMIT
> 10</sql>
>
> Obviously you can mess with the number at the end until you discover what
> the right number ought to be.
>
> Let us know what happens (it may just all break!)
> d.
>
> > -----Original Message-----
> > From: David Shpil [mailto:david@harmeron.com]
> > Sent: Thursday, March 14, 2002 10:17 AM
> > To: James Developers List
> > Subject: Spool processing (was 'Spool manager on M$ SQL' at James-User)
> >
> >
> > Guys,
> >
> > I'm still trying to get spool manager working better under some load.
> > As I mentioned before, message rate of more then 4 per sec overloads
> > the spool manager and dramaticly increases local delivery time. If the
> > message rate remains the same for some time, a local delivery time
> > continues to grow.
> >
> > IMHO, the problem is that Spool Manager grabs all messages available
from
> > the spool and then processes them. In my case, message rate is
relatively
> > high
> > and it causes the manager to grab more and more messages every time.
> >
> > So, first thing I would like to achieve is to provide kind of "quality
of
> > service"
> > for local delivery. In other words: I want to have minimal "local
delivery
> > message rate",
> > regardless of "incoming message rate". This might be done by
> > making a spool
> > manager
> > to grab limited number of messages ...
> >
> > Am I going in right direction? If I am, what changes should be done at
> > sqlResouces.xml file?
> >
> > TIA
> >
> > David.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
RE: Spool processing (was 'Spool manager on M$ SQL' at James-User)
Posted by Danny Angus <da...@thought.co.uk>.
I think you are on to something here, you could hide messages from the spool
manager, try to use a LIMIT in the SQL ..
in this block:
<sqlDefs name="org.apache.james.mailrepository.JDBCSpoolRepository">
change this:
<sql name="listMessagesSQL">SELECT message_name, message_state,
last_updated FROM ${table} WHERE repository_name = ? ORDER BY last_updated
ASC</sql>
to this:
<sql name="listMessagesSQL">SELECT message_name, message_state, last_updated
FROM ${table} WHERE repository_name = ? ORDER BY last_updated ASC LIMIT
10</sql>
Obviously you can mess with the number at the end until you discover what
the right number ought to be.
Let us know what happens (it may just all break!)
d.
> -----Original Message-----
> From: David Shpil [mailto:david@harmeron.com]
> Sent: Thursday, March 14, 2002 10:17 AM
> To: James Developers List
> Subject: Spool processing (was 'Spool manager on M$ SQL' at James-User)
>
>
> Guys,
>
> I'm still trying to get spool manager working better under some load.
> As I mentioned before, message rate of more then 4 per sec overloads
> the spool manager and dramaticly increases local delivery time. If the
> message rate remains the same for some time, a local delivery time
> continues to grow.
>
> IMHO, the problem is that Spool Manager grabs all messages available from
> the spool and then processes them. In my case, message rate is relatively
> high
> and it causes the manager to grab more and more messages every time.
>
> So, first thing I would like to achieve is to provide kind of "quality of
> service"
> for local delivery. In other words: I want to have minimal "local delivery
> message rate",
> regardless of "incoming message rate". This might be done by
> making a spool
> manager
> to grab limited number of messages ...
>
> Am I going in right direction? If I am, what changes should be done at
> sqlResouces.xml file?
>
> TIA
>
> David.
>
>
>
> --
> To unsubscribe, e-mail:
> <ma...@jakarta.apache.org>
> For additional commands, e-mail:
> <ma...@jakarta.apache.org>
>
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>
Spool processing (was 'Spool manager on M$ SQL' at James-User)
Posted by David Shpil <da...@harmeron.com>.
Guys,
I'm still trying to get spool manager working better under some load.
As I mentioned before, message rate of more then 4 per sec overloads
the spool manager and dramaticly increases local delivery time. If the
message rate remains the same for some time, a local delivery time
continues to grow.
IMHO, the problem is that Spool Manager grabs all messages available from
the spool and then processes them. In my case, message rate is relatively
high
and it causes the manager to grab more and more messages every time.
So, first thing I would like to achieve is to provide kind of "quality of
service"
for local delivery. In other words: I want to have minimal "local delivery
message rate",
regardless of "incoming message rate". This might be done by making a spool
manager
to grab limited number of messages ...
Am I going in right direction? If I am, what changes should be done at
sqlResouces.xml file?
TIA
David.
--
To unsubscribe, e-mail: <ma...@jakarta.apache.org>
For additional commands, e-mail: <ma...@jakarta.apache.org>