You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-user@james.apache.org by Charles Benett <ch...@benett1.demon.co.uk> on 2001/07/31 13:15:17 UTC
Input for JDB Mail Repository
In cvs proposal dir is Serge's JDBCMailRepository and
JDBCSpoolRepository.
Serge uses Mssql server with Inet Sprinta (I think) and I have got it
going with MySQL and the mm driver.
This is intended to replace the town db classses in James 1.2.1
At the moment, they new classes need to be re-compiled for different
drivers and dbs.
I want to tidy up the config and move them into the main tree.
So I am looking for input from developers/ users.
1) At the moment the JDBCMailRepository (like the town repository) puts
all messages in one db table.
a) Would it be useful to allows different tables, e.g. one for spam
another for errors etc.?
b) If one had multiple tables, how likely is it that one would have them
in different dbs or on different machines?
c) If one had multiple tables, would we still want all POP3 inboxes to
be in one table or would per-user tables make sense?
Any other suggestions re storing mail in a db would be welcome.
Charles
---------------------------------------------------------------------
To unsubscribe, e-mail: james-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-user-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by "M.P. Willems" <m....@chello.nl>.
Sorry didn't catch the other post.
I also wouldn't recommend a db filesystem combination in such a dynamic
environment as a message reposetory, although in principle it's the same as
splitting the table. ....
Myselve is using a progress v91b and there will be some minor issue
concerning indexing.
----- Original Message -----
From: "Serge Knystautas" <se...@lokitech.com>
To: <ja...@jakarta.apache.org>
Sent: Tuesday, July 31, 2001 5:10 PM
Subject: Re: Input for JDB Mail Repository
> I don't know if you saw my other post, but I've changed (but not
committed)
> the JDBC repository to store the message body separately in the file
system
> (using Avalon's stream repository).
>
> I don't know if putting the message body into another table (put still in
> the DB) is going to help vs just putting it in a DB. I guess I'm used to
a
> DB that does the indexing well enough that the size of the record doesn't
> matter so much, although I do recognize sticking the entire message in the
> same table as everything else somewhat violates traditional DB design.
>
> Serge Knystautas
> Loki Technologies
> http://www.lokitech.com/
> ----- Original Message -----
> From: "M.P. Willems" <m....@chello.nl>
> To: <ja...@jakarta.apache.org>
> Sent: Tuesday, July 31, 2001 11:01 AM
> Subject: Re: Input for JDB Mail Repository
>
>
> > ----- Original Message -----
> > From: "Charles Benett" <ch...@benett1.demon.co.uk>
> > To: "James Dev" <ja...@jakarta.apache.org>; "Jakarta-James User"
> > <ja...@jakarta.apache.org>
> > Sent: Tuesday, July 31, 2001 1:15 PM
> > Subject: Input for JDB Mail Repository
> >
> >
> > > In cvs proposal dir is Serge's JDBCMailRepository and
> > > JDBCSpoolRepository.
> > > Serge uses Mssql server with Inet Sprinta (I think) and I have got it
> > > going with MySQL and the mm driver.
> > > This is intended to replace the town db classses in James 1.2.1
> > >
> > > At the moment, they new classes need to be re-compiled for different
> > > drivers and dbs.
> > > I want to tidy up the config and move them into the main tree.
> > > So I am looking for input from developers/ users.
> > >
> > > 1) At the moment the JDBCMailRepository (like the town repository)
puts
> > > all messages in one db table.
> > > a) Would it be useful to allows different tables, e.g. one for spam
> > > another for errors etc.?
> > > b) If one had multiple tables, how likely is it that one would have
them
> > > in different dbs or on different machines?
> > > c) If one had multiple tables, would we still want all POP3 inboxes to
> > > be in one table or would per-user tables make sense?
> > >
> > > Any other suggestions re storing mail in a db would be welcome.
> >
> >
> > Other suggestion:
> > Maybe there is a argument for changing the message table structure, the
> > records in the table "message" will be allowed to become bulky as a
result
> > of the defenition of the attribute "message body" , thus affecting
> > performance and scalabillity.
> > Maybe it's usefull to give some thoughts on putting those message body's
> > into a seperate table , in order to improve performance, operational
> > maintanance and if desirable allowing the assignment of storage area's
to
> > the table's.
> >
> >
> > > Charles
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> > > For additional commands, e-mail: james-dev-help@jakarta.apache.org
> > >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: james-dev-help@jakarta.apache.org
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by Oki DZ <ok...@pindad.com>.
On Tue, 31 Jul 2001, Serge Knystautas wrote:
> I don't know if putting the message body into another table (put still in
> the DB) is going to help vs just putting it in a DB.
It won't make retrieval any faster, IMO.
And on databases that don't support trigger, there would be some
unnecessary additional complexities in the SQL statements; you'd have to
deal with more than one table for each PreparedStatement.ExecuteQuery().
>I guess I'm used to a
> DB that does the indexing well enough that the size of the record doesn't
> matter so much, although I do recognize sticking the entire message in the
> same table as everything else somewhat violates traditional DB design.
I'm wondering whether having varchar columns would slow down db retrieval.
AFAIK, fixed-width columns would be much better for indexes; but thing
is, ResultSet.getString() would get the whole length when retrieving such
columns; then you'd have to have .trim() for the returned strings. (At
least, James didn't work last time I changed the key column to be char
type. I guess, that's where the problem came from.)
But if code-rewrite permits, I think having fixed-width columns for key
columns would be nicer. In Message table, there is only one key and the
only changes needed is only in JDBCMailRepository.retrieve(), right?.
BTW, I'd like to have a limited POP connection for each user that connects
to the server. Currently, James 1.2 is having problem due to the users
that tries to have more than one connection; the server seems to be hung,
and the only remedy is restarting James.
I think all you need is to have a Hashtable in POP3Handler, put the
connecting remote IP number into the hashtable, and remove the number when
the user disconnects from the server. But then, the contents of the
hashtable should be timed (removed after a certain time has elapsed), so
that those who inadvertently disconnected from the server (eg: due to
network errors) should be able to connect next time. TurbineGlobalCache
comes into mind... Turbine's CachedObject does time out. This feature of
the POP server would have a nice side effect; limiting the user from
attempting to connect to the server too many times in a period of time. I
think, checking the mailbox every 30 seconds is a bit too much; but people
did.
BTW2, contrary to what I have written in this list, TurbineGlobalCache is
not the only single global cache you can have in Turbine. You can have
more by having more cache service running. Sure, you'd need to write a new
class to get the service available (it could be a subclass of
TurbineGlobalCache, with getService() overriden with the one that "gets" a
different service name; which is newly defined by you in Turbine's
properties file). It might not be relevant to write about Turbine in this
list, but I'd like to clear things up.
Oki
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by Serge Knystautas <se...@lokitech.com>.
I don't know if you saw my other post, but I've changed (but not committed)
the JDBC repository to store the message body separately in the file system
(using Avalon's stream repository).
I don't know if putting the message body into another table (put still in
the DB) is going to help vs just putting it in a DB. I guess I'm used to a
DB that does the indexing well enough that the size of the record doesn't
matter so much, although I do recognize sticking the entire message in the
same table as everything else somewhat violates traditional DB design.
Serge Knystautas
Loki Technologies
http://www.lokitech.com/
----- Original Message -----
From: "M.P. Willems" <m....@chello.nl>
To: <ja...@jakarta.apache.org>
Sent: Tuesday, July 31, 2001 11:01 AM
Subject: Re: Input for JDB Mail Repository
> ----- Original Message -----
> From: "Charles Benett" <ch...@benett1.demon.co.uk>
> To: "James Dev" <ja...@jakarta.apache.org>; "Jakarta-James User"
> <ja...@jakarta.apache.org>
> Sent: Tuesday, July 31, 2001 1:15 PM
> Subject: Input for JDB Mail Repository
>
>
> > In cvs proposal dir is Serge's JDBCMailRepository and
> > JDBCSpoolRepository.
> > Serge uses Mssql server with Inet Sprinta (I think) and I have got it
> > going with MySQL and the mm driver.
> > This is intended to replace the town db classses in James 1.2.1
> >
> > At the moment, they new classes need to be re-compiled for different
> > drivers and dbs.
> > I want to tidy up the config and move them into the main tree.
> > So I am looking for input from developers/ users.
> >
> > 1) At the moment the JDBCMailRepository (like the town repository) puts
> > all messages in one db table.
> > a) Would it be useful to allows different tables, e.g. one for spam
> > another for errors etc.?
> > b) If one had multiple tables, how likely is it that one would have them
> > in different dbs or on different machines?
> > c) If one had multiple tables, would we still want all POP3 inboxes to
> > be in one table or would per-user tables make sense?
> >
> > Any other suggestions re storing mail in a db would be welcome.
>
>
> Other suggestion:
> Maybe there is a argument for changing the message table structure, the
> records in the table "message" will be allowed to become bulky as a result
> of the defenition of the attribute "message body" , thus affecting
> performance and scalabillity.
> Maybe it's usefull to give some thoughts on putting those message body's
> into a seperate table , in order to improve performance, operational
> maintanance and if desirable allowing the assignment of storage area's to
> the table's.
>
>
> > Charles
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: james-dev-help@jakarta.apache.org
> >
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by "M.P. Willems" <m....@chello.nl>.
----- Original Message -----
From: "Charles Benett" <ch...@benett1.demon.co.uk>
To: "James Dev" <ja...@jakarta.apache.org>; "Jakarta-James User"
<ja...@jakarta.apache.org>
Sent: Tuesday, July 31, 2001 1:15 PM
Subject: Input for JDB Mail Repository
> In cvs proposal dir is Serge's JDBCMailRepository and
> JDBCSpoolRepository.
> Serge uses Mssql server with Inet Sprinta (I think) and I have got it
> going with MySQL and the mm driver.
> This is intended to replace the town db classses in James 1.2.1
>
> At the moment, they new classes need to be re-compiled for different
> drivers and dbs.
> I want to tidy up the config and move them into the main tree.
> So I am looking for input from developers/ users.
>
> 1) At the moment the JDBCMailRepository (like the town repository) puts
> all messages in one db table.
> a) Would it be useful to allows different tables, e.g. one for spam
> another for errors etc.?
> b) If one had multiple tables, how likely is it that one would have them
> in different dbs or on different machines?
> c) If one had multiple tables, would we still want all POP3 inboxes to
> be in one table or would per-user tables make sense?
>
> Any other suggestions re storing mail in a db would be welcome.
Other suggestion:
Maybe there is a argument for changing the message table structure, the
records in the table "message" will be allowed to become bulky as a result
of the defenition of the attribute "message body" , thus affecting
performance and scalabillity.
Maybe it's usefull to give some thoughts on putting those message body's
into a seperate table , in order to improve performance, operational
maintanance and if desirable allowing the assignment of storage area's to
the table's.
> Charles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by Peter Donald <do...@apache.org>.
On Tue, 31 Jul 2001 21:15, Charles Benett wrote:
> In cvs proposal dir is Serge's JDBCMailRepository and
> JDBCSpoolRepository.
> Serge uses Mssql server with Inet Sprinta (I think) and I have got it
> going with MySQL and the mm driver.
> This is intended to replace the town db classses in James 1.2.1
>
> At the moment, they new classes need to be re-compiled for different
> drivers and dbs.
> I want to tidy up the config and move them into the main tree.
> So I am looking for input from developers/ users.
+1 ;)
> 1) At the moment the JDBCMailRepository (like the town repository) puts
> all messages in one db table.
> a) Would it be useful to allows different tables, e.g. one for spam
> another for errors etc.?
I would have one for spool mail, one for spool mail envelope, and one for local deliveries.
> b) If one had multiple tables, how likely is it that one would have them
> in different dbs or on different machines?
highly unlikely I would think ;)
> c) If one had multiple tables, would we still want all POP3 inboxes to
> be in one table or would per-user tables make sense?
per-user tables would be a nightmare ! ;)
Think what would happen to your DB if you had 300 users ;)
> Any other suggestions re storing mail in a db would be welcome.
>
> Charles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
--
Cheers,
Pete
*-----------------------------------------------------*
| "Faced with the choice between changing one's mind, |
| and proving that there is no need to do so - almost |
| everyone gets busy on the proof." |
| - John Kenneth Galbraith |
*-----------------------------------------------------*
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by Serge Knystautas <se...@lokitech.com>.
Yes, I very much would like to get a major release done soon. The result of
this morning's debugging led me to add saveChanges() during the processing,
and I think I just have to figure out how to work with the new CVS server to
commit. I've also written a "AddFooter" mailet to put a footer on a
message. I'd also like to remove other Town dependencies which are in a few
mailets.
Is there a good way we can define database sources via Avalon? I know
excalibur could do it in theory, but I don't know how. If we could do this,
it would simplify removing these Town dependencies and simplify the
configuration of the repositories.
Serge Knystautas
Loki Technologies
http://www.lokitech.com/
----- Original Message -----
From: "Charles Benett" <ch...@benett1.demon.co.uk>
To: <ja...@jakarta.apache.org>
Cc: "Jakarta-James User" <ja...@jakarta.apache.org>
Sent: Tuesday, July 31, 2001 10:00 AM
Subject: Re: Input for JDB Mail Repository
> Serge Knystautas wrote:
> >
> > Charles,
> >
> > This is a good idea to get feedback, but I've actually taken a stab at
this
> > (<rant>actually I was debugging a freakin' annoying "feature" of
JavaMail...
> > if you set new content on a message, you have to call "saveChanges()"
for it
> > to take effect. what a brilliant design. I think I might change James
so
> > it automatically calls saveChanges after a mailet makes a change as I
think
> > the SMTP Transport engine Sun wrote already is doing this. </rant>).
> >
> > Anyway, here's what I did. If you configure db://conf/spool, this will
look
> > for a configuration file at conf/spool (this should be a file, not a
> > directory). This configuration file contains necessary JDBC connection
> > stuff (driver, URL, username, password), table name, repository name,
all
> > the SQL used by the JDBCMailRepository, and a directory for message
bodies
> > (more later).
> >
> > Right now it's just reading it in as a Properties object (name=value),
but
> > this could be changed to XML (then with includes, you could keep all the
> > JDBC and SQL in one shared XML file, and each conf file would only set
the
> > repository name (and maybe table name). I'm not incredibly pleased with
> > this design as I think I'd rather this all be in the avalon loaded conf
> > files, but what can you do?
> >
> > Two other notes:
> > - I still haven't gotten excalibur (or any JDBC pooler) integrated yet.
> > >From what I've seen of excalibur, it looks like I have to let avalon do
the
> > configuration, so I'm not sure how to back-fit my conf file into an
> > excalibur.
> > - I've written the JDBCMailRepository to store message bodies in the
file
> > system. Even after all this work to not parse messages, a large message
(a
> > meg or more) would still bring my JDBC driver to a crawl. Now, the
> > JDBCMailRepository will store all delivery information and the message's
> > headers to the database, and the message body separately to the file
system.
> > It's hopefully written in such a way that you could turn this off and
on, as
> > it should already read a message that is entirely stored in the database
and
> > so you'd just have to change how it saves messages.
> >
> > Anyway, I was still testing and hadn't committed anything yet. It looks
> > pretty stable, but I thought this morning's error was something
seriously
> > wrong with the parsing-avoidance code.
>
> Great! I'm quite happy to leave it alone if you're making progress.
> My not-so-hidden agenda is to get your dbmail stuff finished and into
> the main tree, update to the latest Avalon stuff (the Avalon Framework
> bits had a full release yesturday) and start thinking about a release of
> James.
>
> Charles
---------------------------------------------------------------------
To unsubscribe, e-mail: james-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-user-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by Serge Knystautas <se...@lokitech.com>.
Yes, I very much would like to get a major release done soon. The result of
this morning's debugging led me to add saveChanges() during the processing,
and I think I just have to figure out how to work with the new CVS server to
commit. I've also written a "AddFooter" mailet to put a footer on a
message. I'd also like to remove other Town dependencies which are in a few
mailets.
Is there a good way we can define database sources via Avalon? I know
excalibur could do it in theory, but I don't know how. If we could do this,
it would simplify removing these Town dependencies and simplify the
configuration of the repositories.
Serge Knystautas
Loki Technologies
http://www.lokitech.com/
----- Original Message -----
From: "Charles Benett" <ch...@benett1.demon.co.uk>
To: <ja...@jakarta.apache.org>
Cc: "Jakarta-James User" <ja...@jakarta.apache.org>
Sent: Tuesday, July 31, 2001 10:00 AM
Subject: Re: Input for JDB Mail Repository
> Serge Knystautas wrote:
> >
> > Charles,
> >
> > This is a good idea to get feedback, but I've actually taken a stab at
this
> > (<rant>actually I was debugging a freakin' annoying "feature" of
JavaMail...
> > if you set new content on a message, you have to call "saveChanges()"
for it
> > to take effect. what a brilliant design. I think I might change James
so
> > it automatically calls saveChanges after a mailet makes a change as I
think
> > the SMTP Transport engine Sun wrote already is doing this. </rant>).
> >
> > Anyway, here's what I did. If you configure db://conf/spool, this will
look
> > for a configuration file at conf/spool (this should be a file, not a
> > directory). This configuration file contains necessary JDBC connection
> > stuff (driver, URL, username, password), table name, repository name,
all
> > the SQL used by the JDBCMailRepository, and a directory for message
bodies
> > (more later).
> >
> > Right now it's just reading it in as a Properties object (name=value),
but
> > this could be changed to XML (then with includes, you could keep all the
> > JDBC and SQL in one shared XML file, and each conf file would only set
the
> > repository name (and maybe table name). I'm not incredibly pleased with
> > this design as I think I'd rather this all be in the avalon loaded conf
> > files, but what can you do?
> >
> > Two other notes:
> > - I still haven't gotten excalibur (or any JDBC pooler) integrated yet.
> > >From what I've seen of excalibur, it looks like I have to let avalon do
the
> > configuration, so I'm not sure how to back-fit my conf file into an
> > excalibur.
> > - I've written the JDBCMailRepository to store message bodies in the
file
> > system. Even after all this work to not parse messages, a large message
(a
> > meg or more) would still bring my JDBC driver to a crawl. Now, the
> > JDBCMailRepository will store all delivery information and the message's
> > headers to the database, and the message body separately to the file
system.
> > It's hopefully written in such a way that you could turn this off and
on, as
> > it should already read a message that is entirely stored in the database
and
> > so you'd just have to change how it saves messages.
> >
> > Anyway, I was still testing and hadn't committed anything yet. It looks
> > pretty stable, but I thought this morning's error was something
seriously
> > wrong with the parsing-avoidance code.
>
> Great! I'm quite happy to leave it alone if you're making progress.
> My not-so-hidden agenda is to get your dbmail stuff finished and into
> the main tree, update to the latest Avalon stuff (the Avalon Framework
> bits had a full release yesturday) and start thinking about a release of
> James.
>
> Charles
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by Charles Benett <ch...@benett1.demon.co.uk>.
Serge Knystautas wrote:
>
> Charles,
>
> This is a good idea to get feedback, but I've actually taken a stab at this
> (<rant>actually I was debugging a freakin' annoying "feature" of JavaMail...
> if you set new content on a message, you have to call "saveChanges()" for it
> to take effect. what a brilliant design. I think I might change James so
> it automatically calls saveChanges after a mailet makes a change as I think
> the SMTP Transport engine Sun wrote already is doing this. </rant>).
>
> Anyway, here's what I did. If you configure db://conf/spool, this will look
> for a configuration file at conf/spool (this should be a file, not a
> directory). This configuration file contains necessary JDBC connection
> stuff (driver, URL, username, password), table name, repository name, all
> the SQL used by the JDBCMailRepository, and a directory for message bodies
> (more later).
>
> Right now it's just reading it in as a Properties object (name=value), but
> this could be changed to XML (then with includes, you could keep all the
> JDBC and SQL in one shared XML file, and each conf file would only set the
> repository name (and maybe table name). I'm not incredibly pleased with
> this design as I think I'd rather this all be in the avalon loaded conf
> files, but what can you do?
>
> Two other notes:
> - I still haven't gotten excalibur (or any JDBC pooler) integrated yet.
> >From what I've seen of excalibur, it looks like I have to let avalon do the
> configuration, so I'm not sure how to back-fit my conf file into an
> excalibur.
> - I've written the JDBCMailRepository to store message bodies in the file
> system. Even after all this work to not parse messages, a large message (a
> meg or more) would still bring my JDBC driver to a crawl. Now, the
> JDBCMailRepository will store all delivery information and the message's
> headers to the database, and the message body separately to the file system.
> It's hopefully written in such a way that you could turn this off and on, as
> it should already read a message that is entirely stored in the database and
> so you'd just have to change how it saves messages.
>
> Anyway, I was still testing and hadn't committed anything yet. It looks
> pretty stable, but I thought this morning's error was something seriously
> wrong with the parsing-avoidance code.
Great! I'm quite happy to leave it alone if you're making progress.
My not-so-hidden agenda is to get your dbmail stuff finished and into
the main tree, update to the latest Avalon stuff (the Avalon Framework
bits had a full release yesturday) and start thinking about a release of
James.
Charles
>
> Serge Knystautas
> Loki Technologies
> http://www.lokitech.com/
> ----- Original Message -----
> From: "Charles Benett" <ch...@benett1.demon.co.uk>
> To: "James Dev" <ja...@jakarta.apache.org>; "Jakarta-James User"
> <ja...@jakarta.apache.org>
> Sent: Tuesday, July 31, 2001 7:15 AM
> Subject: Input for JDB Mail Repository
>
> > In cvs proposal dir is Serge's JDBCMailRepository and
> > JDBCSpoolRepository.
> > Serge uses Mssql server with Inet Sprinta (I think) and I have got it
> > going with MySQL and the mm driver.
> > This is intended to replace the town db classses in James 1.2.1
> >
> > At the moment, they new classes need to be re-compiled for different
> > drivers and dbs.
> > I want to tidy up the config and move them into the main tree.
> > So I am looking for input from developers/ users.
> >
> > 1) At the moment the JDBCMailRepository (like the town repository) puts
> > all messages in one db table.
> > a) Would it be useful to allows different tables, e.g. one for spam
> > another for errors etc.?
> > b) If one had multiple tables, how likely is it that one would have them
> > in different dbs or on different machines?
> > c) If one had multiple tables, would we still want all POP3 inboxes to
> > be in one table or would per-user tables make sense?
> >
> > Any other suggestions re storing mail in a db would be welcome.
> >
> > Charles
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: james-dev-help@jakarta.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by Charles Benett <ch...@benett1.demon.co.uk>.
Serge Knystautas wrote:
>
> Charles,
>
> This is a good idea to get feedback, but I've actually taken a stab at this
> (<rant>actually I was debugging a freakin' annoying "feature" of JavaMail...
> if you set new content on a message, you have to call "saveChanges()" for it
> to take effect. what a brilliant design. I think I might change James so
> it automatically calls saveChanges after a mailet makes a change as I think
> the SMTP Transport engine Sun wrote already is doing this. </rant>).
>
> Anyway, here's what I did. If you configure db://conf/spool, this will look
> for a configuration file at conf/spool (this should be a file, not a
> directory). This configuration file contains necessary JDBC connection
> stuff (driver, URL, username, password), table name, repository name, all
> the SQL used by the JDBCMailRepository, and a directory for message bodies
> (more later).
>
> Right now it's just reading it in as a Properties object (name=value), but
> this could be changed to XML (then with includes, you could keep all the
> JDBC and SQL in one shared XML file, and each conf file would only set the
> repository name (and maybe table name). I'm not incredibly pleased with
> this design as I think I'd rather this all be in the avalon loaded conf
> files, but what can you do?
>
> Two other notes:
> - I still haven't gotten excalibur (or any JDBC pooler) integrated yet.
> >From what I've seen of excalibur, it looks like I have to let avalon do the
> configuration, so I'm not sure how to back-fit my conf file into an
> excalibur.
> - I've written the JDBCMailRepository to store message bodies in the file
> system. Even after all this work to not parse messages, a large message (a
> meg or more) would still bring my JDBC driver to a crawl. Now, the
> JDBCMailRepository will store all delivery information and the message's
> headers to the database, and the message body separately to the file system.
> It's hopefully written in such a way that you could turn this off and on, as
> it should already read a message that is entirely stored in the database and
> so you'd just have to change how it saves messages.
>
> Anyway, I was still testing and hadn't committed anything yet. It looks
> pretty stable, but I thought this morning's error was something seriously
> wrong with the parsing-avoidance code.
Great! I'm quite happy to leave it alone if you're making progress.
My not-so-hidden agenda is to get your dbmail stuff finished and into
the main tree, update to the latest Avalon stuff (the Avalon Framework
bits had a full release yesturday) and start thinking about a release of
James.
Charles
>
> Serge Knystautas
> Loki Technologies
> http://www.lokitech.com/
> ----- Original Message -----
> From: "Charles Benett" <ch...@benett1.demon.co.uk>
> To: "James Dev" <ja...@jakarta.apache.org>; "Jakarta-James User"
> <ja...@jakarta.apache.org>
> Sent: Tuesday, July 31, 2001 7:15 AM
> Subject: Input for JDB Mail Repository
>
> > In cvs proposal dir is Serge's JDBCMailRepository and
> > JDBCSpoolRepository.
> > Serge uses Mssql server with Inet Sprinta (I think) and I have got it
> > going with MySQL and the mm driver.
> > This is intended to replace the town db classses in James 1.2.1
> >
> > At the moment, they new classes need to be re-compiled for different
> > drivers and dbs.
> > I want to tidy up the config and move them into the main tree.
> > So I am looking for input from developers/ users.
> >
> > 1) At the moment the JDBCMailRepository (like the town repository) puts
> > all messages in one db table.
> > a) Would it be useful to allows different tables, e.g. one for spam
> > another for errors etc.?
> > b) If one had multiple tables, how likely is it that one would have them
> > in different dbs or on different machines?
> > c) If one had multiple tables, would we still want all POP3 inboxes to
> > be in one table or would per-user tables make sense?
> >
> > Any other suggestions re storing mail in a db would be welcome.
> >
> > Charles
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> > For additional commands, e-mail: james-dev-help@jakarta.apache.org
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: james-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-user-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by Serge Knystautas <se...@lokitech.com>.
Charles,
This is a good idea to get feedback, but I've actually taken a stab at this
(<rant>actually I was debugging a freakin' annoying "feature" of JavaMail...
if you set new content on a message, you have to call "saveChanges()" for it
to take effect. what a brilliant design. I think I might change James so
it automatically calls saveChanges after a mailet makes a change as I think
the SMTP Transport engine Sun wrote already is doing this. </rant>).
Anyway, here's what I did. If you configure db://conf/spool, this will look
for a configuration file at conf/spool (this should be a file, not a
directory). This configuration file contains necessary JDBC connection
stuff (driver, URL, username, password), table name, repository name, all
the SQL used by the JDBCMailRepository, and a directory for message bodies
(more later).
Right now it's just reading it in as a Properties object (name=value), but
this could be changed to XML (then with includes, you could keep all the
JDBC and SQL in one shared XML file, and each conf file would only set the
repository name (and maybe table name). I'm not incredibly pleased with
this design as I think I'd rather this all be in the avalon loaded conf
files, but what can you do?
Two other notes:
- I still haven't gotten excalibur (or any JDBC pooler) integrated yet.
>From what I've seen of excalibur, it looks like I have to let avalon do the
configuration, so I'm not sure how to back-fit my conf file into an
excalibur.
- I've written the JDBCMailRepository to store message bodies in the file
system. Even after all this work to not parse messages, a large message (a
meg or more) would still bring my JDBC driver to a crawl. Now, the
JDBCMailRepository will store all delivery information and the message's
headers to the database, and the message body separately to the file system.
It's hopefully written in such a way that you could turn this off and on, as
it should already read a message that is entirely stored in the database and
so you'd just have to change how it saves messages.
Anyway, I was still testing and hadn't committed anything yet. It looks
pretty stable, but I thought this morning's error was something seriously
wrong with the parsing-avoidance code.
Serge Knystautas
Loki Technologies
http://www.lokitech.com/
----- Original Message -----
From: "Charles Benett" <ch...@benett1.demon.co.uk>
To: "James Dev" <ja...@jakarta.apache.org>; "Jakarta-James User"
<ja...@jakarta.apache.org>
Sent: Tuesday, July 31, 2001 7:15 AM
Subject: Input for JDB Mail Repository
> In cvs proposal dir is Serge's JDBCMailRepository and
> JDBCSpoolRepository.
> Serge uses Mssql server with Inet Sprinta (I think) and I have got it
> going with MySQL and the mm driver.
> This is intended to replace the town db classses in James 1.2.1
>
> At the moment, they new classes need to be re-compiled for different
> drivers and dbs.
> I want to tidy up the config and move them into the main tree.
> So I am looking for input from developers/ users.
>
> 1) At the moment the JDBCMailRepository (like the town repository) puts
> all messages in one db table.
> a) Would it be useful to allows different tables, e.g. one for spam
> another for errors etc.?
> b) If one had multiple tables, how likely is it that one would have them
> in different dbs or on different machines?
> c) If one had multiple tables, would we still want all POP3 inboxes to
> be in one table or would per-user tables make sense?
>
> Any other suggestions re storing mail in a db would be welcome.
>
> Charles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by Serge Knystautas <se...@lokitech.com>.
Charles,
This is a good idea to get feedback, but I've actually taken a stab at this
(<rant>actually I was debugging a freakin' annoying "feature" of JavaMail...
if you set new content on a message, you have to call "saveChanges()" for it
to take effect. what a brilliant design. I think I might change James so
it automatically calls saveChanges after a mailet makes a change as I think
the SMTP Transport engine Sun wrote already is doing this. </rant>).
Anyway, here's what I did. If you configure db://conf/spool, this will look
for a configuration file at conf/spool (this should be a file, not a
directory). This configuration file contains necessary JDBC connection
stuff (driver, URL, username, password), table name, repository name, all
the SQL used by the JDBCMailRepository, and a directory for message bodies
(more later).
Right now it's just reading it in as a Properties object (name=value), but
this could be changed to XML (then with includes, you could keep all the
JDBC and SQL in one shared XML file, and each conf file would only set the
repository name (and maybe table name). I'm not incredibly pleased with
this design as I think I'd rather this all be in the avalon loaded conf
files, but what can you do?
Two other notes:
- I still haven't gotten excalibur (or any JDBC pooler) integrated yet.
>From what I've seen of excalibur, it looks like I have to let avalon do the
configuration, so I'm not sure how to back-fit my conf file into an
excalibur.
- I've written the JDBCMailRepository to store message bodies in the file
system. Even after all this work to not parse messages, a large message (a
meg or more) would still bring my JDBC driver to a crawl. Now, the
JDBCMailRepository will store all delivery information and the message's
headers to the database, and the message body separately to the file system.
It's hopefully written in such a way that you could turn this off and on, as
it should already read a message that is entirely stored in the database and
so you'd just have to change how it saves messages.
Anyway, I was still testing and hadn't committed anything yet. It looks
pretty stable, but I thought this morning's error was something seriously
wrong with the parsing-avoidance code.
Serge Knystautas
Loki Technologies
http://www.lokitech.com/
----- Original Message -----
From: "Charles Benett" <ch...@benett1.demon.co.uk>
To: "James Dev" <ja...@jakarta.apache.org>; "Jakarta-James User"
<ja...@jakarta.apache.org>
Sent: Tuesday, July 31, 2001 7:15 AM
Subject: Input for JDB Mail Repository
> In cvs proposal dir is Serge's JDBCMailRepository and
> JDBCSpoolRepository.
> Serge uses Mssql server with Inet Sprinta (I think) and I have got it
> going with MySQL and the mm driver.
> This is intended to replace the town db classses in James 1.2.1
>
> At the moment, they new classes need to be re-compiled for different
> drivers and dbs.
> I want to tidy up the config and move them into the main tree.
> So I am looking for input from developers/ users.
>
> 1) At the moment the JDBCMailRepository (like the town repository) puts
> all messages in one db table.
> a) Would it be useful to allows different tables, e.g. one for spam
> another for errors etc.?
> b) If one had multiple tables, how likely is it that one would have them
> in different dbs or on different machines?
> c) If one had multiple tables, would we still want all POP3 inboxes to
> be in one table or would per-user tables make sense?
>
> Any other suggestions re storing mail in a db would be welcome.
>
> Charles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: james-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-user-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by Michel Verbist <mv...@cisco.com>.
I don't know if it's already in there or not, but...
a separate table for domains perhaps. So users would belong to a domain
(1-to-m from tblDomain to tblUser).
Very handy when handling multiple domains.
Sorry if it appears to be already in there, I didn't check.
----- Original Message -----
From: "Charles Benett" <ch...@benett1.demon.co.uk>
To: "James Dev" <ja...@jakarta.apache.org>; "Jakarta-James User"
<ja...@jakarta.apache.org>
Sent: Tuesday, July 31, 2001 1:15 PM
Subject: Input for JDB Mail Repository
> In cvs proposal dir is Serge's JDBCMailRepository and
> JDBCSpoolRepository.
> Serge uses Mssql server with Inet Sprinta (I think) and I have got it
> going with MySQL and the mm driver.
> This is intended to replace the town db classses in James 1.2.1
>
> At the moment, they new classes need to be re-compiled for different
> drivers and dbs.
> I want to tidy up the config and move them into the main tree.
> So I am looking for input from developers/ users.
>
> 1) At the moment the JDBCMailRepository (like the town repository) puts
> all messages in one db table.
> a) Would it be useful to allows different tables, e.g. one for spam
> another for errors etc.?
> b) If one had multiple tables, how likely is it that one would have them
> in different dbs or on different machines?
> c) If one had multiple tables, would we still want all POP3 inboxes to
> be in one table or would per-user tables make sense?
>
> Any other suggestions re storing mail in a db would be welcome.
>
> Charles
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by Charles Benett <ch...@benett1.demon.co.uk>.
Oki DZ wrote:
>
> On Tue, 31 Jul 2001, Charles Benett wrote:
>
> > Any other suggestions re storing mail in a db would be welcome.
>
> I already have one about configuration on one of my other postings.
>
> This one regarding James.java; it needs to be tidied up, IMO, especially
> on (the loading of) IMAP storage. I haven't had a chance to look into the
> IMAP source, but I see that the loading is not as simple as
> compMgr.lookup(); MailStore and UsersStore be done just like that (well,
> they load the appropriate repositories). What makes the difference between
> AvalonMailRepository and IMAP storage? Both are for storing the messages
> right? If IMAP storage is supposed to have different interface than
> MailRepository, then having an IMAPStore class would be nice (which would
> then it loads IMAPRepository).
>
> Well, I don't know the exact inner workings of the IMAP server as a whole,
> so I might suggesting something irrelevant; but all I know is that the
> IMAP things work with a different "tone" (design pattern?) compared to the
> MailStore and UsersStore. And that's a bit confusing.
>
> BTW, the names... yes the names... how would one guess that SimpleSystem
> is related to IMAP? You already have some classes with NNTP...
The IMAP code pre-dates the NNTP code and the MailStore/ UserStore
stuff, which is why it doesn't fit.
What I think we need is for the MailStore class to serve up POP3
MailRepositories, SpoolRepositories or (IMAP) mailboxes as required.
When I started the IMAP stuff I thought to include IMAP mailbox
referrals but now I think I would forget that until we get a working
IMAP system.
And, currently, I don't have much time to do anything :-)
Charles
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by Oki DZ <ok...@pindad.com>.
On Tue, 31 Jul 2001, Charles Benett wrote:
> At the moment, they new classes need to be re-compiled for different
> drivers and dbs.
Recompiling the source to get the class working (for different dbs) is not
the right way, I believe.
> I want to tidy up the config and move them into the main tree.
> So I am looking for input from developers/ users.
What about defining the driver's name in config.xml? It would be nicer and
more flexible. The SQL statements are already outside ("softwired"?) the
source, that's nice.
> 1) At the moment the JDBCMailRepository (like the town repository) puts
> all messages in one db table.
> a) Would it be useful to allows different tables, e.g. one for spam
> another for errors etc.?
It might be.
If you have the table names, column names, and driver names in the config
file, then having different repositories in different tables needs no
rewriting the class. It would be about having the right entries (elements)
in the config.xml and some table creation in the databases. I think the
point is about the ease in the configuration. If it's not that difficult,
having one, two, three or more tables would be all right. Ones may have
reasons for have them more than one. eg: if the spam repository were in
its own table, then removing the spam message would just: "delete from
<spam-table>;" (ie: no need to have "where" this and that).
> b) If one had multiple tables, how likely is it that one would have them
> in different dbs or on different machines?
Admin skills show-off...?
> c) If one had multiple tables, would we still want all POP3 inboxes to
> be in one table or would per-user tables make sense?
One table per user for POP mailboxes is a bit too much, I believe.
> Any other suggestions re storing mail in a db would be welcome.
I already have one about configuration on one of my other postings.
This one regarding James.java; it needs to be tidied up, IMO, especially
on (the loading of) IMAP storage. I haven't had a chance to look into the
IMAP source, but I see that the loading is not as simple as
compMgr.lookup(); MailStore and UsersStore be done just like that (well,
they load the appropriate repositories). What makes the difference between
AvalonMailRepository and IMAP storage? Both are for storing the messages
right? If IMAP storage is supposed to have different interface than
MailRepository, then having an IMAPStore class would be nice (which would
then it loads IMAPRepository).
Well, I don't know the exact inner workings of the IMAP server as a whole,
so I might suggesting something irrelevant; but all I know is that the
IMAP things work with a different "tone" (design pattern?) compared to the
MailStore and UsersStore. And that's a bit confusing.
BTW, the names... yes the names... how would one guess that SimpleSystem
is related to IMAP? You already have some classes with NNTP...
Oki
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: Input for JDB Mail Repository
Posted by Oki DZ <ok...@pindad.com>.
On Tue, 31 Jul 2001, Charles Benett wrote:
> At the moment, they new classes need to be re-compiled for different
> drivers and dbs.
Recompiling the source to get the class working (for different dbs) is not
the right way, I believe.
> I want to tidy up the config and move them into the main tree.
> So I am looking for input from developers/ users.
What about defining the driver's name in config.xml? It would be nicer and
more flexible. The SQL statements are already outside ("softwired"?) the
source, that's nice.
> 1) At the moment the JDBCMailRepository (like the town repository) puts
> all messages in one db table.
> a) Would it be useful to allows different tables, e.g. one for spam
> another for errors etc.?
It might be.
If you have the table names, column names, and driver names in the config
file, then having different repositories in different tables needs no
rewriting the class. It would be about having the right entries (elements)
in the config.xml and some table creation in the databases. I think the
point is about the ease in the configuration. If it's not that difficult,
having one, two, three or more tables would be all right. Ones may have
reasons for have them more than one. eg: if the spam repository were in
its own table, then removing the spam message would just: "delete from
<spam-table>;" (ie: no need to have "where" this and that).
> b) If one had multiple tables, how likely is it that one would have them
> in different dbs or on different machines?
Admin skills show-off...?
> c) If one had multiple tables, would we still want all POP3 inboxes to
> be in one table or would per-user tables make sense?
One table per user for POP mailboxes is a bit too much, I believe.
> Any other suggestions re storing mail in a db would be welcome.
I already have one about configuration on one of my other postings.
This one regarding James.java; it needs to be tidied up, IMO, especially
on (the loading of) IMAP storage. I haven't had a chance to look into the
IMAP source, but I see that the loading is not as simple as
compMgr.lookup(); MailStore and UsersStore be done just like that (well,
they load the appropriate repositories). What makes the difference between
AvalonMailRepository and IMAP storage? Both are for storing the messages
right? If IMAP storage is supposed to have different interface than
MailRepository, then having an IMAPStore class would be nice (which would
then it loads IMAPRepository).
Well, I don't know the exact inner workings of the IMAP server as a whole,
so I might suggesting something irrelevant; but all I know is that the
IMAP things work with a different "tone" (design pattern?) compared to the
MailStore and UsersStore. And that's a bit confusing.
BTW, the names... yes the names... how would one guess that SimpleSystem
is related to IMAP? You already have some classes with NNTP...
Oki
---------------------------------------------------------------------
To unsubscribe, e-mail: james-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-user-help@jakarta.apache.org