You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Serge Knystautas <se...@lokitech.com> on 2001/08/01 06:10:42 UTC
DB code committed
Made the change tonight to let the JDBC repository store the message body in
either the file system or in the database. Also committed all my changes,
so hopefully other people can take it and start testing it with other
databases.
The big thing that's still missing is connection pooling... maybe someone
more familiar can take a crack at integrating it with excalibur. Maybe as
part of the upgrade to Avalon 4.0...
I still have to tweak my AddFooter mailet... hopefully can add that later
this week.
Serge Knystautas
Loki Technologies
http://www.lokitech.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: DB code committed
Posted by Serge Knystautas <se...@lokitech.com>.
I like the second approach as well (a single configuration/list of
DataSources that the individual repository configurations could use). I
can't imagine defining datasources like this could be that hard for an app
server platform... But likewise I don't know how you do this.
Serge Knystautas
Loki Technologies
http://www.lokitech.com/
----- Original Message -----
From: "Darrell DeBoer" <li...@bigdaz.com>
To: <ja...@jakarta.apache.org>
Sent: Wednesday, August 01, 2001 6:40 PM
Subject: Re: DB code committed
> Hi,
>
> Regarding using Avalon for handling db connections:
>
> I was looking at this stuff last weekend, although I didn't get too far.
My
> understanding of the nitty-gritty of Avalon is a bit limited, so forgive
me
> if I'm off track.
>
> I reckon using Avalon to manage Database connections would make sense, to
get
> the built in pooling features if nothing else. My understanding is that we
> could use Excalibur JdbcDataSource for our connection management, but it
> seems that this would currently require configuring a datasource at each
> point where one is required (ie on a "per-repository" basis - User, Spool,
> Inbox, etc), even if they all use the same db. This could get pretty
arduous,
> especially since we call every mailing list a separate "repository".
>
> Possibly better would be to use the Cornerstone DataSourceSelector block
to
> load JdbcDataSource components, so that all datasources could be
configured
> in a single config section, and referred to by name elsewhere in the
config
> file. Different Mail Repositories (spool, inbox, error, spam) could use
the
> same or different datasource, and be configured to use the same table with
an
> index column (as is currently), or separate tables.
>
> This is the theory, now to the implementation. I've tried both of these
> approaches for the JdbcUsersRepository but ran into problems. I haven't
spent
> a great deal of time tracking them down, but if anyone has any advice I'd
> appreciate it.
>
> My main problem is that whenever I fiddle with the James config file (eg
try
> to set a DataSourceSelector block) I get this (unhelpful) exception
message
> on Phoenix startup:
>
> Phoenix 3.1a1-dev
>
> There was an uncaught exception:
> ---------------------------------------------------------
> java.lang.OutOfMemoryError
> <<no stack trace available>>
> ---------------------------------------------------------
>
> The final log file entry is:
> Tue Jul 31 22:23:22 EST 2001 [INFO ] <<Phoenix>> (main): Deploying Block
> file (/home/daz/jakarta/jakarta-james/dist/apps/james/blocks/james.bar) as
> james
>
> I plan to have a better look, but if anyone knows what's going on, it
would
> help me out. (I can replicate this problem by simply renaming the
> <pop3server> element to <pop3server1> and attempting to start James).
>
> ciao and TIA
> Daz
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: DB code committed
Posted by Darrell DeBoer <li...@bigdaz.com>.
Thanks for the pointers, I'm going to try to get the DataSourceSelector block
running in a sample app - if that works, I'll try to move it into James.
On Thu, 2 Aug 2001 13:00, Oki DZ wrote:
> On Thu, 2 Aug 2001, Darrell DeBoer wrote:
> > I reckon using Avalon to manage Database connections would make sense, to
> > get the built in pooling features if nothing else. My understanding is
> > that we could use Excalibur JdbcDataSource for our connection management,
> > but it seems that this would currently require configuring a datasource
> > at each point where one is required (ie on a "per-repository" basis -
> > User, Spool, Inbox, etc), even if they all use the same db. This could
> > get pretty arduous, especially since we call every mailing list a
> > separate "repository".
>
> Well, it should be like that, shouldn't it? I mean, to get a db
> connection, you'd have a Connection object in each repository.
>
> > Possibly better would be to use the Cornerstone DataSourceSelector block
> > to load JdbcDataSource components, so that all datasources could be
> > configured in a single config section, and referred to by name elsewhere
> > in the config file. Different Mail Repositories (spool, inbox, error,
> > spam) could use the same or different datasource, and be configured to
> > use the same table with an index column (as is currently), or separate
> > tables.
>
> This one is better, I think (compared to have a configuration section in
> config.xml for each repository).
>
> > This is the theory, now to the implementation. I've tried both of these
> > approaches for the JdbcUsersRepository but ran into problems. I haven't
> > spent a great deal of time tracking them down, but if anyone has any
> > advice I'd appreciate it.
>
> I had the same problem too. Excalibur's JdbcDataSource needed a
> configuration section for itself, even though you could use it without any
> compilation error; just a runtime error, saying that the object wasn't
> initialized. Last time I asked in the avalon-dev list. There is a FAQ
> entry in Cocoon 2's site regarding JdbcDataSource:
> http://xml.apache.org/cocoon2/datasources.html (I didn't know what was
> going on; the JdbcDataSource's doc was only available in Cocoon project's
> site. There was none in Excalibur's doc. Interesting... maybe it was a
> way to "attract" people to the other Apache projects' sites <those
> developers winking>).
>
> It seems that using JdbcDataSource is not that difficult; the
> documentation is there in the above link, loud and clear. But I already
> implemented the db pooling using one of the Turbine services, so I guess
> I'm stuck with it. It was pretty tedious to have the services available
> for James; you'd have to write an Avalon block to load the TurbineConfig
> class, and wrote some classes to get the services. I think it was
> worthwhile though, so that I know how to write Avalon blocks (and get them
> loaded at appropriate time; ie: the db connection pooling service should
> be available before the AvalonMailStore block gets loaded).
>
> I guess, using JdbcDataSource in James is naturally The Right Thing(tm).
>
> > My main problem is that whenever I fiddle with the James config file (eg
> > try to set a DataSourceSelector block) I get this (unhelpful) exception
> > message on Phoenix startup:
> >
> > Phoenix 3.1a1-dev
> >
> > There was an uncaught exception:
>
> ....
>
> I experienced that exceptions many times. Basically, if you had
> Configuration.getChild("some-element") and Avalon couldn't find it, then
> you'd have the exceptions. This surely is just one of the possible causes,
> though.
>
> > I plan to have a better look, but if anyone knows what's going on, it
> > would help me out. (I can replicate this problem by simply renaming the
> > <pop3server> element to <pop3server1> and attempting to start James).
>
> Yes, the POPServer couldn't find its configuration section.
>
> I think Phoenix should have a better error reporting problem (not just
> waiting the JVM stack to be overflowed); having its XML parser printing
> out the line number where it couldn't get the appropriate input elements
> would be really helpful.
>
> Oki
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: DB code committed
Posted by Oki DZ <ok...@pindad.com>.
On Thu, 2 Aug 2001, Darrell DeBoer wrote:
> I reckon using Avalon to manage Database connections would make sense, to get
> the built in pooling features if nothing else. My understanding is that we
> could use Excalibur JdbcDataSource for our connection management, but it
> seems that this would currently require configuring a datasource at each
> point where one is required (ie on a "per-repository" basis - User, Spool,
> Inbox, etc), even if they all use the same db. This could get pretty arduous,
> especially since we call every mailing list a separate "repository".
Well, it should be like that, shouldn't it? I mean, to get a db
connection, you'd have a Connection object in each repository.
> Possibly better would be to use the Cornerstone DataSourceSelector block to
> load JdbcDataSource components, so that all datasources could be configured
> in a single config section, and referred to by name elsewhere in the config
> file. Different Mail Repositories (spool, inbox, error, spam) could use the
> same or different datasource, and be configured to use the same table with an
> index column (as is currently), or separate tables.
This one is better, I think (compared to have a configuration section in
config.xml for each repository).
> This is the theory, now to the implementation. I've tried both of these
> approaches for the JdbcUsersRepository but ran into problems. I haven't spent
> a great deal of time tracking them down, but if anyone has any advice I'd
> appreciate it.
I had the same problem too. Excalibur's JdbcDataSource needed a
configuration section for itself, even though you could use it without any
compilation error; just a runtime error, saying that the object wasn't
initialized. Last time I asked in the avalon-dev list. There is a FAQ
entry in Cocoon 2's site regarding JdbcDataSource:
http://xml.apache.org/cocoon2/datasources.html (I didn't know what was
going on; the JdbcDataSource's doc was only available in Cocoon project's
site. There was none in Excalibur's doc. Interesting... maybe it was a
way to "attract" people to the other Apache projects' sites <those
developers winking>).
It seems that using JdbcDataSource is not that difficult; the
documentation is there in the above link, loud and clear. But I already
implemented the db pooling using one of the Turbine services, so I guess
I'm stuck with it. It was pretty tedious to have the services available
for James; you'd have to write an Avalon block to load the TurbineConfig
class, and wrote some classes to get the services. I think it was
worthwhile though, so that I know how to write Avalon blocks (and get them
loaded at appropriate time; ie: the db connection pooling service should
be available before the AvalonMailStore block gets loaded).
I guess, using JdbcDataSource in James is naturally The Right Thing(tm).
> My main problem is that whenever I fiddle with the James config file (eg try
> to set a DataSourceSelector block) I get this (unhelpful) exception message
> on Phoenix startup:
>
> Phoenix 3.1a1-dev
>
> There was an uncaught exception:
....
I experienced that exceptions many times. Basically, if you had
Configuration.getChild("some-element") and Avalon couldn't find it, then
you'd have the exceptions. This surely is just one of the possible causes,
though.
> I plan to have a better look, but if anyone knows what's going on, it would
> help me out. (I can replicate this problem by simply renaming the
> <pop3server> element to <pop3server1> and attempting to start James).
Yes, the POPServer couldn't find its configuration section.
I think Phoenix should have a better error reporting problem (not just
waiting the JVM stack to be overflowed); having its XML parser printing
out the line number where it couldn't get the appropriate input elements
would be really helpful.
Oki
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: DB code committed
Posted by Darrell DeBoer <li...@bigdaz.com>.
Hi,
Regarding using Avalon for handling db connections:
I was looking at this stuff last weekend, although I didn't get too far. My
understanding of the nitty-gritty of Avalon is a bit limited, so forgive me
if I'm off track.
I reckon using Avalon to manage Database connections would make sense, to get
the built in pooling features if nothing else. My understanding is that we
could use Excalibur JdbcDataSource for our connection management, but it
seems that this would currently require configuring a datasource at each
point where one is required (ie on a "per-repository" basis - User, Spool,
Inbox, etc), even if they all use the same db. This could get pretty arduous,
especially since we call every mailing list a separate "repository".
Possibly better would be to use the Cornerstone DataSourceSelector block to
load JdbcDataSource components, so that all datasources could be configured
in a single config section, and referred to by name elsewhere in the config
file. Different Mail Repositories (spool, inbox, error, spam) could use the
same or different datasource, and be configured to use the same table with an
index column (as is currently), or separate tables.
This is the theory, now to the implementation. I've tried both of these
approaches for the JdbcUsersRepository but ran into problems. I haven't spent
a great deal of time tracking them down, but if anyone has any advice I'd
appreciate it.
My main problem is that whenever I fiddle with the James config file (eg try
to set a DataSourceSelector block) I get this (unhelpful) exception message
on Phoenix startup:
Phoenix 3.1a1-dev
There was an uncaught exception:
---------------------------------------------------------
java.lang.OutOfMemoryError
<<no stack trace available>>
---------------------------------------------------------
The final log file entry is:
Tue Jul 31 22:23:22 EST 2001 [INFO ] <<Phoenix>> (main): Deploying Block
file (/home/daz/jakarta/jakarta-james/dist/apps/james/blocks/james.bar) as
james
I plan to have a better look, but if anyone knows what's going on, it would
help me out. (I can replicate this problem by simply renaming the
<pop3server> element to <pop3server1> and attempting to start James).
ciao and TIA
Daz
On Wed, 1 Aug 2001 14:10, Serge Knystautas wrote:
> Made the change tonight to let the JDBC repository store the message body
> in either the file system or in the database. Also committed all my
> changes, so hopefully other people can take it and start testing it with
> other databases.
>
> The big thing that's still missing is connection pooling... maybe someone
> more familiar can take a crack at integrating it with excalibur. Maybe as
> part of the upgrade to Avalon 4.0...
>
> I still have to tweak my AddFooter mailet... hopefully can add that later
> this week.
>
> Serge Knystautas
> Loki Technologies
> http://www.lokitech.com/
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: DB code committed
Posted by Charles Benett <ch...@benett1.demon.co.uk>.
Serge Knystautas wrote:
>
> Made the change tonight to let the JDBC repository store the message body in
> either the file system or in the database. Also committed all my changes,
> so hopefully other people can take it and start testing it with other
> databases.
Had a go but couldn't get JDBCMailRepository to read the
inbox.properties file - not sure I'm creating it in the right place.
Is it the toplevel conf dir or the app/james/conf dir?
As I had the JDBMailRepository working before the latest change, and I
have the MimeMessage stuff working now with file repositories I am +1
for you to commit the proposal to the main branch. Which may help
getting everything back in sync.
Charles
>
> The big thing that's still missing is connection pooling... maybe someone
> more familiar can take a crack at integrating it with excalibur. Maybe as
> part of the upgrade to Avalon 4.0...
>
> I still have to tweak my AddFooter mailet... hopefully can add that later
> this week.
>
> Serge Knystautas
> Loki Technologies
> http://www.lokitech.com/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: DB code committed
Posted by Darrell DeBoer <li...@bigdaz.com>.
Thanks for the suggestions, Oki
I contributed the UsersJdbcRepository a couple of months back, but
unfortunately I haven't had much more time to work on it since. The
implemenation leaves a lot of room for improvements - hard-coded SQL, no
connection pooling, etc.
So, your suggestions for changes are very welcome, especially from a
useability standpoint. But remember, if you're in a hurry, the fastest way to
get changes made is to submit code, patches, or even documentation; there is
always way too much to do and way too little time...
Thanks
Daz
On Thu, 2 Aug 2001 16:06, Oki DZ wrote:
> On Wed, 1 Aug 2001, Serge Knystautas wrote:
> > I still have to tweak my AddFooter mailet... hopefully can add that later
> > this week.
>
> As if I hadn't asked to much.
> When you're done with the mailet...
>
> To have something like this in the config.xml for UsersJDBCRepository
> would be really neat. So that the repository would not be tied to the
> table and column names hardwired in the UsersJDBCRepository. Basically, it
> would be good to have one's can have the names suit his/her own liking.
>
> <users-store>
> <!-- MySQL server via mm mysql driver -->
> <repository name="LocalUsers"
> class="com.pindad.james.userrepository.UsersMySQLRepository">
> <destination>
> <datasource>
> <table name="Users2">
> <columns>
> <username>username</username>
> <aliasname>aliasname</aliasname>
> <forward-to>forwardto</forward-to>
> <password>password</password>
> <real-name>name</real-name>
> <!-- the following might not be needed
> <unit-name>unit</unit-name>
> <email-service>email</email-service>
> <proxy-service>squid</proxy-service>
> -->
> </columns>
> </table>
> </datasource>
> </destination>
> <!-- this one too
> <db-pooler>
> <service>com.pindad.james.util.TurbineServices</service>
> </db-pooler>
> -->
> </repository>
> </users-store>
>
> UsersJDBCRepository has been available for a long time, but when it was, I
> already had a production James with a users' table that conforms to
> UsersTownRepository. So, the switch wasn't quite an option. But then I
> wanted to know how James works, so I started small with reimplementing the
> UsersJDBCRepository. Some of the methods in UsersRepository are not yet
> implemented, but having the users' table needn't any change, the
> reimplementation was worthwhile, I think.
>
> There's something in UsersJDBCRepository that would make it better, I
> believe; useForwarding and useAlias could be removed. Having the values as
> nulls to represent no forward address and no alias name would be
> sufficient. And also, storing the algorithm used by the password cipher is
> a bit too much; I believe most of the admins would like to have all the
> password encrypted with the same algorithm for all the user passwords.
>
> BTW, aliases are for aliasing the usernames, right? So I think the column
> shouldn't be that wide. At most, it should be as wide as the username's.
> but then, how aliasing is supposed to be working? I tried to telnet to
> James at port 25, and typing "rcpt to: <username>" (without the braces),
> and it seemed not to be working.
>
> Oki
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: DB code committed
Posted by Danny Angus <da...@thought.co.uk>.
Oki wrote:
>I tried to telnet to
> James at port 25, and typing "rcpt to: <username>" (without the braces),
> and it seemed not to be working.
Oki, you need the <> braces !
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: DB code committed
Posted by Oki DZ <ok...@pindad.com>.
On Wed, 1 Aug 2001, Serge Knystautas wrote:
> I still have to tweak my AddFooter mailet... hopefully can add that later
> this week.
As if I hadn't asked to much.
When you're done with the mailet...
To have something like this in the config.xml for UsersJDBCRepository
would be really neat. So that the repository would not be tied to the
table and column names hardwired in the UsersJDBCRepository. Basically, it
would be good to have one's can have the names suit his/her own liking.
<users-store>
<!-- MySQL server via mm mysql driver -->
<repository name="LocalUsers"
class="com.pindad.james.userrepository.UsersMySQLRepository">
<destination>
<datasource>
<table name="Users2">
<columns>
<username>username</username>
<aliasname>aliasname</aliasname>
<forward-to>forwardto</forward-to>
<password>password</password>
<real-name>name</real-name>
<!-- the following might not be needed
<unit-name>unit</unit-name>
<email-service>email</email-service>
<proxy-service>squid</proxy-service>
-->
</columns>
</table>
</datasource>
</destination>
<!-- this one too
<db-pooler>
<service>com.pindad.james.util.TurbineServices</service>
</db-pooler>
-->
</repository>
</users-store>
UsersJDBCRepository has been available for a long time, but when it was, I
already had a production James with a users' table that conforms to
UsersTownRepository. So, the switch wasn't quite an option. But then I
wanted to know how James works, so I started small with reimplementing the
UsersJDBCRepository. Some of the methods in UsersRepository are not yet
implemented, but having the users' table needn't any change, the
reimplementation was worthwhile, I think.
There's something in UsersJDBCRepository that would make it better, I
believe; useForwarding and useAlias could be removed. Having the values as
nulls to represent no forward address and no alias name would be
sufficient. And also, storing the algorithm used by the password cipher is
a bit too much; I believe most of the admins would like to have all the
password encrypted with the same algorithm for all the user passwords.
BTW, aliases are for aliasing the usernames, right? So I think the column
shouldn't be that wide. At most, it should be as wide as the username's.
but then, how aliasing is supposed to be working? I tried to telnet to
James at port 25, and typing "rcpt to: <username>" (without the braces),
and it seemed not to be working.
Oki
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org