You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Darrell DeBoer <dd...@bigdaz.com> on 2001/08/15 07:27:20 UTC
[COMMIT]: UserStore proposal
G'day,
I've added a new "userstore" proposal to the proposals directory. I've
haven't made any modifications to the main CVS tree; the proposal directory
contains all of the changes, including a modified Ant buildfile. (There were
too many changes to cleanly add it to the existing build file. I reckon that
keeping a separate buildfile for a proposal is probably cleaner, as well).
The following detail is taken from the proposal README.
enjoy
Daz
PROPOSAL: - Modifications to UserStore.
=================================
GOALS:
- use Avalon and Cornerstone DataSource components for connection serving and
pooling (done)
- Remove hard-coded SQL statements from UsersJdbcRepository (done)
- 'SqlResources.java' - detect db product from jdbc connection and select
appropriate SQL statements from SQL definition file for specific product
(done)
- Simpler to create database-backed UserRepository implementations for
different User implementations (done)
- Consolidate existing UserRepository implementations - refactor out common
functionality
- Simplify UserRepository specification in config - make it URL:// based,
like MailRepository.
- Have UserStore serve up repository implementations based on: storage, User
implementation, and location.
OTHER GOALS (reuse development in JdbcMailRepository):
- use Avalon and Cornerstone DataSource components in JdbcMailRepository
- Use SqlResources.java to provide db-specific SQL to JdbcMailRepository
- Automatic table generation for JdbcMailRepository
TO BUILD:
../build.sh -buildfile proposals/userstore/build.xml
I haven't modified the original buildfile, but instead provided a build.xml
within the proposals dir. (I think this is a cleaner approach, since we don't
pollute the "real" buildfile.)
The "javadocs" target builds javadoc from the build/src directory, so it
includes docs for the proposal.
TO USE:
The main configuration is setting up the "database-connections" section of
the config file. There's an example there using MySql - I haven't yet tested
on other databases (although the SQL statements haven't changed much, so I
imagine it will still work on other platforms).
The only properties you should need to set are:
<driver>-- Class name of database driver to use --</driver>
<dburl>-- the jdbc connection string for your database</dburl>
<user>-- database user --</user>
<password>-- database password --</password>
TEST INSTRUCTIONS:
- Telnet to the remote manager: "telnet localhost 4555".
- Do some user management - type "help" for options.
- type "use list-james", to switch to the repository for this list.
- list the users
- send an email to "james-on@localhost"
- list the users again
(note: some user management commands fail for repositories other than
"LocalUsers").
NOTES:
I've added an "AbstractJdbcUsersRepository", which takes care of most of the
work of a JdbcUsersRepository, making it pretty easy to add new ones. The
abstract implementation doesn't have knowledge of actual User
implementations; this is restricted to overridden methods in concrete
repository implementations.
The AbstractJdbcUsersRepository obtains SQL statements via an "SqlResources"
object, which reads an sql definition file, finds the appropriate <sqlDefs>
element, and provides the sql strings contained. In addition, the
SqlResources class handles 2 other things:
a) Parameter replacement in SQL (eg replace all occurances of ${table}
within an sql statement with the parameter value for "table". Currently, all
parameters are taken from the configuration <sqlParameters> element. It is
also possible to define parameters (defaults, if you like) within the sql
definition file itself (a <parameters> element).
b) Examines the Jdbc Connection to determine what database product is
being used. SQL statements specific to a db product (eg mysql) can then be
used automatically. (Detection is done by regexp matches on
Connection.getMetaData.getDatabaseProductName())
I've added 3 concrete subclasses of AbstractJdbcUserRepository: for
DefaultUser, DefaultJamesUser, and "ListUser" (which for now is nothing more
than a name). These give an example of how little work there is to implement
a new repository. The ListUsersJdbcRepository can store multiple lists of
names in a single table.
I've made a simple modification to "RemoteManagerHandler", to allow testing.
The "use [userRepositoryName]" command will switch the Remote manager to
manage the named repository. This isn't really intended for production, makes
for easier testing.
The "james-config.xml" included in the proposal sets up 4 jdbc repositories:
- "localUsers" - a JamesUsersJdbcRepository.
- "list-james" - a ListUsersJdbcRepository, used by the ListServ mailet.
- "list-test" - another ListUsersJdbcRepositor, for testing.
- "default-users" - a DefaultUsersJdbcRepository, for testing.
Note that in order for the Avalon DataSource components to work, I've included
an upgraded "avalon-excalibur.jar" in the proposal.
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: [COMMIT]: UserStore proposal
Posted by Serge Knystautas <se...@lokitech.com>.
----- Original Message -----
From: "Darrell DeBoer" <dd...@bigdaz.com>
> Unfortunately, I think there are problems with getting user Inboxes to
work
> with the new JdbcMailRepository. The problem is that, to find an Inbox for
a
> user, James adds the username to the end of the DestinationUrl. eg it asks
> for something like:
> db://../conf/inbox.propertiesusername
> which obviously won't work, since there's no file called that.
Good point. This clearly must be the issue Charles was running into.
> A few changes I'd like to see in the configuration/setup of the
> JdbcMailRepository, if others agree.
> - Get rid of the separate database .properties files.
> - Move "real" config out of the properties file and into the main
config.xml
> - Move SQL statements (which really should be considered part of the code
> base) for all databases into a common SqlResources file (that's the
> motivation behind SqlResources.java).
> - Add automatic table generation, so that all that's needed for
installation
> is a valid database and dbURL.
+1 to all points. I really wasn't happy with the way I did the
configuration. Looking over the way you did this (at least from a
configuration standpoint... I didn't get a chance to get into the actual
code yet), it looks fantastic. The way you can override specific statements
for a given database is perfect, as I think there's only one SQL statement
that changes (the one that calculates message size) and dreaded having
multiple sets of SQL statements for each provider.
With this approach, we can go back to a db://<resource>/<repository_name>
approach, and I'll leave it to you to figure out how the <resource> uses the
approach resource, table name, etc... Again, I didn't look at the User JDBC
code, but it seems like you should be able to do something like this.
+1 as well if you want to do this yourself and merge into the main branch...
if you have any questions on specifics of the JDBC mail repository, feel
free to ask as I should be pretty responsive via email for the next week or
so. I'll focus then on checking for performance hotspots, some
mailets/matchers, and maybe even some documentation (gasp).
Serge Knystautas
Loki Technologies
http://www.lokitech.com/
----------------------------------------------------
This message was brought to you by MailHive.net
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: [COMMIT]: UserStore proposal
Posted by Darrell DeBoer <dd...@bigdaz.com>.
On Wed, 15 Aug 2001 23:39, Serge Knystautas wrote:
> This sounds great! To me, the connection pooling is the last big issue to
> getting the release out, and these changes really look like a step in the
> right direction (also configuration seems to be better). I'm not sure of
> the quality of the code (since you're putting it in as a proposal), but I'd
> love to get the JDBC mail repository modified to use this approach and get
> this proposal in the main branch soon. I've been busy the past few days
> but hope to work more on new mailets/matchers, some performance testing,
> and documentation later this week.
Hi Serge,
I think the code in the proposal is at least as good as the current
UsersJdbcRepository implementation, which is in the main branch. (Whether
that's saying something good about my new code, or bad about the old stuff -
that's an exercise for the reader ;-). I just though it was standard practice
to put big chunks of new code into a proposal before shifting to the main
tree. The XML parsing of the SqlResources.xml is a little clunky (but it
works); I was hoping to find a nice DOM wrapper somewhere in Jakarta that I
could reuse (or use JDOM?).
It's very simple to convert the JdbcMailRepository to use Avalon connection
pooling - I've got it working at home (for Mail Spool only - see below), and
I could add it to the proposals if you like.
Unfortunately, I think there are problems with getting user Inboxes to work
with the new JdbcMailRepository. The problem is that, to find an Inbox for a
user, James adds the username to the end of the DestinationUrl. eg it asks
for something like:
db://../conf/inbox.propertiesusername
which obviously won't work, since there's no file called that.
A few changes I'd like to see in the configuration/setup of the
JdbcMailRepository, if others agree.
- Get rid of the separate database .properties files.
- Move "real" config out of the properties file and into the main config.xml
- Move SQL statements (which really should be considered part of the code
base) for all databases into a common SqlResources file (that's the
motivation behind SqlResources.java).
- Add automatic table generation, so that all that's needed for installation
is a valid database and dbURL.
I'm happy to help out on the above changes (and the move to Avalon), but I'd
like to get some feedback from others before starting. Whether these are in
the very next release, or slip into an Alpha2 release, I'm not too concerned.
ciao
Daz
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
Re: [COMMIT]: UserStore proposal
Posted by Serge Knystautas <se...@lokitech.com>.
Darrell,
This sounds great! To me, the connection pooling is the last big issue to
getting the release out, and these changes really look like a step in the
right direction (also configuration seems to be better). I'm not sure of
the quality of the code (since you're putting it in as a proposal), but I'd
love to get the JDBC mail repository modified to use this approach and get
this proposal in the main branch soon. I've been busy the past few days but
hope to work more on new mailets/matchers, some performance testing, and
documentation later this week.
Serge Knystautas
Loki Technologies
http://www.lokitech.com/
----- Original Message -----
From: "Darrell DeBoer" <dd...@bigdaz.com>
To: <ja...@jakarta.apache.org>
Sent: Wednesday, August 15, 2001 1:27 AM
Subject: [COMMIT]: UserStore proposal
> G'day,
>
> I've added a new "userstore" proposal to the proposals directory. I've
> haven't made any modifications to the main CVS tree; the proposal
directory
> contains all of the changes, including a modified Ant buildfile. (There
were
> too many changes to cleanly add it to the existing build file. I reckon
that
> keeping a separate buildfile for a proposal is probably cleaner, as well).
>
> The following detail is taken from the proposal README.
> enjoy
> Daz
>
> PROPOSAL: - Modifications to UserStore.
> =================================
>
> GOALS:
> - use Avalon and Cornerstone DataSource components for connection serving
and
> pooling (done)
> - Remove hard-coded SQL statements from UsersJdbcRepository (done)
> - 'SqlResources.java' - detect db product from jdbc connection and select
> appropriate SQL statements from SQL definition file for specific product
> (done)
> - Simpler to create database-backed UserRepository implementations for
> different User implementations (done)
> - Consolidate existing UserRepository implementations - refactor out
common
> functionality
> - Simplify UserRepository specification in config - make it URL:// based,
> like MailRepository.
> - Have UserStore serve up repository implementations based on: storage,
User
> implementation, and location.
>
> OTHER GOALS (reuse development in JdbcMailRepository):
> - use Avalon and Cornerstone DataSource components in JdbcMailRepository
> - Use SqlResources.java to provide db-specific SQL to JdbcMailRepository
> - Automatic table generation for JdbcMailRepository
>
> TO BUILD:
> .../build.sh -buildfile proposals/userstore/build.xml
>
> I haven't modified the original buildfile, but instead provided a
build.xml
> within the proposals dir. (I think this is a cleaner approach, since we
don't
> pollute the "real" buildfile.)
>
> The "javadocs" target builds javadoc from the build/src directory, so it
> includes docs for the proposal.
>
> TO USE:
> The main configuration is setting up the "database-connections" section of
> the config file. There's an example there using MySql - I haven't yet
tested
> on other databases (although the SQL statements haven't changed much, so I
> imagine it will still work on other platforms).
> The only properties you should need to set are:
> <driver>-- Class name of database driver to use --</driver>
> <dburl>-- the jdbc connection string for your database</dburl>
> <user>-- database user --</user>
> <password>-- database password --</password>
>
> TEST INSTRUCTIONS:
> - Telnet to the remote manager: "telnet localhost 4555".
> - Do some user management - type "help" for options.
> - type "use list-james", to switch to the repository for this list.
> - list the users
> - send an email to "james-on@localhost"
> - list the users again
> (note: some user management commands fail for repositories other than
> "LocalUsers").
>
> NOTES:
> I've added an "AbstractJdbcUsersRepository", which takes care of most of
the
> work of a JdbcUsersRepository, making it pretty easy to add new ones. The
> abstract implementation doesn't have knowledge of actual User
> implementations; this is restricted to overridden methods in concrete
> repository implementations.
>
> The AbstractJdbcUsersRepository obtains SQL statements via an
"SqlResources"
> object, which reads an sql definition file, finds the appropriate
<sqlDefs>
> element, and provides the sql strings contained. In addition, the
> SqlResources class handles 2 other things:
> a) Parameter replacement in SQL (eg replace all occurances of ${table}
> within an sql statement with the parameter value for "table". Currently,
all
> parameters are taken from the configuration <sqlParameters> element. It is
> also possible to define parameters (defaults, if you like) within the sql
> definition file itself (a <parameters> element).
> b) Examines the Jdbc Connection to determine what database product is
> being used. SQL statements specific to a db product (eg mysql) can then be
> used automatically. (Detection is done by regexp matches on
> Connection.getMetaData.getDatabaseProductName())
>
> I've added 3 concrete subclasses of AbstractJdbcUserRepository: for
> DefaultUser, DefaultJamesUser, and "ListUser" (which for now is nothing
more
> than a name). These give an example of how little work there is to
implement
> a new repository. The ListUsersJdbcRepository can store multiple lists of
> names in a single table.
>
> I've made a simple modification to "RemoteManagerHandler", to allow
testing.
> The "use [userRepositoryName]" command will switch the Remote manager to
> manage the named repository. This isn't really intended for production,
makes
> for easier testing.
> The "james-config.xml" included in the proposal sets up 4 jdbc
repositories:
> - "localUsers" - a JamesUsersJdbcRepository.
> - "list-james" - a ListUsersJdbcRepository, used by the ListServ mailet.
> - "list-test" - another ListUsersJdbcRepositor, for testing.
> - "default-users" - a DefaultUsersJdbcRepository, for testing.
>
> Note that in order for the Avalon DataSource components to work, I've
included
> an upgraded "avalon-excalibur.jar" in the proposal.
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org