You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Robert Burrell Donkin <ro...@gmail.com> on 2008/02/02 12:13:52 UTC
[modules] Backends
i've been thinking on the recent threads on modules eg [1]
i think that the current system is ok for protocol integration (SMTP,
POP3, etc) but backends don't really fit very well. for example,
avalon-user-function, torque-mailboxmanagerfunction etc don't make as
much sense. i am still convinced that we need to move out
implementations from core-library and into modules with dependencies
that can be managed easily.
i now wonder whether it might be better to aggregate backend classes
according to the technologies they use. so (for example) any backend
code that uses torque would be in a torque-backend module, any code
that uses avalon to store data in a avalon-backend module and so on.
any backend implementations that just use java would be moved into a
base-backend module.
- robert
[1] http://mail-archives.apache.org/mod_mbox/james-server-dev/200801.mbox/%3c32710174.110571200396021368.JavaMail.root@elysia.void.it%3e
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [modules] Backends
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Feb 3, 2008 5:50 PM, Stefano Bagnara <ap...@bago.org> wrote:
> Robert Burrell Donkin ha scritto:
> >> I run a few grep to see what modules/code import each class/package in
> >> core-library and here are some considerations:
> >>
> >> org.apache.james.security.*
> >> - they are only used by spoolmanager-function, and only by smime mailets
> >> (org.apache.james.transport.mailets.smime.*).
> >
> > the security stuff would probably be better as an independent sub-project
>
> BTW They are generic mailets with no dependencies on JAMES Server (IIRC).
> So we might move all of this together with smime mailets, and, if the
> project will increase its size, then create an ad-hoc product for the
> security stuff.
IMHO a separate product would be much better
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [modules] Backends
Posted by Stefano Bagnara <ap...@bago.org>.
Robert Burrell Donkin ha scritto:
>> I run a few grep to see what modules/code import each class/package in
>> core-library and here are some considerations:
>>
>> org.apache.james.security.*
>> - they are only used by spoolmanager-function, and only by smime mailets
>> (org.apache.james.transport.mailets.smime.*).
>
> the security stuff would probably be better as an independent sub-project
BTW They are generic mailets with no dependencies on JAMES Server (IIRC).
So we might move all of this together with smime mailets, and, if the
project will increase its size, then create an ad-hoc product for the
security stuff.
>> org.apache.james.util.junkscore.*
>> - is only used by smtpserver-function (we might move this in the
>> function itself)
>
> +1
DONE
>> org.apache.james.util.urirbl.*
>> - this is smtpserver-function stuff. maybe we can move this package
>> there, for the moment.
>
> +1
DONE
>> org.apache.james.util.MatcherInverter
>> - belongs to the util.mailet package. should we move it there?
>
> +1
DONE
>> org.apache.james.context.*
>> - this is a function code. It is the Avalon/Phoenix implementation of
>> our FileSystem service. Should be moved away from this module (phoenix
>> deployment?)
>
> something like that
DONE (to phoenix-deployment, ATM)
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [modules] Backends
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Feb 2, 2008 6:39 PM, Stefano Bagnara <ap...@bago.org> wrote:
> Robert Burrell Donkin ha scritto:
> > i now wonder whether it might be better to aggregate backend classes
> > according to the technologies they use. so (for example) any backend
> > code that uses torque would be in a torque-backend module, any code
> > that uses avalon to store data in a avalon-backend module and so on.
> > any backend implementations that just use java would be moved into a
> > base-backend module.
<snip>
> Our JDBC based repositories also depends on avalon because they supports
> both db and dbfile "protocols".
yes - IMHO they're all avalon backends
> BTW I think we should at least (as a first step) separate backend code
> from other core library classes. Core-library contains too much
> unrelated code ATM.
+1
> I run a few grep to see what modules/code import each class/package in
> core-library and here are some considerations:
>
> org.apache.james.security.*
> - they are only used by spoolmanager-function, and only by smime mailets
> (org.apache.james.transport.mailets.smime.*).
the security stuff would probably be better as an independent sub-project
> org.apache.james.management.*
> - this package is imported by remotemanager-function and by core-library
> itself (org.apache.james.services). There is a cyclic dependency issue
> between org.apache.james.management and org.apache.james.service that
> should be resolved as a priority.
+1
> org.apache.james.domain.*
> - backend implementation: contains XML and JDBC version (and common
> abstract code). no other module depends on this.
i
> org.apache.james.mailboxmanager.*
> - imap-mailbox-processor-function, imapserver-function,
> torque-mailboxmanager-function depends on this.
> - sieve mailets in "spoolmanager-function" depends on this (I would like
> to see this dependency removed)
> - phoenix-deployment depends on this because of
> PhoenixImapProcessorFactory and TorqueMailboxManagerFactory
> I think the whole mailboxmanager package could be moved to a different
> library module as no other core-library class depends on it, but I'm not
> sure the current api-library-function-deployment layers let us to do
> this right now, unless we move some class down to an api module.
a mailbox API makes a lot of sense: given the number of modules, we'll
probably end up moving some code into separate sub-projects. mailbox
(including IMAP and so on) is a good candidate.
> org.apache.james.core.*
> - this is a problematic package. This contains some core James object
> like MailImpl/MimeMessageWrapper/MailHeaders but also contains common
> components for current protocol functions
> (AbstractJamesHandler/AbstractJamesService).
>
> org.apache.james.util.watchdog.*
> org.apache.james.util.connection.*
> - they are related to AbstractJamesHandler/AbstractJamesService.
>
> org.apache.james.util.junkscore.*
> - is only used by smtpserver-function (we might move this in the
> function itself)
+1
> org.apache.james.util.Lock
> - belongs to the repository logic
+1
> org.apache.james.util.mail.mdn.*
> - only used by sieve mailets. This is almost standalone utility code,
> maybe we should move this where we move sieve mailets, or give them a
> custom module (this depends on how we'll reorganize mailets). It imports
> org.apache.james.util.mail.MimeMultipartReport and the
> org.apache.james.util.mail.handlers.* classes.
the sieve mailets are a bit of an issue but i agree that it belongs with them
> org.apache.james.util.mail.dsn.DSNStatus
> - utility class used by smtpserver and by the DSNBounce mailet. To make
> DSNBounce a non-James-specific mailet we have to move this somewhere
> else, but where??
>
> org.apache.james.util.mail.MimeMultipartReport
> org.apache.james.util.mail.handlers.*
> - they are used by DNSBounce and by the mdn code. Something along the
> line of the DSNStatus destiny.
i'd like to separate the DNS code into domain
> org.apache.james.util.JDBCBayesianAnalyzer
> org.apache.james.util.BayesianAnalyzer
> - they deserve their own package. And maybe they belong to the backend
> stuff.
maybe the bayesian stuff would be better in a separate sub-project
> org.apache.james.util.urirbl.*
> - this is smtpserver-function stuff. maybe we can move this package
> there, for the moment.
+1
> org.apache.james.util.POP3BeforeSMTPHelper
> - this is "weird" because it links pop3function to the smtpfunction
> (smtpserver.core.POP3BeforeSMTPHandler, pop3server.PassCmdHandler). It
> is an smtp specific feature)
>
> org.apache.james.util.DotStuffingInputStream
> org.apache.james.util.SchedulerNotifyInputStream
> org.apache.james.util.CharTerminatedInputStream
> org.apache.james.util.ExtraDotOutputStream
> org.apache.james.util.InternetPrintWriter
> org.apache.james.util.TimeConverter
> org.apache.james.util.Base64
> org.apache.james.util.CRLFTerminatedReader
> org.apache.james.util.SqlResources
> - utility classes. maybe we should remove oro dependency from
> timeconverter. Some of them are specific to our
> AbstractJamesHandler/AbstractJamesService stuff)
>
> org.apache.james.util.MatcherInverter
> - belongs to the util.mailet package. should we move it there?
+1
> org.apache.james.util.NetMatcher
> - used by many mailets and by some smtpserver code. it depends on the
> DNSServer, and I wonder if it should better depends on a modified Mailet
> API (including the getByName method).
i'd like to split out the DNS into domain modules
> org.apache.james.util.dbcp.JdbcDataSource
> - this is avalon/excalibur function code. it implements the
> org.apache.avalon.excalibur.datasource.DataSourceComponent interface and
> is only used via that service. belongs to the backend.
+1
> org.apache.james.util.io.*
> - maybe we should remove this in favor of commons-io library. This is
> pure java, no deps, utility code for IO. I guess this was once avalon
> code, that has been integrated in james because of avalon death but also
> found his way in jakarta commons.
+1
quite possibly commons may take any missing functions
> org.apache.james.util.mailet.*
> - this code is only used by some of our mailets.
move into standard mailets sub-projects
> org.apache.james.context.*
> - this is a function code. It is the Avalon/Phoenix implementation of
> our FileSystem service. Should be moved away from this module (phoenix
> deployment?)
something like that
> org.apache.james.services.*
> - ehm.. time is running out and this package is tricky.. so let's delay
> it for a while ;-)
i would like to push these into API modules grouped by semantic function. eg
MailetLoader -> spool [extension]
MailProcessor -> spool [extension]
BayesianAnalyzerManagementService -> bayesian mailet [management]
DomainListManagementService -> domain [management]
FileSystem ???
JamesConnectionManager ??? management
MailRepository -> mail [extension though ATM service]
MailServer -> mail [service]
ManageableDomainList -> domain [management]
MatcherLoader -> spool [extension]
ProcessorManagementService -> spool [management]
SpoolManagementService -> spool [management]
SpoolRepository -> spool[extension]
note that spool and mail repositories are going cause difficulties
with coupling: may need some refactoring
i would like to see all APIs repackaged into sub-packages of
org.apache.james.api grouped first by functional area and then into
service, extension and management APIs
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [modules] Backends
Posted by Stefano Bagnara <ap...@bago.org>.
Robert Burrell Donkin ha scritto:
> On Feb 2, 2008 6:39 PM, Stefano Bagnara <ap...@bago.org> wrote:
>> Our torque function also depends on avalon services.
>
> not since i stripped out the dependency ;-)
>
> the avalon-based configuration is in pheonix but the module has been
> clean for a few months
Thank you!
It's clear that I'm no more so up-to-date wrt trunk content as I was 1
year ago ;-)
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [modules] Backends
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Feb 2, 2008 6:39 PM, Stefano Bagnara <ap...@bago.org> wrote:
> Robert Burrell Donkin ha scritto:
> > i now wonder whether it might be better to aggregate backend classes
> > according to the technologies they use. so (for example) any backend
> > code that uses torque would be in a torque-backend module, any code
> > that uses avalon to store data in a avalon-backend module and so on.
> > any backend implementations that just use java would be moved into a
> > base-backend module.
>
> This could not be so easy.
>
> Our torque function also depends on avalon services.
not since i stripped out the dependency ;-)
the avalon-based configuration is in pheonix but the module has been
clean for a few months
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [modules] Backends
Posted by Stefano Bagnara <ap...@bago.org>.
Robert Burrell Donkin ha scritto:
> i now wonder whether it might be better to aggregate backend classes
> according to the technologies they use. so (for example) any backend
> code that uses torque would be in a torque-backend module, any code
> that uses avalon to store data in a avalon-backend module and so on.
> any backend implementations that just use java would be moved into a
> base-backend module.
This could not be so easy.
Our torque function also depends on avalon services.
Our JDBC based repositories also depends on avalon because they supports
both db and dbfile "protocols".
BTW I think we should at least (as a first step) separate backend code
from other core library classes. Core-library contains too much
unrelated code ATM.
I run a few grep to see what modules/code import each class/package in
core-library and here are some considerations:
org.apache.james.security.*
- they are only used by spoolmanager-function, and only by smime mailets
(org.apache.james.transport.mailets.smime.*).
org.apache.james.management.*
- this package is imported by remotemanager-function and by core-library
itself (org.apache.james.services). There is a cyclic dependency issue
between org.apache.james.management and org.apache.james.service that
should be resolved as a priority.
org.apache.james.domain.*
- backend implementation: contains XML and JDBC version (and common
abstract code). no other module depends on this.
org.apache.james.mailboxmanager.*
- imap-mailbox-processor-function, imapserver-function,
torque-mailboxmanager-function depends on this.
- sieve mailets in "spoolmanager-function" depends on this (I would like
to see this dependency removed)
- phoenix-deployment depends on this because of
PhoenixImapProcessorFactory and TorqueMailboxManagerFactory
I think the whole mailboxmanager package could be moved to a different
library module as no other core-library class depends on it, but I'm not
sure the current api-library-function-deployment layers let us to do
this right now, unless we move some class down to an api module.
org.apache.james.core.*
- this is a problematic package. This contains some core James object
like MailImpl/MimeMessageWrapper/MailHeaders but also contains common
components for current protocol functions
(AbstractJamesHandler/AbstractJamesService).
org.apache.james.util.watchdog.*
org.apache.james.util.connection.*
- they are related to AbstractJamesHandler/AbstractJamesService.
org.apache.james.util.junkscore.*
- is only used by smtpserver-function (we might move this in the
function itself)
org.apache.james.util.Lock
- belongs to the repository logic
org.apache.james.util.mail.mdn.*
- only used by sieve mailets. This is almost standalone utility code,
maybe we should move this where we move sieve mailets, or give them a
custom module (this depends on how we'll reorganize mailets). It imports
org.apache.james.util.mail.MimeMultipartReport and the
org.apache.james.util.mail.handlers.* classes.
org.apache.james.util.mail.dsn.DSNStatus
- utility class used by smtpserver and by the DSNBounce mailet. To make
DSNBounce a non-James-specific mailet we have to move this somewhere
else, but where??
org.apache.james.util.mail.MimeMultipartReport
org.apache.james.util.mail.handlers.*
- they are used by DNSBounce and by the mdn code. Something along the
line of the DSNStatus destiny.
org.apache.james.util.JDBCBayesianAnalyzer
org.apache.james.util.BayesianAnalyzer
- they deserve their own package. And maybe they belong to the backend
stuff.
org.apache.james.util.urirbl.*
- this is smtpserver-function stuff. maybe we can move this package
there, for the moment.
org.apache.james.util.POP3BeforeSMTPHelper
- this is "weird" because it links pop3function to the smtpfunction
(smtpserver.core.POP3BeforeSMTPHandler, pop3server.PassCmdHandler). It
is an smtp specific feature)
org.apache.james.util.DotStuffingInputStream
org.apache.james.util.SchedulerNotifyInputStream
org.apache.james.util.CharTerminatedInputStream
org.apache.james.util.ExtraDotOutputStream
org.apache.james.util.InternetPrintWriter
org.apache.james.util.TimeConverter
org.apache.james.util.Base64
org.apache.james.util.CRLFTerminatedReader
org.apache.james.util.SqlResources
- utility classes. maybe we should remove oro dependency from
timeconverter. Some of them are specific to our
AbstractJamesHandler/AbstractJamesService stuff)
org.apache.james.util.MatcherInverter
- belongs to the util.mailet package. should we move it there?
org.apache.james.util.NetMatcher
- used by many mailets and by some smtpserver code. it depends on the
DNSServer, and I wonder if it should better depends on a modified Mailet
API (including the getByName method).
org.apache.james.util.dbcp.JdbcDataSource
- this is avalon/excalibur function code. it implements the
org.apache.avalon.excalibur.datasource.DataSourceComponent interface and
is only used via that service. belongs to the backend.
org.apache.james.util.io.*
- maybe we should remove this in favor of commons-io library. This is
pure java, no deps, utility code for IO. I guess this was once avalon
code, that has been integrated in james because of avalon death but also
found his way in jakarta commons.
org.apache.james.util.mailet.*
- this code is only used by some of our mailets.
org.apache.james.context.*
- this is a function code. It is the Avalon/Phoenix implementation of
our FileSystem service. Should be moved away from this module (phoenix
deployment?)
org.apache.james.services.*
- ehm.. time is running out and this package is tricky.. so let's delay
it for a while ;-)
HTH,
Stefano
PS: maybe all of this message is OT, and the subject should be "what do
we have in core-library?" ;-)
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [modules] Backends
Posted by Robert Burrell Donkin <ro...@gmail.com>.
On Feb 2, 2008 8:04 PM, Danny Angus <da...@apache.org> wrote:
> On Feb 2, 2008 11:13 AM, Robert Burrell Donkin
> <ro...@gmail.com> wrote:
>
> > i now wonder whether it might be better to aggregate backend classes
> > according to the technologies they use. so (for example) any backend
> > code that uses torque would be in a torque-backend module, any code
> > that uses avalon to store data in a avalon-backend module and so on.
> > any backend implementations that just use java would be moved into a
> > base-backend module.
>
> I liketo use a pattern that has agnostic API interface module and has
> technology specific implementations.
> Then for service X you could have X-torque-impl or X-hibernate-impl
> etc. You can then also have technology specific shared code, which
> might be used across the same technology for a number of different
> module impls and anything that is technology neutral is util code for
> the API. You kind of get two hierarchies, one functional one
> technological.
>
> Does that make sense in this context? It preserves the "purity" of
> module splits by function, but also allows for the need to support
> different technologies in implementation.
this was my first intention
this will produce a very finely grained component model with lots of
modules most with a handful of classes in. inevitably, library code
will need to be shared between implementations which use the same
technology. so, library modules will also be needed. this will add up
to a *lot* of modules. it can be hard to understand systems with
scores of modules.
whether this is reasonable depends a lot on the granularity of the API
components. even then, it may be necessary to be bold in completely
decoupling areas of the server and moving them into separate projects
completely de-coupled from server. for example, we might need to move
mailbox (IMAPs, mailbox managers etc) into a completely separate
sub-project.
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [modules] Backends
Posted by Danny Angus <da...@apache.org>.
On Feb 2, 2008 11:13 AM, Robert Burrell Donkin
<ro...@gmail.com> wrote:
> i now wonder whether it might be better to aggregate backend classes
> according to the technologies they use. so (for example) any backend
> code that uses torque would be in a torque-backend module, any code
> that uses avalon to store data in a avalon-backend module and so on.
> any backend implementations that just use java would be moved into a
> base-backend module.
I liketo use a pattern that has agnostic API interface module and has
technology specific implementations.
Then for service X you could have X-torque-impl or X-hibernate-impl
etc. You can then also have technology specific shared code, which
might be used across the same technology for a number of different
module impls and anything that is technology neutral is util code for
the API. You kind of get two hierarchies, one functional one
technological.
Does that make sense in this context? It preserves the "purity" of
module splits by function, but also allows for the need to support
different technologies in implementation.
d.
d.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org