You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Stefano Bagnara <ap...@bago.org> on 2006/05/20 15:24:31 UTC
James 2.4 and repositories...
Just read again an old thread (august 2005) about repositories, and I
think this is the right moment to do something...
The first is a concrete change, ready to go, the second and third are
topics related to refactorings I thought in past.
1) Remove the MailRepository interface and up-merge MailRepositories in
SpoolRepositories. SpoolRepository is a superset of MailRepository, they
share the same storage formats, there's no need to keep them both: we
always can provide SpoolRepositories when people asks for
MailRepositories. This is only a first step, preparing for the
introduction of a MessageRepository interface (see also imap).
2) Do some experimentation on the spoolrepository interface:
- MessageKey created while storing, and not userprovided.
- Split store in insert / update methods.
- Introduce message moving/cloning between repositories:
moveFrom/moveTo/copyFrom/copyTo: this way further optimizations could be
introduced at repository level when the 2 repositories share the storage
medium.
- Introduce transaction awarness: JMS approach could help us in this
task, they already solved this problem, maybe it's better to start from
there than inventing new things.
3) Users Repository analysis for refactoring:
- Split Users Profile services from Authentication services
- Optional support for plain-text (or bidirectional encription) in
userrepositories to provide digest based authentications (not possible
with our SHA passwords).
- Decide how to deprecate the current "James-SHA" hash in favor of "more
used" standard SHA (don't Base64, simple hex string), and support MD5
also: this would make James more easy to integrate with shell scripts
(md5 and sha command line tools do not create Base64 hashes, but
hexstrings of the hash). Imho we could keep the current "SHA" under the
"JSHA" name to simplify the upgrade to the new method, and start using
SHA for the 40 [0-9a-z] chars sequence.
- By default our UsersRepositories will provide also the
AuthenticationService, but separating the 2 services we simplify the
integration with third party autenthication mechanisms, and improve the
modularity.
- Users should never created directly from outside and we should
deprecate usersrepository methods to store users created from outside:
at most we should provide a factory method in the service interface (I
already removed the last usage fixing JAMES-505 today, so we should
simply deprecate)
As I often say, I write this only as an high level overview of what can
be done, but we can't go too much in details for points 2 and 3 because
we'll find concrete problems/solutions only working in the code.
Comments?
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: James 2.4 and repositories...
Posted by Joachim Draeger <jd...@joachim-draeger.de>.
Stefano Bagnara wrote:
> 1) Remove the MailRepository interface and up-merge MailRepositories in
> SpoolRepositories. SpoolRepository is a superset of MailRepository, they
> share the same storage formats, there's no need to keep them both: we
> always can provide SpoolRepositories when people asks for
> MailRepositories. This is only a first step, preparing for the
> introduction of a MessageRepository interface (see also imap).
Maybe it would be helpful to split up the capabilities for the
MessageRepository.
Maybe the basic repository has only a public void store(Message m)
method and doesn't care where they go. This would be enough for delivery
and very easy to implement for custom needs.
The advanced one has all capabilities needed for imap.
Joachim
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: James 2.4 and repositories...
Posted by Stefano Bagnara <ap...@bago.org>.
Noel J. Bergman wrote:
> Stefano Bagnara wrote:
>
> This is way past 2.4, but other than that ... :-)
>
>> 1) Remove the MailRepository interface and up-merge MailRepositories in
>> SpoolRepositories.
>
> Although today the SpoolRepository is a specialized MailRepository, they are
> actually quite different, and should be separate. The spool could end up
> being JMS, JDBC, JavaSpace, whereas the MailRepository will have different
> structural and retrieval semantics.
Maybe I've not explained what I intended: currently there is no need for
the 2 interfaces. We can join them to a single (the large one) interface
and then introduce a new interface for the MessageRepository (keeping in
account IMAP needs). First simplify what you have, then introduce new
things.
I also think that, when we'll have SpoolRepository and a new
MessageRepository we should start considering the use of JMS for the
spool (clustering, multitier, transaction support, xa transactions even
mixed with jdbc).
>> 2) Do some experimentation on the spoolrepository interface
>
> Agreed. So lets talk about Spool and Stores, and *then* see whether or not
> they end up still being the same.
>
>> - MessageKey created while storing, and not userprovided.
>
> And separate Mail and Message. Mail is the RFC 2821 and similar attributes,
> including JAMES meta-data, and has a Message. Message is the RFC 2822 DATA
> (MimeMessage).
Agreed.
The new "MessageRepository" would only store Message (MimeMessage) and
not the whole Mail object (we don't need the envelope in the message
repository).
>> - Split store in insert / update methods.
>
> Clarify, please
Now we user store(Mail) either to insert messages and update messages in
a repository. If the message key is already existing then we update,
otherwise we insert. if we let the repository to create the message keys
that method is no more clear, so we should use 2 different methods to
insert and update messages (mails) in the repository.
>> - Introduce message moving/cloning between repositories
>
> Yes, but first look at what happens when we separate Mail and Message. We
> may have multiple Mail sharing the same Message. Also, ideally, we don't
> want to move messages; once we have the Message in its store, we move the
> smaller, lighter, Mail.
Yes, this is an implementation detail but should fit well in the changes
above.
>> - Introduce transaction awarness: JMS approach could help us in this
>> task, they already solved this problem, maybe it's better to start
>> from there than inventing new things.
>
> I have looked at it ... believe me, I have looked at JMS, JavaSpaces, LDAP,
> etc. But although we can get distribution and transactional benefits, I am
> still looking at plain old JDBC as being the most efficient. For one thing,
> although JMS has message attributes, we don't get nice indexing on attribute
> queries. And the query semantics aren't quite a match with our needs.
I never thought at JMS for efficiency: I think that transactions,
clustering and similar features always comes at a price. But I used JMS
for a while and we can learn something from that.
>> 3) Users Repository analysis for refactoring
>
> I'd like for us to consider adopting JNDI/LDAP. Unlike message stores,
> where we want to support mbox, maildir, and other legacy message stores, we
> do not really have the same requirement for user repositories. We can
> standardize on one format, either using JNDI or JDBC. Both ApacheDS and
> Derby can be embedded to ensure that we always have at least one
> implementation. The LDAP approach makes the most sense to me.
> Authentication comes via the same package.
Why not JAAS for Authentication?
BTW, I think we simply have to separate the services interfaces, then we
can create JNDI/LDAP implementations or JAAS bridges adn more.
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: James 2.4 and repositories...
Posted by Markus Kühn <ty...@hotmail.com>.
----- Original Message -----
From: "Stefano Bagnara" <ap...@bago.org>
To: "James Developers List" <se...@james.apache.org>
Sent: Monday, May 22, 2006 1:37 AM
Subject: Re: James 2.4 and repositories...
> Markus Kühn wrote:
>> My motivation is that if I want to add or change something in the future
>> it is easier for me if I don't have to catch up big changes resulting
>> from many different changes like refactoring or splitting on method into
>> 2 and so on.
>
> Refactoring is the heart of programming. If you don't need new things you
> can use the old one.
>
> If we don't change anything we can't move on.
Sure.
> We just branched 2.3 so we can even feel more free to change things in
> trunk.
>
> Btw you probably don't understand my proposal, because my intended goal
> was to keep it simple for people like you that have locally branched from
> James 2.[012] to upgrade to a future 3.0.
So, maybe I misunderstood. You seem to already take this aspect into
consideration and also agree that it is not always worth making things just
more readable.
Markus
I
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: James 2.4 and repositories...
Posted by Stefano Bagnara <ap...@bago.org>.
Markus Kühn wrote:
> My motivation is that if I want to add or change something in the future
> it is easier for me if I don't have to catch up big changes resulting
> from many different changes like refactoring or splitting on method into
> 2 and so on.
Refactoring is the heart of programming. If you don't need new things
you can use the old one.
If we don't change anything we can't move on.
We just branched 2.3 so we can even feel more free to change things in
trunk.
Btw you probably don't understand my proposal, because my intended goal
was to keep it simple for people like you that have locally branched
from James 2.[012] to upgrade to a future 3.0.
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: James 2.4 and repositories...
Posted by Markus Kühn <ty...@hotmail.com>.
----- Original Message -----
From: "Stefano Bagnara" <ap...@bago.org>
To: "James Developers List" <se...@james.apache.org>
Sent: Monday, May 22, 2006 12:59 AM
Subject: Re: James 2.4 and repositories...
> I don't consider myself a new coder on James, but I think that merging 2
> interfaces that are currently used to store and retrieve the same
> identical object in the same identical persistent area is a good thing.
>
> If I proposed it is because I think it is really of benefit: I don't take
> James as a sandbox for proof programming ;-)
> IMAP is not another story: I started with this SIMPLE proposal because I
> was thinking at what we need for IMAP. The first thing is a
> MessageRepository supporting IMAP: I don't want to bring "old" things in
> this new MessageRepository so we should first clean up the current things.
> We currently store Mail objects where we should only store MimeMessages:
> I'm trying to fix this.
>
> If you think it's better to keep the 2 interfaces please tell me your
> motivations.
>
> Stefano
My motivation is that if I want to add or change something in the future it
is easier for me if I don't have to catch up big changes resulting from many
different changes like refactoring or splitting on method into 2 and so on.
Markus
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: James 2.4 and repositories...
Posted by Stefano Bagnara <ap...@bago.org>.
Markus Kühn wrote:
> ----- Original Message ----- From: "Stefano Bagnara" <ap...@bago.org>
> To: "James Developers List" <se...@james.apache.org>
> Sent: Monday, May 22, 2006 12:35 AM
> Subject: Re: James 2.4 and repositories...
>
>> Sorry, I don't understand your points.
>>
>> In fact I'm talking of changes to implement new features (imap).
>>
>> Stefano
>
> IMAP is another story. You suggested merging 2 interfaces and doing some
> refactoring.
>
> If it is really of benefit, it is ok, but changing code in order to make
> it more readable is good for new coders but not so good for others.
>
> Markus
I don't consider myself a new coder on James, but I think that merging 2
interfaces that are currently used to store and retrieve the same
identical object in the same identical persistent area is a good thing.
If I proposed it is because I think it is really of benefit: I don't
take James as a sandbox for proof programming ;-)
IMAP is not another story: I started with this SIMPLE proposal because I
was thinking at what we need for IMAP. The first thing is a
MessageRepository supporting IMAP: I don't want to bring "old" things in
this new MessageRepository so we should first clean up the current
things. We currently store Mail objects where we should only store
MimeMessages: I'm trying to fix this.
If you think it's better to keep the 2 interfaces please tell me your
motivations.
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: James 2.4 and repositories...
Posted by Markus Kühn <ty...@hotmail.com>.
----- Original Message -----
From: "Stefano Bagnara" <ap...@bago.org>
To: "James Developers List" <se...@james.apache.org>
Sent: Monday, May 22, 2006 12:35 AM
Subject: Re: James 2.4 and repositories...
> Sorry, I don't understand your points.
>
> In fact I'm talking of changes to implement new features (imap).
>
> Stefano
IMAP is another story. You suggested merging 2 interfaces and doing some
refactoring.
If it is really of benefit, it is ok, but changing code in order to make it
more readable is good for new coders but not so good for others.
Markus
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: James 2.4 and repositories...
Posted by Stefano Bagnara <ap...@bago.org>.
Markus Kühn wrote:
> ----- Original Message ----- From: "Stefano Bagnara" <ap...@bago.org>
> To: "James Developers List" <se...@james.apache.org>
> Sent: Sunday, May 21, 2006 4:18 PM
> Subject: Re: James 2.4 and repositories...
>
>> The reasoning behind the choice to merge MailRepository to
>> SpoolRepository and introduce a new MessageRepository instead of
>> migrating MailRepository to MessageRepository is:
>>
>> 1st. Greater backward compatibility: if we completely refactor the
>> MailRepository much more "third party" code will become deprecated
>>
>> 2nd. Avoid confusion: MessageRepository will only store MimeMessage
>> and not the envelope informations of the Mail object. MailRepository
>> currently store both, so people could expect this to be also true for
>> a refactored MailRepository.
>
> I won't be confused if you don't change anything, that does not add
> functionality.
>
> If there are new/changed interfaces, new names and the code is actually
> doing the same as before, it makes it harder and not easier for those
> who know the current code.
>
> Markus
Sorry, I don't understand your points.
In fact I'm talking of changes to implement new features (imap).
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: James 2.4 and repositories...
Posted by Markus Kühn <ty...@hotmail.com>.
----- Original Message -----
From: "Stefano Bagnara" <ap...@bago.org>
To: "James Developers List" <se...@james.apache.org>
Sent: Sunday, May 21, 2006 4:18 PM
Subject: Re: James 2.4 and repositories...
> The reasoning behind the choice to merge MailRepository to SpoolRepository
> and introduce a new MessageRepository instead of migrating MailRepository
> to MessageRepository is:
>
> 1st. Greater backward compatibility: if we completely refactor the
> MailRepository much more "third party" code will become deprecated
>
> 2nd. Avoid confusion: MessageRepository will only store MimeMessage and
> not the envelope informations of the Mail object. MailRepository currently
> store both, so people could expect this to be also true for a refactored
> MailRepository.
I won't be confused if you don't change anything, that does not add
functionality.
If there are new/changed interfaces, new names and the code is actually
doing the same as before, it makes it harder and not easier for those who
know the current code.
Markus
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: James 2.4 and repositories...
Posted by Stefano Bagnara <ap...@bago.org>.
Noel J. Bergman wrote:
> Stefano Bagnara wrote:
>
> This is way past 2.4, but other than that ... :-)
>
>> 1) Remove the MailRepository interface and up-merge MailRepositories in
>> SpoolRepositories.
>
> Although today the SpoolRepository is a specialized MailRepository, they are
> actually quite different, and should be separate. The spool could end up
> being JMS, JDBC, JavaSpace, whereas the MailRepository will have different
> structural and retrieval semantics.
The reasoning behind the choice to merge MailRepository to
SpoolRepository and introduce a new MessageRepository instead of
migrating MailRepository to MessageRepository is:
1st. Greater backward compatibility: if we completely refactor the
MailRepository much more "third party" code will become deprecated
2nd. Avoid confusion: MessageRepository will only store MimeMessage and
not the envelope informations of the Mail object. MailRepository
currently store both, so people could expect this to be also true for a
refactored MailRepository.
As Joachim suggests we could even introduce super-interfaces containing
only subsets of the SpoolRepository or the MessageRepository interface
to simplify the introduction of limited implementations: I agree because
this would enhance encapsulation and readability of the code. I think
this is an easy task to do when the right interfaces are in place.
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
RE: James 2.4 and repositories...
Posted by "Noel J. Bergman" <no...@devtech.com>.
Stefano Bagnara wrote:
This is way past 2.4, but other than that ... :-)
> 1) Remove the MailRepository interface and up-merge MailRepositories in
> SpoolRepositories.
Although today the SpoolRepository is a specialized MailRepository, they are
actually quite different, and should be separate. The spool could end up
being JMS, JDBC, JavaSpace, whereas the MailRepository will have different
structural and retrieval semantics.
> 2) Do some experimentation on the spoolrepository interface
Agreed. So lets talk about Spool and Stores, and *then* see whether or not
they end up still being the same.
> - MessageKey created while storing, and not userprovided.
And separate Mail and Message. Mail is the RFC 2821 and similar attributes,
including JAMES meta-data, and has a Message. Message is the RFC 2822 DATA
(MimeMessage).
> - Split store in insert / update methods.
Clarify, please
> - Introduce message moving/cloning between repositories
Yes, but first look at what happens when we separate Mail and Message. We
may have multiple Mail sharing the same Message. Also, ideally, we don't
want to move messages; once we have the Message in its store, we move the
smaller, lighter, Mail.
> - Introduce transaction awarness: JMS approach could help us in this
> task, they already solved this problem, maybe it's better to start
> from there than inventing new things.
I have looked at it ... believe me, I have looked at JMS, JavaSpaces, LDAP,
etc. But although we can get distribution and transactional benefits, I am
still looking at plain old JDBC as being the most efficient. For one thing,
although JMS has message attributes, we don't get nice indexing on attribute
queries. And the query semantics aren't quite a match with our needs.
> 3) Users Repository analysis for refactoring
I'd like for us to consider adopting JNDI/LDAP. Unlike message stores,
where we want to support mbox, maildir, and other legacy message stores, we
do not really have the same requirement for user repositories. We can
standardize on one format, either using JNDI or JDBC. Both ApacheDS and
Derby can be embedded to ensure that we always have at least one
implementation. The LDAP approach makes the most sense to me.
Authentication comes via the same package.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org