[STATUS] - API refactoring

[Danny Angus <da...@gmail.com>]

Hi,

I've started refactoring in server/sandbox/mailet-refactorings

Initial comits moved user and mail repositories into the API.

This is because it is virtually impossible to do anything with the API
without those entities.

User repo depends upon User, so I moved that too.

While I was at it I refactored JamesUser. JamesUser is badly named
because it has two responsibilities, forwarding and aliasing, so I
split ForwardingUser and AliasedUser out of it and, on the basis that
they are reasonably sensible email concepts, moved them into the API
as well.

d.

Re: [STATUS] - API refactoring

[Danny Angus <da...@gmail.com>]

I didn't vote on any of this at the time because my votes then would
all have been +0 or -1 and I didn't want to stop anyone from doing
anything that would move James forwards.

Furthermore I think it was too much to decide in one go so I just
tuned out of it.

I still think it is too much for one thread to consider.

But if you insist...

On 10/28/06, Stefano Bagnara <ap...@bago.org> wrote:
> Danny Angus wrote:
> -----
> B1. Remove Avalon from "API Components"

> So I think we all agree on this.
>
> -------
> E. Specific API Components issues:
> E1. Use JNDI to lookup datasources

+1 to making them available through JNDI

But... I don't think they should be *required* just available where they exist.

> E2. Use JNDI to lookup users/mail repositories, the store and any other
> James component.

-1 repositories should be made available by the api directly


> E3. Add datasource, repositories, store and any other used service to
> the MailetContext API (this also mean adding the interfaces for this
> objects to the Mailet APIs)

+0.5 *some* of these, which represent first class entities in the
model should, but peripheral services need a more flexible approach
(JNDI and an extension mechanism perhaps) I think I'm in agreement
with Noel.

> E4. Use Dependency Injection (setter based, constructor based, enabling
> interfaces, service locator injection) to automatically satisfy
> components dependencies.

I think this is only relevant to James implementation,
-1 to imposing too much IoC on mailets

> E5. Keep the ServiceManager as a property stored in the MailetContext.

-1

> As you can see we have topics where it seems there is consensus (remove
> ServiceManager from the MailetContext and Remove Avalon from "API
> Components") and others where we have opposite preferences.
> Almost every other question received a bunch of +1 but at least a -1: so
> you know this is really a minefield.

I've always known that, and have been thinking about this for more
years than you could guess! I don't think my approach in the sandbox
is going against the consensus.


> That said I really appreciate your effort on this issue: all this text
> is just to update you on what happened in the months you were more busy
> and what informations I collected about this issue.

I've been reading all the mail, but thanks anyway.

d.

Re: [STATUS] - API refactoring

[Stefano Bagnara <ap...@bago.org>]

Danny Angus wrote:
> On 10/28/06, Stefano Bagnara <ap...@bago.org> wrote:
> 
>> Now you know it will be even harder than 2 years ago to find consensus
>> on this topic, but hopefully the poll gave you few hints on possible
>> compromises.
> 
> Compromise my ass, this is my sandbox fork Muahahahh ! ;-)
> What I mean is, this project doesn't have to achieve consensus itself,
> as long as it helps us to understand what the underlying problems are
> which have resulted in no agreement for such a long time.

++1 You know I'm a fan of "code based proposals".
I just put a "beware the DI fanatics" and a "beware the JNDI haters" 
signs near your sandbox :-P

Stefano

> After this fork is all done we won't have Mailet v3, we'll have a plan
> for Mailet v3, which might well include decisions which still need to
> be bottomed out.
> 
> d.

Re: [STATUS] - API refactoring

[Danny Angus <da...@gmail.com>]

On 10/28/06, Stefano Bagnara <ap...@bago.org> wrote:

> Now you know it will be even harder than 2 years ago to find consensus
> on this topic, but hopefully the poll gave you few hints on possible
> compromises.



Compromise my ass, this is my sandbox fork Muahahahh ! ;-)
What I mean is, this project doesn't have to achieve consensus itself,
as long as it helps us to understand what the underlying problems are
which have resulted in no agreement for such a long time.
After this fork is all done we won't have Mailet v3, we'll have a plan
for Mailet v3, which might well include decisions which still need to
be bottomed out.

d.

Re: [STATUS] - API refactoring

[Stefano Bagnara <ap...@bago.org>]

Danny Angus wrote:
> On 10/28/06, Stefano Bagnara <ap...@bago.org> wrote:
>> What is confusing? The sentences you quoted or the results of the poll?
>> Let me know if I can help clearing up things.
> 
> Partly the sentences, but also the poll itself. I think it is much
> clearer to everyone if a POLL or VOTE is conducted with only one or
> two simple questions and best way to do that is to have a big
> discussion before hand. Use the POLL and the VOTE to confirm the
> things people have suggested/agreed in the discussion.

I considered that POLL a real success (excluding the unfortunate choice, 
my mistake, to use [VOTE] instead of [POLL] as the prefix): it would 
have taken hundreds of messages to have a similar overview of 
preferences of the PMC team.

> It was useful to me to review it all though.
> 
> d.

This was the goal :-)
Now you know it will be even harder than 2 years ago to find consensus 
on this topic, but hopefully the poll gave you few hints on possible 
compromises.

Stefano

Re: [STATUS] - API refactoring

[Danny Angus <da...@gmail.com>]

On 10/28/06, Stefano Bagnara <ap...@bago.org> wrote:
> Danny Angus wrote:
> > Stefano,
> >
> > This is *very* confusing!!
>
> What is confusing? The sentences you quoted or the results of the poll?
> Let me know if I can help clearing up things.

Partly the sentences, but also the poll itself. I think it is much
clearer to everyone if a POLL or VOTE is conducted with only one or
two simple questions and best way to do that is to have a big
discussion before hand. Use the POLL and the VOTE to confirm the
things people have suggested/agreed in the discussion.

It was useful to me to review it all though.

d.

Re: [STATUS] - API refactoring

[Stefano Bagnara <ap...@bago.org>]

Danny Angus wrote:
> Stefano,
> 
> This is *very* confusing!!

What is confusing? The sentences you quoted or the results of the poll?
Let me know if I can help clearing up things.

Stefano

>> I think it is "in-topic" to revamp part ot the result of a POLL I
>> started 3 months ago ([VOTE] Wiring/Dependencies/Lifecycle Management
>> for next James). I see you didn't write your opinion, so you can add
>> them now to update us on your preferences.
>>
>> To be clear I think we all consider this "votes" as preferences and not
>> as a real vote (it was made clear on the vote thread that it was more a
>> POLL than a vote), but I think it is important to keep this in mind if
>> you're working on a proposal and you're looking for consensus ;-)

Re: [STATUS] - API refactoring

[Danny Angus <da...@gmail.com>]

Stefano,

This is *very* confusing!!

> I think it is "in-topic" to revamp part ot the result of a POLL I
> started 3 months ago ([VOTE] Wiring/Dependencies/Lifecycle Management
> for next James). I see you didn't write your opinion, so you can add
> them now to update us on your preferences.
>
> To be clear I think we all consider this "votes" as preferences and not
> as a real vote (it was made clear on the vote thread that it was more a
> POLL than a vote), but I think it is important to keep this in mind if
> you're working on a proposal and you're looking for consensus ;-)

Re: [STATUS] - API refactoring

[Stefano Bagnara <ap...@bago.org>]

Danny Angus wrote:
> My proposal is that we do indeed move a whole lot of our service 
> interfaces.
> But... (read this first before replying...)
> 
> We need to have an extensible mechanism which allows 3rd parties to
> add services, which means that we're really talking about using JNDI

I think it is "in-topic" to revamp part ot the result of a POLL I 
started 3 months ago ([VOTE] Wiring/Dependencies/Lifecycle Management 
for next James). I see you didn't write your opinion, so you can add 
them now to update us on your preferences.

To be clear I think we all consider this "votes" as preferences and not 
as a real vote (it was made clear on the vote thread that it was more a 
POLL than a vote), but I think it is important to keep this in mind if 
you're working on a proposal and you're looking for consensus ;-)

-----
B1. Remove Avalon from "API Components"

+0.2 Stefano - if we introduce a james specific DI to supports this 
components and add Lifecycle to the component API.
+1 Bernd
+0.8 Soren
+0.8 Norman - That whould be cool.. I don't like the idea of need 
knowing about avalon to write mailets or smtphandler etc..
+0.8 Vincenzo
+1 Serge
------
So I think we all agree on this.

-------
E. Specific API Components issues:
E1. Use JNDI to lookup datasources

-0 Stefano - I'm still undecided about this: I think that using DI would 
be better.
Bernd - tough question. my brief insight into this topic tells me that 
it's very wrong how it is today, but exposing the interfaces directly? 
_That's_ where I'd like to see JNDI! But I don't think I am aware of 
enough discussion concerning this topic to vote it.
Noel - Again, premature and possibly container dependent.
+0.5 Soren
+0 Norman - like i said no expirences with JNDI
+0.2 Vincenzo
-1 Serge

E2. Use JNDI to lookup users/mail repositories, the store and any other 
James component.

-0.5 Stefano
Bernd - tough question. my brief insight into this topic tells me that 
it's very wrong how it is today, but exposing the interfaces directly? 
_That's_ where I'd like to see JNDI! But I don't think I am aware of 
enough discussion concerning this topic to vote it.
Noel - Again, premature and possibly container dependent.
+0.5 Soren
+0 Norman
+0.2 Vincenzo
-1 Serge

E3. Add datasource, repositories, store and any other used service to 
the MailetContext API (this also mean adding the interfaces for this 
objects to the Mailet APIs)

-0 Stefano - The API would grow a lot and we'll limit extensibility.
Bernd - tough question. my brief insight into this topic tells me that 
it's very wrong how it is today, but exposing the interfaces directly? 
_That's_ where I'd like to see JNDI! But I don't think I am aware of 
enough discussion concerning this topic to vote it.
Noel - Probably not.  That would require the MailetContext to expand to 
understand every possible service, and require every possible service. 
But some of them?  Perhaps.
-0.9 Soren
-0 Norman
+0.5 Vincenzo - for a proper subset (which one :-) ?)
-0.8 Vincenzo - for everything
-1 Serge

E4. Use Dependency Injection (setter based, constructor based, enabling 
interfaces, service locator injection) to automatically satisfy 
components dependencies.

+1 Stefano - Imo the right way
+1 Bernd - (but I'm also +1 to _optionally_support_ JNDI)
-? Noel - Where?  For the Mailet API?  -1.  Elsewhere in JAMES?  To be 
determined.
+0.8 Soren
+0.8 Norman
+0 Vincenzo
+1 Serge - setter or construction based

E5. Keep the ServiceManager as a property stored in the MailetContext.

-0.9 Stefano - Imho this is a bad practice and introduce a "hidden" 
dependency.
-0.5 Bernd
-0.8 Soren
-0.5 Norman - Seems to be more a hack..
-0.8 Vincenzo
-1 Serge
------


As you can see we have topics where it seems there is consensus (remove 
ServiceManager from the MailetContext and Remove Avalon from "API 
Components") and others where we have opposite preferences.
Almost every other question received a bunch of +1 but at least a -1: so 
you know this is really a minefield.

That said I really appreciate your effort on this issue: all this text 
is just to update you on what happened in the months you were more busy 
and what informations I collected about this issue.

Stefano

Re: [STATUS] - API refactoring

[Danny Angus <da...@gmail.com>]

On 10/28/06, Stefano Bagnara <ap...@bago.org> wrote:
> Danny Angus wrote:
> > Hi,
> >
> > I've started refactoring in server/sandbox/mailet-refactorings
> >
> > Initial comits moved user and mail repositories into the API.
>
> I remember that in past there was no mail repositories in the APIs. Then
> for a lot of months I saw the repositories added to the APIs, and later
> they have been removed again.

Noel wanted to use JNDI for looking up services, so we abandoned the
changes which had been made to the trunk but not released.



> I think that moving only Mail and UsersRepository does not fix any
> mailet: maybe we have at least to move the cornerstone.Store (previously
> we were using james.services.MailStore) and UsersStore.

It doesn't completely fix them because we need to provide a lookup for
the services.

>
> But I'm not sure this is the right way to go unless we want to move
> almost *everything* we have in "services" to the mailet apis.

My proposal is that we do indeed move a whole lot of our service interfaces.
But... (read this first before replying...)

We need to have an extensible mechanism which allows 3rd parties to
add services, which means that we're really talking about using JNDI

However the entities, the "things" which are handled by the services,
need to be in the API so that people can make services which work with
each other. So User and Mail go in.

Because User and Mail persistance need to be available to almost every
conievable application it makes sense to add a standard service
interface for them, otherwise you wouldnlt know how to get and put
them in your mailet.

How James chooses to implement them is not the concern of the API, and
moving the James interfaces is only a first step. They can be changed
to be more focused on the basic get and put responsibility, and less
influenced by James internals.


> In the past weeks we talked about JamesUsers and the way we was
> currently supporting Aliasing and Forwarding and we agreed that it was a
> bit weird (you have to create a new "virtual" user to enable aliasing
> for another user).

<snip> I get it.

> I have not analyzed again this thing now, but I think we now don't need
> the "JamesUser" or the Alias/Forward thing in the APIs as all of this is
> hidden behind the UsersRepository and VirtualUserTable services.

Ok, in which case this could go out again as long as it doesn't
prevent *other* mailet containers which ight be written from providing
the same functions without mailets having to access any server
internals.

d.

Re: [STATUS] - API refactoring

[Stefano Bagnara <ap...@bago.org>]

Danny Angus wrote:
> Hi,
> 
> I've started refactoring in server/sandbox/mailet-refactorings
> 
> Initial comits moved user and mail repositories into the API.

I remember that in past there was no mail repositories in the APIs. Then 
for a lot of months I saw the repositories added to the APIs, and later 
they have been removed again.

I was not following so much the list to remember why this happened: can 
you tell us what was the historical cause of this?

> This is because it is virtually impossible to do anything with the API
> without those entities.
> 
> User repo depends upon User, so I moved that too.

I think that moving only Mail and UsersRepository does not fix any 
mailet: maybe we have at least to move the cornerstone.Store (previously 
we were using james.services.MailStore) and UsersStore.

But I'm not sure this is the right way to go unless we want to move 
almost *everything* we have in "services" to the mailet apis.

> While I was at it I refactored JamesUser. JamesUser is badly named
> because it has two responsibilities, forwarding and aliasing, so I
> split ForwardingUser and AliasedUser out of it and, on the basis that
> they are reasonably sensible email concepts, moved them into the API
> as well.
> 
> d.

In the past weeks we talked about JamesUsers and the way we was 
currently supporting Aliasing and Forwarding and we agreed that it was a 
bit weird (you have to create a new "virtual" user to enable aliasing 
for another user).

Few months ago I isolated the aliasing/forwarding code in James.java and 
UsersRepositoryAliasingForwarding mailet. Few weeks ago, introducing the 
VirtualUserTable service we decided that its behaviour was really 
similar to what we did in UsersRepositoryAliasingForwarding so we 
started a refactoring that brought us to isolate the use of 
aliasing/forwarding INSIDE the specific implementation of 
UsersRepository: the implementation also provide a VirtualUserTable 
implementation that give you access to the alias/forward informations.

Alias and Forward have been unified to a single concept: when the 
resulting address is local it is considered an alias, if the resulting 
address is remote it is considered a forward.

I have not analyzed again this thing now, but I think we now don't need 
the "JamesUser" or the Alias/Forward thing in the APIs as all of this is 
hidden behind the UsersRepository and VirtualUserTable services.

Stefano