You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Stefano Bagnara <ap...@bago.org> on 2007/03/22 21:02:54 UTC
Re: [MODULARISATION] Stage Two: division into modules [WAS Re: Server
modularisation and directory layout questions]
40 days passed without comments/opinions so today I found a bit of time
to experiment with the modular build system and I committed it.
I extracted a mailet-api module and the mailet-library module from the
phoenix-deployment and it worked fine.
Then I moved all of the source code (escluding Main.java) to the
core-library module. I had to move to core-library/build.xml the
Constants.java properties based source pre-processing.
I also had to add hacks because this code depends on mailet-library
module but being it a library itself was not possible to depends on it.
Then I extracted most of the functions modules in their own folder:
smtpserver, pop3server, fetchmail, nntpserver, remotemanager, imapserver
and spoolmanager (maybe to be splitten in mailetcontainer/transport)
worked out of the box.
Running ant from the root did not work because the default target is
"lite" and it expects modules to be already built and only run
deployments, so I switched the default target to Full.
Please review the commits and let me know if this is what you intended
with your proposal.
I have a bunch of problems to proceed:
1) we can reorganize stuff but libraries will depends on other
libraries: how can we manage intra-libraries dependencies? We will
refactor when we have cross-dependency but we cannot avoid simple
dependency. The same apply to APIs (e.g: the MailRepository apis depends
on the Mailet apis). We will probably need an "util/common" library to
be imported by every other library, too.
2) How should we manage javadocs creation and sdk creation in this
multimodule structure? I think the "move" broke them.
Once this steps are solved the following step is to split the
core-library into multiple libraries and to move around 3-4 classes
belonging to function-modules packages but we need them in the libraries.
Once we are done with this we should try to isolate the use of avalon
from the api/library layer and isolate it to the function modules.
The last step is to refactor the function modules to move the avalon
stuff into a specific avalon-deployment module and the
phoenix-deployment will depend on this module (we need intra module
dependency here too).
WDYT?
robert burrell donkin ha scritto:
> (moved stage two discussion into a new thread)
>
> i've created some spaces on
> http://wiki.apache.org/james/Development/Modularisation to help
> develop ideas about the actual movement of code (stage two of the
> proposal, stage one being preparations). please feel free to dive in
> an update :-)
> [...]
>> >> Function Modules
>> >> - smtpserver
>> >> - pop3server
>> >> - fetchmail
>> >> - remotemanager
>> >> - nntpserver
>> >> - imapserver
>> >> - transport (maybe to be named spoolmanager)
>> >> - mailetcontainer: to contain the James.java and some of the
>> >> transport/*loader* stuff.
>> >
>> > sounds reasonable
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [MODULARISATION] Stage Two: division into modules [WAS Re: Server modularisation and directory layout questions]
Posted by Danny Angus <da...@apache.org>.
That's generous of you, feel free to wade right in there if you feel like it.
I hereby withdraw my right to moan if you don't do what I was
expecting, and replace it with a promise to pull my finger out :-)
d.
On 3/23/07, robert burrell donkin <ro...@gmail.com> wrote:
> On 3/23/07, Danny Angus <da...@apache.org> wrote:
> > On 3/23/07, robert burrell donkin <ro...@gmail.com> wrote:
> >
> > > (other than energy) what issues prevent the mailet-api being moved out
> > > into a separate subproject?
> >
> > Nothing.
> >
> > Needs web pages, and a build, but nothing challenging technically.
>
> probably easier to do that than add special cases to the server build.
> i'm willing to take a look at pulling something basic together.
>
> - robert
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [MODULARISATION] Stage Two: division into modules [WAS Re: Server modularisation and directory layout questions]
Posted by robert burrell donkin <ro...@gmail.com>.
On 3/23/07, Danny Angus <da...@apache.org> wrote:
> On 3/23/07, robert burrell donkin <ro...@gmail.com> wrote:
>
> > (other than energy) what issues prevent the mailet-api being moved out
> > into a separate subproject?
>
> Nothing.
>
> Needs web pages, and a build, but nothing challenging technically.
probably easier to do that than add special cases to the server build.
i'm willing to take a look at pulling something basic together.
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [MODULARISATION] Stage Two: division into modules [WAS Re: Server modularisation and directory layout questions]
Posted by Danny Angus <da...@apache.org>.
On 3/23/07, robert burrell donkin <ro...@gmail.com> wrote:
> (other than energy) what issues prevent the mailet-api being moved out
> into a separate subproject?
Nothing.
Needs web pages, and a build, but nothing challenging technically.
d.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [MODULARISATION] Stage Two: division into modules [WAS Re: Server modularisation and directory layout questions]
Posted by robert burrell donkin <ro...@gmail.com>.
On 3/23/07, Danny Angus <da...@apache.org> wrote:
> On 3/22/07, Stefano Bagnara <ap...@bago.org> wrote:
>
> > Maybe we can solve this once we move the mailet-api into its own
> > project. But this must be investigated while splitting/moving modules.
>
> I hope I'll have some time to look at that, and review the
> modularisation, this w/e.
> I've been pretty full-on at work these last few weeks, really sorry.
>
> I do have an incentive though, I need to make sure that the things i
> want to say in my apachecon presentation are true!
(other than energy) what issues prevent the mailet-api being moved out
into a separate subproject?
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [MODULARISATION] Stage Two: division into modules [WAS Re: Server modularisation and directory layout questions]
Posted by Danny Angus <da...@apache.org>.
On 3/22/07, Stefano Bagnara <ap...@bago.org> wrote:
> Maybe we can solve this once we move the mailet-api into its own
> project. But this must be investigated while splitting/moving modules.
I hope I'll have some time to look at that, and review the
modularisation, this w/e.
I've been pretty full-on at work these last few weeks, really sorry.
I do have an incentive though, I need to make sure that the things i
want to say in my apachecon presentation are true!
d.
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [MODULARISATION] Stage Two: division into modules [WAS Re: Server modularisation and directory layout questions]
Posted by robert burrell donkin <ro...@gmail.com>.
On 3/23/07, Stefano Bagnara <ap...@bago.org> wrote:
> robert burrell donkin ha scritto:
> >> Running ant from the root did not work because the default target is
> >> "lite" and it expects modules to be already built and only run
> >> deployments, so I switched the default target to Full.
> >
> > i suspected that this might need tuning
> >
> > there are two general schools of thought
>
> Well, I have no strong preferences. My only request is that "ant" from
> an out-of-the-box checkout should be able to build our artifacts.
> So either we commit snapshots jars of the modules so that ant will
> simply "combine" them into the final artifact or we use the full goal or
> we at least use lazy building of the modules.
>
> In the mean time the "full" solves the nighly build issue.
>
> > [..]
> > i planned to spend this weekend working on the modularisation so
> > expect more comments then
>
> I hope I didn't "steal" some task you already planned to do. If this is
> the case feel free to revert my experiments and apply "your way" :-)
i'm very, very happy for other people to jump in :-)
too many projects: too little energy
> >> I have a bunch of problems to proceed:
> >>
> >> 1) we can reorganize stuff but libraries will depends on other
> >> libraries: how can we manage intra-libraries dependencies? We will
> >> refactor when we have cross-dependency but we cannot avoid simple
> >> dependency. The same apply to APIs (e.g: the MailRepository apis depends
> >> on the Mailet apis). We will probably need an "util/common" library to
> >> be imported by every other library, too.
> >
> > i'd hoped that we could avoid a complex dependency graph by dividing
> > into broad layers with standard dependencies between layers. i'll take
> > a look at the codebase this weekend and see whether this is
> > reasonable.
>
> Maybe we can solve this once we move the mailet-api into its own
> project. But this must be investigated while splitting/moving modules.
yep
<snip>
> >> Once this steps are solved the following step is to split the
> >> core-library into multiple libraries and to move around 3-4 classes
> >> belonging to function-modules packages but we need them in the libraries.
> >
> > not sure i understand this. i'll take a look at the code and ask if it
> > still isn't clear...
>
> We have some class from imapserver.debug package used in the core
> package. We have classes from fetchmail package used in core lib. If you
> look at the current structure I left them in their original package but
> I moved them to the right module, but, as you suggested, having splitted
> packages is no good, so it is better to move them into different packages.
+1
i'd like to see the core interfaces and associated classes pulled out
from core-library into a core-api
hopefully this might allow the implementations to be split up in to
separate libraries or functions.
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [MODULARISATION] Stage Two: division into modules [WAS Re: Server
modularisation and directory layout questions]
Posted by Stefano Bagnara <ap...@bago.org>.
robert burrell donkin ha scritto:
>> Running ant from the root did not work because the default target is
>> "lite" and it expects modules to be already built and only run
>> deployments, so I switched the default target to Full.
>
> i suspected that this might need tuning
>
> there are two general schools of thought
Well, I have no strong preferences. My only request is that "ant" from
an out-of-the-box checkout should be able to build our artifacts.
So either we commit snapshots jars of the modules so that ant will
simply "combine" them into the final artifact or we use the full goal or
we at least use lazy building of the modules.
In the mean time the "full" solves the nighly build issue.
> [..]
> i planned to spend this weekend working on the modularisation so
> expect more comments then
I hope I didn't "steal" some task you already planned to do. If this is
the case feel free to revert my experiments and apply "your way" :-)
>> I have a bunch of problems to proceed:
>>
>> 1) we can reorganize stuff but libraries will depends on other
>> libraries: how can we manage intra-libraries dependencies? We will
>> refactor when we have cross-dependency but we cannot avoid simple
>> dependency. The same apply to APIs (e.g: the MailRepository apis depends
>> on the Mailet apis). We will probably need an "util/common" library to
>> be imported by every other library, too.
>
> i'd hoped that we could avoid a complex dependency graph by dividing
> into broad layers with standard dependencies between layers. i'll take
> a look at the codebase this weekend and see whether this is
> reasonable.
Maybe we can solve this once we move the mailet-api into its own
project. But this must be investigated while splitting/moving modules.
>> 2) How should we manage javadocs creation and sdk creation in this
>> multimodule structure? I think the "move" broke them.
>
> it's easy to put them back again but i'd prefer to hear opinions
> before i implement
>
> one approach is for all modules to create src and javadoc jars of
> their own (useful for IDEs) and then unpack them in deploy. another is
> to javadoc from the deployment across all modules.
I prefer to have each jar with its own javadocs, and maybe we can unpack
them all in a single package in the final distribution, but I don't know
if this is easy to do.
>> Once this steps are solved the following step is to split the
>> core-library into multiple libraries and to move around 3-4 classes
>> belonging to function-modules packages but we need them in the libraries.
>
> not sure i understand this. i'll take a look at the code and ask if it
> still isn't clear...
We have some class from imapserver.debug package used in the core
package. We have classes from fetchmail package used in core lib. If you
look at the current structure I left them in their original package but
I moved them to the right module, but, as you suggested, having splitted
packages is no good, so it is better to move them into different packages.
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [MODULARISATION] Stage Two: division into modules [WAS Re: Server modularisation and directory layout questions]
Posted by robert burrell donkin <ro...@gmail.com>.
On 3/22/07, Stefano Bagnara <ap...@bago.org> wrote:
> robert burrell donkin ha scritto:
> > i took a look at the avalon coupling. IIRC the major issue is going to
> > be logging.
>
> Wrong: the LogEnabled issue can be solved with a simple search and
> replace and is trivial once we agree on the solution (agreement is not
> trivial here)
>
> The major issue instead is the code using ContainerUtil to propagate
> avalon lifecycles to child objects.
this is just the same general problem as logging: in each case the
question is how to propogate container specific configurations through
the object graph. in the case of logging, we want to propogate logging
settings throughout the object graph. in the case of lifecycles, we
want to propogate settings.
> Almost an year I took care to replace every
> ---
> if (object instanceof ServiceEnablingInterface) {
> ((ServiceEnablingInterface) object).serviceEnable(service);
> }
> ---
> with the equivalent ContainerUtil.serviceEnable(object, service), so it
> should be easy to run a find in files for the ContainerUtil object to
> understand what is happening.
i like the pico-container approach to lifecycles: define your own
lifecycle interfaces appropriate to the problem and then glue together
with byte-code engineering.
but you're right that construction of child objects is an issue
> Imho the key is to create factory services so that you never inject the
> lifecycle but instead you ask a factory to provide you a lifecycled
> object. Once we'll have removed the ContainerUtil usage with top level
> avalon services it will be trivial to extract the avalon interfaces to
> avalon-specific extensions of the "pojo" services.
the design issue that will need to be overcome with factory services
is how to inject these services whilst avoiding unwanted coupling. not
impossible, but possibly messy (i don't know the codebase well enough
to make a good judgement about that).
i had been thinking thinking about just using protected factory
methods within each POJO for the moment. it's not great from a design
perspective but is quick and reliable.
a more elegant solution would be to use aspects for container support.
cross-cuts may save a lot of legwork.
but i need to take a closer look at the code...
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [MODULARISATION] Stage Two: division into modules [WAS Re: Server
modularisation and directory layout questions]
Posted by Stefano Bagnara <ap...@bago.org>.
robert burrell donkin ha scritto:
> i took a look at the avalon coupling. IIRC the major issue is going to
> be logging.
Wrong: the LogEnabled issue can be solved with a simple search and
replace and is trivial once we agree on the solution (agreement is not
trivial here)
The major issue instead is the code using ContainerUtil to propagate
avalon lifecycles to child objects.
Almost an year I took care to replace every
---
if (object instanceof ServiceEnablingInterface) {
((ServiceEnablingInterface) object).serviceEnable(service);
}
---
with the equivalent ContainerUtil.serviceEnable(object, service), so it
should be easy to run a find in files for the ContainerUtil object to
understand what is happening.
Imho the key is to create factory services so that you never inject the
lifecycle but instead you ask a factory to provide you a lifecycled
object. Once we'll have removed the ContainerUtil usage with top level
avalon services it will be trivial to extract the avalon interfaces to
avalon-specific extensions of the "pojo" services.
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [MODULARISATION] Stage Two: division into modules [WAS Re: Server modularisation and directory layout questions]
Posted by robert burrell donkin <ro...@gmail.com>.
On 3/22/07, Stefano Bagnara <ap...@bago.org> wrote:
> 40 days passed without comments/opinions so today I found a bit of time
> to experiment with the modular build system and I committed it.
>
> I extracted a mailet-api module and the mailet-library module from the
> phoenix-deployment and it worked fine.
great :-)
> Then I moved all of the source code (escluding Main.java) to the
> core-library module. I had to move to core-library/build.xml the
> Constants.java properties based source pre-processing.
> I also had to add hacks because this code depends on mailet-library
> module but being it a library itself was not possible to depends on it.
>
> Then I extracted most of the functions modules in their own folder:
> smtpserver, pop3server, fetchmail, nntpserver, remotemanager, imapserver
> and spoolmanager (maybe to be splitten in mailetcontainer/transport)
> worked out of the box.
>
> Running ant from the root did not work because the default target is
> "lite" and it expects modules to be already built and only run
> deployments, so I switched the default target to Full.
i suspected that this might need tuning
there are two general schools of thought
> Please review the commits and let me know if this is what you intended
> with your proposal.
it's not so much what i intended but whether i can see any improvements :-)
i planned to spend this weekend working on the modularisation so
expect more comments then
> I have a bunch of problems to proceed:
>
> 1) we can reorganize stuff but libraries will depends on other
> libraries: how can we manage intra-libraries dependencies? We will
> refactor when we have cross-dependency but we cannot avoid simple
> dependency. The same apply to APIs (e.g: the MailRepository apis depends
> on the Mailet apis). We will probably need an "util/common" library to
> be imported by every other library, too.
i'd hoped that we could avoid a complex dependency graph by dividing
into broad layers with standard dependencies between layers. i'll take
a look at the codebase this weekend and see whether this is
reasonable.
> 2) How should we manage javadocs creation and sdk creation in this
> multimodule structure? I think the "move" broke them.
it's easy to put them back again but i'd prefer to hear opinions
before i implement
one approach is for all modules to create src and javadoc jars of
their own (useful for IDEs) and then unpack them in deploy. another is
to javadoc from the deployment across all modules.
> Once this steps are solved the following step is to split the
> core-library into multiple libraries and to move around 3-4 classes
> belonging to function-modules packages but we need them in the libraries.
not sure i understand this. i'll take a look at the code and ask if it
still isn't clear...
> Once we are done with this we should try to isolate the use of avalon
> from the api/library layer and isolate it to the function modules.
>
> The last step is to refactor the function modules to move the avalon
> stuff into a specific avalon-deployment module and the
> phoenix-deployment will depend on this module (we need intra module
> dependency here too).
i took a look at the avalon coupling. IIRC the major issue is going to
be logging.
- robert
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org