You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Simon Funnell <si...@googlemail.com> on 2009/11/20 18:51:56 UTC
As Above, So Below
Hi,
I've been studying the code base and evaluating which approach is the
best to take. At present, I will try to clarify the framework
architecture I am using as 'evangelising' might actually be quite
effective. Firstly, there is nothing new to learn, the architecture is
exactly the same as the internet. Applications are networks of machines
and each machine on the network is a component of one or more
applications. Each machine/component specialises in doing one small job
very well, for example you could have a network of machines/components
that do something like the following.
Machine 1's job is to listen on a port for new client connections, when
a new client comes along, it is accepted and the resulting socket is
contextualised. A context is basically an object that maintains state,
such as application/session/request or whatever is relevant. This
context/state is then dispatched to Machine 2.
Machine 2 will now alter the context state by reading bytes from the
socket and buffering them in some suitable form before dispatching to
Machine 3.
Machine 3 is responsible for taking the buffered content and parsing
into some sort object model, like an email. If an error occurs this
machine will forward it to an exception handling machine which
implements the exception policy of the system (Machine 4?). Otherwise,
it is sent to the next machine (Machine 5?) which does some sort of
matching or something
And so on.
Now if we want to do some logging at any time, we just put a logging
machine in between Machine 1 and 2 or between 2 and 3 to check socket
state, buffer state or some other combination of properties.
To relate this somewhat to James, you are currently using Guice to wire
components together and.this is equivalent to defining a network. With
the design I am following there is not really a configuration as such,
they are basically replaced with policy statements and change logs,
which are a kind of configuration. Its fair to say James would be a
network of machines defined by a policy statement. These policy
statements can be large or small and applied to new or already running
systems. Newer policy statements always supersede older ones (unless a
prior policy forbids it). James already has a fairly substantial default
policy statement like thing, its configuration files. However, to
understand the benefits of policy statements, instead of configurations,
it is worth looking at a small example.
In this example I have an installation of James, it does not do any
logging at present but I want to do some. And this is what I want to do:
For the next two days, between the hours of the 3-5 pm, if a client
using the SMTP service invokes the Expn command and has a IP address
matching some address log some information to this file in this format
and if its between 3.45pm and 4.00pm send me an email with such and such
information.
This policy is expressed (but doesn't have to be) as an XML document.
The document is evaluated and the necessary machines are added to the
network to implement the policy. How the policy is defined and
implemented is system independent. The new machines added to the network
that implement the policy usually simply fit in between two existing
machines and do stuff like filtering, logging, security and anything
that makes logical sense (putting a logging machine that records
information about a http request in an smtp service doesn't make logical
sense). Here, Java is used to create very small, highly robust units of
functionality with XML policy statements (closer to natural language and
easy to learn for users) stating how the machines should interact. The
sophistication of policy statements is increased as the size and
collective ability of a community of machines/components in a given
installation grows.
The current mailets and matchers are sort of ideal for use in this
system as they can (in principle) be easily wrapped/adapted or what not
to become a network machine. A machine here is simply an object with a
single method (I've been thinking about using the reflection api's to
extract individual methods from existing James classes and assign them
to individual machines but its not straight forward as the existing
methods in James classes tend to mix concerns together, like application
logic and writing to an output in a single method)
I hope this is of interest, declarative logging is really easy to
implement and requires no learning, only a subtle shift in approach.
James could be implemented as a guice module and a policy statement, in
fact it would probably would be fairly easy to generate guice modules
from a policy statement. Its not a direct translation but they are not
dissimilar in nature. Also I am actually thinking of employing guice
myself to do a few things, I am not in competition with guice. I am not
in competition with spring either because its got a load of stuff with
it mine doesn't. That said, Platformed is not 'just another framework',
it has features (an architecture) that make it distinct.
I'm thinking about a re-factoring strategy but I am still learning James
under the hood. Ideally I would like to convince you of the benefits,
they are fairly obvious.
Regards,
Simon
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: As Above, So Below
Posted by Norman Maurer <no...@apache.org>.
Hi Simon,
im still confused, But sometimes a little Bit Code can help others to
get it.... So please show us some code.
Bye
Norman
2009/11/20, Simon Funnell <si...@googlemail.com>:
> Hi,
>
> I've been studying the code base and evaluating which approach is the
> best to take. At present, I will try to clarify the framework
> architecture I am using as 'evangelising' might actually be quite
> effective. Firstly, there is nothing new to learn, the architecture is
> exactly the same as the internet. Applications are networks of machines
> and each machine on the network is a component of one or more
> applications. Each machine/component specialises in doing one small job
> very well, for example you could have a network of machines/components
> that do something like the following.
>
> Machine 1's job is to listen on a port for new client connections, when
> a new client comes along, it is accepted and the resulting socket is
> contextualised. A context is basically an object that maintains state,
> such as application/session/request or whatever is relevant. This
> context/state is then dispatched to Machine 2.
>
> Machine 2 will now alter the context state by reading bytes from the
> socket and buffering them in some suitable form before dispatching to
> Machine 3.
>
> Machine 3 is responsible for taking the buffered content and parsing
> into some sort object model, like an email. If an error occurs this
> machine will forward it to an exception handling machine which
> implements the exception policy of the system (Machine 4?). Otherwise,
> it is sent to the next machine (Machine 5?) which does some sort of
> matching or something
>
> And so on.
>
> Now if we want to do some logging at any time, we just put a logging
> machine in between Machine 1 and 2 or between 2 and 3 to check socket
> state, buffer state or some other combination of properties.
>
> To relate this somewhat to James, you are currently using Guice to wire
> components together and.this is equivalent to defining a network. With
> the design I am following there is not really a configuration as such,
> they are basically replaced with policy statements and change logs,
> which are a kind of configuration. Its fair to say James would be a
> network of machines defined by a policy statement. These policy
> statements can be large or small and applied to new or already running
> systems. Newer policy statements always supersede older ones (unless a
> prior policy forbids it). James already has a fairly substantial default
> policy statement like thing, its configuration files. However, to
> understand the benefits of policy statements, instead of configurations,
> it is worth looking at a small example.
>
> In this example I have an installation of James, it does not do any
> logging at present but I want to do some. And this is what I want to do:
>
> For the next two days, between the hours of the 3-5 pm, if a client
> using the SMTP service invokes the Expn command and has a IP address
> matching some address log some information to this file in this format
> and if its between 3.45pm and 4.00pm send me an email with such and such
> information.
>
> This policy is expressed (but doesn't have to be) as an XML document.
> The document is evaluated and the necessary machines are added to the
> network to implement the policy. How the policy is defined and
> implemented is system independent. The new machines added to the network
> that implement the policy usually simply fit in between two existing
> machines and do stuff like filtering, logging, security and anything
> that makes logical sense (putting a logging machine that records
> information about a http request in an smtp service doesn't make logical
> sense). Here, Java is used to create very small, highly robust units of
> functionality with XML policy statements (closer to natural language and
> easy to learn for users) stating how the machines should interact. The
> sophistication of policy statements is increased as the size and
> collective ability of a community of machines/components in a given
> installation grows.
>
> The current mailets and matchers are sort of ideal for use in this
> system as they can (in principle) be easily wrapped/adapted or what not
> to become a network machine. A machine here is simply an object with a
> single method (I've been thinking about using the reflection api's to
> extract individual methods from existing James classes and assign them
> to individual machines but its not straight forward as the existing
> methods in James classes tend to mix concerns together, like application
> logic and writing to an output in a single method)
>
> I hope this is of interest, declarative logging is really easy to
> implement and requires no learning, only a subtle shift in approach.
> James could be implemented as a guice module and a policy statement, in
> fact it would probably would be fairly easy to generate guice modules
> from a policy statement. Its not a direct translation but they are not
> dissimilar in nature. Also I am actually thinking of employing guice
> myself to do a few things, I am not in competition with guice. I am not
> in competition with spring either because its got a load of stuff with
> it mine doesn't. That said, Platformed is not 'just another framework',
> it has features (an architecture) that make it distinct.
>
> I'm thinking about a re-factoring strategy but I am still learning James
> under the hood. Ideally I would like to convince you of the benefits,
> they are fairly obvious.
>
> Regards,
>
> Simon
>
>
>
>
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
The Point
Posted by Simon Funnell <si...@googlemail.com>.
Eric MacAdie wrote:
> I run James on a VPS host with about 300 MB of memory. Now you are
> talking about multiple machines.
No, they are not literally machines as such, that's just a synonym. A
'machine' in Platformed is just part of the terminology used in the API,
a 'machine' is basically just a Java object with a single method. So a
network of machines is basically a collection of single method objects
(like a servlet filter/matcher/mailet). Here's an example of how I might
adapt James matchers to my framework:
public class MatcherVirtualMachine implements VirtualMachine{
//Wraps an existing matcher
private final Matcher matcher;
@Inject .
public MatcherVirtualMachine( Matcher theMatcher ){
matcher = theMatcher
}
//Framework to component adaption.
public void operate( SessionState state ){
Property property = state.getProperty( "hostname" );
boolean matches = matcher.match( property );
//I realise this is not how the matcher api works.
//Its just to illustrate.
}
}
> It sounds to me like you want to complicate James unnecessarily to try
> out some new framework (that nobody else on the list seems to get).
No, I would like James components to be more reusable (the point?). This
is not about my framework, its about creating future proof James
components. James was previously tied to Avalon and now you are
suggesting tying it another framework? This is not solving the problem,
its repeating it.
> Why should James be the Guinea pig for some new framework? I think
> that the effort to "Guiceyfy" should just continue as is.
Sorry but I find that a bit offensive, you clearly are not reading very
carefully. I am not asking anyone to use this framework, I am asking
that James users (of which I am one) have the opportunity to choose
which framework they use (this idea is at the core of modern software
development). I chose open source because of collaboration, not
competition. Please make James components more reusable, if not for 3,
then for 4. I don't need you to use my framework and I am not here to
sell it, I use James on a day to day basis and would like to contribute
my experience and knowledge, I would like to see James grow. Would you
exclude blind people from your website because you don't want to support
them? Of course not, instead what you do is improve your knowledge base
and make a better website. You need to be clear and make sure you are
not saying I can't use James components because you don't want to make
them more modular, this is profoundly wrong. Its not rocket science or
magic, it just requires engineering discipline. Sorry if I am a little
angry, I should not rant.
Guice is a great product, I am not saying don't use it. The problem is
not with guice, spring or platformed, its with component design. Its
important the problem is solved otherwise you might end up creating a
set of classes that are not tied to the Avalon framework, but still use
the same flawed design patterns, that would be basically moving the
problem from place A to place B.
> have talked about getting Avalon out of James for years, and finally
> the Guice effort is making some progress. Now you want to throw a
> wrench into everything with some new framework. Why? Let's just do one
> change at a time.
Again, this is missing the point, its about creating framework
independent components. At the moment if I want to use James components
I have to literally 'hack' them into bits to use them because I cant
'plug' them in. I am not saying don't use guice, I am saying structure
James code in a way that allows other people to use it in the way they
want, rather than forcing them to do it your way. I am asking you to
basically write code in a more modular fashion (this is what we are all
employed for isn't it?). You don't need to know about the framework,
only that James components (its not specific to James components) are
not as modular as they could be. I don't want to throw a wrench in, I
want to remove the one that is already in there (it was there before I
came along and any experienced programmer knows its there). I picked
James because I actually use James and want to continue using it.
One thing at a time is false economics, you want to aim for 'once' (its
basically about good api design, if you can solve the problem once,
people will pay you lots of money). I don't want to get in the way of
version 3, there looks some really cool stuff in it and if the effort to
do that is going well, it should carry on. However, if James components
are not designed for the future now they will have to be re-written
again some time soon (and if this is best for the community, is what
should be done). Why not just write them once and get it done with,
there are some very intelligent people working on James.
As a professional, I would advise the community to carry on exactly how
they are and get a release done if that is the collective objective, I
will help with documentation in that case. However I would also advise
keeping your mind 'open', there are frameworks out there that are 'very'
good (.mil) and you should probably broaden your horizons. Platformed
is, for all other purposes, a very primitive passive neural net (there
are no learning components to make it active...yet), each machine is
effectively a neuron. Its not a new design, its been around for some
time. Its just an open source implementation of how things are done in
some private (very well funded) arenas. Guice is an injection framework,
Platformed in an on-demand server application framework, they are not
the same thing. I am thinking of using Guice to do some things in
Platformed.
Hi Simon,
im still confused, But sometimes a little Bit Code can help others to
get it.... So please show us some code.
Bye
Norman
I have probably confused matters, too much/too little, I apologise. I am
assuming you mean show you how I would write James components to be more
flexible. I have been working on a strategy to rewrite the SMTP package
but I think this is the wrong approach. The reason why is discussed here.
How To Design A Good API and Why it Matters Google Tech Talks January
24, 2007
http://video.google.co.uk/videoplay?docid=-3733345136856180693&hl=en#
This is a great presentation and well worth a watch if you have not
watched it before, I highly recommend it.
I am prepared to put my framework aside for the moment and contribute
documentation to James. This will help me to learn more about the
specifics of James and having a broader view of the current James
architecture (I can do diagrams) will be useful to everyone. In this way
I can add value without stalling the current effort.
How does that sound?
Thanks for your patience,
Simon
>
> Eric MacAdie
>
>
>
> Simon Funnell wrote:
>> Hi,
>>
>> I've been studying the code base and evaluating which approach is the
>> best to take. At present, I will try to clarify the framework
>> architecture I am using as 'evangelising' might actually be quite
>> effective. Firstly, there is nothing new to learn, the architecture
>> is exactly the same as the internet. Applications are networks of
>> machines and each machine on the network is a component of one or
>> more applications. Each machine/component specialises in doing one
>> small job very well, for example you could have a network of
>> machines/components that do something like the following.
>>
>> Machine 1's job is to listen on a port for new client connections,
>> when a new client comes along, it is accepted and the resulting
>> socket is contextualised. A context is basically an object that
>> maintains state, such as application/session/request or whatever is
>> relevant. This context/state is then dispatched to Machine 2.
>>
>> Machine 2 will now alter the context state by reading bytes from the
>> socket and buffering them in some suitable form before dispatching to
>> Machine 3.
>>
>> Machine 3 is responsible for taking the buffered content and parsing
>> into some sort object model, like an email. If an error occurs this
>> machine will forward it to an exception handling machine which
>> implements the exception policy of the system (Machine 4?).
>> Otherwise, it is sent to the next machine (Machine 5?) which does
>> some sort of matching or something
>>
>> And so on.
>>
>> Now if we want to do some logging at any time, we just put a logging
>> machine in between Machine 1 and 2 or between 2 and 3 to check socket
>> state, buffer state or some other combination of properties.
>>
>> To relate this somewhat to James, you are currently using Guice to
>> wire components together and.this is equivalent to defining a
>> network. With the design I am following there is not really a
>> configuration as such, they are basically replaced with policy
>> statements and change logs, which are a kind of configuration. Its
>> fair to say James would be a network of machines defined by a policy
>> statement. These policy statements can be large or small and applied
>> to new or already running systems. Newer policy statements always
>> supersede older ones (unless a prior policy forbids it). James
>> already has a fairly substantial default policy statement like thing,
>> its configuration files. However, to understand the benefits of
>> policy statements, instead of configurations, it is worth looking at
>> a small example.
>>
>> In this example I have an installation of James, it does not do any
>> logging at present but I want to do some. And this is what I want to do:
>>
>> For the next two days, between the hours of the 3-5 pm, if a client
>> using the SMTP service invokes the Expn command and has a IP address
>> matching some address log some information to this file in this
>> format and if its between 3.45pm and 4.00pm send me an email with
>> such and such information.
>>
>> This policy is expressed (but doesn't have to be) as an XML document.
>> The document is evaluated and the necessary machines are added to the
>> network to implement the policy. How the policy is defined and
>> implemented is system independent. The new machines added to the
>> network that implement the policy usually simply fit in between two
>> existing machines and do stuff like filtering, logging, security and
>> anything that makes logical sense (putting a logging machine that
>> records information about a http request in an smtp service doesn't
>> make logical sense). Here, Java is used to create very small, highly
>> robust units of functionality with XML policy statements (closer to
>> natural language and easy to learn for users) stating how the
>> machines should interact. The sophistication of policy statements is
>> increased as the size and collective ability of a community of
>> machines/components in a given installation grows.
>>
>> The current mailets and matchers are sort of ideal for use in this
>> system as they can (in principle) be easily wrapped/adapted or what
>> not to become a network machine. A machine here is simply an object
>> with a single method (I've been thinking about using the reflection
>> api's to extract individual methods from existing James classes and
>> assign them to individual machines but its not straight forward as
>> the existing methods in James classes tend to mix concerns together,
>> like application logic and writing to an output in a single method)
>>
>> I hope this is of interest, declarative logging is really easy to
>> implement and requires no learning, only a subtle shift in approach.
>> James could be implemented as a guice module and a policy statement,
>> in fact it would probably would be fairly easy to generate guice
>> modules from a policy statement. Its not a direct translation but
>> they are not dissimilar in nature. Also I am actually thinking of
>> employing guice myself to do a few things, I am not in competition
>> with guice. I am not in competition with spring either because its
>> got a load of stuff with it mine doesn't. That said, Platformed is
>> not 'just another framework', it has features (an architecture) that
>> make it distinct.
>>
>> I'm thinking about a re-factoring strategy but I am still learning
>> James under the hood. Ideally I would like to convince you of the
>> benefits, they are fairly obvious.
>>
>> Regards,
>>
>> Simon
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
> For additional commands, e-mail: server-dev-help@james.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: As Above, So Below
Posted by Eric MacAdie <er...@MacAdie.net>.
I run James on a VPS host with about 300 MB of memory. Now you are
talking about multiple machines. It sounds to me like you want to
complicate James unnecessarily to try out some new framework (that
nobody else on the list seems to get). Why should James be the Guinea
pig for some new framework? I think that the effort to "Guiceyfy" should
just continue as is.
People have talked about getting Avalon out of James for years, and
finally the Guice effort is making some progress. Now you want to throw
a wrench into everything with some new framework. Why? Let's just do one
change at a time.
Eric MacAdie
Simon Funnell wrote:
> Hi,
>
> I've been studying the code base and evaluating which approach is the
> best to take. At present, I will try to clarify the framework
> architecture I am using as 'evangelising' might actually be quite
> effective. Firstly, there is nothing new to learn, the architecture is
> exactly the same as the internet. Applications are networks of
> machines and each machine on the network is a component of one or more
> applications. Each machine/component specialises in doing one small
> job very well, for example you could have a network of
> machines/components that do something like the following.
>
> Machine 1's job is to listen on a port for new client connections,
> when a new client comes along, it is accepted and the resulting socket
> is contextualised. A context is basically an object that maintains
> state, such as application/session/request or whatever is relevant.
> This context/state is then dispatched to Machine 2.
>
> Machine 2 will now alter the context state by reading bytes from the
> socket and buffering them in some suitable form before dispatching to
> Machine 3.
>
> Machine 3 is responsible for taking the buffered content and parsing
> into some sort object model, like an email. If an error occurs this
> machine will forward it to an exception handling machine which
> implements the exception policy of the system (Machine 4?). Otherwise,
> it is sent to the next machine (Machine 5?) which does some sort of
> matching or something
>
> And so on.
>
> Now if we want to do some logging at any time, we just put a logging
> machine in between Machine 1 and 2 or between 2 and 3 to check socket
> state, buffer state or some other combination of properties.
>
> To relate this somewhat to James, you are currently using Guice to
> wire components together and.this is equivalent to defining a network.
> With the design I am following there is not really a configuration as
> such, they are basically replaced with policy statements and change
> logs, which are a kind of configuration. Its fair to say James would
> be a network of machines defined by a policy statement. These policy
> statements can be large or small and applied to new or already running
> systems. Newer policy statements always supersede older ones (unless a
> prior policy forbids it). James already has a fairly substantial
> default policy statement like thing, its configuration files. However,
> to understand the benefits of policy statements, instead of
> configurations, it is worth looking at a small example.
>
> In this example I have an installation of James, it does not do any
> logging at present but I want to do some. And this is what I want to do:
>
> For the next two days, between the hours of the 3-5 pm, if a client
> using the SMTP service invokes the Expn command and has a IP address
> matching some address log some information to this file in this format
> and if its between 3.45pm and 4.00pm send me an email with such and
> such information.
>
> This policy is expressed (but doesn't have to be) as an XML document.
> The document is evaluated and the necessary machines are added to the
> network to implement the policy. How the policy is defined and
> implemented is system independent. The new machines added to the
> network that implement the policy usually simply fit in between two
> existing machines and do stuff like filtering, logging, security and
> anything that makes logical sense (putting a logging machine that
> records information about a http request in an smtp service doesn't
> make logical sense). Here, Java is used to create very small, highly
> robust units of functionality with XML policy statements (closer to
> natural language and easy to learn for users) stating how the machines
> should interact. The sophistication of policy statements is increased
> as the size and collective ability of a community of
> machines/components in a given installation grows.
>
> The current mailets and matchers are sort of ideal for use in this
> system as they can (in principle) be easily wrapped/adapted or what
> not to become a network machine. A machine here is simply an object
> with a single method (I've been thinking about using the reflection
> api's to extract individual methods from existing James classes and
> assign them to individual machines but its not straight forward as the
> existing methods in James classes tend to mix concerns together, like
> application logic and writing to an output in a single method)
>
> I hope this is of interest, declarative logging is really easy to
> implement and requires no learning, only a subtle shift in approach.
> James could be implemented as a guice module and a policy statement,
> in fact it would probably would be fairly easy to generate guice
> modules from a policy statement. Its not a direct translation but they
> are not dissimilar in nature. Also I am actually thinking of employing
> guice myself to do a few things, I am not in competition with guice. I
> am not in competition with spring either because its got a load of
> stuff with it mine doesn't. That said, Platformed is not 'just another
> framework', it has features (an architecture) that make it distinct.
>
> I'm thinking about a re-factoring strategy but I am still learning
> James under the hood. Ideally I would like to convince you of the
> benefits, they are fairly obvious.
>
> Regards,
>
> Simon
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org