You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Marco Beelen <ja...@mcbeelen.net> on 2005/07/13 01:58:10 UTC
[ GSoC - failfast SMTP ] Architecture and suggestion
Hello Anagha ( and of course many others ),
Although I'm not an active developer for james yet ( besides being an
active user, which already is the case and very satisfied one ), I've
been monitoring the mailing-list for some while and do have some ideas
about the architecture of the product.
Your design for the fail-fast SMTPServer triggered me to post my ideas.
I'll start by some general outline of my vision about the architecture
and finish with a suggestion for the Summer of Code proposal.
Juan Carlos Murillo posted a message about James as a distributed
enterprise application suite, which I agree upon.
( http://www.mail-archive.com/server-dev@james.apache.org/msg04325.html )
I think implementations for a handler of a network protocol ( SMTP,
POP3, IMAP and NNTP ) should be loosly coupled from spoolmanager and
user- or mail-repositories. Access to the spoolmanager,
user-repositories and mailrepositories should be able to be done eighter
locally within the same JVM ( single-machine-configuration ) of remotely
( possible using JMS ) in a 'clustered enterprice environment'.
In order to make this work, we would have to seperate the front-end (
the actual handling of the network protocol ) from the back-end (
reading and writing to JamesConfiguration, UserRepositories,
MailRepositories and the SpoolManager ). During the startup of a Server
it should be configured on how to access the backend. Dependancy
Injection ( by usage of Spring ) seems like a logical solution for that
part.
By doing this you could concentrate on the actual SMTP-protocol when
writing an implementating for a SMTPProtocolHandler and delegate
backend-interaction to a implementation of some 'JamesBackendInterface'.
This could simplify the implementation of the failfast SMTP-server,
especially since your design for the ProtocolHandler looks very simular
to the design of MINA.
"MINA is a network application framework to resolve all the issues
concerned with implementing any protocol stack in Java without
sacrificing performance nor scalability." MINA has been developed as a
sub-project of the directory project of the ASF. ( I consider this as an
indication of the quality of your design. )
Therefor I would like to suggest that you take a look at Mina and
consider it as the basis for your implementation.
This is something which should be agreed upon by you and the active
james-committers.
Suggested materials for reading about MINA:
http://directory.apache.org/subprojects/network/index.html
http://wiki.apache.org/directory/MinaTutorial
I know this is properably a fuzzy description about my ideas for the
architecture and I'm willing to clearify any questions about it.
I'll start working on some UML-diagrams to improve the 'readibility' and
put those up for discussion in a seperate threat.
With kind regards,
Marco Beelen
The Netherlands ( GMT +1:00 )
Anagha Mudigonda wrote:
>Hi,
>I did a little more work on the design and fixed some
>mistakes. Attached is an updated design.
>
>awaiting comments ...
>-- anagha
>
>--- Danny Angus <Da...@slc.co.uk> wrote:
>
>
>
>------------------------------------------------------------------------
>
>
>Introduction:The redesign of the SMTP protocol handler has the following design goals
> - to enable in-protocol handling
> - to have enough flexibility to implement SMTP protocol extensions without changing protocol handling code.
>
>The new framework is designed to address these. The framework contains the following classes
> - CommandHandler
> - ConnectionHandler
> - SMTPSession
> - SMTPProtocolHandler
> - SMTPOutputStream
> - SMTPInputStream
> - an XML configuration file.
>
>The main change:
> - Introduction of SMTPSession as a new interface.
> - Things are more CommandHandler centric. The command handler has access to the SMTPSession. So, the command handler now has freedom to close a message or even close a session. But the SMTPSession interface is implemented in the james core code.
>
>Class ConnectionHandler
>-----------------------
>1. Registers to handle a connection
>2. Registers to handle a message
>
>interface ConnectionHandler
>{
>void init(String configStr);
>void processConnection(String local_hostname, int local_port, String remote_hostname, int remote_port);
>void setSMTPSession(SMTPSession session);
>void processMessage(Mail Obj);
>}
>
>Class CommandHandler
>--------------------
>
>A Commandhandler class
>
>1. Registers to handle a command ( may process or validate the command arguments)
>2. Registers to handle a message (validate message or modify message header or body)(this is needed for situations like SPF failure, where the headers are updated if SPF fails.)
>
>interface CommandHandler
>{
> void init(String configStr);
> void processCommand(String cmdString);
> void setSMTPSession(SMTPSession session);
> void processMessage(Mail Obj);
>}
>
>Description: There can be more than one command handler registered for a command.
>CommandHandlers can be registered to validate commands or modify message.
>CommandHandler's *process* methods can trigger events like message start , message end and message abort
>and session end, session abort.
>
>Class SMTPSession
>-----------------
>The SMTP protocol session maintains the state information and ensures that
>the commands are executed in proper order. Also it provides controlled
>access to I/O streams.
>
>The SMTP session can be in one of the 4 states:
>1. SESSION_START ( as soon as the client connects to server)
>2. SESSION_END ( as soon as the client Sends QUIT command or closes connection)
>3. MESSAGE_START ( as soon as the client issues MAIL FROM command)
>4. MESSAGE_END ( as soon as the client sends message after DATA command)
>
>These states are more like triggers. For instance, all the message handlers get triggered on MESSAGE_END.
>
>interface SMTPSession
>{
>
> //Write Response
> void WriteResponse(String respString);
>
>//Register connection handlers
>void registerConnectionHandler(ConnectionHandler connHandler);
>//Register connectionhandlers to handle Message
>void registerMessageHandler(ConnectionHandler connHandler);
>
> //Register commandhandlers to command
> void registerCommandHandler(String cmdPrefix, CommandHandler cmdHandler);
> //Register commandhandlers to handle Message
> void registerMessageHandler(CommandHandler cmdHandler);
>
> //Command sequence handling
> void addCommandSequenceMap(string command, String[] predecessorCmds, string failMessage); //
> bool isCommandAllowed(String cmdString);
>
>
> //State information
> String getSender();
> String[] getRecepients();
> String setSender(String sender);
> String addRecepient(String recepient);
> String addRecepient(String[] recepients);
> String removeRecepient(String[] recepient);
> String removeRecepient(String recepient);
>
> //Message state modifier
> void startMessage();
> void endMessage();
> void abortMessage();
>
> //Message state modifier
> void endSession();
> void startSession();
> void abortSession();
>
> int getState(); //SESSION_START, SESSION_END, MESSAGE_BEGIN, MESSAGE_END
>
> void reset();
>
> SMTPInputStream getInputStream(); // object with no permission to close stream
> SMTPOutputStream getOutputStream(); // Output stream object with no permission to close stream
>}
>
>
>SMTPSession is per connection and is accessible to all commandhandlers.
>If a command handler needs to modify the message, it registers with SMTPSession object.
>And SMTPsession object registers after the message is received passes the message to
>all the registered commandhandlers for processing.
>
>class SMTPProtocolHandler
>-------------------------
>
>class SMTPProtocolHandler implements SMTPSession
>{
> //load all handlers and configure these
> //configure the command sequence
>
> //wait for SMTP connection
> //Create session state object
> //check if the command is allowed else respond with the failure message
>
> //then loop thru the command handlers registered with the command and process the command
>
> //When Message end is triggered, call all the handlers registered for the message
>
>}
>
>
>XML Configuration
>-----------------
>
> <smtpserver enabled="true">
> <!-- port 25 is the well-known/IANA registered port for SMTP -->
> <port>25</port>
>
> <!-- NEW CONFIGURATION -->
>
> <Protocolhandler class="org.apache.james.SMTPHandler">
> <Session class=org.apache.james.SMTPSession>
> <command name="MAIL" commandSequence="HELO,EHLO"/>
> <command name="RCPT" commandSequence="RCPT"/>
> <event sessionStartCommand=""/>
> <event sessionEndCommand="QUIT"/>
> <event MessageBeginCommand="MAIL"/>
> <event MessageEndCommand="DATA"/>
> </Session>
>
> <!-- registers with Protocol handler to handle "MAIL" command -->
> <CommandHandler class="org.apache.james.MAILCommandHandler">
> config..
> </CommandHandler>
>
> <!-- registers with protocol handler to handle "MAIL" command
> also registers with protocol handler to handle Message (SPFhandler adds a SPF details into Message header)
> -->
> <CommandHandler name="SPFHandler">
> config...
> </CommandHandler>
>
> <CommandHandler name="org.apache.james.RCPTCommandHandler">
> config
> </CommandHandler>
>
> <CommandHandler name="RecepientValidator">
> config
> </CommandHandler>
>
> </protocolhandler>
> </smtpserver>
>
>Since the classes are loaded based on XML configuration, the default handlers can be replaced by custom handlers whenever needed.
>
>
>
>Ease of use of the framework
>------------------------------
>
>1. New commands can be added (for eg: STARTTLS)
> Add new command handlers and modifiying the config to include new command sequence.
>
>2. New Validations (SPF validation)
> Add a new command handler.
>
>
>
>
>------------------------------------------------------------------------
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
>For additional commands, e-mail: server-dev-help@james.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [ GSoC - failfast SMTP ] Architecture and suggestion
Posted by Mike Heath <mh...@avaloninc.com>.
On Fri, 2005-07-15 at 10:23 +0200, Stefano Bagnara wrote:
> I think that you could look better to Mina and talk with Mike Heath that was
> developing an SMTP implementation over Mina. (search for "Mina based SMTP
> handler" in the archives): this probably will make your work easier and it
> would be good for the Apache community to share common libraries.
>
> E.g: if we use Mina we will get JMX for the protocol handlers for free as
> soon as they implement it (already in roadmap for Mina 1.1).
My MINA based SMTP protocol handler is at http://hausmail.safehaus.org/
The code depends on MINA trunk and won't compile at the moment because
of some changes to MINA. I should have this issue resolved over the
weekend though.
HausMail implements STARTTLS and already has some fast fail support if
TLS is required. It is also very fast and optimized for performance.
It does zero copy all the way up the protocol stack so it can receive
data very efficiently.
I haven't tested HausMail for load and performance much yet but I have
access to some of the super computers at Brigham Young University as
part of my grad school studies and I plan to take full advantage of
them.
There are issues to using MINA though. Using TLS with MINA requires
Java 5.0. Also, Noel doesn't seem to excited to depend heavily on MINA
(which, for the record, I don't agree with). I think the advantages of
MINA heavily out weight the cons. Aside from SMTP, I honestly can't see
JAMES having a scalable IMAP server without using a Staged Event Driven
Architecture (SEDA) architecture and MINA gives us that for free.
If there are questions about HausMail and/or MINA, I'm more than happy
to try and answer them.
-Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [ GSoC - failfast SMTP ] Architecture and suggestion
Posted by Stefano Bagnara <ap...@bago.org>.
> From what I understand you are proposing an architecture
> based on MINA. Currently JAMES is built on Avalon 4.0 server
> frame work. I have designed the protocol handler framework in
> such a that it can be ported to MINA as well.
>
> The IN protocol framework expects the following 3 things from
> the underlying server kernel
>
> ProtocolHandler to be run as separate thread
> Socket object is passed to it
> A mechanism to read the XML config file
>
> In future if JAMES migrates to MINA architecture, I think
> James.java can be modified and no change need be done to the
> protocol handler framework that I am proposing.
You are comparing MINA to Phoenix. I think a more appropriate comparison
would be "your Protocolo Handler Framework" against "MINA". MINA is a
network handling framework and not a container. MINA will support
integration with few containers in future (Spring, OSGi).
IMHO most of your proposal is really similar to what MINA already do.
I think that you could look better to Mina and talk with Mike Heath that was
developing an SMTP implementation over Mina. (search for "Mina based SMTP
handler" in the archives): this probably will make your work easier and it
would be good for the Apache community to share common libraries.
E.g: if we use Mina we will get JMX for the protocol handlers for free as
soon as they implement it (already in roadmap for Mina 1.1).
Stefano
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org
Re: [ GSoC - failfast SMTP ] Architecture and suggestion
Posted by Anagha Mudigonda <an...@yahoo.com>.
Hi Marco,
Thank you for your comments.
>From what I understand you are proposing an
architecture based on MINA. Currently JAMES is built
on Avalon 4.0 server frame work. I have designed the
protocol handler framework in such a that it can be
ported to MINA as well.
The IN protocol framework expects the following 3
things from the underlying server kernel
ProtocolHandler to be run as separate thread
Socket object is passed to it
A mechanism to read the XML config file
In future if JAMES migrates to MINA architecture, I
think James.java can be modified and no change need be
done to the protocol handler framework that I am
proposing.
Best Regards
Anagha
--- Marco Beelen <ja...@mcbeelen.net> wrote:
> Hello Anagha ( and of course many others ),
>
> Although I'm not an active developer for james yet (
> besides being an
> active user, which already is the case and very
> satisfied one ), I've
> been monitoring the mailing-list for some while and
> do have some ideas
> about the architecture of the product.
>
> Your design for the fail-fast SMTPServer triggered
> me to post my ideas.
> I'll start by some general outline of my vision
> about the architecture
> and finish with a suggestion for the Summer of Code
> proposal.
>
> Juan Carlos Murillo posted a message about James as
> a distributed
> enterprise application suite, which I agree upon.
> (
>
http://www.mail-archive.com/server-dev@james.apache.org/msg04325.html
> )
>
> I think implementations for a handler of a network
> protocol ( SMTP,
> POP3, IMAP and NNTP ) should be loosly coupled from
> spoolmanager and
> user- or mail-repositories. Access to the
> spoolmanager,
> user-repositories and mailrepositories should be
> able to be done eighter
> locally within the same JVM (
> single-machine-configuration ) of remotely
> ( possible using JMS ) in a 'clustered enterprice
> environment'.
> In order to make this work, we would have to
> seperate the front-end (
> the actual handling of the network protocol ) from
> the back-end (
> reading and writing to JamesConfiguration,
> UserRepositories,
> MailRepositories and the SpoolManager ). During the
> startup of a Server
> it should be configured on how to access the
> backend. Dependancy
> Injection ( by usage of Spring ) seems like a
> logical solution for that
> part.
>
> By doing this you could concentrate on the actual
> SMTP-protocol when
> writing an implementating for a SMTPProtocolHandler
> and delegate
> backend-interaction to a implementation of some
> 'JamesBackendInterface'.
>
> This could simplify the implementation of the
> failfast SMTP-server,
> especially since your design for the ProtocolHandler
> looks very simular
> to the design of MINA.
>
> "MINA is a network application framework to resolve
> all the issues
> concerned with implementing any protocol stack in
> Java without
> sacrificing performance nor scalability." MINA has
> been developed as a
> sub-project of the directory project of the ASF. ( I
> consider this as an
> indication of the quality of your design. )
>
> Therefor I would like to suggest that you take a
> look at Mina and
> consider it as the basis for your implementation.
>
> This is something which should be agreed upon by you
> and the active
> james-committers.
>
> Suggested materials for reading about MINA:
>
http://directory.apache.org/subprojects/network/index.html
> http://wiki.apache.org/directory/MinaTutorial
>
>
>
> I know this is properably a fuzzy description about
> my ideas for the
> architecture and I'm willing to clearify any
> questions about it.
> I'll start working on some UML-diagrams to improve
> the 'readibility' and
> put those up for discussion in a seperate threat.
>
> With kind regards,
> Marco Beelen
> The Netherlands ( GMT +1:00 )
>
>
>
> Anagha Mudigonda wrote:
>
> >Hi,
> >I did a little more work on the design and fixed
> some
> >mistakes. Attached is an updated design.
> >
> >awaiting comments ...
> >-- anagha
> >
> >--- Danny Angus <Da...@slc.co.uk> wrote:
> >
> >
> >
>
>------------------------------------------------------------------------
> >
> >
> >Introduction:The redesign of the SMTP protocol
> handler has the following design goals
> > - to enable in-protocol handling
> > - to have enough flexibility to implement SMTP
> protocol extensions without changing protocol
> handling code.
> >
> >The new framework is designed to address these. The
> framework contains the following classes
> > - CommandHandler
> > - ConnectionHandler
> > - SMTPSession
> > - SMTPProtocolHandler
> > - SMTPOutputStream
> > - SMTPInputStream
> > - an XML configuration file.
> >
> >The main change:
> > - Introduction of SMTPSession as a new interface.
> > - Things are more CommandHandler centric. The
> command handler has access to the SMTPSession. So,
> the command handler now has freedom
> to close a message or even close a session. But the
> SMTPSession interface is implemented in the
> james core code.
> >
> >Class ConnectionHandler
> >-----------------------
> >1. Registers to handle a connection
> >2. Registers to handle a message
> >
> >interface ConnectionHandler
> >{
> >void init(String configStr);
> >void processConnection(String local_hostname, int
> local_port, String remote_hostname, int
> remote_port);
> >void setSMTPSession(SMTPSession session);
> >void processMessage(Mail Obj);
> >}
> >
> >Class CommandHandler
> >--------------------
> >
> >A Commandhandler class
> >
> >1. Registers to handle a command ( may process or
> validate the command arguments)
> >2. Registers to handle a message (validate message
> or modify message header or body)(this is needed for
> situations like SPF failure, where the headers are
> updated if SPF fails.)
> >
> >interface CommandHandler
> >{
> > void init(String configStr);
> > void processCommand(String cmdString);
> > void setSMTPSession(SMTPSession session);
> > void processMessage(Mail Obj);
> >}
> >
> >Description: There can be more than one command
> handler registered for a command.
> >CommandHandlers can be registered to validate
> commands or modify message.
> >CommandHandler's *process* methods can trigger
> events like message start , message end and message
> abort
> >and session end, session abort.
> >
> >Class SMTPSession
> >-----------------
> >The SMTP protocol session maintains the state
> information and ensures that
> >the commands are executed in proper order. Also it
> provides controlled
> >access to I/O streams.
> >
> >The SMTP session can be in one of the 4 states:
> >1. SESSION_START ( as soon as the client connects
> to server)
> >2. SESSION_END ( as soon as the client Sends QUIT
> command or closes connection)
>
=== message truncated ===
Asatoma Satgamaya (Lead me From falsehood to truth)
Tamasoma Jyotirgamaya (From darkness to light)
Mryutorma Amrutangamaya (From death to immortality)
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com
---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org