[James Wiki] Update of "ChauLe" by ChauLe

[Apache Wiki <wi...@apache.org>]

Dear Wiki user,

You have subscribed to a wiki page or wiki category on "James Wiki" for change notification.

The following page has been changed by ChauLe:
http://wiki.apache.org/james/ChauLe

New page:
Hello!

I'm Le Dinh Bao Chau. I study at http://www.ntu.edu.sg/sce/ School of Computer Engineering.
I'm keen of programming and I have been programming for seven years. 

My friends tell me that I tend to think and code a bit too academically but I hope this will only help me to participate in the summer of code program. Here's my proposal.

=== Problem ===
James mail server performs reasonably well under normal load but heavy
spam traffic and dos attacks significantly degrade performance. Under
the current design the main process of decision making about each particular
mail message happens after the message has been fully received and filed
to disk or database. Combating spam requires us to adopt an alternative
strategy - we may wish to just cut TCP connections short based on IP and
SMTP commands given before we even start to accept the message.

Another issue is that under the current design very little control
is possible over a running instance of James mail server - the main
configuration method is to edit configuration manually and then restart
(which takes significant time).

=== Proposed solutions ===
Following FastFail proposal by DannyAngus and NoelBergman I propose
to implement a drop-in replacement for
[http://james.apache.org/javadocs/org/apache/james/smtpserver/SMTPServer.html SMTPServer]
block.

In spirit the new block will follow the design of
[http://james.apache.org/javadocs/org/apache/james/transport/JamesSpoolManager.html JamesSpoolManager].
While the later hosts a family of matcher - mailet pairs forming a "tree of responsibility" the
new SMTP block will host a sequence of "protocolLets" - actors for line by line processing
of SMTP protocol messages.

Technically however it is going to be a bit different.

Firstly I intend to allow re-configuration, reordering, addition and removal of "protocolLets" via JMX. To facilitate this I'm going to find a way to map:
 * "protocolLets" bean-style configuration interface
 * JMX allowed CompositeData and TabularData
 * XML representation thereof (xstream-style)
 * possibly Phoenix configuration objects
onto each other.

As configuration of "protocolLets" and their order is changed via JMX this immediately affects the running block - in fact I imagine that we would take down "protocolLets" configured with old values and instantiated "protocolLets" with new configuration to avoid concurrency issues. Same applies to the protocolLet[] array representing their sequence. Each protocolLet is going to be stateless and thus tread-safe. Perhaps we can design JMX configuration interface so that all the reordering and reconfiguration will take effect in a batch as partially configured "protocolLets" may cause server failures.

A key point here is to let the "protocolLets" see other blocks visible to the new SMTP block itself. This will simplify implementing such features as communicating outer processes (perl-written policies taken from postfix) or talking TCP to outer world hosts (RBL) - none of these actions actually belongs to "protocolLets" - little stateless decision makers.

Then I'm going to add a JMX method for saving this configuration on disk. (Does Tomcat do that already?).
One interesting possibility would be to allow new SMTP block to save directly to SVN - thus making
provision for possible failures.

If timeframe permits we may consider doing beanshell integration - there's nothing impossible in "protocolLets" being written in beanshell, or probably jython, javascript. In this case the code of the "protocolLet" becomes its configuration.

I believe that this "go from getter/setters via JMX CompositeData to XML will let some fresh wind into the old dynamic server reconfiguration problem.

Technical implementation of these features still needs to be explored. In essence "protocolLets" are quite similar to blocks but Phoenix machinery most likely can not be reused as it is. I see the following options to investigate and choose from:
 * go for some quirks and tricks and make use of existing Phoenix internals as they are
 * create a patched Phoenix (and push for new version being rolled out with these patches)
 * run.. say Fortress inside new SMTP block (I'm trying to pick a container living inside ASF)
 * DIY container plumbing pico-container style (shouldn't be too much code) closely along the lines of the current [http://james.apache.org/javadocs/org/apache/james/transport/JamesSpoolManager.html JamesSpoolManager] implementation

=== Advantages for ASF ===
 * A solid foundation for implementing a feature heavily demanded by current internet realities - SMTP fast fail
 * Possible a new insight on the issue of reconfiguring components inside IoC containers

=== Deliverables ===
Totally revamped drop-in replacement for the existing [http://james.apache.org/javadocs/org/apache/james/smtpserver/SMTPServer.html SMTPServer]. Possibly patches or pieces of coded targeted for ASF containers.

=== Schedule ===
 * start active coding around June the 22nd, the expected end of examination session
 * deliver version 0.1 by July the 12th, this version should prove that configuration and more importantly reconfiguration of "protocolLets" along the path: "inspect class via reflection - build JMX type descriptor - configure via JMX - save to XML" is feasible and desirable
 * deliver version 0.2 by July 25 - "protocolLets" actually able to see other blocks visible to SMTP server (they get them via setter methods), the new SMTP block actually able to accept main and feed it into the existing processor pipeline
 * deliver version 0.5 by August 14 - some protocolLets actually make sense and do some useful Spam blocking

After August 14 fix react to bug reports and requests for enhancements. If time permits look into integration with scripting languages.

=== Appendix - Expected protocolLet interface ===
(Subject to change)
{{{
  interface ConnectionHandler (
    RulesResponce onConnection(Socket)
  }

  interface CommandsHandler {
    RulesResponce Map getCommands()
  }

  interface CommandHandler {
    RulesResponce onCommand(...)
  }

  interface MessageHandler {
    RulesResponce onMessage(...)
  }

  RulesResponse {
    boolean getSuccess();
    SMTPResponseCode getSMTPResponseCode();
    DSNResponseCode getDSNResponseCode();
    String getMessage();
    boolean isResponseMultiline();
  }
}}}

Which has been shamelessly compiled out of FailFast and [http://mail-archives.apache.org/mod_mbox/james-server-dev/200506.mbox/%3cOF3C9E9454.0BC764AD-ON80257019.002CAB01-80257019.002E29DE@slc.co.uk%3e  "07 Jun 2005 message on server-dev”]