You are viewing a plain text version of this content. The canonical link for it is here.
Posted to server-dev@james.apache.org by Sid Stuart <si...@weaselworks.com> on 2003/04/20 16:49:15 UTC
Supporting multiple "root processor"s in James 1.3?
Hi,
This suggestion was sparked by Kenny Smith's letter on the James Dev
mailing list on April 16. In it, he requested a mechanism to reinject
mail delivered to the error repository.
My first thought was he needed a tool to resubmit the mail to the
JamesSpoolManager to be run through the "root processor" again. But that
submits the message to some tests, such as spammerlist matching, that
really don't need to be done and to other tests that might generate
erroneous or duplicate error messages.
The cleanest solution would be to have a second "root processor",
defined specifically to deal with mail queued in the error repository.
This root-error-message-processor could then be run on a regular
interval to scan the error-message repository. Changes in the
configuration, such as new aliases in Mr. Smith's case, would then allow
the mail to be delivered. One nice aspect of this solution is that
amailet could be defined that would delete error messages after a
defined period of time.
I hesitated to suggest this solution though as it did not seem like a
significant enough requirement to mandate such a major architectural
change. Then I saw a note on the inclusion of the CMU Sieve mechanism on
the James Wiki V3 Plans page.
(http://nagoya.apache.org/wiki/apachewiki.cgi?JamesV3)
<http://nagoya.apache.org/wiki/apachewiki.cgi?JamesV3>
Sieve defines a language to "parse, sort and filter" a user's mail on
the server. The mailet package makes implementation of this feature
quite simple. One simply writes a mailet that runs the user's sieve
script before final mail delivery.
The problem with this is that one would also like to run new or edited
sieve scripts on mail that has already been delivered. This creates the
need for a "root processor" that will process delivered messages. With
this second feature request, it starts to look a more generalized
message processor facility is needed.
Sid
Re: Supporting multiple "root processor"s in James 1.3?
Posted by Sid Stuart <si...@weaselworks.com>.
Noel J. Bergman wrote:
>You are engaging in a fiction. There is absolutely nothing magical about
>the ROOT processor. There is no "tree", non-looping or otherwise. ROOT is
>simply the default entry point for new messages, whether created internally
>or spooled via the SMTP handler. All that I'm contemplating is adding a
>convenience method and a new XML element to override the default.
>
First off, let me say, I agree with what you are contemplating. I think
it is a good idea. It is a part of what I was proposing in the letter
that started this thread. I was using the phrase root processor to mean
entry point as I thought that would make more sense to the reader.
Apparently I was mistaken.
>>The problem is it is not obvious whether a mailet
>>sets a new processor queue and which processor it
>>does call when defining the XML tree.
>>
>>The Mailet that does this is called ToProcessor. Quite obvious, except for
>>the ones that cause messages to start over at ROOT.
>>
Yes, I understand the function of the ToProcessor mailet, it is
transparent in what it does and I have no problems with it. The concern
I was expounding on (poorly) was the method defined in the Mail
interface called setState(String state). My understanding from previous
discussions is that this method allows the mailet to redirect the
processing to a new processor. My concern was that this adds obscure
connections to the graph that would make loops hard to see/debug. My
suggestion was to change setState to be a non-argument method that would
pick up the value of the new state from a parameter in the configuration
file. This would make it easier to see/debug infinite loops, should they
occur. It would also allow the names of processor's to be changed
without requiring a recompile of any mailets that use the method. (Not
that I am suggesting that any processor's be renamed, heaven forbid. :)
>>I very much believe having multiple root processors that
>>act on messages with different attributes (spooled,
>>delivered, dead-letter...) to be a big win.
>>
>>
>
>Minus the misleading term "root", my configuration is already pretty much
>that way. All of the tools are already present.
>
Okay, to use your terminology, I think there should be different sets of
directed graphs defined in the config file and a mechanism provided to
initiate processing by the matchers/mailets of one of those graphs on
different sets of messages: those that are being delivered (already
exists), those that have been delivered and those that have ended up in
the dead-letter repository. If I understand your proposal to provide
multiple entry points correctly, that will allow different sets of
directed graphs to be defined in the config file. So that will solve
half of the problem.
The other half is to have the capability to select a set of messages to
run through one of those graphs.
If the tools to do that exist, would you please point me to where they
are documented, because I have missed it. Better yet, if you could give
an example of how to solve Mr. Smith's need to reinsert dead-letter
messages back into the system, it would help both him and me.
>By the way, you keep referring to James 1.2 and 1.3. We're shipping v2.1.x,
>and planning version 3.
>
Yeah, I have a hard time with numbers, something to do with being
slightly dyslexic. My apologies for the mistake.
Sid
RE: Supporting multiple "root processor"s in James 1.3?
Posted by "Noel J. Bergman" <no...@devtech.com>.
> > you just need an option to specify which processor to inject the mail
> > into, e.g., the "transport" processor.
> I worry a bit about programatic selection of processors.
> The root processor defines a (hopefully) non-looping tree
> of processors.
You are engaging in a fiction. There is absolutely nothing magical about
the ROOT processor. There is no "tree", non-looping or otherwise. ROOT is
simply the default entry point for new messages, whether created internally
or spooled via the SMTP handler. All that I'm contemplating is adding a
convenience method and a new XML element to override the default.
> If a mailet selects a processor that is a point in the tree
> above where the mailet it normally called, it would introduce
> a processing loop.
There isn't a tree; there is a directed graph. And loops are more than
possible, although the conditions are often terminating. Turn on DEBUG for
the spool manager sometime, and watch the message flow.
> The problem is it is not obvious whether a mailet
> sets a new processor queue and which processor it
> does call when defining the XML tree.
The Mailet that does this is called ToProcessor. Quite obvious, except for
the ones that cause messages to start over at ROOT.
> I suggest requiring mailets that forward to new processors
> pick up the name of the processor from the XML definition.
ToProcessor already has that element. The SMTP server would also gain it.
And, so far I have heard not a good justification for renaming things,
although there is a need for better examples of what already exists.
> Also, while I have not made a strong case for it, I want
> to be able to use mailets to define a custom processing
> mechanism for mail that has been delivered.
That's what Mailets do.
> I very much believe having multiple root processors that
> act on messages with different attributes (spooled,
> delivered, dead-letter...) to be a big win.
Minus the misleading term "root", my configuration is already pretty much
that way. All of the tools are already present.
By the way, you keep referring to James 1.2 and 1.3. We're shipping v2.1.x,
and planning version 3.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Supporting multiple "root processor"s in James 1.3?
Posted by Danny Angus <da...@apache.org>.
I agree with Sid about that, but still think that being able to specify
which is "root" for each SMTP vhost would go a long way to supporting
address based hosting for SMTP.
-----Original Message-----
From: Sid Stuart [mailto:sid@weaselworks.com]
Sent: 20 April 2003 22:20
To: James Developers List
Subject: Re: Supporting multiple "root processor"s in James 1.3?
Noel J. Bergman wrote:
The cleanest solution would be to have a second "root processor",
defined specifically to deal with mail queued in the error repository.
No, you just need an option to specify which processor to inject the mail
into, e.g., the "transport" processor.
--- Noel
I haven't checked to see how this works, but I worry a bit about
programatic selection of processors. The root processor defines a
(hopefully) non-looping tree of processors. If a mailet selects a processor
that is a point in the tree above where the mailet it normally called, it
would introduce a processing loop.
The problem is it is not obvious whether a mailet sets a new processor
queue and which processor it does call when defining the XML tree. This is
not too much of a problem now, but could lead to a fragile system as the
number of mailets grows. One would need to understand the details of each
when building the tree in an XML definition to be sure loops are not
created.
I suggest requiring mailets that forward to new processors pick up the
name of the processor from the XML definition. In other words, change
setQueue to not accept an argument, but to use a parameter specified in the
XML file. (The name of the method could be refactored at the same time. :)
That way, the flow of control is defined in one location and loop detection
should be easy to do during the XML parsing phase.
Also, while I have not made a strong case for it, I want to be able to use
mailets to define a custom processing mechanism for mail that has been
delivered. I very much believe having multiple root processors that act on
messages with different attributes (spooled, delivered, dead-letter...) to
be a big win.
Sid
Re: Supporting multiple "root processor"s in James 1.3?
Posted by Sid Stuart <si...@weaselworks.com>.
Noel J. Bergman wrote:
>>The cleanest solution would be to have a second "root processor",
>>defined specifically to deal with mail queued in the error repository.
>>
>>
>
>No, you just need an option to specify which processor to inject the mail
>into, e.g., the "transport" processor.
>
> --- Noel
>
>
I haven't checked to see how this works, but I worry a bit about
programatic selection of processors. The root processor defines a
(hopefully) non-looping tree of processors. If a mailet selects a
processor that is a point in the tree above where the mailet it normally
called, it would introduce a processing loop.
The problem is it is not obvious whether a mailet sets a new processor
queue and which processor it does call when defining the XML tree. This
is not too much of a problem now, but could lead to a fragile system as
the number of mailets grows. One would need to understand the details of
each when building the tree in an XML definition to be sure loops are
not created.
I suggest requiring mailets that forward to new processors pick up the
name of the processor from the XML definition. In other words, change
setQueue to not accept an argument, but to use a parameter specified in
the XML file. (The name of the method could be refactored at the same
time. :) That way, the flow of control is defined in one location and
loop detection should be easy to do during the XML parsing phase.
Also, while I have not made a strong case for it, I want to be able to
use mailets to define a custom processing mechanism for mail that has
been delivered. I very much believe having multiple root processors that
act on messages with different attributes (spooled, delivered,
dead-letter...) to be a big win.
Sid
RE: Supporting multiple "root processor"s in James 1.3?
Posted by Danny Angus <da...@apache.org>.
+1 nice idea Noel, have several different roots, but shared error/spam/remote delivery
d.
> -----Original Message-----
> From: Noel J. Bergman [mailto:noel@devtech.com]
> Sent: 20 April 2003 17:46
> To: James Developers List
> Subject: RE: Supporting multiple "root processor"s in James 1.3?
>
>
> > > The cleanest solution would be to have a second "root processor",
> > > defined specifically to deal with mail queued in the error repository.
>
> > No, you just need an option to specify which processor to
> inject the mail
> > into, e.g., the "transport" processor.
>
> In fact, we could add an optional configuration element to the smtp server
> block indicating which processor to consider the "root". That
> would provide
> additional flexibility when we allow multiple smtp server blocks listening
> on different bind addresses.
>
> --- Noel
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: james-dev-help@jakarta.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Supporting multiple "root processor"s in James 1.3?
Posted by "Noel J. Bergman" <no...@devtech.com>.
> > The cleanest solution would be to have a second "root processor",
> > defined specifically to deal with mail queued in the error repository.
> No, you just need an option to specify which processor to inject the mail
> into, e.g., the "transport" processor.
In fact, we could add an optional configuration element to the smtp server
block indicating which processor to consider the "root". That would provide
additional flexibility when we allow multiple smtp server blocks listening
on different bind addresses.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org
RE: Supporting multiple "root processor"s in James 1.3?
Posted by "Noel J. Bergman" <no...@devtech.com>.
> The cleanest solution would be to have a second "root processor",
> defined specifically to deal with mail queued in the error repository.
No, you just need an option to specify which processor to inject the mail
into, e.g., the "transport" processor.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: james-dev-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-dev-help@jakarta.apache.org