Exception handling in James (was also "Matchers & X Window")

[Vincenzo Gianferrari Pini <vi...@praxis.it>]

Oki,

here is a summary of the situation.

IsInfected is part of a small set of matchers and mailets that either I wrote or someone else wrote and I modified/extended and, coming from other contributions, I considered correct and useful to make available. IsInfected in particular has been first written by Cesar Bonadio.

It is *not* part of the James project, although it could become in the future if of interest and agreed (and code cleaned). Because of this it is unrelated to James distributions and versions.

I'm now also helping in the James project, and I've become "sensible" to the issue of what and where catching Throwable-s. So I have made some small enhancements and fixes to James in this direction, starting from James version 2.2.0a5.

So, *just by chance being in those two sides*, it happens that I changed IsInfected from 1.1.5/15 to 1.1.6/16 to be in sync with the changes I introduced to James and available in James 2.2.0a5.

=============================================================================================

Now, a summary to the changes in James related to exception management, as could be of interest to all users. They were essentially a new feature, that Noel and I call "onException [control]" in other threads in james-dev, and a fix to a pre-existing situation that arose yesterday from your problem.

To recall (for the non Java programmers), here is the class hierarchy of Trowable:

Throwable
	Exception
		...
		MessagingException
		...
		RuntimeException
		...
	Error
		...

The major distinctions are (quoting from Sun Javadoc):

1) "An Error is a subclass of Throwable that indicates serious problems that a reasonable application should not try to catch. Most such errors are abnormal conditions."
A standalone program should just not catch it and terminate. A server application like James should catch it and, if "fatal" (i.e. of general scope and not just related to the processing of a single Mail object for example) should terminate, otherwise should recover as possible (for instance sending the offending/offended Mail object to the error processor) and continue.

2) "The class Exception and its subclasses are a form of Throwable that indicates conditions that a reasonable application might want to catch."
With the exclusion of RuntimeException-s (see below), an Exception arise when an un-anticipated "event", caused by "external agents" (like IO error) occurs that can and should be managed somewhere in the code stack.
The "best practice" should consist in having the code catch the exception as down as possible in the code stack and take an appropriate action as specifically as possible. Such action can also consist, if appropriate, in rethrowing up the exception or in embedding in a new exception and throw it. The java compiler helps, compelling a class either to catch an exception if thrown from below (unless it is a RuntimeException) or to declare a "throws".

3) "RuntimeException is the superclass of those exceptions that can be thrown during the normal operation of the Java Virtual Machine. A method is not required to declare in its throws clause any subclasses of RuntimeException that might be thrown during the execution of the method but not caught."
It looks similar to Error above, but the key point here is that a RuntimeException is (or should be) a programming error: something that can be fixed in the code, as for example an IndexOutOfBoundsException that appropriate controls in the code can avoid.

4) MessagingException is "The base class for all exceptions thrown by the Messaging classes."
Very generic.
The key point for us is that the Matcher and the Mailet interfaces declare respectively the match(Mail) and the service(Mail) methods as throwing MessagingException. So it is the communication channel for exceptions between mailets and matchers and the rest of James.

Now, talking about matchers and mailets, there are three types of people involved in coding in a James environment: (i) the James developers, (ii) the Matcher and Mailet developers, and (iii) the administrators that deploy a server and maintain their own config.xml files.

The James developers, taken in a broad sense, can be considered as a coordinated group of people, and we can consider exception management in the James code as following the same guidelines (to improve/enforce/cleanup perhaps).

The James code invokes matchers and mailets, that can be foreign code. So James must protect itself from Throwable-s arising from below, from code that not necessarily follows all the rules. Each matcher or mailet follows its own exception management rules (those in the James distribution should be coherent with the rest of the James code.

At this level the following is anyhow always true: as said before, the Matcher and the Mailet interfaces declare respectively the match(Mail) and the service(Mail) methods as throwing MessagingException. As a consequence, when a matcher or a mailet gets an exception while processing a Mail object (not during initialization - this is another story) it can do the following things: if the exception is a MessagingException it can either catch it or let it flow up (to LinearProcessor, part of James' spool management); if the exception is any other Exception (but not some kind of RuntimeException), the java compiler forces the programmer either to catch it somewhere and take the appropriate action, or to embed in a new MessagingExceptionand and throw it up.

In the middle stays the administrator of the James server instance deployed, that coding his config.xml decides to use or not a specific matcher or mailet, and is coding a kind of "program" using <mailet match="..." class="...">, equivalent to if (...) {...}.
He is though, before James 2.2.0a5, unable to "code" its own exception management, and he may need to, as he is the only person knowing if matching because of an exception is bad or not etc. In 2.2.0a5 we added a mechanism (the "onException") that allows him to do that, optionally coding (the attribute values are case insensitive):

	<mailet match="..." class="..." [onMatchException="{noMatch|matchAll|error|<aProcessorName>}"] [onMailetException="{ignore|error|<aProcessorName>}"]>

The default is equivalent to code on{Match|Mailet}Exception="error", that means sending to the error processor, as before.

This config.xml "code" catches any MessagingException thrown by the underlying matcher or mailet (that may embed any Throwable catched below), but if a RuntimeException or an Error flows up it will not be considered here. By the way, it is an open issue whether to catch here also RuntimeException or even Throwable: any comments?

A possible endless loop was introduced here, fixed in 2.2.0a6 (see http://archives.apache.org/eyebrowse/ReadMsg?listName=james-dev@jakarta.apache.org&msgId=739634). So don't use 2.2.0a5!.

An example taken from my system (using James 2.2.0a7 and it.praxis.james.jar 1.1.6/16) is the following:

         <!-- Check for viruses -->
         <mailet match='IsInfected="C:\Program Files\Common Files\Network Associates\VirusScan Engine\4.0.xx\scan" /analyze /noboot /nomem /noexpire /unzip /report %reportFile% %targetDir%\*.tt, D:\Viruscheck, false, 13' class="ToProcessor" onMatchException="noMatch">
            <processor> virus </processor>
         </mailet>

        <!-- Check attachment extensions for possible viruses -->
         <mailet match="AttachmentFileNameIs=*.exe,*.com,*.bat,*.pif,*.scr,*.vbs,*.eml,*.avi,*.mp3,*.mpeg,*.shs" class="Bounce" onMatchException="noMatch">
            <inline>heads</inline>
            <attachment>none</attachment>
            <passThrough>false</passThrough>
            <prefix>[REJECTED]</prefix>
            <notice>
The Security Policy of XYZ Corp. does not allow to forward messages containing attachments having any of the extensions .exe, .com, .bat, .pif, .scr, .vbs, .eml, .avi, .mp3, .mpeg, .shs, therefore your message has been rejected.

Please don't reply to this e-mail as it has been automatically sent by the antivirus system.

Regards, Postmaster xyz.com
-----------------------------
            </notice>
         </mailet>

The onMatchException attributes were coded because some malformed messages, typical of viruses, are causing some javax methods to throw exceptions; my choice was to not match and continue, but it could be done differently, perhaps the first one using noMatch, and the second one sending to a special "may-contain-executables" processor, that bounces back a less "resolute" message.

Now, what happens if a RuntimeError flows up? It will not be catched at config.xml level, but JamesSpoolManager.process(MailImpl mail) will catch it and have the Mail object sent to the error processor.

Finally, what happens if an Error flows up? Before James 2.2.0a7, it was catched a level up in the stack, in JamesSpoolManager.run(), that was reporting the exception and leaving it in the spool (I said yesterday erroneously "lost", but is a kind of). This was a pre-existing problem.
This is what was happening to you.
In James 2.2.0a7 JamesSpoolManager.process(MailImpl mail) will catch any Throwable, and have the Mail object sent to the error processor.
JamesSpoolManager.run() still can catch Throwables that flow up from other spool related operations, but will no longer receive from below any Throwable originating from matchers or mailets.

This has been a long "summary", but I wrote it to fix some ideas, thinking that it can be useful to other James users (Marco, it may answer your questions about the "onException" control).

=============================================================================================

The IsInfected antivirus matcher and the JDBCBayesianAnalysis mailet in it.praxis.james.jar 1.1.6/16 just take advantage of this new onException feature.

Why you, Oki, got the java.lang.InternalError thrown up by the IsInfected matcher is another story, related to Sun code.

And whether or not IsInfected should try to scan "innocent" attachments like .gif files is still another story: could be an option or not to look for viruses hidden in "innocent" attachments, but has nothing to do with James development and support.

I hope that all this helps.

Vincenzo

> -----Original Message-----
> From: Oki DZ [mailto:okidz@pindad.com]
> Sent: venerdi 20 giugno 2003 8.17
> To: James Users List
> Subject: RE: Matchers & X Window
> 
> 
> On Thu, 19 Jun 2003, Noel J. Bergman wrote:
> > By the way, v2.2.0a7 is available.  :-)
> 
> It's already installed and running. Too bad that I couldn't recreate the
> problem, so I guess, I'm just going to be waiting for it to happen again,
> and see the logs.
> 
Based on what I said before, you could still get the exception, but the message would go to the "error" processor.


> BTW, it's already in the morning I think; time to sleep. (Just in case...)
> 
> Oki
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: james-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: james-user-help@jakarta.apache.org