Email Forensics for JAMES

[Anagha Mudigonda <an...@gmail.com>]

Hi Guys,

After classes started, I was not able to contribute much towards the
protocol handler (JAMES fastfail). I will start working on some of the
filters Stefano suggested pretty soon.

I am doing my MS project on *email forensics* using JAMES. Basically this
would involve collecting forensic data from the SMTP session, from the
POP/IMAP client, from the DNS Server(if needed) saving it and infering
something useful at a later time.
 To start, I will begin by collecting the SMTP session data, like the MAIL,
RCPT commands with corresponding parameters and so on.

The data collected could be used for Anomoly detection / intrusion detection
or just for compliance.

I was wondering if I could get some inputs from this forum regarding this.

Since a lot of people have a lot of experience with e-mail I would be glad
to get some suggestions or thoughts regarding this.
 Best Regards
Anagha.

RE: Email Forensics for JAMES

[Danny Angus <Da...@slc.co.uk>]

I agree with Jason..

> Logging is the key to this
...
> James is currently quite poor at logging at the mailet & matcher level
...


Mabey you should be looking at how we can enable an extensible logging
framework within James that treats messages as the logged items rather than
classes?

In other words;
log entries classified by message so that log analysis tools can quickly
extract whole histories for individual messages.
individually configurable logging (levels, locations) for mailets/matchers
that also fits in with the above.

For troubleshooting of rare issues in a high traffic (1.2 million hits/day
+) web application we use carefully designed log format which allows a
custom analysis tool to reconstruct the picture of all the concurrent
requests in progress at a single time point, this has a graphical (swing)
tree display showing the start and end time, and URI for each request
allowing the investigator to drill into the detailed log messages generated
by that request as it passes through the application. We rely on
one-thread-per-request to connect these things together, but mail logging
could use either a James issued UID or the mail UID in the headers.

Hope that helps a bit.





***************************************************************************
The information in this e-mail is confidential and for use by the addressee(s) only. If you are not the intended recipient (or responsible for delivery of the message to the intended recipient) please notify us immediately on 0141 306 2050 and delete the message from your computer. You may not copy or forward it or use or disclose its contents to any other person. As Internet communications are capable of data corruption Student Loans Company Limited does not accept any  responsibility for changes made to this message after it was sent. For this reason it may be inappropriate to rely on advice or opinions contained in an e-mail without obtaining written confirmation of it. Neither Student Loans Company Limited or the sender accepts any liability or responsibility for viruses as it is your responsibility to scan attachments (if any). Opinions and views expressed in this e-mail are those of the sender and may not reflect the opinions and views of The Student Loans Company Limit
 ed.

This footnote also confirms that this email message has been swept for the presence of computer viruses.

**************************************************************************

---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org

RE: Email Forensics for JAMES

[Jason Webb <jw...@inovem.com>]

Logging is the key to this (duh).
James is currently quite poor at logging at the mailet & matcher level
because of the patchy implementation. A matcher and mailet should always log
the fact that they have "seen" a message (i.e. matched or processed it). At
the moment lots of them don't.

The commercial MLM I wrote had lots of logging in it so we could trace the
following:
Did the MLM receive the message?
What rules did the message cause to fire?
Who was the message then sent to?
Did they receive it or did it bounce?
How long did this take?

We provide a web based app written in ColdFusion that allows an admin to
trace any message using a simple point an click interface.

I don't know about Sendmail in depth but Qmail has very good logging and
lots of utilities to help you calculate throughput (qmailanalog) so QMail
might be a good place to start.
In the case of Qmail we used Dan Bernstein's tcpd which allowed us to detect
stale connections and DDOS attacks and close down the connections quickly.
We then looked for patterns of IP addresses in these attacks and then I
called the ISP and abused them :) If they were in country where that was
useless then we just blocked the range of the offending ISP. We also looked
at the SMTP logs so we could work out if people were just trying to clog the
system up with bad emails (not a valid group or badly addressed)

I've not done much with POP3/IMAP logging other to look for connection
problems of individual users. I assume some of the more complex intrusion
detection systems might monitor the port/log files looking for attack
patterns.

Hope this helps,

-- Jason

> -----Original Message-----
> From: Anagha Mudigonda [mailto:anaghamudigonda@gmail.com]
> Sent: 25 October 2005 04:12
> To: James Developers List
> Subject: Email Forensics for JAMES
> 
> Hi Guys,
> 
> After classes started, I was not able to contribute much towards the
> protocol handler (JAMES fastfail). I will start working on some of the
> filters Stefano suggested pretty soon.
> 
> I am doing my MS project on *email forensics* using JAMES. Basically this
> would involve collecting forensic data from the SMTP session, from the
> POP/IMAP client, from the DNS Server(if needed) saving it and infering
> something useful at a later time.
>  To start, I will begin by collecting the SMTP session data, like the
> MAIL,
> RCPT commands with corresponding parameters and so on.
> 
> The data collected could be used for Anomoly detection / intrusion
> detection
> or just for compliance.
> 
> I was wondering if I could get some inputs from this forum regarding this.
> 
> Since a lot of people have a lot of experience with e-mail I would be glad
> to get some suggestions or thoughts regarding this.
>  Best Regards
> Anagha.


---------------------------------------------------------------------
To unsubscribe, e-mail: server-dev-unsubscribe@james.apache.org
For additional commands, e-mail: server-dev-help@james.apache.org