Testing MD5-Sum of the Subject against a dnsbl

[Scott Trady <sc...@googlemail.com>]

Hello,

This is what it realised:

I setup some spamstraps reporting the Subjects to a Central Server.
This Server generates a rule file periodicaly and the clients loads this
file into SA.

The Disadvantages of this System are:
- Time delay (I can't refresh/update the Database every 5Minutes)
- Reload if SA needed
- growing Rules file.

So i would like to realise the following:

- Building a DNS-Bl with the MD5-Summes of the Subjects.
(This part is done by using rbldnsd and a zone MD5-sum.mydom)

NOW:

How can I add Test to SA doing :
- generating the MD5 Sum of the subject and doing a dnsquery against to
RBL-Zone ?

Is there a way to realise this in SA.

Best STrady

Re: Testing MD5-Sum of the Subject against a dnsbl

[Blaine Fleming <gr...@digital-z.com>]

Benny Pedersen wrote:
>> That is a good starting point for writing a plugin to do something
>> similar but the OP wants to hash the subject not the body.
>>     
>
> subject is part of the body
>   
Correct me if I'm wrong, but I believe that ixhash splits the part after 
the blank line (body) and hashes it using the appropriate method(s).  
Last I checked, the "Subject:" line is in the header, before the blank 
in the part it discards.

>> I started doing this a while ago in addition to using ixhash.
>>     
>
> how ?
>   
Wrote a plugin that gets the exact subject line from SA, hashes it then 
queries a remote server.  Still debating of subject hashing is worth it 
in the end as this rule overlaps with several others such as CRM114, 
ixhash and some other custom rules.

>> After engineering a Client-Server plugin to provide realtime hash stats
>>     
> it helps to push
>   
>> spam over the threshold without having too many false positives.
>>     
>
> with the hashhack-server.pl ?
The only part I found valuable for my environment was the regular 
expressions showing how it generates the hashes.  Otherwise, I tossed it 
all out and rolled my own.  I learned quick that exporting the data to a 
rbldns zone was too slow so I took the approach of using a UDP messaging 
system to directly query a central server that contains all the data 
collected by my spamtraps.  Doing this boosted the hash hit rate 
substatially over DNS data that was dumped every 5 minutes.  I'm still 
revising the architecture but plan on releasing all the code shortly.  
In other words, I hate perl and the code looks like crap so let me clean 
it up before I totally embarrass myself! :-)

Hope that all makes sense as I'm not really good at explaining things.  
I'm more of a lock-him-in-the-back-and-let-him-code type of person.

--Blaine

Re: Testing MD5-Sum of the Subject against a dnsbl

[Benny Pedersen <me...@junc.org>]

>>> Is there a way to realise this in SA.
>> http://ixhash.sourceforge.net/
> That is a good starting point for writing a plugin to do something
> similar but the OP wants to hash the subject not the body.

subject is part of the body

> I started doing this a while ago in addition to using ixhash.

how ?

> After engineering a Client-Server plugin to provide realtime hash stats
it helps to push
> spam over the threshold without having too many false positives.

with the hashhack-server.pl ?

> The false positives I received were typically monthly notices like the
Verizon
> Wireless statement notice that hit thousands of boxes at the
> same time.

super :)

> I strongly recommend being careful with the scoring of the subject hash
> and be sure to account for missing subjects.  I find a missing subject is
> more common in ham than spam.

adjust score and hits to do there best is what i do, over 3 hits and score
on 5 :-)

Re: Testing MD5-Sum of the Subject against a dnsbl

[Blaine Fleming <gr...@digital-z.com>]

Oops, still trying to get used to Thunderbird and didn't post this to 
the list
Benny Pedersen wrote:
>> Is there a way to realise this in SA.
>>     
>
> http://ixhash.sourceforge.net/
>
>   
That is a good starting point for writing a plugin to do something 
similar but the OP wants to hash the subject not the body.  I started 
doing this a while ago in addition to using ixhash.  After engineering a 
Client-Server plugin to provide realtime hash stats it helps to push 
spam over the threshold without having too many false positives.  The 
false positives I received were typically monthly notices like the 
Verizon Wireless statement notice that hit thousands of boxes at the 
same time.

I strongly recommend being careful with the scoring of the subject hash 
and be sure to account for missing subjects.  I find a missing subject 
is more common in ham than spam.

--Blaine

Re: Testing MD5-Sum of the Subject against a dnsbl

[Benny Pedersen <me...@junc.org>]

> Is there a way to realise this in SA.

http://ixhash.sourceforge.net/