You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Fabio Sangiovanni <sa...@nweb.it> on 2013/06/21 10:27:50 UTC
uridnsbl checks on domains in headers
Hi everybody,
I've configured my MSA (Postfix) so that a copy of submitted mail is
sent (BCC'd) to a postfix/amavisd-new/spamassassin system for
out-of-band antispam analysis.
The MSA is set to write envelope from/rcpt addresses in custom headers.
Is it possibile to check this addresses' domains against uridns lists? I
necessarily need to do this on the out-of-band system, I can't do it on
the system that receives mail in the first place.
Any suggestions are greatly appreciated.
Thanks,
Fabio
Re: uridnsbl checks on domains in headers
Posted by Axb <ax...@gmail.com>.
On 06/21/2013 05:29 PM, Fabio Sangiovanni wrote:
> I'm using amavisd-new to pass messages to SA.
> Envelope recipients are in the mail message, as payloads of my custom
> X-header. That's why I asked for a way to check headers against URI BLs.
> I'm considering filtering out bad recipient domains using before-queue
> restrictions directly in postfix (they support dns checks).
Assuming you're an ISP:
...curious why you're so concerned about recipients listed in, for
example, DBL. (unless you have a backscatter issue).
but then...
Re: uridnsbl checks on domains in headers
Posted by Fabio Sangiovanni <sa...@nweb.it>.
Il 21/06/13 17:16, Axb ha scritto:
> On 06/21/2013 05:07 PM, Fabio Sangiovanni wrote:
>> Il 21/06/13 16:27, Axb ha scritto:
>>> This is possible against standard headers.
>>> you can see how it's done in 20_dnsbl_tests.cf DNS_FROM_AHBL_RHSBL
>>>
>> Ok, so I assume there's no way to force checks against custom headers.
>> Plus, I'm more interested in check against envelope recipients.
>>> Why do you need it to check with your dedicated headers?
>>> BCC shows the original env-sender (unless you're hacking it away)
>>>
>> As stated above, I need checks for the original envelope recipients (the
>> bcc makes impossibile to retrieve them in other ways).
>
> What glue are you using with SA?
> afaik, the SA BL eval has no knowledge of rcpt but you could use a
> milter or a Postfix filter to do this.
> It would be way more efficient that using SA.
I'm using amavisd-new to pass messages to SA.
Envelope recipients are in the mail message, as payloads of my custom
X-header. That's why I asked for a way to check headers against URI BLs.
I'm considering filtering out bad recipient domains using before-queue
restrictions directly in postfix (they support dns checks).
Fabio
Re: uridnsbl checks on domains in headers
Posted by Axb <ax...@gmail.com>.
On 06/21/2013 05:07 PM, Fabio Sangiovanni wrote:
> Il 21/06/13 16:27, Axb ha scritto:
>> This is possible against standard headers.
>> you can see how it's done in 20_dnsbl_tests.cf DNS_FROM_AHBL_RHSBL
>>
> Ok, so I assume there's no way to force checks against custom headers.
> Plus, I'm more interested in check against envelope recipients.
>> Why do you need it to check with your dedicated headers?
>> BCC shows the original env-sender (unless you're hacking it away)
>>
> As stated above, I need checks for the original envelope recipients (the
> bcc makes impossibile to retrieve them in other ways).
What glue are you using with SA?
afaik, the SA BL eval has no knowledge of rcpt but you could use a
milter or a Postfix filter to do this.
It would be way more efficient that using SA.
Re: uridnsbl checks on domains in headers
Posted by Fabio Sangiovanni <sa...@nweb.it>.
Il 21/06/13 16:27, Axb ha scritto:
> This is possible against standard headers.
> you can see how it's done in 20_dnsbl_tests.cf DNS_FROM_AHBL_RHSBL
>
Ok, so I assume there's no way to force checks against custom headers.
Plus, I'm more interested in check against envelope recipients.
> Why do you need it to check with your dedicated headers?
> BCC shows the original env-sender (unless you're hacking it away)
>
As stated above, I need checks for the original envelope recipients (the
bcc makes impossibile to retrieve them in other ways).
> Make sure you're not violating BL's TOS by hammering mirros with the
> wrong type of query.
Of course, we have datafeed subscriptions to avoid that.
Thanks,
Fabio
Re: uridnsbl checks on domains in headers
Posted by Axb <ax...@gmail.com>.
On 06/21/2013 03:21 PM, Fabio Sangiovanni wrote:
> Il 21/06/13 14:19, Martin Gregorie ha scritto:
>> Assuming that the copy is sent to a maildir format mailbox you can
>> periodically run a shell script something this:
>>
>> for m in maildir/*
>> do
>> spamc <$m | rescanned_results_filter
>> mv $m scanned_dir
>> done
>>
>> This could be a second pass through your normal SA system or could
>> equally be using a differently configured copy of spamd. So, rescanning
>> the BCCed messages is rather easy. This is basically how I run new and
>> revised rules against my spam collection.
>>
>> The more important questions concern what you want to do with the BCCed
>> mail after its second scan and what you'll do with the results scanning
>> it the second time.
>>
>>
>> Martin
> I normally already scan the BCCed message *only*. The main submission
> channel doesn't have an antispam system on its own; instead, an
> out-of-band antispam stack (postfix + amavisd-new + spamassassin) is in
> place; it receives BCCed messages and takes care of checks in an
> unobtrusive way, just for screening purposes.
> What I'm trying to do is to implement uribldns checks against one single
> type of header, containing an email address.
> Example:
>
> [mail headers]
> [...]
> X-My-Own-Header: <lo...@domain.tld>
> [...]
>
> [mail body]
>
>
> I need rules to have spamassassin check domain.tld against some URIBL
> lists (eg. Spamhaus DBL). Is this even possible?
This is possible against standard headers.
you can see how it's done in 20_dnsbl_tests.cf DNS_FROM_AHBL_RHSBL
Why do you need it to check with your dedicated headers?
BCC shows the original env-sender (unless you're hacking it away)
Make sure you're not violating BL's TOS by hammering mirros with the
wrong type of query.
Re: uridnsbl checks on domains in headers
Posted by Fabio Sangiovanni <sa...@nweb.it>.
Il 21/06/13 16:49, Martin Gregorie ha scritto:
> On Fri, 2013-06-21 at 15:21 +0200, Fabio Sangiovanni wrote:
>
>> I normally already scan the BCCed message *only*. The main submission
>> channel doesn't have an antispam system on its own; instead, an
>> out-of-band antispam stack (postfix + amavisd-new + spamassassin) is in
>> place; it receives BCCed messages and takes care of checks in an
>> unobtrusive way, just for screening purposes.
>>
> Understood.
>
>> What I'm trying to do is to implement uribldns checks against one single
>> type of header, containing an email address.
>> Example:
>>
>> [mail headers]
>> [...]
>> X-My-Own-Header: <lo...@domain.tld>
>> [...]
>>
>> [mail body]
>>
>> I need rules to have spamassassin check domain.tld against some URIBL
>> lists (eg. Spamhaus DBL). Is this even possible?
>>
> I don't see why not.
>
> If you want to stay inside the SA framework you can write a rule that
> triggers on your own header and processes its payload. The processing
> might be as simple as calling the existing URIBL module: I've not looked
> at it to see if that would be possible. More likely you'd need to modify
> the Perl URIBL module and change it to work with your payload format
> rather than the parts of a message that it currently uses.
Yes, my idea is to work with the SA framework if possible. I think I've
read the docs quite thoroughly, in particular
http://spamassassin.apache.org/full/3.3.x/doc/Mail_SpamAssassin_Conf.html,
but couldn't come up with a solution yet. I may be missing something
anyway :)
As far as I can see, this should be the relevant part:
http://spamassassin.apache.org/full/3.3.x/doc/Mail_SpamAssassin_Conf.html#name_of_eval_method
But then I'm quite stuck in writing the actual rule (no skills in perl).
Same thing about modifying URIBL module. Honestly I tought this was an
easier task! :)
> Alternatively, as you only want to look at your header, consider
> omitting SA entirely and using simple code to ignore everything except
> your header. For instance, you might decide to follow the high volume
> route of taking the periodic FTP downloads from the URIBLs. This would
> let you use a straightforward program along these lines:
>
> Start by loading the URIBL lists into array(s)
> repeat
> {
> if there are waiting messages
> {
> for each message
> {
> extract the URI from X-My-Own-Header
> if URI matches array content
> {
> take your blacklisting action
> }
>
> delete the message (done last so there's no data loss)
> }
> }
> else
> {
> sleep a few seconds
> }
> }
>
> This would be stopped and restarted each time a new set of BL lists were
> downloaded, but would be so fast the pause should not matter. Thats the
> sort of thing that could be designed and written in well under a day
> using awk, Perl or Python.
>
> Or you could take a little longer and write something in Perl, C or Java
> that would notice a new download appearing, pause while it automatically
> updated its blacklist array(s), then continue with its comparisons.
>
> HTH
>
> Martin
That's interesting, I'll surely think about it.
Thanks,
Fabio
Re: uridnsbl checks on domains in headers
Posted by Martin Gregorie <ma...@gregorie.org>.
On Fri, 2013-06-21 at 15:21 +0200, Fabio Sangiovanni wrote:
> I normally already scan the BCCed message *only*. The main submission
> channel doesn't have an antispam system on its own; instead, an
> out-of-band antispam stack (postfix + amavisd-new + spamassassin) is in
> place; it receives BCCed messages and takes care of checks in an
> unobtrusive way, just for screening purposes.
>
Understood.
> What I'm trying to do is to implement uribldns checks against one single
> type of header, containing an email address.
> Example:
>
> [mail headers]
> [...]
> X-My-Own-Header: <lo...@domain.tld>
> [...]
>
> [mail body]
>
> I need rules to have spamassassin check domain.tld against some URIBL
> lists (eg. Spamhaus DBL). Is this even possible?
>
I don't see why not.
If you want to stay inside the SA framework you can write a rule that
triggers on your own header and processes its payload. The processing
might be as simple as calling the existing URIBL module: I've not looked
at it to see if that would be possible. More likely you'd need to modify
the Perl URIBL module and change it to work with your payload format
rather than the parts of a message that it currently uses.
Alternatively, as you only want to look at your header, consider
omitting SA entirely and using simple code to ignore everything except
your header. For instance, you might decide to follow the high volume
route of taking the periodic FTP downloads from the URIBLs. This would
let you use a straightforward program along these lines:
Start by loading the URIBL lists into array(s)
repeat
{
if there are waiting messages
{
for each message
{
extract the URI from X-My-Own-Header
if URI matches array content
{
take your blacklisting action
}
delete the message (done last so there's no data loss)
}
}
else
{
sleep a few seconds
}
}
This would be stopped and restarted each time a new set of BL lists were
downloaded, but would be so fast the pause should not matter. Thats the
sort of thing that could be designed and written in well under a day
using awk, Perl or Python.
Or you could take a little longer and write something in Perl, C or Java
that would notice a new download appearing, pause while it automatically
updated its blacklist array(s), then continue with its comparisons.
HTH
Martin
Re: uridnsbl checks on domains in headers
Posted by Fabio Sangiovanni <sa...@nweb.it>.
Il 21/06/13 14:19, Martin Gregorie ha scritto:
> Assuming that the copy is sent to a maildir format mailbox you can
> periodically run a shell script something this:
>
> for m in maildir/*
> do
> spamc <$m | rescanned_results_filter
> mv $m scanned_dir
> done
>
> This could be a second pass through your normal SA system or could
> equally be using a differently configured copy of spamd. So, rescanning
> the BCCed messages is rather easy. This is basically how I run new and
> revised rules against my spam collection.
>
> The more important questions concern what you want to do with the BCCed
> mail after its second scan and what you'll do with the results scanning
> it the second time.
>
>
> Martin
I normally already scan the BCCed message *only*. The main submission
channel doesn't have an antispam system on its own; instead, an
out-of-band antispam stack (postfix + amavisd-new + spamassassin) is in
place; it receives BCCed messages and takes care of checks in an
unobtrusive way, just for screening purposes.
What I'm trying to do is to implement uribldns checks against one single
type of header, containing an email address.
Example:
[mail headers]
[...]
X-My-Own-Header: <lo...@domain.tld>
[...]
[mail body]
I need rules to have spamassassin check domain.tld against some URIBL
lists (eg. Spamhaus DBL). Is this even possible?
Thanks,
Fabio
Re: uridnsbl checks on domains in headers
Posted by Martin Gregorie <ma...@gregorie.org>.
On Fri, 2013-06-21 at 10:27 +0200, Fabio Sangiovanni wrote:
> Hi everybody,
>
> I've configured my MSA (Postfix) so that a copy of submitted mail is
> sent (BCC'd) to a postfix/amavisd-new/spamassassin system for
> out-of-band antispam analysis.
> The MSA is set to write envelope from/rcpt addresses in custom headers.
> Is it possibile to check this addresses' domains against uridns lists? I
> necessarily need to do this on the out-of-band system, I can't do it on
> the system that receives mail in the first place.
> Any suggestions are greatly appreciated.
>
Assuming that the copy is sent to a maildir format mailbox you can
periodically run a shell script something this:
for m in maildir/*
do
spamc <$m | rescanned_results_filter
mv $m scanned_dir
done
This could be a second pass through your normal SA system or could
equally be using a differently configured copy of spamd. So, rescanning
the BCCed messages is rather easy. This is basically how I run new and
revised rules against my spam collection.
The more important questions concern what you want to do with the BCCed
mail after its second scan and what you'll do with the results scanning
it the second time.
Martin