You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Luis Hernán Otegui <lu...@gmail.com> on 2007/06/08 23:04:23 UTC
These are getting through SA...
Hi, could somebody run this mail trough SA and give me the scores?
They aren't scoring very much here...
Return-Path: <bu...@medienkulturhaus.at>
X-Original-To: user@domain.com
Delivered-To: user@user@domain.com
Received: from localhost (localhost [127.0.0.1])
by nahuel.biol.unlp.edu.ar (Postfix) with ESMTP id 660BE7B1FE;
Fri, 8 Jun 2007 17:25:09 -0300 (ART)
X-Virus-Scanned: by amavisd-new-2.4.4 (20061120) (Debian) at biol.unlp.edu.ar
X-Spam-Score: 3.964
X-Spam-Level: ***
X-Spam-Status: No, score=3.964 tagged_above=-100 required=5
tests=[BAYES_99=3.5, HTML_30_40=0.463, HTML_MESSAGE=0.001]
Received: from nahuel.biol.unlp.edu.ar ([127.0.0.1])
by localhost (nahuel.biol.unlp.edu.ar [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id C9PabKr0nGQL; Fri, 8 Jun 2007 17:25:08 -0300 (ART)
Received: from formiak.senix.pl (formiak.senix.pl [83.12.246.226])
by nahuel.biol.unlp.edu.ar (Postfix) with ESMTP id 2A4A87E47;
Fri, 8 Jun 2007 17:25:03 -0300 (ART)
Received: from [83.12.246.226] by mail.medienkulturhaus.at; Fri, 8 Jun
2007 20:25:53 -0100
Date: Fri, 8 Jun 2007 20:25:53 -0100
From: "Deana Adams" <bu...@medienkulturhaus.at>
X-Mailer: The Bat! (v2.00.3) Business
Reply-To: budmedienkulturhaussow@medienkulturhaus.at
X-Priority: 3 (Normal)
Message-ID: <96...@medienkulturhaus.at>
To: internet@biol.unlp.edu.ar
Subject: Can you imagine that you are healthy?
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----------4B80CCCC51A29EFD"
------------4B80CCCC51A29EFD
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 7bit
LegalRXMedications drug store introduce all medications that you have
a need in in order to renew your health at low price.
We work across the globe with clients from Europe, America and Asia.
At the time you got no need to search for chemist's shop at your local area.
We surely bring pills of the best quality to the distant parts of the world.
Visit our site to obtain drugs you instantly need direct to your location.
http://teethcat.hk/
We're ratified by VISA & VeriSign then we provide effective and
dependable purchase.
------------4B80CCCC51A29EFD
Content-Type: text/html; charset=iso-8859-1
Content-Transfer-Encoding: 7bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<HTML><HEAD><TITLE></TITLE>
</HEAD>
<BODY>
<b><font color="#00CC33"><em>LegalRXMedications</em></font> drug store
introduce all medications that you have a need in in order to renew
your health at low price. <br>
We work across the globe with clients from Europe, America and Asia. <br>
At the time you got no need to search for chemist's shop at your local area.<br>
We surely bring pills of the best quality to the distant parts of the world.
<br>
<br>
<a href="http://teethcat.hk/"><em>Visit our site to obtain drugs you
instantly need direct to your location.</em></a></b>
<br>
<font color="#D9EDFF">http://teethcat.hk/</font>
<br><b>We're ratified by <font color="#FF0000"><em>VISA</em></font> &
<font color="#FF0000"><em>VeriSign</em></font> then we provide
effective and dependable purchase.
</b>
</BODY></HTML>
------------4B80CCCC51A29EFD--
--
-------------------------------------------------
GNU-GPL: "May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-------------------------------------------------
RE: These are getting through SA...
Posted by "Randal, Phil" <pr...@herefordshire.gov.uk>.
Well caught, Mark!
I'd come to similar conclusions even without digging into the code when
I saw DNS-related strangeness when I was testing SA 3.2.0 a few weeks
back.
I'll second your request that SA process all results it has collected on
timeout, instead of discarding them.
Cheers,
Phil
-----Original Message-----
From: Mark Martinec [mailto:Mark.Martinec+sa@ijs.si]
Sent: 12 June 2007 17:20
To: users@spamassassin.apache.org
Subject: Re: These are getting through SA...
Luis,
> > Namely with 22 RBL results coming back, the last one
> > (which was the crucial URIBL test) had a timeout of 0
> > and was ignored even though dns result did arrive.
> >
> > Moreover, there is a bug in Mail::SpamAssassin::Dns, where
> > a late-spawned URIBL queries (which only start after Razor,
> > DCC and Pyzor are run) are being timed against start time
> > of the first wave of plain RBL dns queries, which are fired-off
> > seconds earlier, so there is a good chance that URIBL queries
> > time out in 0 seconds and their resultes are never collected.
I submitted a problem report, with a proposed patch:
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
Things are much more predictable now.
Mark
Re: These are getting through SA...
Posted by Mark Martinec <Ma...@ijs.si>.
Luis,
> > Namely with 22 RBL results coming back, the last one
> > (which was the crucial URIBL test) had a timeout of 0
> > and was ignored even though dns result did arrive.
> >
> > Moreover, there is a bug in Mail::SpamAssassin::Dns, where
> > a late-spawned URIBL queries (which only start after Razor,
> > DCC and Pyzor are run) are being timed against start time
> > of the first wave of plain RBL dns queries, which are fired-off
> > seconds earlier, so there is a good chance that URIBL queries
> > time out in 0 seconds and their resultes are never collected.
I submitted a problem report, with a proposed patch:
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
Things are much more predictable now.
Mark
Re: These are getting through SA...
Posted by Mark Martinec <Ma...@ijs.si>.
Bill,
> Mark, I patched Dns.pm but this didn't resolve the issue for me.
> You can test with the sample messages I posted to bugzilla:
> http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506
Yes, it is the same problem as I describe in
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
but to fix it requires my 'feature request' to be implemented:
Now for the last part - a feature request. I think that no attempt
has been made to collect already received DNS responses when
timeout is reached. Given an asynchronous nature of DNS lookups
in this module, I think it would be worthwhile to collect whatever
is still in the IP receive queue after a timeout.
The problem with BOTNET is that it tries to do a reverse DNS
lookup on 66.17.235.109, which has broken DNS servers and
none are reachable, so it hangs in sub get_rdns query($ip,'PTR','IN')
for 24 seconds.
After BOTNET finally times out, the Dns.pm harvest_dnsbl_queries
tries to collect its RBL results, but abandons all attempts right away
because it sees that 24 seconds has passed by and just declares
them timed out, despite the fact that DNS results are waiting
in TCP/IP received queue.
Mark
Re: These are getting through SA...
Posted by Mark Martinec <Ma...@ijs.si>.
> ...a bug pause here...
bug -> big
(29 seconds)
Re: These are getting through SA...
Posted by Bill Landry <bi...@inetmsg.com>.
Mark Martinec wrote the following on 6/15/2007 10:41 AM -0800:
> Bill,
>
>
>>> There is now an additional patch at:
>>> http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
>>> which should fix this.
>>>
>
>
>> Mark, thanks for the patches. However, even with both Dns.pm patches
>> applied, unless I set "rbl_timeout" to a high enough time interval, SA
>> still misses the URIBL test results in the sample messages I posted in
>> bugzilla 5506.
>> For example, if I set "rbl_timeout 15", I get no URIBL hit results.
>> However, if I set "rbl_timeout 60", I do get URIBL hit results. This is
>> because the 60 interval is long enough for Botnet's (in this case) RDNS
>> test to timeout before the "rbl_timeout" interval.
>>
>
> Don't know. I was using your test case (test1.txt) to cause a Botnet hang,
> had rbl_timeout at 15, and I did get the RBL hits.
>
> Try spamassassin from a command line with -t -D.
> If you applied both patches, there should be something
> like the following in the debug output.
>
> 19:26:55.558 8.010 0.000 [13051] dbg: Botnet: no trusted relays
> 19:26:55.558 8.010 0.000 [13051] dbg: Botnet: get_relay didn't find RDNS
> ...a bug pause here...
> 19:27:24.558 37.010 28.999 [13051] dbg: Botnet: IP is '66.17.235.109'
> 19:27:24.558 37.010 0.001 [13051] dbg: Botnet: RDNS is ''
> 19:27:24.559 37.011 0.001 [13051] dbg: Botnet: HELO is 'xxxxxx'
> 19:27:24.559 37.011 0.000 [13051] dbg: Botnet: BADDNS miss
> ...
> 19:27:26.269 38.721 0.002 [13051] dbg: rules: running meta tests; score so far=12.753
> 19:27:26.269 38.721 0.001 [13051] dbg: rules: compiled meta tests
> 19:27:26.271 38.723 0.002 [13051] dbg: check: running tests for priority: 500
> 19:27:26.274 38.726 0.002 [13051] dbg: async: select found 1 socks ready
> 19:27:26.275 38.727 0.001 [13051] dbg: uridnsbl: query for xxx took 35 seconds to look up (dob.sibl.support-intelligence.net:xxx)
> ...
> 19:27:26.281 38.733 0.001 [13051] dbg: async: queries completed: 15 started: 2
> 19:27:26.283 38.735 0.001 [13051] dbg: async: queries active: at Fri Jun 15 19:27:26 2007
>
> vvvvvvvvvvv
> 19:27:26.283 38.735 0.001 [13051] dbg: dns: harvest_dnsbl_queries: on extended
> time, overdue by 30.000 s, still 1.200 s
> ^^^^^^^^^^^^
>
> 19:27:26.295 38.747 0.011 [13051] dbg: async: select found 1 socks ready
> 19:27:26.297 38.749 0.002 [13051] dbg: async: queries completed: 1 started: 2
> 19:27:26.297 38.749 0.000 [13051] dbg: async: queries active: URI-A=1 at Fri
> ...
>
> 0.1 RDNS_NONE Delivered to trusted network by a host with no rDNS
> 3.4 HEADER_SPAM Bulk email fingerprint (header-based) found
> 0.1 BOTNET Relay might be a spambot or virusbot
> [botnet0.7,ip=66.17.235.109,nordns]
> 0.0 DKIM_POLICY_SIGNSOME Domain Keys Identified Mail: policy says domain
> signs some mails
> 0.0 BOTNET_NORDNS Relay's IP address has no PTR record
> [botnet_nordns,ip=66.17.235.109]
> 1.2 HTML_IMAGE_ONLY_28 BODY: HTML: images with 2400-2800 bytes of words
> 0.2 HTML_MESSAGE BODY: HTML included in message
> 3.0 BAYES_95 BODY: Bayesian spam probability is 95 to 99%
> [score: 0.9658]
> 1.5 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
> 0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
> 2.8 DCC_CHECK Listed in DCC (http://rhyolite.com/anti-spam/dcc/)
> 2.0 URIBL_BLACK Contains an URL listed in the URIBL blacklist
> [URIs: xxx]
> 1.5 URIBL_OB_SURBL Contains an URL listed in the OB SURBL blocklist
> [URIs: xxx]
> 0.0 DIGEST_MULTIPLE Message hits more than one network digest check
> 0.5 BOTNET_OTHER BOTNET_OTHER
> -1.2 AWL AWL: From: address is in the auto white-list
>
> (I had to strike the uri in the log to be able to post the message :)
>
> Mark
>
Mark, I'm really sorry if I caused you to unnecessarily spin your wheels
on this, I applied the second patch to the wrong SA version of Dns.pm.
The patches work fine. Again, my sincere apologies... :-(
Bill
Re: These are getting through SA...
Posted by Mark Martinec <Ma...@ijs.si>.
Bill,
> > There is now an additional patch at:
> > http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
> > which should fix this.
> Mark, thanks for the patches. However, even with both Dns.pm patches
> applied, unless I set "rbl_timeout" to a high enough time interval, SA
> still misses the URIBL test results in the sample messages I posted in
> bugzilla 5506.
> For example, if I set "rbl_timeout 15", I get no URIBL hit results.
> However, if I set "rbl_timeout 60", I do get URIBL hit results. This is
> because the 60 interval is long enough for Botnet's (in this case) RDNS
> test to timeout before the "rbl_timeout" interval.
Don't know. I was using your test case (test1.txt) to cause a Botnet hang,
had rbl_timeout at 15, and I did get the RBL hits.
Try spamassassin from a command line with -t -D.
If you applied both patches, there should be something
like the following in the debug output.
19:26:55.558 8.010 0.000 [13051] dbg: Botnet: no trusted relays
19:26:55.558 8.010 0.000 [13051] dbg: Botnet: get_relay didn't find RDNS
...a bug pause here...
19:27:24.558 37.010 28.999 [13051] dbg: Botnet: IP is '66.17.235.109'
19:27:24.558 37.010 0.001 [13051] dbg: Botnet: RDNS is ''
19:27:24.559 37.011 0.001 [13051] dbg: Botnet: HELO is 'xxxxxx'
19:27:24.559 37.011 0.000 [13051] dbg: Botnet: BADDNS miss
...
19:27:26.269 38.721 0.002 [13051] dbg: rules: running meta tests; score so far=12.753
19:27:26.269 38.721 0.001 [13051] dbg: rules: compiled meta tests
19:27:26.271 38.723 0.002 [13051] dbg: check: running tests for priority: 500
19:27:26.274 38.726 0.002 [13051] dbg: async: select found 1 socks ready
19:27:26.275 38.727 0.001 [13051] dbg: uridnsbl: query for xxx took 35 seconds to look up (dob.sibl.support-intelligence.net:xxx)
...
19:27:26.281 38.733 0.001 [13051] dbg: async: queries completed: 15 started: 2
19:27:26.283 38.735 0.001 [13051] dbg: async: queries active: at Fri Jun 15 19:27:26 2007
vvvvvvvvvvv
19:27:26.283 38.735 0.001 [13051] dbg: dns: harvest_dnsbl_queries: on extended
time, overdue by 30.000 s, still 1.200 s
^^^^^^^^^^^^
19:27:26.295 38.747 0.011 [13051] dbg: async: select found 1 socks ready
19:27:26.297 38.749 0.002 [13051] dbg: async: queries completed: 1 started: 2
19:27:26.297 38.749 0.000 [13051] dbg: async: queries active: URI-A=1 at Fri
...
0.1 RDNS_NONE Delivered to trusted network by a host with no rDNS
3.4 HEADER_SPAM Bulk email fingerprint (header-based) found
0.1 BOTNET Relay might be a spambot or virusbot
[botnet0.7,ip=66.17.235.109,nordns]
0.0 DKIM_POLICY_SIGNSOME Domain Keys Identified Mail: policy says domain
signs some mails
0.0 BOTNET_NORDNS Relay's IP address has no PTR record
[botnet_nordns,ip=66.17.235.109]
1.2 HTML_IMAGE_ONLY_28 BODY: HTML: images with 2400-2800 bytes of words
0.2 HTML_MESSAGE BODY: HTML included in message
3.0 BAYES_95 BODY: Bayesian spam probability is 95 to 99%
[score: 0.9658]
1.5 MIME_HTML_ONLY BODY: Message only has text/html MIME parts
0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
2.8 DCC_CHECK Listed in DCC (http://rhyolite.com/anti-spam/dcc/)
2.0 URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: xxx]
1.5 URIBL_OB_SURBL Contains an URL listed in the OB SURBL blocklist
[URIs: xxx]
0.0 DIGEST_MULTIPLE Message hits more than one network digest check
0.5 BOTNET_OTHER BOTNET_OTHER
-1.2 AWL AWL: From: address is in the auto white-list
(I had to strike the uri in the log to be able to post the message :)
Mark
Re: These are getting through SA...
Posted by Bill Landry <bi...@inetmsg.com>.
Mark Martinec wrote the following on 6/15/2007 2:34 PM -0800:
>> So far so good with Mark's patches - although I am awaiting his
>> follow-up regarding a possible bug...
>>
>
> Not sure I understand this. My fixes make SA more robust when
> plugins misbehave. The Botnet problem still causes the mail
> processing to take half a minute, but now without ill-effects
> on other checks.
>
> Mark
>
I was referring to your e-mail to the list just previous to this one
where you stated:
=====
> ...a bug pause here...
bug -> big
(29 seconds)
=====
I misunderstood this to mean that you had found a bug in one of your
patches. Please disregard.
And thanks again for the patches! They do indeed make SA more robust by
resolving the issue of lost DNS query results due to high timeout
thresholds in plugins.
Bill
Re: These are getting through SA...
Posted by Mark Martinec <Ma...@ijs.si>.
Bill,
> Hmmm, once I patched the correct SA version Dns.pm file, Mark's patches
> worked fine. However, perhaps my error caused Mark to find a bug, as
> noted by his follow-up e-mail, which might have gone undetected
> otherwise. :-)
Indeed, thanks! (but there were two other similar reports as well)
> Also, I'm not sure if John Rudd is still supporting Botnet or not, since
> I have sent him 3 e-mails to the address listed in Botnet.pm off-list
> over the past week about this, and asked him if he would consider adding
> user configurable timeout values, but have not received any responses
> back from him.
It would definitely be worth shortening a dns timeout limit in Botnet,
or make it configurable (or both).
> So far so good with Mark's patches - although I am awaiting his
> follow-up regarding a possible bug...
Not sure I understand this. My fixes make SA more robust when
plugins misbehave. The Botnet problem still causes the mail
processing to take half a minute, but now without ill-effects
on other checks.
Mark
Re: These are getting through SA...
Posted by Bill Landry <bi...@inetmsg.com>.
Daniel J McDonald wrote the following on 6/15/2007 3:37 PM -0800:
> On Fri, 2007-06-15 at 15:27 -0700, Bill Landry wrote:
>
>> Daniel J McDonald wrote the following on 6/15/2007 2:54 PM -0800:
>>
>>> On Fri, 2007-06-15 at 22:08 +0100, Randal, Phil wrote:
>>>
>>> And a few others... Might as well be completely consistent. Try this
>>> patch:
>>> --- Botnet.pm.orig 2007-06-15 16:47:33.000000000 -0500
>>> +++ Botnet.pm 2007-06-15 16:52:13.000000000 -0500
>>>
>
>
>> Daniel, here is a snippet of my debug output with this patch applied to
>> Botnet.pm, version 0.7:
>>
>> [23898] warn: plugin: failed to parse plugin
>> /etc/mail/spamassassin/Botnet.pm: Global symbol "$name" requires
>> explicit package name at /etc/mail/spamassassin/Botnet.pm line 848.
>> [23898] warn: Global symbol "$type" requires explicit package name at
>> /etc/mail/spamassassin/Botnet.pm line 848.
>> [23898] warn: Missing right curly or square bracket at
>> /etc/mail/spamassassin/Botnet.pm line 875, at end of line
>> [23898] warn: syntax error at /etc/mail/spamassassin/Botnet.pm line 875,
>> at EOF
>> [23898] warn: Compilation failed in require at
>> /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PluginHandler.pm line 97.
>>
>>
> Oops, yanked an entra line. Try this one instead:
> --- Botnet.pm.orig 2007-06-15 16:47:33.000000000 -0500
> +++ Botnet.pm 2007-06-15 17:35:33.000000000 -0500
> @@ -703,7 +703,16 @@
> ($type =~ /^(?:A|MX)$/) &&
> (defined $max) &&
> ($max =~ /^-?\d+$/) ) {
> - $resolver = Net::DNS::Resolver->new();
> + $resolver = Net::DNS::Resolver->new(
> + udp_timeout => 3,
> + tcp_timeout => 3,
> + retrans => 0,
> + retry => 1,
> + persistent_tcp=>0,
> + persistent_udp=>0,
> + dnsrch=>0,
> + defnames=>0,
> + );
> if ($query = $resolver->search($name, $type)) {
> # found matches
> $i = 0;
> @@ -826,7 +835,16 @@
> sub get_rdns {
> my ($ip) = @_;
> my ($query, @answer, $rr);
> - my $resolver = Net::DNS::Resolver->new();
> + my $resolver = Net::DNS::Resolver->new(
> + udp_timeout => 3,
> + tcp_timeout => 3,
> + retrans => 0,
> + retry => 1,
> + persistent_tcp=>0,
> + persistent_udp=>0,
> + dnsrch=>0,
> + defnames=>0,
> + );
> my $name = "";
>
> if ($query = $resolver->query($ip, 'PTR', 'IN')) {
>
>
Beautiful, works like a charm!
Thanks Daniel!
Bill
Re: These are getting through SA...
Posted by Daniel J McDonald <da...@austinenergy.com>.
On Fri, 2007-06-15 at 15:27 -0700, Bill Landry wrote:
> Daniel J McDonald wrote the following on 6/15/2007 2:54 PM -0800:
> > On Fri, 2007-06-15 at 22:08 +0100, Randal, Phil wrote:
> >
> > And a few others... Might as well be completely consistent. Try this
> > patch:
> > --- Botnet.pm.orig 2007-06-15 16:47:33.000000000 -0500
> > +++ Botnet.pm 2007-06-15 16:52:13.000000000 -0500
> Daniel, here is a snippet of my debug output with this patch applied to
> Botnet.pm, version 0.7:
>
> [23898] warn: plugin: failed to parse plugin
> /etc/mail/spamassassin/Botnet.pm: Global symbol "$name" requires
> explicit package name at /etc/mail/spamassassin/Botnet.pm line 848.
> [23898] warn: Global symbol "$type" requires explicit package name at
> /etc/mail/spamassassin/Botnet.pm line 848.
> [23898] warn: Missing right curly or square bracket at
> /etc/mail/spamassassin/Botnet.pm line 875, at end of line
> [23898] warn: syntax error at /etc/mail/spamassassin/Botnet.pm line 875,
> at EOF
> [23898] warn: Compilation failed in require at
> /usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PluginHandler.pm line 97.
>
Oops, yanked an entra line. Try this one instead:
--- Botnet.pm.orig 2007-06-15 16:47:33.000000000 -0500
+++ Botnet.pm 2007-06-15 17:35:33.000000000 -0500
@@ -703,7 +703,16 @@
($type =~ /^(?:A|MX)$/) &&
(defined $max) &&
($max =~ /^-?\d+$/) ) {
- $resolver = Net::DNS::Resolver->new();
+ $resolver = Net::DNS::Resolver->new(
+ udp_timeout => 3,
+ tcp_timeout => 3,
+ retrans => 0,
+ retry => 1,
+ persistent_tcp=>0,
+ persistent_udp=>0,
+ dnsrch=>0,
+ defnames=>0,
+ );
if ($query = $resolver->search($name, $type)) {
# found matches
$i = 0;
@@ -826,7 +835,16 @@
sub get_rdns {
my ($ip) = @_;
my ($query, @answer, $rr);
- my $resolver = Net::DNS::Resolver->new();
+ my $resolver = Net::DNS::Resolver->new(
+ udp_timeout => 3,
+ tcp_timeout => 3,
+ retrans => 0,
+ retry => 1,
+ persistent_tcp=>0,
+ persistent_udp=>0,
+ dnsrch=>0,
+ defnames=>0,
+ );
my $name = "";
if ($query = $resolver->query($ip, 'PTR', 'IN')) {
> Bill
Re: These are getting through SA...
Posted by Bill Landry <bi...@inetmsg.com>.
Daniel J McDonald wrote the following on 6/15/2007 2:54 PM -0800:
> On Fri, 2007-06-15 at 22:08 +0100, Randal, Phil wrote:
>
>> Bill,
>>
>> The problem is that Botnet uses Net::DNS::Resolver's default retry and
>> timeout values, which are way too high.
>>
>> Spamassassin's DnsResolver.pm uses these values:
>>
>> udp_timeout:3
>> tcp_timeout:3
>> retrans:0
>> retry:1
>>
>
> And a few others... Might as well be completely consistent. Try this
> patch:
> --- Botnet.pm.orig 2007-06-15 16:47:33.000000000 -0500
> +++ Botnet.pm 2007-06-15 16:52:13.000000000 -0500
> @@ -703,7 +703,16 @@
> ($type =~ /^(?:A|MX)$/) &&
> (defined $max) &&
> ($max =~ /^-?\d+$/) ) {
> - $resolver = Net::DNS::Resolver->new();
> + $resolver = Net::DNS::Resolver->new(
> + udp_timeout => 3,
> + tcp_timeout => 3,
> + retrans => 0,
> + retry => 1,
> + persistent_tcp=>0,
> + persistent_udp=>0,
> + dnsrch=>0,
> + defnames=>0,
> + );
> if ($query = $resolver->search($name, $type)) {
> # found matches
> $i = 0;
> @@ -826,7 +835,18 @@
> sub get_rdns {
> my ($ip) = @_;
> my ($query, @answer, $rr);
> - my $resolver = Net::DNS::Resolver->new();
> + my $resolver = Net::DNS::Resolver->new(
> + udp_timeout => 3,
> + tcp_timeout => 3,
> + retrans => 0,
> + retry => 1,
> + persistent_tcp=>0,
> + persistent_udp=>0,
> + dnsrch=>0,
> + defnames=>0,
> + );
> + if ($query = $resolver->search($name, $type)) {
> + # found matches
> my $name = "";
>
> if ($query = $resolver->query($ip, 'PTR', 'IN')) {
>
>
Daniel, here is a snippet of my debug output with this patch applied to
Botnet.pm, version 0.7:
[23898] warn: plugin: failed to parse plugin
/etc/mail/spamassassin/Botnet.pm: Global symbol "$name" requires
explicit package name at /etc/mail/spamassassin/Botnet.pm line 848.
[23898] warn: Global symbol "$type" requires explicit package name at
/etc/mail/spamassassin/Botnet.pm line 848.
[23898] warn: Missing right curly or square bracket at
/etc/mail/spamassassin/Botnet.pm line 875, at end of line
[23898] warn: syntax error at /etc/mail/spamassassin/Botnet.pm line 875,
at EOF
[23898] warn: Compilation failed in require at
/usr/lib/perl5/site_perl/5.8.8/Mail/SpamAssassin/PluginHandler.pm line 97.
Bill
RE: These are getting through SA...
Posted by Daniel J McDonald <da...@austinenergy.com>.
On Fri, 2007-06-15 at 22:08 +0100, Randal, Phil wrote:
> Bill,
>
> The problem is that Botnet uses Net::DNS::Resolver's default retry and
> timeout values, which are way too high.
>
> Spamassassin's DnsResolver.pm uses these values:
>
> udp_timeout:3
> tcp_timeout:3
> retrans:0
> retry:1
And a few others... Might as well be completely consistent. Try this
patch:
--- Botnet.pm.orig 2007-06-15 16:47:33.000000000 -0500
+++ Botnet.pm 2007-06-15 16:52:13.000000000 -0500
@@ -703,7 +703,16 @@
($type =~ /^(?:A|MX)$/) &&
(defined $max) &&
($max =~ /^-?\d+$/) ) {
- $resolver = Net::DNS::Resolver->new();
+ $resolver = Net::DNS::Resolver->new(
+ udp_timeout => 3,
+ tcp_timeout => 3,
+ retrans => 0,
+ retry => 1,
+ persistent_tcp=>0,
+ persistent_udp=>0,
+ dnsrch=>0,
+ defnames=>0,
+ );
if ($query = $resolver->search($name, $type)) {
# found matches
$i = 0;
@@ -826,7 +835,18 @@
sub get_rdns {
my ($ip) = @_;
my ($query, @answer, $rr);
- my $resolver = Net::DNS::Resolver->new();
+ my $resolver = Net::DNS::Resolver->new(
+ udp_timeout => 3,
+ tcp_timeout => 3,
+ retrans => 0,
+ retry => 1,
+ persistent_tcp=>0,
+ persistent_udp=>0,
+ dnsrch=>0,
+ defnames=>0,
+ );
+ if ($query = $resolver->search($name, $type)) {
+ # found matches
my $name = "";
if ($query = $resolver->query($ip, 'PTR', 'IN')) {
--
Daniel J McDonald, CCIE # 2495, CISSP # 78281, CNX
Austin Energy
http://www.austinenergy.com
Re: These are getting through SA...
Posted by John Rudd <jr...@ucsc.edu>.
John Rudd wrote:
> Bill Landry wrote:
>
>>
>> Also, I'm not sure if John Rudd is still supporting Botnet or not,
>> since I have sent him 3 e-mails to the address listed in Botnet.pm
>> off-list over the past week about this, and asked him if he would
>> consider adding user configurable timeout values, but have not
>> received any responses back from him.
>
> I am, but I haven't received those email messages from you.
>
Looks like they were marked as spam, because the site you mentioned was
listed on 3 different URIBL's.
I'll look into the options you presented.
Re: These are getting through SA...
Posted by Bill Landry <bi...@inetmsg.com>.
John Rudd wrote the following on 6/15/2007 3:00 PM -0800:
> Bill Landry wrote:
>
>>
>> Also, I'm not sure if John Rudd is still supporting Botnet or not,
>> since I have sent him 3 e-mails to the address listed in Botnet.pm
>> off-list over the past week about this, and asked him if he would
>> consider adding user configurable timeout values, but have not
>> received any responses back from him.
>
> I am, but I haven't received those email messages from you.
>
John, could they have gotten caught in you spam filters possibly? I
sent messages to you at the same e-mail address you used for posting to
this list, two on 6/8/07 and one on 6/12/07. They would have come from
the same e-mail address I am using here to post this message to the
sa-users list. I could try resending them to you if you would like...?
Bill
Re: These are getting through SA...
Posted by John Rudd <jr...@ucsc.edu>.
Bill Landry wrote:
>
> Also, I'm not sure if John Rudd is still supporting Botnet or not, since
> I have sent him 3 e-mails to the address listed in Botnet.pm off-list
> over the past week about this, and asked him if he would consider adding
> user configurable timeout values, but have not received any responses
> back from him.
I am, but I haven't received those email messages from you.
Re: These are getting through SA...
Posted by Bill Landry <bi...@inetmsg.com>.
Randal, Phil wrote the following on 6/15/2007 2:08 PM -0800:
> Bill,
>
> The problem is that Botnet uses Net::DNS::Resolver's default retry and
> timeout values, which are way too high.
>
> Spamassassin's DnsResolver.pm uses these values:
>
> udp_timeout:3
> tcp_timeout:3
> retrans:0
> retry:1
>
> try
>
> export RES_OPTIONS="udp_timeout:3 tcp_timeout:3 retrans:0 retry:1"
>
> before invoking spamassassin and see what happens.
>
> The proper solution would be to fix Botnet.pm to use sensible values for
> these parameters instead of the defaults.
>
> Cheers,
>
> Phil
>
Hmmm, once I patched the correct SA version Dns.pm file, Mark's patches
worked fine. However, perhaps my error caused Mark to find a bug, as
noted by his follow-up e-mail, which might have gone undetected
otherwise. :-)
Also, I'm not sure if John Rudd is still supporting Botnet or not, since
I have sent him 3 e-mails to the address listed in Botnet.pm off-list
over the past week about this, and asked him if he would consider adding
user configurable timeout values, but have not received any responses
back from him.
So far so good with Mark's patches - although I am awaiting his
follow-up regarding a possible bug...
Bill
RE: These are getting through SA...
Posted by "Randal, Phil" <pr...@herefordshire.gov.uk>.
Bill,
The problem is that Botnet uses Net::DNS::Resolver's default retry and
timeout values, which are way too high.
Spamassassin's DnsResolver.pm uses these values:
udp_timeout:3
tcp_timeout:3
retrans:0
retry:1
try
export RES_OPTIONS="udp_timeout:3 tcp_timeout:3 retrans:0 retry:1"
before invoking spamassassin and see what happens.
The proper solution would be to fix Botnet.pm to use sensible values for
these parameters instead of the defaults.
Cheers,
Phil
-----Original Message-----
From: Bill Landry [mailto:bill@inetmsg.com]
Sent: 15 June 2007 18:18
To: users@spamassassin.apache.org
Subject: Re: These are getting through SA...
Mark, thanks for the patches. However, even with both Dns.pm patches
applied, unless I set "rbl_timeout" to a high enough time interval, SA
still misses the URIBL test results in the sample messages I posted in
bugzilla 5506.
For example, if I set "rbl_timeout 15", I get no URIBL hit results.
However, if I set "rbl_timeout 60", I do get URIBL hit results. This is
because the 60 interval is long enough for Botnet's (in this case) RDNS
test to timeout before the "rbl_timeout" interval.
Bill
Re: These are getting through SA...
Posted by Bill Landry <bi...@inetmsg.com>.
Mark Martinec wrote the following on 6/15/2007 3:36 AM -0800:
> Phil, Bill,
>
>
>> Mark, I patched Dns.pm but this didn't resolve the issue for me.
>> You can test with the sample messages I posted to bugzilla:
>> http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506
>>
>
>
>> I was getting this sort of symptom without using Botnet.
>> It's almost as if something's deadlocking somewhere in SA
>> (until the timeout kicks in).
>>
>
>
>>> If the time spent by Razor+dcc+Botnet+(not sure what else)
>>> is longer than rbl_timeout, then replies to RBL queries are
>>> thrown away by mistake.
>>>
>
> There is now an additional patch at:
> http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
> which should fix this.
>
> Mark
>
Mark, thanks for the patches. However, even with both Dns.pm patches
applied, unless I set "rbl_timeout" to a high enough time interval, SA
still misses the URIBL test results in the sample messages I posted in
bugzilla 5506.
For example, if I set "rbl_timeout 15", I get no URIBL hit results.
However, if I set "rbl_timeout 60", I do get URIBL hit results. This is
because the 60 interval is long enough for Botnet's (in this case) RDNS
test to timeout before the "rbl_timeout" interval.
Bill
Re: These are getting through SA...
Posted by Mark Martinec <Ma...@ijs.si>.
Phil, Bill,
> Mark, I patched Dns.pm but this didn't resolve the issue for me.
> You can test with the sample messages I posted to bugzilla:
> http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506
> I was getting this sort of symptom without using Botnet.
> It's almost as if something's deadlocking somewhere in SA
> (until the timeout kicks in).
>> If the time spent by Razor+dcc+Botnet+(not sure what else)
>> is longer than rbl_timeout, then replies to RBL queries are
>> thrown away by mistake.
There is now an additional patch at:
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
which should fix this.
Mark
Re: These are getting through SA...
Posted by Mark Martinec <Ma...@ijs.si>.
Phil,
> What happens if Botnet is patched to use Mail::SpamAssassin::DnsResolver
> instead of Net::DNS::Resolver?
> I'm musuing about Net::DNS::Resolver's default timeouts and retries...
> Phil (probably barking up the wrong tree)
It would do good if Botnet would impose a time limit on its DNS queries.
It would also sidestep the Dns.pm problem, but not fix it.
If the time spent by Razor+dcc+Botnet+(not sure what else)
is longer than rbl_timeout, then replies to RBL queries are
thrown away by mistake.
Mark
RE: These are getting through SA...
Posted by "Randal, Phil" <pr...@herefordshire.gov.uk>.
What happens if Botnet is patched to use Mail::SpamAssassin::DnsResolver
instead of Net::DNS::Resolver?
I'm musuing about Net::DNS::Resolver's default timeouts and retries...
Phil (probably barking up the wrong tree)
--
Phil Randal
Network Engineer
Herefordshire Council
Hereford, UK
> -----Original Message-----
> From: Bill Landry [mailto:bill@inetmsg.com]
> Sent: 12 June 2007 23:30
> To: users@spamassassin.apache.org
> Subject: Re: These are getting through SA...
>
> Mark Martinec wrote the following on 6/12/2007 3:05 PM -0800:
> > Bill,
> >
> >
> >> Mark, just curious if you are running Botnet? I found that some
> >> messages cause the Botnet RDNS test to timeout after
> hanging for about
> >> 30 seconds, and then network test randomly fail (primarily URIBL
> >> tests). I found that if I disable Botnet, then all
> network tests will
> >> run fine on the very same messages.
> >>
> >
> > Thanks, looks like the same cause (Botnet runs with Razor,
> dcc, etc.,
> > before the first and the second round of DNS launches).
> Please try the patch
> > attached to
> http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
> > (applies to SA 2.3.1 or 2.3.0), it is likely to fix these
> symptoms too.
> >
> > Mark
> >
> Mark, I patched Dns.pm but this didn't resolve the issue for me. You
> can test with the sample messages I posted to bugzilla:
>
> http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506
>
> The only way I can get the URIBL tests to report hits it to run the
> messages through SA without the headers (samples without headers also
> posted to the bugzilla).
>
> Bill
>
Re: These are getting through SA...
Posted by Bill Landry <bi...@inetmsg.com>.
Mark Martinec wrote the following on 6/12/2007 3:05 PM -0800:
> Bill,
>
>
>> Mark, just curious if you are running Botnet? I found that some
>> messages cause the Botnet RDNS test to timeout after hanging for about
>> 30 seconds, and then network test randomly fail (primarily URIBL
>> tests). I found that if I disable Botnet, then all network tests will
>> run fine on the very same messages.
>>
>
> Thanks, looks like the same cause (Botnet runs with Razor, dcc, etc.,
> before the first and the second round of DNS launches). Please try the patch
> attached to http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
> (applies to SA 2.3.1 or 2.3.0), it is likely to fix these symptoms too.
>
> Mark
>
Mark, I patched Dns.pm but this didn't resolve the issue for me. You
can test with the sample messages I posted to bugzilla:
http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5506
The only way I can get the URIBL tests to report hits it to run the
messages through SA without the headers (samples without headers also
posted to the bugzilla).
Bill
Re: These are getting through SA...
Posted by Mark Martinec <Ma...@ijs.si>.
Bill,
> Mark, just curious if you are running Botnet? I found that some
> messages cause the Botnet RDNS test to timeout after hanging for about
> 30 seconds, and then network test randomly fail (primarily URIBL
> tests). I found that if I disable Botnet, then all network tests will
> run fine on the very same messages.
Thanks, looks like the same cause (Botnet runs with Razor, dcc, etc.,
before the first and the second round of DNS launches). Please try the patch
attached to http://issues.apache.org/SpamAssassin/show_bug.cgi?id=5511
(applies to SA 2.3.1 or 2.3.0), it is likely to fix these symptoms too.
Mark
RE: These are getting through SA...
Posted by "Randal, Phil" <pr...@herefordshire.gov.uk>.
Bill,
I was getting this sort of symptom without using Botnet.
It's almost as if something's deadlocking somewhere in SA (until the
timeout kicks in).
Phil
-----Original Message-----
From: Bill Landry [mailto:bill@inetmsg.com]
Sent: 12 June 2007 22:47
To: users@spamassassin.apache.org
Subject: Re: These are getting through SA...
Mark, just curious if you are running Botnet? I found that some
messages cause the Botnet RDNS test to timeout after hanging for about
30 seconds, and then network test randomly fail (primarily URIBL
tests). I found that if I disable Botnet, then all network tests will
run fine on the very same messages.
Bill
Re: These are getting through SA...
Posted by Bill Landry <bi...@inetmsg.com>.
Mark Martinec wrote the following on 6/12/2007 3:53 AM -0800:
> Luis,
>
>
>> I don't have any URIBL rules firing up (SA 3.2.0 from source here,
>> most of the other relevant info is in the header of the mail I sent
>> before to test). Where did you get them?
>> [...]
>> But the main difference between the live run and the ones I did with
>> SA by itself (both as root and as user amavis) is the URIDNSBL hit.
>> [...]
>> From this debug, I see Amavis loading up the URIDNSBL plugin at startup,
>> but lately it simply doesn't fire up on any spammy link (I googled
>> for them, since the DDoS attack blocked the website).
>>
>
> I came across the same issue yesterday, with the same type
> of a spam message, which would mostly get hits from URIBL tests,
> but lots of other RBL checks come back emptyhanded.
>
> On the first appearance it seems that SA under amavisd-new didn't
> fire on DNSBL tests, but spamassassin from a command line did.
>
> Investigating the problem more thoroughly turned out that even
> a command line SA check behaved intermittently, sometimes
> returning URIBL_BLACK, URIBL_JP_SURBL, etc, and sometimes
> none of these URIBL tests - they were timing out.
>
> What is your setting for rbl_timeout ?
>
> Mine was fairly low, 5 seconds, and I find the dynamic timeout
> (for rbl_timeout) cutback logic (man Mail::SpamAssassin::Conf)
> does not work as advertised:
>
> In addition, whenever the effective timeout is lowered due to addi-
> tional query results returning, the remaining queries are always
> given at least one more second before timing out
>
> Namely with 22 RBL results coming back, the last one
> (which was the crucial URIBL test) had a timeout of 0
> and was ignored even though dns result did arrive.
>
> Moreover, there is a bug in Mail::SpamAssassin::Dns, where
> a late-spawned URIBL queries (which only start after Razor,
> DCC and Pyzor are run) are being timed against start time
> of the first wave of plain RBL dns queries, which are fired-off
> seconds earlier, so there is a good chance that URIBL queries
> time out in 0 seconds and their resultes are never collected.
> The problem is made worse when for example Razor itself also
> times out (thus extending time between the two rounds of
> dns queries being sent).
>
> Luis, check your DNS if it is responponding quickly,
> try extending rbl_timeout to maybe 10 seconds, see if
> there are many timeouts in RBL, URIBL, Razor or DCC queries.
>
> Mark
>
Mark, just curious if you are running Botnet? I found that some
messages cause the Botnet RDNS test to timeout after hanging for about
30 seconds, and then network test randomly fail (primarily URIBL
tests). I found that if I disable Botnet, then all network tests will
run fine on the very same messages.
Bill
Re: These are getting through SA...
Posted by Luis Hernán Otegui <lu...@gmail.com>.
Well, I dint't have rbl_timeout set, but after your mail, I did. The
DNSs I have set in resolv.conf are mine, they both cache and work as
internal and external resolvers. But the UNLP NOC got screwed in the
last days, so setting the timeout a little higher wont't hurt. Thanks
for the suggestion.
However, I upgraded to Amavis 2.5.1 yesterday (and rebuilt the AWL and
the Bayes SQL databases, because they got corrupted) and everythig
got back to normal. Updated several modules as Amavis required, and
everything got back to the usual behavior. URIBL rules got fired (on
several mails), and Razor and Pyzor got me results again.
Additionally, SA stopped complaining about some minor issues when
running sa-compile.
Thanks again,
Luix
2007/6/12, Mark Martinec <Ma...@ijs.si>:
> Luis,
>
> > I don't have any URIBL rules firing up (SA 3.2.0 from source here,
> > most of the other relevant info is in the header of the mail I sent
> > before to test). Where did you get them?
> >[...]
> > But the main difference between the live run and the ones I did with
> > SA by itself (both as root and as user amavis) is the URIDNSBL hit.
> >[...]
> > From this debug, I see Amavis loading up the URIDNSBL plugin at startup,
> > but lately it simply doesn't fire up on any spammy link (I googled
> > for them, since the DDoS attack blocked the website).
>
> I came across the same issue yesterday, with the same type
> of a spam message, which would mostly get hits from URIBL tests,
> but lots of other RBL checks come back emptyhanded.
>
> On the first appearance it seems that SA under amavisd-new didn't
> fire on DNSBL tests, but spamassassin from a command line did.
>
> Investigating the problem more thoroughly turned out that even
> a command line SA check behaved intermittently, sometimes
> returning URIBL_BLACK, URIBL_JP_SURBL, etc, and sometimes
> none of these URIBL tests - they were timing out.
>
> What is your setting for rbl_timeout ?
>
> Mine was fairly low, 5 seconds, and I find the dynamic timeout
> (for rbl_timeout) cutback logic (man Mail::SpamAssassin::Conf)
> does not work as advertised:
>
> In addition, whenever the effective timeout is lowered due to addi-
> tional query results returning, the remaining queries are always
> given at least one more second before timing out
>
> Namely with 22 RBL results coming back, the last one
> (which was the crucial URIBL test) had a timeout of 0
> and was ignored even though dns result did arrive.
>
> Moreover, there is a bug in Mail::SpamAssassin::Dns, where
> a late-spawned URIBL queries (which only start after Razor,
> DCC and Pyzor are run) are being timed against start time
> of the first wave of plain RBL dns queries, which are fired-off
> seconds earlier, so there is a good chance that URIBL queries
> time out in 0 seconds and their resultes are never collected.
> The problem is made worse when for example Razor itself also
> times out (thus extending time between the two rounds of
> dns queries being sent).
>
> Luis, check your DNS if it is responponding quickly,
> try extending rbl_timeout to maybe 10 seconds, see if
> there are many timeouts in RBL, URIBL, Razor or DCC queries.
>
> Mark
>
--
-------------------------------------------------
GNU-GPL: "May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-------------------------------------------------
Re: These are getting through SA...
Posted by Mark Martinec <Ma...@ijs.si>.
Luis,
> I don't have any URIBL rules firing up (SA 3.2.0 from source here,
> most of the other relevant info is in the header of the mail I sent
> before to test). Where did you get them?
>[...]
> But the main difference between the live run and the ones I did with
> SA by itself (both as root and as user amavis) is the URIDNSBL hit.
>[...]
> From this debug, I see Amavis loading up the URIDNSBL plugin at startup,
> but lately it simply doesn't fire up on any spammy link (I googled
> for them, since the DDoS attack blocked the website).
I came across the same issue yesterday, with the same type
of a spam message, which would mostly get hits from URIBL tests,
but lots of other RBL checks come back emptyhanded.
On the first appearance it seems that SA under amavisd-new didn't
fire on DNSBL tests, but spamassassin from a command line did.
Investigating the problem more thoroughly turned out that even
a command line SA check behaved intermittently, sometimes
returning URIBL_BLACK, URIBL_JP_SURBL, etc, and sometimes
none of these URIBL tests - they were timing out.
What is your setting for rbl_timeout ?
Mine was fairly low, 5 seconds, and I find the dynamic timeout
(for rbl_timeout) cutback logic (man Mail::SpamAssassin::Conf)
does not work as advertised:
In addition, whenever the effective timeout is lowered due to addi-
tional query results returning, the remaining queries are always
given at least one more second before timing out
Namely with 22 RBL results coming back, the last one
(which was the crucial URIBL test) had a timeout of 0
and was ignored even though dns result did arrive.
Moreover, there is a bug in Mail::SpamAssassin::Dns, where
a late-spawned URIBL queries (which only start after Razor,
DCC and Pyzor are run) are being timed against start time
of the first wave of plain RBL dns queries, which are fired-off
seconds earlier, so there is a good chance that URIBL queries
time out in 0 seconds and their resultes are never collected.
The problem is made worse when for example Razor itself also
times out (thus extending time between the two rounds of
dns queries being sent).
Luis, check your DNS if it is responponding quickly,
try extending rbl_timeout to maybe 10 seconds, see if
there are many timeouts in RBL, URIBL, Razor or DCC queries.
Mark
Re: These are getting through SA...
Posted by Luis Hernán Otegui <lu...@gmail.com>.
What I copied and pasted into my message was the original spammy
message (the source of it) as IMP showed it. The posterior ALL_TRUSTED
occured because it has already been scanned and tagged by my servers.
But the main difference between the live run and the ones I did with
SA by itself (both as root and as user amavis) is the URIDNSBL hit.
Well, the blaming on Net::DNS wasn't an easy way out. I ran Amavis in
debug mode and spotted out some warnings about the use of (.) in
concatenation string in Util.pm (not literally, i'll post the correct
output on monday, when I get back to work). From this debug, I see
Amavis loading up the URIDNSBL plugin at startup, but lately it simply
doesn't fire up on any spammy link (I googled for them, since the DDoS
attack blocked the website).
Anyway, seems like my perl installation came out buggy (upgraded from
source to 5.8.8 before upgrading SA from 3.1.8 to 3.2.0), and it is
messing things up. Lately some errors with Net::SMTP came out when
reporting to SpamCop, so I guess I'll have to start it all over again
from scratch, but this time making sure all compiles ok.
Thanks,
Luis
2007/6/8, guenther <gu...@rudersport.de>:
> On Fri, 2007-06-08 at 18:46 -0300, Luis Hernán Otegui wrote:
> > OK, i?ve been googlin' around, and it seems like an issue between
> > Amavis (or MailScanner, for waht I've found) and some unsupported
> > versions of Net::DNS, because when I run the message through SA by
> > itself, this comes out:
>
> Whatever you manually fed SA was even more borked than the inline
> copy-n-paste of a message in your OP. Looking briefly at your original
> paste, I do see these:
>
> > Date: Fri, 8 Jun 2007 20:25:53 -0100
> > From: "Deana Adams" <bu...@medienkulturhaus.at>
> > Message-ID: <96...@medienkulturhaus.at>
> > To: internet@biol.unlp.edu.ar
> > Subject: Can you imagine that you are healthy?
>
> However, your manual run hit hard on...
>
> > 0.0 MISSING_MID Missing Message-Id: header
> > 0.0 MISSING_DATE Missing Date: header
> > 1.3 MISSING_HEADERS Missing To: header
> > 1.8 MISSING_SUBJECT Missing Subject: header
> > 2.5 FM_NO_FROM_OR_TO FM_NO_FROM_OR_TO
> > 0.5 FM_NO_TO FM_NO_TO
>
> The "-1.8 ALL_TRUSTED" seems to support the assumption that you fed a
> body only. Could be due to the exact details how you did it, though.
> Also, this run didn't identify a HTML part at all...
>
> The only difference that accounts for the spamminess in the second run
> is the URIBL_BLACK hit. Maybe an oops, maybe a misconfiguration, maybe
> due to not running in real time, but long after.
>
> > So I'm blaming it on Amavis... (Net::DNS 0.59 here)...
>
> I don't see much evidence for this, yet. ;)
>
> guenther
>
>
> --
> char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
> main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
> (c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
>
>
--
-------------------------------------------------
GNU-GPL: "May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-------------------------------------------------
Re: These are getting through SA...
Posted by guenther <gu...@rudersport.de>.
On Fri, 2007-06-08 at 18:46 -0300, Luis Hernán Otegui wrote:
> OK, i?ve been googlin' around, and it seems like an issue between
> Amavis (or MailScanner, for waht I've found) and some unsupported
> versions of Net::DNS, because when I run the message through SA by
> itself, this comes out:
Whatever you manually fed SA was even more borked than the inline
copy-n-paste of a message in your OP. Looking briefly at your original
paste, I do see these:
> Date: Fri, 8 Jun 2007 20:25:53 -0100
> From: "Deana Adams" <bu...@medienkulturhaus.at>
> Message-ID: <96...@medienkulturhaus.at>
> To: internet@biol.unlp.edu.ar
> Subject: Can you imagine that you are healthy?
However, your manual run hit hard on...
> 0.0 MISSING_MID Missing Message-Id: header
> 0.0 MISSING_DATE Missing Date: header
> 1.3 MISSING_HEADERS Missing To: header
> 1.8 MISSING_SUBJECT Missing Subject: header
> 2.5 FM_NO_FROM_OR_TO FM_NO_FROM_OR_TO
> 0.5 FM_NO_TO FM_NO_TO
The "-1.8 ALL_TRUSTED" seems to support the assumption that you fed a
body only. Could be due to the exact details how you did it, though.
Also, this run didn't identify a HTML part at all...
The only difference that accounts for the spamminess in the second run
is the URIBL_BLACK hit. Maybe an oops, maybe a misconfiguration, maybe
due to not running in real time, but long after.
> So I'm blaming it on Amavis... (Net::DNS 0.59 here)...
I don't see much evidence for this, yet. ;)
guenther
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: These are getting through SA...
Posted by Luis Hernán Otegui <lu...@gmail.com>.
OK, i?ve been googlin' around, and it seems like an issue between
Amavis (or MailScanner, for waht I've found) and some unsupported
versions of Net::DNS, because when I run the message through SA by
itself, this comes out:
Content analysis details: (9.7 points, 5.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
-1.8 ALL_TRUSTED Passed through trusted hosts only via SMTP
3.5 BAYES_99 BODY: Bayesian spam probability is 99 to 100%
[score: 0.9999]
0.0 MISSING_MID Missing Message-Id: header
0.0 MISSING_DATE Missing Date: header
1.3 MISSING_HEADERS Missing To: header
2.0 URIBL_BLACK Contains an URL listed in the URIBL blacklist
[URIs: teethcat.hk]
1.8 MISSING_SUBJECT Missing Subject: header
2.5 FM_NO_FROM_OR_TO FM_NO_FROM_OR_TO
0.5 FM_NO_TO FM_NO_TO
So I'm blaming it on Amavis... (Net::DNS 0.59 here)...
I'll post this issue to the Amavis list.
Luis
2007/6/8, Daryl C. W. O'Shea <sp...@dostech.ca>:
> If you've got the current update from updates.spamassassin.org you've
> got a working set of rules for URIBL_BLACK and URIBL_GREY. It turns out
> that they didn't hit for Raymond either, so you won't see them in this case.
>
> Daryl
>
>
> Luis Hernán Otegui wrote:
> > Well, right now I'm running these commands to get updates:
> >
> > sa-update --gpgkey <GPGKEY> --channel saupdates.openprotect.com
> >
> > sa-update --gpgkey <GPGKEY> --channel updates.spamassassin.org
> >
> > sa-update doesn't download URIBL_BLACK and URIBL_GREY
> >
> > What am I doing wrong?
> >
> >
> > Luis
> >
> > 2007/6/8, Daryl C. W. O'Shea <sp...@dostech.ca>:
> >> Luis Hernán Otegui wrote:
> >> > Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
> >> > source here, most of the other relevant info is in the header of the
> >> > mail I sent before to test). Where did you get them?
> >>
> >> Run sa-update to get URIBL_BLACK and URIBL_GREY.
> >>
> >> Daryl
> >>
> >
> >
>
>
--
-------------------------------------------------
GNU-GPL: "May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-------------------------------------------------
Re: These are getting through SA...
Posted by Bill Landry <bi...@inetmsg.com>.
Daryl C. W. O'Shea wrote the following on 6/8/2007 2:41 PM -0800:
> If you've got the current update from updates.spamassassin.org you've
> got a working set of rules for URIBL_BLACK and URIBL_GREY. It turns
> out that they didn't hit for Raymond either, so you won't see them in
> this case.
>
> Daryl
>
>
> Luis Hernán Otegui wrote:
>> Well, right now I'm running these commands to get updates:
>>
>> sa-update --gpgkey <GPGKEY> --channel saupdates.openprotect.com
>>
>> sa-update --gpgkey <GPGKEY> --channel updates.spamassassin.org
>>
>> sa-update doesn't download URIBL_BLACK and URIBL_GREY
>>
>> What am I doing wrong?
>>
>>
>> Luis
>>
>> 2007/6/8, Daryl C. W. O'Shea <sp...@dostech.ca>:
>>> Luis Hernán Otegui wrote:
>>> > Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
>>> > source here, most of the other relevant info is in the header of the
>>> > mail I sent before to test). Where did you get them?
>>>
>>> Run sa-update to get URIBL_BLACK and URIBL_GREY.
>>>
>>> Daryl
For what it's worth, and maybe it's different than the issues you are
seeing, but I have been finding with SA 3.2.0, that some messages are
not firing on URIBL tests. However, if the headers of the message are
removed, the URIBL tests run fine. I posted a bug report to bugzilla
(Bug 5506). I am running SA on Fedora 7 with perl 5.8.8 and was doing
my test strictly with SA:
spamassassin -t < test.txt
I did this because I wanted to take amavisd-new out of the picture.
Also, someone mentioned Net::DNS, my version is 0.59.
Bill
Re: These are getting through SA...
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
If you've got the current update from updates.spamassassin.org you've
got a working set of rules for URIBL_BLACK and URIBL_GREY. It turns out
that they didn't hit for Raymond either, so you won't see them in this case.
Daryl
Luis Hernán Otegui wrote:
> Well, right now I'm running these commands to get updates:
>
> sa-update --gpgkey <GPGKEY> --channel saupdates.openprotect.com
>
> sa-update --gpgkey <GPGKEY> --channel updates.spamassassin.org
>
> sa-update doesn't download URIBL_BLACK and URIBL_GREY
>
> What am I doing wrong?
>
>
> Luis
>
> 2007/6/8, Daryl C. W. O'Shea <sp...@dostech.ca>:
>> Luis Hernán Otegui wrote:
>> > Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
>> > source here, most of the other relevant info is in the header of the
>> > mail I sent before to test). Where did you get them?
>>
>> Run sa-update to get URIBL_BLACK and URIBL_GREY.
>>
>> Daryl
>>
>
>
Re: These are getting through SA...
Posted by Luis Hernán Otegui <lu...@gmail.com>.
Well, right now I'm running these commands to get updates:
sa-update --gpgkey <GPGKEY> --channel saupdates.openprotect.com
sa-update --gpgkey <GPGKEY> --channel updates.spamassassin.org
sa-update doesn't download URIBL_BLACK and URIBL_GREY
What am I doing wrong?
Luis
2007/6/8, Daryl C. W. O'Shea <sp...@dostech.ca>:
> Luis Hernán Otegui wrote:
> > Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
> > source here, most of the other relevant info is in the header of the
> > mail I sent before to test). Where did you get them?
>
> Run sa-update to get URIBL_BLACK and URIBL_GREY.
>
> Daryl
>
--
-------------------------------------------------
GNU-GPL: "May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-------------------------------------------------
Re: These are getting through SA...
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
Luis Hernán Otegui wrote:
> Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
> source here, most of the other relevant info is in the header of the
> mail I sent before to test). Where did you get them?
Run sa-update to get URIBL_BLACK and URIBL_GREY.
Daryl
Re: These are getting through SA...
Posted by Raymond Dijkxhoorn <ra...@prolocation.net>.
Hi!
> Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
> source here, most of the other relevant info is in the header of the
> mail I sent before to test). Where did you get them?
>> X-Prolocation-MailScanner-SpamCheck: spam, SpamAssassin (not cached,
>> score=14.999, required 5, BAD_URI 1.50, CM_META_HK_NOW 2.50,
>> CM_SPACED_DATE 0.50, PROLO_BLACK_DNSBL 3.00,
>> RAZOR2_CF_RANGE_51_100 0.50, RAZOR2_CF_RANGE_E8_51_100 1.50,
>> RAZOR2_CHECK 0.50, SPF_PASS -0.00, URIBL_JP_SURBL 4.00)
Most likely since we get SURBL updates and so earlier then most of you
do. Could be timings issues.
Bye,
Raymond.
Re: These are getting through SA...
Posted by Luis Hernán Otegui <lu...@gmail.com>.
Hi, Raymond, I don't have any URIBL rules firing up (SA 3.2.0 from
source here, most of the other relevant info is in the header of the
mail I sent before to test). Where did you get them?
Thanks,
Luis
2007/6/8, Raymond Dijkxhoorn <ra...@prolocation.net>:
> Hi!
>
> > They aren't scoring very much here...
> >
> > Return-Path: <bu...@medienkulturhaus.at>
> > X-Original-To: user@domain.com
> > Delivered-To: user@user@domain.com
> > Received: from localhost (localhost [127.0.0.1])
> > by nahuel.biol.unlp.edu.ar (Postfix) with ESMTP id 660BE7B1FE;
> > Fri, 8 Jun 2007 17:25:09 -0300 (ART)
>
> X-Prolocation-MailScanner-SpamCheck: spam, SpamAssassin (not cached,
> score=14.999, required 5, BAD_URI 1.50, CM_META_HK_NOW 2.50,
> CM_SPACED_DATE 0.50, PROLO_BLACK_DNSBL 3.00,
> RAZOR2_CF_RANGE_51_100 0.50, RAZOR2_CF_RANGE_E8_51_100 1.50,
> RAZOR2_CHECK 0.50, SPF_PASS -0.00, URIBL_JP_SURBL 4.00)
>
> Allmost 15, not bad i think.
>
> Bye,
> Raymond.
>
--
-------------------------------------------------
GNU-GPL: "May The Source Be With You...
Linux Registered User #448382.
When I grow up, I wanna be like Theo...
-------------------------------------------------
Re: These are getting through SA...
Posted by Raymond Dijkxhoorn <ra...@prolocation.net>.
Hi!
> They aren't scoring very much here...
>
> Return-Path: <bu...@medienkulturhaus.at>
> X-Original-To: user@domain.com
> Delivered-To: user@user@domain.com
> Received: from localhost (localhost [127.0.0.1])
> by nahuel.biol.unlp.edu.ar (Postfix) with ESMTP id 660BE7B1FE;
> Fri, 8 Jun 2007 17:25:09 -0300 (ART)
X-Prolocation-MailScanner-SpamCheck: spam, SpamAssassin (not cached,
score=14.999, required 5, BAD_URI 1.50, CM_META_HK_NOW 2.50,
CM_SPACED_DATE 0.50, PROLO_BLACK_DNSBL 3.00,
RAZOR2_CF_RANGE_51_100 0.50, RAZOR2_CF_RANGE_E8_51_100 1.50,
RAZOR2_CHECK 0.50, SPF_PASS -0.00, URIBL_JP_SURBL 4.00)
Allmost 15, not bad i think.
Bye,
Raymond.
Re: These are getting through SA...
Posted by arni <ma...@arni.name>.
Luis Hernán Otegui schrieb:
> Hi, could somebody run this mail trough SA and give me the scores?
> They aren't scoring very much here...
>
Hi, your mailing probably broke half of the email so these scores are
only an estimate - if you want me to try again attach the mail as a raw
text (or .eml as many clients call it)
X-Spam-Report:
* 0.0 MISSING_MID Missing Message-Id: header
* 0.0 MISSING_DATE Missing Date: header
* 2.5 MISSING_HB_SEP Missing blank line between message header
and body
* 0.0 DKIM_POLICY_SIGNSOME Domain Keys Identified Mail: policy
says domain
* signs some mails
* 1.3 MISSING_HEADERS Missing To: header
* 0.0 BAYES_50 BODY: Bayesian spam probability is 40 to 60%
* [score: 0.5000]
* 1.5 RAZOR2_CF_RANGE_E8_51_100 Razor2 gives engine 8
confidence level
* above 50%
* [cf: 100]
* 0.5 RAZOR2_CHECK Listed in Razor2 (http://razor.sf.net/)
* 0.5 RAZOR2_CF_RANGE_51_100 Razor2 gives confidence level
above 50%
* [cf: 100]
* 2.0 URIBL_BLACK Contains an URL listed in the URIBL blacklist
* [URIs: teethcat.hk]
* 1.8 MISSING_SUBJECT Missing Subject: header
arni
Re: These are getting through SA...
Posted by ian douglas <id...@w98.us>.
> X-Spam-Status: No, score=3.964 tagged_above=-100 required=5
> tests=[BAYES_99=3.5, HTML_30_40=0.463, HTML_MESSAGE=0.001]
To me, it looks like enough tokens were seen to flag it as BAYES_99, but
that the host and IP it came from didn't trigger any RBL hits, which
left your point score well under your 5.0 tolerance.
I can't speak for others, but I personally set my 'required' level to
3.5, so anything SpamAssassin flags as BAYES_99 automatically gets
flagged as spam. And since I use {SPAM _SCORE(0)_} as a subject rewrite,
I can catch any messages that score over another threshold (for me, it's
10.0) and filter them to /dev/null if they score that high -- the only
things that score that high are messages with bayes scores of 80-100
plus RBL hits.
Just my $0.02.