You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Michael Hutchinson <mh...@manux.co.nz> on 2009/04/08 01:09:29 UTC
20_dnsbl_tests.cf
Hello everyone,
Does anyone know of a way to perform individual debug tests on the
DNSBL's listed in 20_dnsbl_tests.cf? In essence I need to see failures
and/or timeouts.
I have made some changes to my SA 3.1.7 20_dnsbl_tests.cf when I
compared it to the 3.2.5 release. I basically just removed 2 DNSBL
lookups that are redundant. This is done in attempt to solve an issue
random scan times of 30 seconds plus.
There does not appear to be any common rule firing against the E-Mails
that take 30+ seconds to scan.
I have not managed to replicate the long scan time by testing
Spamassassin locally with network tests enabled.
Any pointers would be greatly appreciated ;)
Thanks and Cheers,
Michael Hutchinson
Manux Solutions Limited
RE: 20_dnsbl_tests.cf
Posted by Michael Hutchinson <mh...@manux.co.nz>.
Hello Dave,
> -----Original Message-----
> From: Dave Koontz [mailto:dkoontz@mbc.edu]
> Sent: Wednesday, 8 April 2009 11:34 a.m.
> To: Michael Hutchinson
> Cc: users@spamassassin.apache.org
> Subject: Re: 20_dnsbl_tests.cf
>
> Michael Hutchinson wrote ... (4/7/2009 7:09 PM):
> > I have made some changes to my SA 3.1.7 20_dnsbl_tests.cf when I
> > compared it to the 3.2.5 release. I basically just removed 2 DNSBL
> > lookups that are redundant. This is done in attempt to solve an
issue
> > random scan times of 30 seconds plus.
> When was the last time you used sa-update? Not that it will be but so
> effective on a 3.1.x install.
>
> Is there a particular reason you can not upgrade this sever to 3.2.x?
> 3.1.7 is quite old now, and many rbls have gone away or changed since
> then. Two immediately changes come to mind, spamhaus changed to their
> zen rbl, and whois is gone. I believe in addition to these,
> list.dsbl.org is now gone. I am sure others here can give you more
> changes or reasons to update! ;-)
Can't update SA until another 20 days or so. Need to get this server
running normally again.
I have changed the SBLXBL list to ZEN. I have removed DSBL and WHOIS.
Cheers,
Mike
Re: 20_dnsbl_tests.cf
Posted by Dave Koontz <dk...@mbc.edu>.
Michael Hutchinson wrote ... (4/7/2009 7:09 PM):
> I have made some changes to my SA 3.1.7 20_dnsbl_tests.cf when I
> compared it to the 3.2.5 release. I basically just removed 2 DNSBL
> lookups that are redundant. This is done in attempt to solve an issue
> random scan times of 30 seconds plus.
When was the last time you used sa-update? Not that it will be but so
effective on a 3.1.x install.
Is there a particular reason you can not upgrade this sever to 3.2.x?
3.1.7 is quite old now, and many rbls have gone away or changed since
then. Two immediately changes come to mind, spamhaus changed to their
zen rbl, and whois is gone. I believe in addition to these,
list.dsbl.org is now gone. I am sure others here can give you more
changes or reasons to update! ;-)
RE: 20_dnsbl_tests.cf
Posted by Mark <ad...@asarian-host.net>.
-----Original Message-----
From: Michael Hutchinson [mailto:mhutchinson@manux.co.nz]
Sent: woensdag 8 april 2009 5:46
To: users@spamassassin.apache.org
Subject: RE: 20_dnsbl_tests.cf
> > Upgrade to 3.2.x.
>
> > Seriously, 3.1.7 is vastly to old to be very effective, it was
> > released over 2 years ago.
> Unfortunately I cannot just upgrade SA. We are bound to a max
> version of 3.1.7 due to our Operating System version.
Why? It's Perl. :) Even if you couldn't upgrade Perl itself, that's not
needed, either (unless it's < 5.6.1). Just grab the SA tarball from CPAN,
install, add your customizations, and voila, you're done!
> > You might want to fire up CPAN and upgrade Net::DNS.
> [choke]. The last time I used CPAN for upgrading anything on this
> box, it broke Spamassassin rather badly and I had to spend several
> hours restoring it to it's former glory from backups (and removing
> additional Perl modules that got installed on my system, but aren't
> compatible with SA 3.1.7). Is there another way to upgrade that
> Module without using CPAN, and giving myself some kind of instant
> "revert to Net::DNS" fallback ability if it fails?
Just don't run CPAN in its auto-install mode, if you fear it might screw
things up. Again, just grab:
http://search.cpan.org/~olaf/Net-DNS-0.65/
Untar, run:
perl ./Makefile.PL
make
make test
make install
And you're done. Easy-peasy.
- Mark
RE: 20_dnsbl_tests.cf
Posted by Michael Hutchinson <mh...@manux.co.nz>.
Hello Matt, thanks for the response.
> -----Original Message-----
> From: Matt Kettler [mailto:mkettler_sa@verizon.net]
> Sent: Wednesday, 8 April 2009 11:26 a.m.
> To: Michael Hutchinson
> Cc: users@spamassassin.apache.org
> Subject: Re: 20_dnsbl_tests.cf
>
> Michael Hutchinson wrote:
> > Hello everyone,
> >
> > Does anyone know of a way to perform individual debug tests on the
> > DNSBL's listed in 20_dnsbl_tests.cf? In essence I need to see
> failures
> > and/or timeouts.
> >
> > I have made some changes to my SA 3.1.7 20_dnsbl_tests.cf when I
> > compared it to the 3.2.5 release.
> This is, in general, a very bad idea. Those files get dropped or
> deleted
> when you run sa-update. So now you have to make sure you never run
> sa-update or your changes might get nuked.
>
> Better to make your over-rides in a file in /etc/mail/spamassassin.
Yes, I understand and had thought that as well. Considering my SA is
version 3.1.7, no updates are coming out for it at the moment anyway.
So, I would only run SA-update to get 3rd party rules (ie SARE) but I
understand there are no updates for those rulesets either, so probably
won't run sa-update until we have upgraded the server.
I know I can override the scores in /etc/mail/spamassassin.. But how
would I disable any one specific DNSBL test from there? (didn't see a
way to do it before, hence the edits of the cf file directly). (And I
know I can't run sa-update now).
> > I basically just removed 2 DNSBL
> > lookups that are redundant.
> Which ones?
Heh.. the list is a bit longer than I might have previously suggested:
This one got nuked:
header RCVD_IN_NJABL_DUL eval:check_rbl('njabl-lastexternal',
'combined.njabl.org.', '127.0.0.3')
describe RCVD_IN_NJABL_DUL NJABL: dialup sender did non-local SMTP
tflags RCVD_IN_NJABL_DUL net
#reuse RCVD_IN_NJABL_DUL
SBL_XBL got changed to ZEN. No biggie there.
PBL added:
# PBL is the Policy Block List: http://www.spamhaus.org/pbl/
header RCVD_IN_PBL eval:check_rbl('zen-lastexternal',
'zen.spamhaus.org.', '127.0.0.1[01]')
describe RCVD_IN_PBL Received via a relay in Spamhaus PBL
tflags RCVD_IN_PBL net
#reuse RCVD_IN_PBL T_RCVD_IN_PBL_WITH_NJABL_DUL RCVD_IN_NJABL_DUL
These nuked:
header DNS_FROM_RFC_POST eval:check_rbl_sub('rfci_envfrom',
'127.0.0.3')
describe DNS_FROM_RFC_POST Envelope sender in
postmaster.rfc-ignorant.org
tflags DNS_FROM_RFC_POST net
#reuse DNS_FROM_RFC_POST
header DNS_FROM_RFC_ABUSE eval:check_rbl_sub('rfci_envfrom',
'127.0.0.4')
describe DNS_FROM_RFC_ABUSE Envelope sender in
abuse.rfc-ignorant.org
tflags DNS_FROM_RFC_ABUSE net
#reuse DNS_FROM_RFC_ABUSE
header DNS_FROM_RFC_WHOIS eval:check_rbl_sub('rfci_envfrom',
'127.0.0.5')
describe DNS_FROM_RFC_WHOIS Envelope sender in
whois.rfc-ignorant.org
tflags DNS_FROM_RFC_WHOIS net
#reuse DNS_FROM_RFC_WHOIS
And these got nuked too:
# CompleteWhois blacklists
header __RCVD_IN_WHOIS eval:check_rbl('whois',
'combined-HIB.dnsiplists.completewhois.com.')
tflags __RCVD_IN_WHOIS net
header RCVD_IN_WHOIS_BOGONS eval:check_rbl_sub('whois', '127.0.0.2')
describe RCVD_IN_WHOIS_BOGONS CompleteWhois: sender on bogons IP block
tflags RCVD_IN_WHOIS_BOGONS net
header RCVD_IN_WHOIS_HIJACKED eval:check_rbl_sub('whois', '127.0.0.3')
describe RCVD_IN_WHOIS_HIJACKED CompleteWhois: sender on hijacked IP
block
tflags RCVD_IN_WHOIS_HIJACKED net
header RCVD_IN_WHOIS_INVALID eval:check_rbl('whois-lastexternal',
'combined-HIB.dnsiplists.completewhois.com.', '127.0.0.4')
describe RCVD_IN_WHOIS_INVALID CompleteWhois: sender on invalid IP
block
tflags RCVD_IN_WHOIS_INVALID net
#reuse RCVD_IN_WHOIS_INVALID RCVD_IN_RFC_IPWHOIS
# another domain-based blacklist
header DNS_FROM_SECURITYSAGE eval:check_rbl_envfrom('securitysage',
'blackhole.securitysage.com.')
describe DNS_FROM_SECURITYSAGE Envelope sender in
blackholes.securitysage.com
tflags DNS_FROM_SECURITYSAGE net
#reuse DNS_FROM_SECURITYSAGE
I have refrained from adding any new ones, apart from the PBL.
> > This is done in attempt to solve an issue
> > random scan times of 30 seconds plus.
> >
> > There does not appear to be any common rule firing against the E-
> Mails
> > that take 30+ seconds to scan.
> > I have not managed to replicate the long scan time by testing
> > Spamassassin locally with network tests enabled.
> >
> > Any pointers would be greatly appreciated ;)
> >
> Upgrade to 3.2.x.
>
> Seriously, 3.1.7 is vastly to old to be very effective, it was
released
> over 2 years ago.
Agreed, but I need to fix this server. We have an upgrade plan that will
see us build a new Mail Server to replace this one after the end of the
month. I have pushed to get this to happen sooner, but it will not.
Unfortunately I cannot just upgrade SA. We are bound to a max version of
3.1.7 due to our Operating System version. There is no upgrade path
left, it will get a complete rebuild.
> It also has security holes in it (CVE-2007-0451
> <http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-0451> and
> CVE-2007-2873
> <http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-2873>).
Hmm.. may have to see about working around those, this server has to
operate for at least another 20 days.
Thanks and Cheers,
Michael Hutchinson
Re: 20_dnsbl_tests.cf
Posted by Matt Kettler <mk...@verizon.net>.
Michael Hutchinson wrote:
> Hello everyone,
>
> Does anyone know of a way to perform individual debug tests on the
> DNSBL's listed in 20_dnsbl_tests.cf? In essence I need to see failures
> and/or timeouts.
>
> I have made some changes to my SA 3.1.7 20_dnsbl_tests.cf when I
> compared it to the 3.2.5 release.
This is, in general, a very bad idea. Those files get dropped or deleted
when you run sa-update. So now you have to make sure you never run
sa-update or your changes might get nuked.
Better to make your over-rides in a file in /etc/mail/spamassassin.
> I basically just removed 2 DNSBL
> lookups that are redundant.
Which ones?
> This is done in attempt to solve an issue
> random scan times of 30 seconds plus.
>
> There does not appear to be any common rule firing against the E-Mails
> that take 30+ seconds to scan.
> I have not managed to replicate the long scan time by testing
> Spamassassin locally with network tests enabled.
>
> Any pointers would be greatly appreciated ;)
>
Upgrade to 3.2.x.
Seriously, 3.1.7 is vastly to old to be very effective, it was released
over 2 years ago.
It also has security holes in it (CVE-2007-0451
<http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-0451> and
CVE-2007-2873
<http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2007-2873>).
RE: 20_dnsbl_tests.cf
Posted by Michael Hutchinson <mh...@manux.co.nz>.
Hello John,
> Upgrading one package from CPAN _shouldn't_ be _that_ intrusive.
> Telling
> it to upgrade everthing is probably a bad idea, though.
I think that last time we used CPAN, I went to upgrade just one package,
and it caught the fact that I would be missing dependencies. It then
went about automatically upgrading all the packages we apparently needed
to support the Perl module I was upgrading. I understand this behaviour
can be switched off, but I'm in no hurry to touch CPAN again thanks :)
> > I don't see where Net::DNS is causing an issue, however..
>
> It probably is not, but I don't pay a great deal of attention
> discussion
> of problems in it, so I'm not sure. Upgrading it shouldn't hurt and
may
> help.
>
> > Is there some debug routine I can throw in to get a general idea of
> how
> > it is performing for all E-Mail?
>
> "it" being DNS? Yes, "-D dns" as you've done, or the more verbose "--
> debug
> area=dns". You can also run "--debug area=dns,all" to see everything
> else
> too.
And with -D I discovered:
Apr 8 22:41:08 tuatara spamd[1291]: dns: timeout for
zen-lastexternal,zen,zen-lastexternal after 4 seconds
Apr 8 22:41:09 tuatara spamd[1292]: dns: timeout for
sorbs-lastexternal,sorbs after 7 seconds
Apr 8 22:41:09 tuatara spamd[1292]: dns: timeout for
zen-lastexternal,zen,zen-lastexternal after 7 seconds
Apr 8 22:41:16 tuatara spamd[1921]: dns: timeout for zen after 5
seconds
Apr 8 22:41:16 tuatara spamd[1921]: dns: timeout for
zen-lastexternal,zen,zen-lastexternal after 5 seconds
Apr 8 22:41:18 tuatara spamd[1291]: dns: timeout for
sorbs-lastexternal,sorbs after 7 seconds
I have managed to replicate this from the command line, so this probably
isn't a spamassassin issue anymore.
So, I've found the problem, or at least part of it. We're going to run
analysis of this via NetPriva and get some more logging happening with
Bind (also running on the mail server), and I will enable --debug
area=dns at your suggestion to see if we can pin this issue for good.
> > If I am correct, this server hasn't used any swap for quite some
> time,
> > but does keep the physical memory well consumed for performance
> reasons.
> > (Debian 3.1 Sarge).
>
> That looks good.
>
> These are superficial suggestions, of course.
They all help, John. Thanks for your response and ideas! :)
Cheers,
Michael Hutchinson
RE: 20_dnsbl_tests.cf
Posted by John Hardin <jh...@impsec.org>.
On Wed, 8 Apr 2009, Michael Hutchinson wrote:
>>> MailServer:~/spamassassin# spamassassin -D dns -t </root/SpamA.txt
>>> [27256] dbg: dns: is Net::DNS::Resolver available? yes
>>> [27256] dbg: dns: Net::DNS version: 0.61
>>
>> You might want to fire up CPAN and upgrade Net::DNS.
>
> [choke]. The last time I used CPAN for upgrading anything on this box,
> it broke Spamassassin rather badly and I had to spend several hours
> restoring it to it's former glory from backups (and removing additional
> Perl modules that got installed on my system, but aren't compatible with
> SA 3.1.7).
Upgrading one package from CPAN _shouldn't_ be _that_ intrusive. Telling
it to upgrade everthing is probably a bad idea, though.
> I don't see where Net::DNS is causing an issue, however..
It probably is not, but I don't pay a great deal of attention discussion
of problems in it, so I'm not sure. Upgrading it shouldn't hurt and may
help.
> Is there some debug routine I can throw in to get a general idea of how
> it is performing for all E-Mail?
"it" being DNS? Yes, "-D dns" as you've done, or the more verbose "--debug
area=dns". You can also run "--debug area=dns,all" to see everything else
too.
>> ...how are you for memory? Those three were all close together in time,
>> maybe (WAG) you're hitting swap?
>
> #free -m -t
> total used free shared buffers
> cached
> Mem: 2027 1945 82 0 117
> 1023
> -/+ buffers/cache: 804 1223
> Swap: 1906 0 1906
> Total: 3933 1945 1988
>
> If I am correct, this server hasn't used any swap for quite some time,
> but does keep the physical memory well consumed for performance reasons.
> (Debian 3.1 Sarge).
That looks good.
These are superficial suggestions, of course.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Sheep have only two speeds: graze and stampede. -- LTC Grossman
-----------------------------------------------------------------------
5 days until Thomas Jefferson's 266th Birthday
RE: 20_dnsbl_tests.cf
Posted by Michael Hutchinson <mh...@manux.co.nz>.
> > MailServer:~/spamassassin# spamassassin -D dns -t </root/SpamA.txt
> > [27256] dbg: dns: is Net::DNS::Resolver available? yes
> > [27256] dbg: dns: Net::DNS version: 0.61
>
> You might want to fire up CPAN and upgrade Net::DNS.
[choke]. The last time I used CPAN for upgrading anything on this box,
it broke Spamassassin rather badly and I had to spend several hours
restoring it to it's former glory from backups (and removing additional
Perl modules that got installed on my system, but aren't compatible with
SA 3.1.7). Is there another way to upgrade that Module without using
CPAN, and giving myself some kind of instant "revert to Net::DNS"
fallback ability if it fails? I fear that I will not be able to upgrade
Net::DNS as our Debian Sarge will be too old to support it. I'll see if
I can manually implement the upgrade, without breaking dependencies and
so forth.
I don't see where Net::DNS is causing an issue, however.. Is there some
debug routine I can throw in to get a general idea of how it is
performing for all E-Mail?
> > See the horrible scantimes. These are logged in between other tests
> that look quite normal:
> >
> > There does not appear to be a common rule that hits the Mail that
> takes
> > too long to scan. I'd say that around 1/4 of all mail, perhaps less
> > (without knowing for sure) takes an excessive amount of time to
scan.
>
> ...how are you for memory? Those three were all close together in
time,
> maybe (WAG) you're hitting swap?
#free -m -t
total used free shared buffers
cached
Mem: 2027 1945 82 0 117
1023
-/+ buffers/cache: 804 1223
Swap: 1906 0 1906
Total: 3933 1945 1988
If I am correct, this server hasn't used any swap for quite some time,
but does keep the physical memory well consumed for performance reasons.
(Debian 3.1 Sarge).
Cheers,
Mike
RE: 20_dnsbl_tests.cf
Posted by John Hardin <jh...@impsec.org>.
On Wed, 8 Apr 2009, Michael Hutchinson wrote:
> I have done this, and appear to have quite a nominal time for those checks:
> MailServer:~/spamassassin# spamassassin -D dns -t </root/SpamA.txt
> [27256] dbg: dns: is Net::DNS::Resolver available? yes
> [27256] dbg: dns: Net::DNS version: 0.61
You might want to fire up CPAN and upgrade Net::DNS.
> Now.. Meat.. Sorry about the address rewrites.. Been told by the boss..
>
> Apr 8 11:47:02 tuatara spamd[23141]: spamd: result: Y 23 - BAYES_99,HTML_MESSAGE,HTML_SHORT_LINK_IMG_3,MIME_HTML_ONLY,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_XBL,TW_AQ,TW_JW,TW_QJ,TW_QK,TW_QZ,TW_YF,URIBL_AB_SURBL,URIBL_BLACK,URIBL_SBL scantime=30.3,size=2233,user=email1@hosteddomain1.co.nz,uid=0,required_score=5.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=56696,mid=(unknown),bayes=0.999998520162275,autolearn=spam
> Apr 8 11:47:08 tuatara spamd[23298]: spamd: result: Y 17 - BAYES_80,HELO_DYNAMIC_IPADDR,HTML_30_40,HTML_MESSAGE,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_PBL,RCVD_IN_SORBS_DUL,RCVD_IN_XBL,UNPARSEABLE_RELAY,URIBL_AB_SURBL scantime=30.4,size=30018,user=email2@hosteddomain2.co.nz,uid=0,required_score=5.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=56703,mid=<00...@disputesle13>,bayes=0.887238649863803,autolearn=spam
> Apr 8 11:47:57 tuatara spamd[22212]: spamd: result: . 0 - AWL,BAYES_00,NO_REAL_NAME scantime=30.0,size=27338,user=email3@hosteddomain3.co.nz,uid=0,required_score=5.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=56719,mid=<58...@akcux371>,bayes=0.000695157869939289,autolearn=no
>
> See the horrible scantimes. These are logged in between other tests that look quite normal:
>
> There does not appear to be a common rule that hits the Mail that takes
> too long to scan. I'd say that around 1/4 of all mail, perhaps less
> (without knowing for sure) takes an excessive amount of time to scan.
...how are you for memory? Those three were all close together in time,
maybe (WAG) you're hitting swap?
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Health Care _is_ a right - the government has no business keeping
you from getting it. But forcing somebody else to pay for your
health care at gunpoint (i.e. through taxation) is _not_ a right.
-----------------------------------------------------------------------
6 days until Thomas Jefferson's 266th Birthday
RE: 20_dnsbl_tests.cf
Posted by Michael Hutchinson <mh...@manux.co.nz>.
> -----Original Message-----
> From: Karsten Bräckelmann [mailto:guenther@rudersport.de]
> Sent: Wednesday, 8 April 2009 11:31 a.m.
> To: users@spamassassin.apache.org
> Subject: Re: 20_dnsbl_tests.cf
>
> On Wed, 2009-04-08 at 11:09 +1200, Michael Hutchinson wrote:
> > Hello everyone,
> >
> > Does anyone know of a way to perform individual debug tests on the
> > DNSBL's listed in 20_dnsbl_tests.cf? In essence I need to see
> failures
> > and/or timeouts.
>
> spamassassin -D. In particular, I believe -D dns should limit it to the
> results you're after. Sorry, from memory, not tested. Too lazy and too
> late this night. ;)
I have done this, and appear to have quite a nominal time for those checks:
MailServer:~/spamassassin# spamassassin -D dns -t </root/SpamA.txt
[27256] dbg: dns: is Net::DNS::Resolver available? yes
[27256] dbg: dns: Net::DNS version: 0.61
[27256] dbg: dns: name server: 127.0.0.1, family: 2, ipv6: 0
[27256] dbg: dns: testing resolver nameservers: 127.0.0.1
[27256] dbg: dns: trying (3) motorola.com...
[27256] dbg: dns: looking up NS for 'motorola.com'
[27256] dbg: dns: NS lookup of motorola.com using 127.0.0.1 succeeded => DNS available (set dns_available to override)
[27256] dbg: dns: is DNS available? 1
[27256] dbg: dns: checking RBL zen.spamhaus.org., set zen-lastexternal
[27256] dbg: dns: checking RBL zen.spamhaus.org., set zen
[27256] dbg: dns: checking RBL sa-other.bondedsender.org., set bsp-untrusted
[27256] dbg: dns: checking RBL plus.bondedsender.org., set ssc-firsttrusted
[27256] dbg: dns: checking RBL combined.njabl.org., set njabl
[27256] dbg: dns: checking RBL bl.spamcop.net., set spamcop
[27256] dbg: dns: checking RBL sa-trusted.bondedsender.org., set bsp-firsttrusted
[27256] dbg: dns: checking RBL zen.spamhaus.org., set zen-lastexternal
[27256] dbg: dns: checking RBL dnsbl.sorbs.net., set sorbs-lastexternal
[27256] dbg: dns: checking RBL dnsbl.sorbs.net., set sorbs
[27256] dbg: dns: checking RBL iadb.isipp.com., set iadb-firsttrusted
This took about 2 seconds or less. So I'm guessing that is quite normal. I suspect that some of these may be failing occasionally, perhaps this suggests some kind of occasional DNS lookup failure...
> > I have made some changes to my SA 3.1.7 20_dnsbl_tests.cf when I
> > compared it to the 3.2.5 release. I basically just removed 2 DNSBL
> > lookups that are redundant. This is done in attempt to solve an issue
> > random scan times of 30 seconds plus.
>
> It would help to let us know about the changes. That way we might
> already be able to tell you, if it possibly could fix such issue.
>
> Other than that -- update. :-)
Can't for reasons already described. Sorry for trying to get you guys to help me fix old technology. Eh.
> > There does not appear to be any common rule firing against the E-
> Mails
> > that take 30+ seconds to scan.
> > I have not managed to replicate the long scan time by testing
> > Spamassassin locally with network tests enabled.
>
> Size? Well, maybe not, given the non-reproducibility. DNS timeouts?
> Possibly. See above...
>
> > Any pointers would be greatly appreciated ;)
>
> Some real meat in your problem description would be appreciated as
> well. ;)
Now.. Meat.. Sorry about the address rewrites.. Been told by the boss..
Apr 8 11:47:02 tuatara spamd[23141]: spamd: result: Y 23 - BAYES_99,HTML_MESSAGE,HTML_SHORT_LINK_IMG_3,MIME_HTML_ONLY,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_XBL,TW_AQ,TW_JW,TW_QJ,TW_QK,TW_QZ,TW_YF,URIBL_AB_SURBL,URIBL_BLACK,URIBL_SBL scantime=30.3,size=2233,user=email1@hosteddomain1.co.nz,uid=0,required_score=5.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=56696,mid=(unknown),bayes=0.999998520162275,autolearn=spam
Apr 8 11:47:08 tuatara spamd[23298]: spamd: result: Y 17 - BAYES_80,HELO_DYNAMIC_IPADDR,HTML_30_40,HTML_MESSAGE,RCVD_IN_BL_SPAMCOP_NET,RCVD_IN_PBL,RCVD_IN_SORBS_DUL,RCVD_IN_XBL,UNPARSEABLE_RELAY,URIBL_AB_SURBL scantime=30.4,size=30018,user=email2@hosteddomain2.co.nz,uid=0,required_score=5.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=56703,mid=<00...@disputesle13>,bayes=0.887238649863803,autolearn=spam
Apr 8 11:47:57 tuatara spamd[22212]: spamd: result: . 0 - AWL,BAYES_00,NO_REAL_NAME scantime=30.0,size=27338,user=email3@hosteddomain3.co.nz,uid=0,required_score=5.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=56719,mid=<58...@akcux371>,bayes=0.000695157869939289,autolearn=no
See the horrible scantimes. These are logged in between other tests that look quite normal:
Apr 8 11:59:49 tuatara spamd[25073]: spamd: result: Y 18 - AWL,BAYES_50,DATE_IN_PAST_24_48,HTML_MESSAGE,HTML_TAG_BALANCE_BODY,TW_YC,UNPARSEABLE_RELAY,URIBL_AB_SURBL,URIBL_BLACK,URIBL_JP_SURBL,URIBL_SBL,URIBL_WS_SURBL scantime=2.6,size=8192,user=clamav,uid=0,required_score=5.0,rhost=localhost.localdomain,raddr=127.0.0.1,rport=57186,mid=<49...@Aloha.wst.pwsrotterdam.nl>,bayes=0.528807234903339,autolearn=no
There does not appear to be a common rule that hits the Mail that takes too long to scan. I'd say that around 1/4 of all mail, perhaps less (without knowing for sure) takes an excessive amount of time to scan.
Cheers,
Mike
Re: 20_dnsbl_tests.cf
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Wed, 2009-04-08 at 11:09 +1200, Michael Hutchinson wrote:
> Hello everyone,
>
> Does anyone know of a way to perform individual debug tests on the
> DNSBL's listed in 20_dnsbl_tests.cf? In essence I need to see failures
> and/or timeouts.
spamassassin -D. In particular, I believe -D dns should limit it to the
results you're after. Sorry, from memory, not tested. Too lazy and too
late this night. ;)
> I have made some changes to my SA 3.1.7 20_dnsbl_tests.cf when I
> compared it to the 3.2.5 release. I basically just removed 2 DNSBL
> lookups that are redundant. This is done in attempt to solve an issue
> random scan times of 30 seconds plus.
It would help to let us know about the changes. That way we might
already be able to tell you, if it possibly could fix such issue.
Other than that -- update. :-)
> There does not appear to be any common rule firing against the E-Mails
> that take 30+ seconds to scan.
> I have not managed to replicate the long scan time by testing
> Spamassassin locally with network tests enabled.
Size? Well, maybe not, given the non-reproducibility. DNS timeouts?
Possibly. See above...
> Any pointers would be greatly appreciated ;)
Some real meat in your problem description would be appreciated as
well. ;)
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}