You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by al...@twobikes.ottawa.on.ca on 2005/03/07 06:11:47 UTC

DB_FILE error causes serious deep recursion/system overloading problem 

I had a serious spamd problem this evening after we rebooted our server
(running SuSE Linux 9.2). We had been running SpamAssassin 3.0.1 since
last October and it had been working very well for several months.

Suddenly, shortly after the upgrade, the load averages for the server shot
through the roof (from 0.09 to 30-40) for no reason we could see. We
narrowed it down to spamd. Once we killed that, everything was fine.

I thought it might be a problem with 3.0.1 so I upgraded to 3.0.2.  The
install went flawlessly, but the problem recurred as soon as I restarted
spamd (though not if I ran spamassassin from the command line).

I tried clearing out the mail queue (had a few leftover refused-delivery
messages there). No effect.

I ran spamd -D from the command line. Based on what I saw (attached
below), I added the lines

# turn off autowhitelisting
use_auto_whitelist 0

to /etc/mail/spamassassin/local.cf

And that solved the problem immediately. I don't use or need
auto-whitelisting so this was no big loss.

What I would really like to know, though, is why it happened, why it
happened now, and what can be done to avoid it in the future.

Anyone have any ideas? Thanks in advance ...

Alayne

-----------------------------------------------------------------------------
debug messages from running spamd before changing local.cf:

# spamd -D
trying to connect to syslog/unix...
no error connecting to syslog/unix
logging enabled:
        facility: mail
        socket:   unix
        output:   syslog
creating INET socket:
        Listen: 128
        LocalAddr: 127.0.0.1
        LocalPort: 783
        Proto: 6
        ReuseAddr: 1
        Type: 1
debug: SpamAssassin version 3.0.2
debug: Score set 0 chosen.
debug: Storable module v2.04 found
debug: Preloading modules with HOME=/tmp/spamd-8866-init
debug: ignore: test message to precompile patterns and load modules
debug: using "/etc/mail/spamassassin/init.pre" for site rules init.pre
debug: config: read file /etc/mail/spamassassin/init.pre
debug: using "/usr/share/spamassassin" for default rules dir
debug: config: read file /usr/share/spamassassin/10_misc.cf
debug: config: read file /usr/share/spamassassin/20_anti_ratware.cf
debug: config: read file /usr/share/spamassassin/20_body_tests.cf
debug: config: read file /usr/share/spamassassin/20_compensate.cf
debug: config: read file /usr/share/spamassassin/20_dnsbl_tests.cf
debug: config: read file /usr/share/spamassassin/20_drugs.cf
debug: config: read file /usr/share/spamassassin/20_fake_helo_tests.cf
debug: config: read file /usr/share/spamassassin/20_head_tests.cf
debug: config: read file /usr/share/spamassassin/20_html_tests.cf
debug: config: read file /usr/share/spamassassin/20_meta_tests.cf
debug: config: read file /usr/share/spamassassin/20_phrases.cf
debug: config: read file /usr/share/spamassassin/20_porn.cf
debug: config: read file /usr/share/spamassassin/20_ratware.cf
debug: config: read file /usr/share/spamassassin/20_uri_tests.cf
debug: config: read file /usr/share/spamassassin/23_bayes.cf
debug: config: read file /usr/share/spamassassin/25_body_tests_es.cf
debug: config: read file /usr/share/spamassassin/25_hashcash.cf
debug: config: read file /usr/share/spamassassin/25_spf.cf
debug: config: read file /usr/share/spamassassin/25_uribl.cf
debug: config: read file /usr/share/spamassassin/30_text_de.cf
debug: config: read file /usr/share/spamassassin/30_text_fr.cf
debug: config: read file /usr/share/spamassassin/30_text_nl.cf
debug: config: read file /usr/share/spamassassin/30_text_pl.cf
debug: config: read file /usr/share/spamassassin/50_scores.cf
debug: config: read file /usr/share/spamassassin/60_whitelist.cf
debug: using "/etc/mail/spamassassin" for site rules dir
debug: config: read file /etc/mail/spamassassin/local.cf
debug: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
debug: plugin: registered
Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
debug: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC
debug: plugin: registered
Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8cbd434)
debug: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC
debug: plugin: registered Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc)
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
implements 'parse_config'
debug: plugin: Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8cbd434)
implements 'parse_config'
debug: config: SpamAssassin failed to parse line, skipping:
rewrite_subject 0
debug: config: SpamAssassin failed to parse line, skipping:
use_terse_report 1
debug: Allowing user rules!
debug: config: SpamAssassin failed to parse line, skipping: cheap Rolex
Watch?
debug: bayes: no dbs present, cannot tie DB R/O:
/tmp/spamd-8866-init/.spamassassin/bayes_toks
debug: Score set 1 chosen.
debug: ---- MIME PARSER START ----
debug: main message type: text/plain
debug: parsing normal part
debug: added part, type: text/plain
debug: ---- MIME PARSER END ----
debug: bayes: no dbs present, cannot tie DB R/O:
/tmp/spamd-8866-init/.spamassassin/bayes_toks
debug: metadata: X-Spam-Relays-Trusted:
debug: metadata: X-Spam-Relays-Untrusted:
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
implements 'parsed_metadata'
debug: is Net::DNS::Resolver available? yes
debug: Net::DNS version: 0.48
debug: trying (3) akamai.com...
debug: looking up NS for 'akamai.com'
debug: NS lookup of akamai.com succeeded => Dns available (set
dns_available to hardcode)
debug: is DNS available? 1
debug: decoding: no encoding detected
debug: URIDNSBL: domains to query:
debug: all '*From' addrs: ignore@compiling.spamassassin.taint.org
debug: Running tests for priority: 0
debug: running header regexp tests; score so far=0
debug: registering glue method for check_hashcash_double_spend
(Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8cbd434))
debug: registering glue method for check_for_spf_helo_pass
(Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
debug: SPF: message was delivered entirely via trusted relays, not
required
debug: registering glue method for check_hashcash_value
(Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8cbd434))
debug: all '*To' addrs:
debug: registering glue method for check_for_spf_softfail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
debug: SPF: message was delivered entirely via trusted relays, not
required
debug: registering glue method for check_for_spf_pass
(Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
debug: registering glue method for check_for_spf_helo_softfail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
debug: registering glue method for check_for_spf_helo_fail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
debug: running body-text per-line regexp tests; score so far=-2.623
debug: running uri tests; score so far=-2.623
debug: registering glue method for check_uridnsbl
(Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4))
debug: Razor2 is not available
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
implements 'check_tick'
debug: running raw-body-text per-line regexp tests; score so far=-2.623
debug: running full-text regexp tests; score so far=-2.623
debug: Razor2 is not available
debug: Current PATH is: /usr/sbin:/bin:/usr/bin:/sbin:/usr/X11R6/bin
debug: Pyzor is not available: pyzor not found
debug: DCCifd is not available: no r/w dccifd socket found.
debug: DCC is not available: no executable dccproc found.
debug: Running tests for priority: 500
debug: RBL: success for 1 of 1 queries
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
implements 'check_post_dnsbl'
debug: running meta tests; score so far=-2.623
debug: running header regexp tests; score so far=-1.053
debug: running body-text per-line regexp tests; score so far=-1.053
debug: running uri tests; score so far=-1.053
debug: running raw-body-text per-line regexp tests; score so far=-1.053
debug: running full-text regexp tests; score so far=-1.053
debug: Running tests for priority: 1000
debug: running meta tests; score so far=-1.053
debug: running header regexp tests; score so far=-1.053
debug: lock: 8866 created
/tmp/spamd-8866-init/.spamassassin/auto-whitelist.lock.pannier.twobikes.ottawa.on.ca.8866
debug: lock: 8866 trying to get lock on
/tmp/spamd-8866-init/.spamassassin/auto-whitelist with 0 retries
debug: lock: 8866 link to
/tmp/spamd-8866-init/.spamassassin/auto-whitelist.lock: link ok
debug: Tie-ing to DB file R/W in
/tmp/spamd-8866-init/.spamassassin/auto-whitelist
Use of uninitialized value in numeric gt (>) at
/usr/lib/perl5/5.8.0/i586-linux-thread-multi/DB_File.pm line 271.
Deep recursion on subroutine "DB_File::AUTOLOAD" at
/usr/lib/perl5/5.8.0/i586-linux-thread-multi/DB_File.pm line 234.

(at this point I killed spamd).

-- 
Alayne McGregor
alayne at twobikes.ottawa.on.ca

"I am not amused by this 21st century. I want to take it back for a full refund,
and can we have a real one, please?" -- Charles Stross (Locus, January, 2005)

Re: More problems was Re: DB_FILE error causes serious deep recursion/system overloading problem

Posted by QQQQ <qq...@usermail.com>.
alayne@twobikes.ottawa.on.ca wrote:

>Well, turning off auto-whitelist does solve the system load problem.
>
>Unfortunately, it's also become clear that it turns off all spam tagging,
>meaning SpamAssassin is useless.
>
>Any ideas, please?
>  
>

Wow!

I turned off AWL and sure enough, I see all the system resources free 
up.  In fact, that second CPU I added is just sitting there doing nothing.

Unlike Alayne, I do not see any errors and I tried to upgrade DB_File 
from cpan but it told me I had the latest version.  I'm running RH 7.3 
with Perl 5.6.1 and all the updates.

I'm going to do what Greg did and rename my AWL.  This sucks though as I 
thought I had it pretty well trained.

QQQQ

RE: More problems was Re: DB_FILE error causes serious deep recursion/system overloading problem

Posted by Greg Allen <ga...@netrox.net>.
The old one was only 12k, I did not use it much and it was disabled before
the upgrade to latest SA. I have never cared much for the concept of AW.
This latest version of SA turns AW on by default however.

I think when SA turned it on, it started failing because it was created with
a very old db_file. It just could not talk to it. So I renamed it to get rid
of the old one. I could have deleted it, but I just wanted to be safe since
I don't know everything going on behind the scenes with the code. :-)

So maybe I will try AW for a little while again and see if I change my
opinion of auto-whitelist. I can always disable it again.



-----Original Message-----
From: qqqq [mailto:qqqq@usermail.com]
Sent: Monday, March 07, 2005 2:58 PM
To: gallen@netrox.net; alayne@twobikes.ottawa.on.ca;
users@spamassassin.apache.org
Subject: Re: More problems was Re: DB_FILE error causes serious deep
recursion/system overloading problem


Greg,

How large was auto-whitelist?
I just renamed mine and it was about 85Megs.  Ironically, I have a VERY old
one in the same directory that was about the same size.

QQQQ


----- Original Message -----
From: "Greg Allen" <ga...@netrox.net>
To: <al...@twobikes.ottawa.on.ca>; <us...@spamassassin.apache.org>
Sent: Monday, March 07, 2005 12:41 AM
Subject: RE: More problems was Re: DB_FILE error causes serious deep
recursion/system overloading problem


| I had a problem with auto-whitelist. I had errors writing to it when I
| tested with spamassassin -D --lint
|
| I went to the auto-whitelist directory
|
| /var/spool/filter/.spamassassin
|
| and renamed the auto-whitelist file to auto-whitelist.old
|
| The system created a new auto-whitelist and so far so good.
|
| I think this may have to do with DB_file upgrade, but the developers would
| know more than me.
|
|
|
|
|
| -----Original Message-----
| From: alayne@twobikes.ottawa.on.ca [mailto:alayne@twobikes.ottawa.on.ca]
| Sent: Monday, March 07, 2005 1:36 AM
| To: users@spamassassin.apache.org
| Subject: More problems was Re: DB_FILE error causes serious deep
| recursion/system overloading problem
|
|
| Well, turning off auto-whitelist does solve the system load problem.
|
| Unfortunately, it's also become clear that it turns off all spam tagging,
| meaning SpamAssassin is useless.
|
| Any ideas, please?
|
| I tried moving out all auto-whitelist files from home
| directories/.spamassassin and from /var/lib/nobody/.spamassassin -- that
| had no effect, either.
|
| Two more quick notes:
|
| The SuSE version is 8.2 (not 9.2) on the server. Perl version is 5.8.0 .
|
| Besides using up almost all the CPU cycles when auto-whitelist was
| enabled, spamd was also eating up almost all the memory.
|
| Alayne
|
| On Mon, 7 Mar 2005 alayne@twobikes.ottawa.on.ca wrote:
|
| > I had a serious spamd problem this evening after we rebooted our server
| > (running SuSE Linux 9.2). We had been running SpamAssassin 3.0.1 since
| > last October and it had been working very well for several months.
| >
| > Suddenly, shortly after the upgrade, the load averages for the server
shot
| > through the roof (from 0.09 to 30-40) for no reason we could see. We
| > narrowed it down to spamd. Once we killed that, everything was fine.
|
| [...]
| --
| Alayne McGregor
| alayne at twobikes.ottawa.on.ca
|
| "I am not amused by this 21st century. I want to take it back for a full
| refund,
| and can we have a real one, please?" -- Charles Stross (Locus, January,
| 2005)
|
|

Re: More problems was Re: DB_FILE error causes serious deep recursion/system overloading problem

Posted by qqqq <qq...@usermail.com>.
Greg,

How large was auto-whitelist?
I just renamed mine and it was about 85Megs.  Ironically, I have a VERY old
one in the same directory that was about the same size.

QQQQ


----- Original Message ----- 
From: "Greg Allen" <ga...@netrox.net>
To: <al...@twobikes.ottawa.on.ca>; <us...@spamassassin.apache.org>
Sent: Monday, March 07, 2005 12:41 AM
Subject: RE: More problems was Re: DB_FILE error causes serious deep
recursion/system overloading problem


| I had a problem with auto-whitelist. I had errors writing to it when I
| tested with spamassassin -D --lint
|
| I went to the auto-whitelist directory
|
| /var/spool/filter/.spamassassin
|
| and renamed the auto-whitelist file to auto-whitelist.old
|
| The system created a new auto-whitelist and so far so good.
|
| I think this may have to do with DB_file upgrade, but the developers would
| know more than me.
|
|
|
|
|
| -----Original Message-----
| From: alayne@twobikes.ottawa.on.ca [mailto:alayne@twobikes.ottawa.on.ca]
| Sent: Monday, March 07, 2005 1:36 AM
| To: users@spamassassin.apache.org
| Subject: More problems was Re: DB_FILE error causes serious deep
| recursion/system overloading problem
|
|
| Well, turning off auto-whitelist does solve the system load problem.
|
| Unfortunately, it's also become clear that it turns off all spam tagging,
| meaning SpamAssassin is useless.
|
| Any ideas, please?
|
| I tried moving out all auto-whitelist files from home
| directories/.spamassassin and from /var/lib/nobody/.spamassassin -- that
| had no effect, either.
|
| Two more quick notes:
|
| The SuSE version is 8.2 (not 9.2) on the server. Perl version is 5.8.0 .
|
| Besides using up almost all the CPU cycles when auto-whitelist was
| enabled, spamd was also eating up almost all the memory.
|
| Alayne
|
| On Mon, 7 Mar 2005 alayne@twobikes.ottawa.on.ca wrote:
|
| > I had a serious spamd problem this evening after we rebooted our server
| > (running SuSE Linux 9.2). We had been running SpamAssassin 3.0.1 since
| > last October and it had been working very well for several months.
| >
| > Suddenly, shortly after the upgrade, the load averages for the server
shot
| > through the roof (from 0.09 to 30-40) for no reason we could see. We
| > narrowed it down to spamd. Once we killed that, everything was fine.
|
| [...]
| --
| Alayne McGregor
| alayne at twobikes.ottawa.on.ca
|
| "I am not amused by this 21st century. I want to take it back for a full
| refund,
| and can we have a real one, please?" -- Charles Stross (Locus, January,
| 2005)
|
|

RE: More problems was Re: DB_FILE error causes serious deep recursion/system overloading problem

Posted by Greg Allen <ga...@netrox.net>.
I had a problem with auto-whitelist. I had errors writing to it when I
tested with spamassassin -D --lint

I went to the auto-whitelist directory

/var/spool/filter/.spamassassin

and renamed the auto-whitelist file to auto-whitelist.old

The system created a new auto-whitelist and so far so good.

I think this may have to do with DB_file upgrade, but the developers would
know more than me.





-----Original Message-----
From: alayne@twobikes.ottawa.on.ca [mailto:alayne@twobikes.ottawa.on.ca]
Sent: Monday, March 07, 2005 1:36 AM
To: users@spamassassin.apache.org
Subject: More problems was Re: DB_FILE error causes serious deep
recursion/system overloading problem


Well, turning off auto-whitelist does solve the system load problem.

Unfortunately, it's also become clear that it turns off all spam tagging,
meaning SpamAssassin is useless.

Any ideas, please?

I tried moving out all auto-whitelist files from home
directories/.spamassassin and from /var/lib/nobody/.spamassassin -- that
had no effect, either.

Two more quick notes:

The SuSE version is 8.2 (not 9.2) on the server. Perl version is 5.8.0 .

Besides using up almost all the CPU cycles when auto-whitelist was
enabled, spamd was also eating up almost all the memory.

Alayne

On Mon, 7 Mar 2005 alayne@twobikes.ottawa.on.ca wrote:

> I had a serious spamd problem this evening after we rebooted our server
> (running SuSE Linux 9.2). We had been running SpamAssassin 3.0.1 since
> last October and it had been working very well for several months.
>
> Suddenly, shortly after the upgrade, the load averages for the server shot
> through the roof (from 0.09 to 30-40) for no reason we could see. We
> narrowed it down to spamd. Once we killed that, everything was fine.

[...]
--
Alayne McGregor
alayne at twobikes.ottawa.on.ca

"I am not amused by this 21st century. I want to take it back for a full
refund,
and can we have a real one, please?" -- Charles Stross (Locus, January,
2005)

More problems was Re: DB_FILE error causes serious deep recursion/system overloading problem

Posted by al...@twobikes.ottawa.on.ca.
Well, turning off auto-whitelist does solve the system load problem.

Unfortunately, it's also become clear that it turns off all spam tagging,
meaning SpamAssassin is useless.

Any ideas, please?

I tried moving out all auto-whitelist files from home
directories/.spamassassin and from /var/lib/nobody/.spamassassin -- that
had no effect, either.

Two more quick notes:

The SuSE version is 8.2 (not 9.2) on the server. Perl version is 5.8.0 .

Besides using up almost all the CPU cycles when auto-whitelist was
enabled, spamd was also eating up almost all the memory.

Alayne

On Mon, 7 Mar 2005 alayne@twobikes.ottawa.on.ca wrote:

> I had a serious spamd problem this evening after we rebooted our server
> (running SuSE Linux 9.2). We had been running SpamAssassin 3.0.1 since
> last October and it had been working very well for several months.
>
> Suddenly, shortly after the upgrade, the load averages for the server shot
> through the roof (from 0.09 to 30-40) for no reason we could see. We
> narrowed it down to spamd. Once we killed that, everything was fine.

[...]
-- 
Alayne McGregor
alayne at twobikes.ottawa.on.ca

"I am not amused by this 21st century. I want to take it back for a full refund,
and can we have a real one, please?" -- Charles Stross (Locus, January, 2005)

Re: DB_FILE error causes serious deep recursion/system overloading problem

Posted by qqqq <qq...@usermail.com>.
This is interesting as I am dealing with a load issue that started at 3.0.1 
as well.
All of a sudden, all of my RAM is being used.  I added a second CPU to the 
box to no avail.
I am using AWL as well and will try to run it without to see what happens.
Unlike you, when I run it in debug mode, I do not get any errors.

You can read my problem with the subject line of "Spamd stops responding"

QQQQ

----- Original Message ----- 
From: <al...@twobikes.ottawa.on.ca>
To: <sp...@incubator.apache.org>
Sent: Sunday, March 06, 2005 10:11 PM
Subject: DB_FILE error causes serious deep recursion/system overloading 
problem


>I had a serious spamd problem this evening after we rebooted our server
> (running SuSE Linux 9.2). We had been running SpamAssassin 3.0.1 since
> last October and it had been working very well for several months.
>
> Suddenly, shortly after the upgrade, the load averages for the server shot
> through the roof (from 0.09 to 30-40) for no reason we could see. We
> narrowed it down to spamd. Once we killed that, everything was fine.
>
> I thought it might be a problem with 3.0.1 so I upgraded to 3.0.2.  The
> install went flawlessly, but the problem recurred as soon as I restarted
> spamd (though not if I ran spamassassin from the command line).
>
> I tried clearing out the mail queue (had a few leftover refused-delivery
> messages there). No effect.
>
> I ran spamd -D from the command line. Based on what I saw (attached
> below), I added the lines
>
> # turn off autowhitelisting
> use_auto_whitelist 0
>
> to /etc/mail/spamassassin/local.cf
>
> And that solved the problem immediately. I don't use or need
> auto-whitelisting so this was no big loss.
>
> What I would really like to know, though, is why it happened, why it
> happened now, and what can be done to avoid it in the future.
>
> Anyone have any ideas? Thanks in advance ...
>
> Alayne
>
> -----------------------------------------------------------------------------
> debug messages from running spamd before changing local.cf:
>
> # spamd -D
> trying to connect to syslog/unix...
> no error connecting to syslog/unix
> logging enabled:
>        facility: mail
>        socket:   unix
>        output:   syslog
> creating INET socket:
>        Listen: 128
>        LocalAddr: 127.0.0.1
>        LocalPort: 783
>        Proto: 6
>        ReuseAddr: 1
>        Type: 1
> debug: SpamAssassin version 3.0.2
> debug: Score set 0 chosen.
> debug: Storable module v2.04 found
> debug: Preloading modules with HOME=/tmp/spamd-8866-init
> debug: ignore: test message to precompile patterns and load modules
> debug: using "/etc/mail/spamassassin/init.pre" for site rules init.pre
> debug: config: read file /etc/mail/spamassassin/init.pre
> debug: using "/usr/share/spamassassin" for default rules dir
> debug: config: read file /usr/share/spamassassin/10_misc.cf
> debug: config: read file /usr/share/spamassassin/20_anti_ratware.cf
> debug: config: read file /usr/share/spamassassin/20_body_tests.cf
> debug: config: read file /usr/share/spamassassin/20_compensate.cf
> debug: config: read file /usr/share/spamassassin/20_dnsbl_tests.cf
> debug: config: read file /usr/share/spamassassin/20_drugs.cf
> debug: config: read file /usr/share/spamassassin/20_fake_helo_tests.cf
> debug: config: read file /usr/share/spamassassin/20_head_tests.cf
> debug: config: read file /usr/share/spamassassin/20_html_tests.cf
> debug: config: read file /usr/share/spamassassin/20_meta_tests.cf
> debug: config: read file /usr/share/spamassassin/20_phrases.cf
> debug: config: read file /usr/share/spamassassin/20_porn.cf
> debug: config: read file /usr/share/spamassassin/20_ratware.cf
> debug: config: read file /usr/share/spamassassin/20_uri_tests.cf
> debug: config: read file /usr/share/spamassassin/23_bayes.cf
> debug: config: read file /usr/share/spamassassin/25_body_tests_es.cf
> debug: config: read file /usr/share/spamassassin/25_hashcash.cf
> debug: config: read file /usr/share/spamassassin/25_spf.cf
> debug: config: read file /usr/share/spamassassin/25_uribl.cf
> debug: config: read file /usr/share/spamassassin/30_text_de.cf
> debug: config: read file /usr/share/spamassassin/30_text_fr.cf
> debug: config: read file /usr/share/spamassassin/30_text_nl.cf
> debug: config: read file /usr/share/spamassassin/30_text_pl.cf
> debug: config: read file /usr/share/spamassassin/50_scores.cf
> debug: config: read file /usr/share/spamassassin/60_whitelist.cf
> debug: using "/etc/mail/spamassassin" for site rules dir
> debug: config: read file /etc/mail/spamassassin/local.cf
> debug: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
> debug: plugin: registered
> Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
> debug: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC
> debug: plugin: registered
> Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8cbd434)
> debug: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC
> debug: plugin: registered Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc)
> debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
> implements 'parse_config'
> debug: plugin: Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8cbd434)
> implements 'parse_config'
> debug: config: SpamAssassin failed to parse line, skipping:
> rewrite_subject 0
> debug: config: SpamAssassin failed to parse line, skipping:
> use_terse_report 1
> debug: Allowing user rules!
> debug: config: SpamAssassin failed to parse line, skipping: cheap Rolex
> Watch?
> debug: bayes: no dbs present, cannot tie DB R/O:
> /tmp/spamd-8866-init/.spamassassin/bayes_toks
> debug: Score set 1 chosen.
> debug: ---- MIME PARSER START ----
> debug: main message type: text/plain
> debug: parsing normal part
> debug: added part, type: text/plain
> debug: ---- MIME PARSER END ----
> debug: bayes: no dbs present, cannot tie DB R/O:
> /tmp/spamd-8866-init/.spamassassin/bayes_toks
> debug: metadata: X-Spam-Relays-Trusted:
> debug: metadata: X-Spam-Relays-Untrusted:
> debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
> implements 'parsed_metadata'
> debug: is Net::DNS::Resolver available? yes
> debug: Net::DNS version: 0.48
> debug: trying (3) akamai.com...
> debug: looking up NS for 'akamai.com'
> debug: NS lookup of akamai.com succeeded => Dns available (set
> dns_available to hardcode)
> debug: is DNS available? 1
> debug: decoding: no encoding detected
> debug: URIDNSBL: domains to query:
> debug: all '*From' addrs: ignore@compiling.spamassassin.taint.org
> debug: Running tests for priority: 0
> debug: running header regexp tests; score so far=0
> debug: registering glue method for check_hashcash_double_spend
> (Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8cbd434))
> debug: registering glue method for check_for_spf_helo_pass
> (Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
> debug: SPF: message was delivered entirely via trusted relays, not
> required
> debug: registering glue method for check_hashcash_value
> (Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8cbd434))
> debug: all '*To' addrs:
> debug: registering glue method for check_for_spf_softfail
> (Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
> debug: SPF: message was delivered entirely via trusted relays, not
> required
> debug: registering glue method for check_for_spf_pass
> (Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
> debug: registering glue method for check_for_spf_helo_softfail
> (Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
> debug: registering glue method for check_for_spf_helo_fail
> (Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
> debug: running body-text per-line regexp tests; score so far=-2.623
> debug: running uri tests; score so far=-2.623
> debug: registering glue method for check_uridnsbl
> (Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4))
> debug: Razor2 is not available
> debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
> implements 'check_tick'
> debug: running raw-body-text per-line regexp tests; score so far=-2.623
> debug: running full-text regexp tests; score so far=-2.623
> debug: Razor2 is not available
> debug: Current PATH is: /usr/sbin:/bin:/usr/bin:/sbin:/usr/X11R6/bin
> debug: Pyzor is not available: pyzor not found
> debug: DCCifd is not available: no r/w dccifd socket found.
> debug: DCC is not available: no executable dccproc found.
> debug: Running tests for priority: 500
> debug: RBL: success for 1 of 1 queries
> debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
> implements 'check_post_dnsbl'
> debug: running meta tests; score so far=-2.623
> debug: running header regexp tests; score so far=-1.053
> debug: running body-text per-line regexp tests; score so far=-1.053
> debug: running uri tests; score so far=-1.053
> debug: running raw-body-text per-line regexp tests; score so far=-1.053
> debug: running full-text regexp tests; score so far=-1.053
> debug: Running tests for priority: 1000
> debug: running meta tests; score so far=-1.053
> debug: running header regexp tests; score so far=-1.053
> debug: lock: 8866 created
> /tmp/spamd-8866-init/.spamassassin/auto-whitelist.lock.pannier.twobikes.ottawa.on.ca.8866
> debug: lock: 8866 trying to get lock on
> /tmp/spamd-8866-init/.spamassassin/auto-whitelist with 0 retries
> debug: lock: 8866 link to
> /tmp/spamd-8866-init/.spamassassin/auto-whitelist.lock: link ok
> debug: Tie-ing to DB file R/W in
> /tmp/spamd-8866-init/.spamassassin/auto-whitelist
> Use of uninitialized value in numeric gt (>) at
> /usr/lib/perl5/5.8.0/i586-linux-thread-multi/DB_File.pm line 271.
> Deep recursion on subroutine "DB_File::AUTOLOAD" at
> /usr/lib/perl5/5.8.0/i586-linux-thread-multi/DB_File.pm line 234.
>
> (at this point I killed spamd).
>
> -- 
> Alayne McGregor
> alayne at twobikes.ottawa.on.ca
>
> "I am not amused by this 21st century. I want to take it back for a full 
> refund,
> and can we have a real one, please?" -- Charles Stross (Locus, January, 
> 2005)
>
>

RE: DB_FILE error causes serious deep recursion/system overloading problem

Posted by Greg Allen <ga...@netrox.net>.
This is not good. You should make sure that you upgraded all of the
supporting files including db_file to latest from cpan.org

run spamassassin -D --lint to see what files it uses and upgrade those.

You need to get your db working again it looks like to me.

-->debug: bayes: no dbs present, cannot tie DB R/O:




-----Original Message-----
From: alayne@twobikes.ottawa.on.ca [mailto:alayne@twobikes.ottawa.on.ca]
Sent: Monday, March 07, 2005 12:12 AM
To: spamassassin-users@incubator.apache.org
Subject: DB_FILE error causes serious deep recursion/system overloading
problem


I had a serious spamd problem this evening after we rebooted our server
(running SuSE Linux 9.2). We had been running SpamAssassin 3.0.1 since
last October and it had been working very well for several months.

Suddenly, shortly after the upgrade, the load averages for the server shot
through the roof (from 0.09 to 30-40) for no reason we could see. We
narrowed it down to spamd. Once we killed that, everything was fine.

I thought it might be a problem with 3.0.1 so I upgraded to 3.0.2.  The
install went flawlessly, but the problem recurred as soon as I restarted
spamd (though not if I ran spamassassin from the command line).

I tried clearing out the mail queue (had a few leftover refused-delivery
messages there). No effect.

I ran spamd -D from the command line. Based on what I saw (attached
below), I added the lines

# turn off autowhitelisting
use_auto_whitelist 0

to /etc/mail/spamassassin/local.cf

And that solved the problem immediately. I don't use or need
auto-whitelisting so this was no big loss.

What I would really like to know, though, is why it happened, why it
happened now, and what can be done to avoid it in the future.

Anyone have any ideas? Thanks in advance ...

Alayne

----------------------------------------------------------------------------
-
debug messages from running spamd before changing local.cf:

# spamd -D
trying to connect to syslog/unix...
no error connecting to syslog/unix
logging enabled:
        facility: mail
        socket:   unix
        output:   syslog
creating INET socket:
        Listen: 128
        LocalAddr: 127.0.0.1
        LocalPort: 783
        Proto: 6
        ReuseAddr: 1
        Type: 1
debug: SpamAssassin version 3.0.2
debug: Score set 0 chosen.
debug: Storable module v2.04 found
debug: Preloading modules with HOME=/tmp/spamd-8866-init
debug: ignore: test message to precompile patterns and load modules
debug: using "/etc/mail/spamassassin/init.pre" for site rules init.pre
debug: config: read file /etc/mail/spamassassin/init.pre
debug: using "/usr/share/spamassassin" for default rules dir
debug: config: read file /usr/share/spamassassin/10_misc.cf
debug: config: read file /usr/share/spamassassin/20_anti_ratware.cf
debug: config: read file /usr/share/spamassassin/20_body_tests.cf
debug: config: read file /usr/share/spamassassin/20_compensate.cf
debug: config: read file /usr/share/spamassassin/20_dnsbl_tests.cf
debug: config: read file /usr/share/spamassassin/20_drugs.cf
debug: config: read file /usr/share/spamassassin/20_fake_helo_tests.cf
debug: config: read file /usr/share/spamassassin/20_head_tests.cf
debug: config: read file /usr/share/spamassassin/20_html_tests.cf
debug: config: read file /usr/share/spamassassin/20_meta_tests.cf
debug: config: read file /usr/share/spamassassin/20_phrases.cf
debug: config: read file /usr/share/spamassassin/20_porn.cf
debug: config: read file /usr/share/spamassassin/20_ratware.cf
debug: config: read file /usr/share/spamassassin/20_uri_tests.cf
debug: config: read file /usr/share/spamassassin/23_bayes.cf
debug: config: read file /usr/share/spamassassin/25_body_tests_es.cf
debug: config: read file /usr/share/spamassassin/25_hashcash.cf
debug: config: read file /usr/share/spamassassin/25_spf.cf
debug: config: read file /usr/share/spamassassin/25_uribl.cf
debug: config: read file /usr/share/spamassassin/30_text_de.cf
debug: config: read file /usr/share/spamassassin/30_text_fr.cf
debug: config: read file /usr/share/spamassassin/30_text_nl.cf
debug: config: read file /usr/share/spamassassin/30_text_pl.cf
debug: config: read file /usr/share/spamassassin/50_scores.cf
debug: config: read file /usr/share/spamassassin/60_whitelist.cf
debug: using "/etc/mail/spamassassin" for site rules dir
debug: config: read file /etc/mail/spamassassin/local.cf
debug: plugin: loading Mail::SpamAssassin::Plugin::URIDNSBL from @INC
debug: plugin: registered
Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
debug: plugin: loading Mail::SpamAssassin::Plugin::Hashcash from @INC
debug: plugin: registered
Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8cbd434)
debug: plugin: loading Mail::SpamAssassin::Plugin::SPF from @INC
debug: plugin: registered Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc)
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
implements 'parse_config'
debug: plugin: Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8cbd434)
implements 'parse_config'
debug: config: SpamAssassin failed to parse line, skipping:
rewrite_subject 0
debug: config: SpamAssassin failed to parse line, skipping:
use_terse_report 1
debug: Allowing user rules!
debug: config: SpamAssassin failed to parse line, skipping: cheap Rolex
Watch?
debug: bayes: no dbs present, cannot tie DB R/O:
/tmp/spamd-8866-init/.spamassassin/bayes_toks
debug: Score set 1 chosen.
debug: ---- MIME PARSER START ----
debug: main message type: text/plain
debug: parsing normal part
debug: added part, type: text/plain
debug: ---- MIME PARSER END ----
debug: bayes: no dbs present, cannot tie DB R/O:
/tmp/spamd-8866-init/.spamassassin/bayes_toks
debug: metadata: X-Spam-Relays-Trusted:
debug: metadata: X-Spam-Relays-Untrusted:
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
implements 'parsed_metadata'
debug: is Net::DNS::Resolver available? yes
debug: Net::DNS version: 0.48
debug: trying (3) akamai.com...
debug: looking up NS for 'akamai.com'
debug: NS lookup of akamai.com succeeded => Dns available (set
dns_available to hardcode)
debug: is DNS available? 1
debug: decoding: no encoding detected
debug: URIDNSBL: domains to query:
debug: all '*From' addrs: ignore@compiling.spamassassin.taint.org
debug: Running tests for priority: 0
debug: running header regexp tests; score so far=0
debug: registering glue method for check_hashcash_double_spend
(Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8cbd434))
debug: registering glue method for check_for_spf_helo_pass
(Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
debug: SPF: message was delivered entirely via trusted relays, not
required
debug: registering glue method for check_hashcash_value
(Mail::SpamAssassin::Plugin::Hashcash=HASH(0x8cbd434))
debug: all '*To' addrs:
debug: registering glue method for check_for_spf_softfail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
debug: SPF: message was delivered entirely via trusted relays, not
required
debug: registering glue method for check_for_spf_pass
(Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
debug: registering glue method for check_for_spf_helo_softfail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
debug: registering glue method for check_for_spf_helo_fail
(Mail::SpamAssassin::Plugin::SPF=HASH(0x8c98ecc))
debug: running body-text per-line regexp tests; score so far=-2.623
debug: running uri tests; score so far=-2.623
debug: registering glue method for check_uridnsbl
(Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4))
debug: Razor2 is not available
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
implements 'check_tick'
debug: running raw-body-text per-line regexp tests; score so far=-2.623
debug: running full-text regexp tests; score so far=-2.623
debug: Razor2 is not available
debug: Current PATH is: /usr/sbin:/bin:/usr/bin:/sbin:/usr/X11R6/bin
debug: Pyzor is not available: pyzor not found
debug: DCCifd is not available: no r/w dccifd socket found.
debug: DCC is not available: no executable dccproc found.
debug: Running tests for priority: 500
debug: RBL: success for 1 of 1 queries
debug: plugin: Mail::SpamAssassin::Plugin::URIDNSBL=HASH(0x84bfec4)
implements 'check_post_dnsbl'
debug: running meta tests; score so far=-2.623
debug: running header regexp tests; score so far=-1.053
debug: running body-text per-line regexp tests; score so far=-1.053
debug: running uri tests; score so far=-1.053
debug: running raw-body-text per-line regexp tests; score so far=-1.053
debug: running full-text regexp tests; score so far=-1.053
debug: Running tests for priority: 1000
debug: running meta tests; score so far=-1.053
debug: running header regexp tests; score so far=-1.053
debug: lock: 8866 created
/tmp/spamd-8866-init/.spamassassin/auto-whitelist.lock.pannier.twobikes.otta
wa.on.ca.8866
debug: lock: 8866 trying to get lock on
/tmp/spamd-8866-init/.spamassassin/auto-whitelist with 0 retries
debug: lock: 8866 link to
/tmp/spamd-8866-init/.spamassassin/auto-whitelist.lock: link ok
debug: Tie-ing to DB file R/W in
/tmp/spamd-8866-init/.spamassassin/auto-whitelist
Use of uninitialized value in numeric gt (>) at
/usr/lib/perl5/5.8.0/i586-linux-thread-multi/DB_File.pm line 271.
Deep recursion on subroutine "DB_File::AUTOLOAD" at
/usr/lib/perl5/5.8.0/i586-linux-thread-multi/DB_File.pm line 234.

(at this point I killed spamd).

--
Alayne McGregor
alayne at twobikes.ottawa.on.ca

"I am not amused by this 21st century. I want to take it back for a full
refund,
and can we have a real one, please?" -- Charles Stross (Locus, January,
2005)