You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by "Dan Mahoney, System Admin" <da...@prime.gushi.org> on 2006/03/09 03:17:52 UTC
Re: [sa-list] Re: [sa-list] Re: Spamd keeps getting hung up!
On Tue, 7 Mar 2006, Theo Van Dinter wrote:
> On Tue, Mar 07, 2006 at 08:14:38PM -0500, Dan Mahoney, System Admin wrote:
>>> =item languages_filename
>>>
>>> If you want to be able to use the language-guessing rule
>>> C<UNWANTED_LANGUAGE_BODY>, and are using C<config_text> instead of
>>> C<rules_filename>, C<site_rules_filename>, and C<userprefs_filename>, you
> [...]
>>
>> I'm NOT using rules_text though, unless this is the default behavior.
>> I got this for a while:
>>
>> Mar 7 19:51:17 quark spamd[81728]: config: failed to parse line,
>> skipping: languages_filename /usr/local/share/spamassassin/languages
>
> rules_text, languages_filename, etc, are parameters to the Mail::SpamAssassin
> object during creation, not config options. So "languages_filename" won't
> really help you unless you're writing your own code.
>
>> I gave up and disabled textcat -- it's a shame, I'd like to use it.
>
> TextCat will use <default path to rules>/languages. Run "spamassassin -D
> --lint" and look for the default rules line:
>
> [22346] dbg: config: using "/usr/share/spamassassin" for default rules dir
Yes, it did. I'll kill that bug off in a few. Thanks. BTW, for some
reason the languages file was MISSING originally. Possibly due to package
maintainer insanity, I've been using SA for a WHILE now.
> I responded with this information in the ticket but you never responded.
I'll deal with the ticket shortly.
[Odd SQL Error]
>
>> The only VERY ODD error I am seeing now (and only for ONE user) is this:
>>
>> Mar 7 20:19:12 quark spamd[83564]: config: failed to parse line,
>> skipping: rewrite_subject_0
>> Mar 7 20:19:12 quark spamd[83564]: config: failed to parse line,
>> skipping: report_header_1
>> Mar 7 20:19:12 quark spamd[83564]: config: failed to parse line,
>> skipping: use_terse_report_1
>> Mar 7 20:19:12 quark spamd[83564]: config: failed to parse line,
>> skipping: defang_mime_0
>> Mar 7 20:19:12 quark spamd[83564]: config: failed to parse line,
>> skipping: auto_learn_0
>
> Why are these odd? None of these are valid config lines for 3.1.
Because my configs are in a SQL database, and for that user, the config
lines look totally normal. For this user only, the SQL interpreter is
adding an underscore where there shouldn't be one. The user doesn't even
have a homedir on the system where spamd runs -- and spamd runs as user
"spamd" who has no prefs of its own.
mysql> select * from userpref WHERE username = 'cww'
-> ;
+----------+------------------+--------------------+--------+
| username | preference | value | prefid |
+----------+------------------+--------------------+--------+
| cww | required_hits | 6 | 600 |
| cww | rewrite_subject | 0 | 601 |
| cww | report_header | 1 | 602 |
| cww | report_safe | 1 | 603 |
| cww | use_terse_report | 1 | 604 |
| cww | defang_mime | 0 | 605 |
| cww | whitelist_from | list@myucomics.com | 606 |
| cww | whitelist_from | *@rwmt.com | 607 |
| cww | whitelist_from | *komando.com | 608 |
| cww | use_bayes | 0 | 609 |
| cww | auto_learn | 0 | 610 |
+----------+------------------+--------------------+--------+
It's definitely that user, but just to be sure, I decided to double check:
mysql> select * from userpref where preference LIKE "%subject%";
+-----------------------+------------------------+---------------+--------+
| username | preference | value | prefid
|
+-----------------------+------------------------+---------------+--------+
| mirthika | rewrite_subject | 0 | 211
|
| computechie | subject_tag | {SpamAssasin} | 230
|
| madge@prime.gushi.org | rewrite_header subject | SPAM | 442
|
| cww | rewrite_subject | 0 | 601
|
+-----------------------+------------------------+---------------+--------+
4 rows in set (0.02 sec)
Here's proof it's that user:
Mar 8 00:04:21 quark spamd[83570]: config: failed to parse line,
skipping: rewrite_subject_0
Mar 8 00:04:21 quark spamd[83570]: config: failed to parse line,
skipping: report_header_1
Mar 8 00:04:21 quark spamd[83570]: config: failed to parse line,
skipping: use_terse_report_1
Mar 8 00:04:21 quark spamd[83570]: config: failed to parse line,
skipping: defang_mime_0
Mar 8 00:04:21 quark spamd[83570]: config: failed to parse line,
skipping: auto_learn_0
Mar 8 00:04:21 quark spamd[83570]: Use of uninitialized value in pattern
match (m//) at /usr/local/lib/perl5/site_perl/5.8.6/Mail/S
pamAssassin/Conf/Parser.pm line 547, <GEN2412> line 2.
Mar 8 00:04:21 quark spamd[83570]: Use of uninitialized value in pattern
match (m//) at /usr/local/lib/perl5/site_perl/5.8.6/Mail/S
pamAssassin/Conf/Parser.pm line 547, <GEN2412> line 2.
Mar 8 00:04:21 quark spamd[83570]: spamd: processing message
<21...@0845dial.com> for cww:58
OCCASIONALLY I see it for other users, like this -- however cww hits ALL
the time:
Mar 8 00:01:57 quark spamd[83570]: config: failed to parse line,
skipping: report_header_1
Mar 8 00:01:57 quark spamd[83570]: config: failed to parse line,
skipping: use_terse_report_0
Mar 8 00:01:57 quark spamd[83570]: config: failed to parse line,
skipping: defang_mime_1
Mar 8 00:01:57 quark spamd[83570]: config: failed to parse line,
skipping: bayes_auto_learn_threshold_spa_6.6
Mar 8 00:01:57 quark spamd[83570]: config: failed to parse line,
skipping: bayes_auto_learn_threshold_non_2.0
Mar 8 00:01:57 quark spamd[83570]: Use of uninitialized value in pattern
match (m//) at /usr/local/lib/perl5/site_perl/5.8.6/Mail/S
pamAssassin/Conf/Parser.pm line 547, <GEN2284> line 2.
Mar 8 00:01:57 quark spamd[83570]: Use of uninitialized value in pattern
match (m//) at /usr/local/lib/perl5/site_perl/5.8.6/Mail/S
pamAssassin/Conf/Parser.pm line 547, <GEN2284> line 2.
Mar 8 00:01:57 quark spamd[83570]: spamd: processing message
<14...@guruweb2> for sdk:58
That's likely the cause of the uninitialized pattern error -- I believe
one of you (Justin/Theo/Daryl) suggested a possible fix -- should I try it
-- it wouldn't fix the fact that these things are being occasionally put
together wrong, but would at least quell the uninitialzed value error :)
[pyzor, razor, dcc]
>> I could see trouble with any ONE of these things -- but all three?
>> Something feels odd.
>
> Do you have a firewall blocking those services from working?
No firewall at all, and those processes hit (as in, generate scores) most
of the time. I'm under the impression if they were going to be killed by
a firewall config, it would be a very all or nothing type thing.
It's a realtek NIC, but that shouldn't matter at all since ALL this box
does is mysql and spam processes (less than a meg). Duplex is 100/full
(auto) on a known good (but unmanaged) switch.
And there was the mention I saw earlier of still-escaping alarms.
However, I haven't had a true lockup since making all these changes.
Usually those happen late at night when I say "they musta fixed it, things
haven't locked up in a while..." :)
I'll let you guys know as soon as I do (it does), however. __alarm__
messages without a corresponding freeze CONCERN me, but not NEARLY as much
as when the system is "of a down".
-Dan
--
"Why are you wearing TWO grounding straps?"
-John Evans, Ezzi Computers August 23, 2001
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
Errors in SQL config.
Posted by "Dan Mahoney, System Admin" <da...@prime.gushi.org>.
On Wed, 8 Mar 2006, Dan Mahoney, System Admin wrote:
> On Wed, 8 Mar 2006, Daryl C. W. O'Shea wrote:
>
>> On 08/03/06 09:17 PM, Dan Mahoney, System Admin wrote:
>>> On Tue, 7 Mar 2006, Theo Van Dinter wrote:
>>>
>>>> On Tue, Mar 07, 2006 at 08:14:38PM -0500, Dan Mahoney, System Admin
>>>> wrote:
>>
>>> [Odd SQL Error]
Nevermind. You said "invalid", and I misunderstood that you said it
meaning obsolete/deprecated, as opposed to "incorrectly formatted".
*foreheadsmack*
Is there a complete list of deprecated options somewhere (along with the
versions they were valid for, and/or what's replaced them?) I know for
the most recent version, it's always in the release notes -- but it would
be good if SA could warn on obsolete options.
Alternatively -- is there an easy way to make use of the
Mail::SpamAssassin::Conf class when importing options INTO my DB?
Sorry to come at you guys with so many issues at once -- the desire to
clean up the original issue is making me take further scrutiny on my logs
than probably ever before.
-Dan
--
"You're a nomad billygoat!"
-Juston, July 18th, 2002
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
Re: Spamd keeps getting hung up!
Posted by "Dan Mahoney, System Admin" <da...@prime.gushi.org>.
On Wed, 8 Mar 2006, Daryl C. W. O'Shea wrote:
> On 08/03/06 09:17 PM, Dan Mahoney, System Admin wrote:
>> On Tue, 7 Mar 2006, Theo Van Dinter wrote:
>>
>>> On Tue, Mar 07, 2006 at 08:14:38PM -0500, Dan Mahoney, System Admin wrote:
>
>> [Odd SQL Error]
> Just because the configs are in a SQL database, doesn't mean that they'll
> work even though they are invalid. ;)
>
> Underscore or not, they're not valid.
That's fine -- but the format they're IN in the DB *IS* valid. Somewhere
between being in the DB and getting read into spamd something's putting an
underscore where there should be whitespace (hence my logic at pasting
those SQL queries -- note that it also ONLY happens for certain prefs,
too -- another user could have identical prefs and be fine)
It's a separate issue from the lockups -- and I'm not sure why it's
happening -- but it's definitely an issue. What action is best to take on
this? Bug report? It goes without saying here, but if anyone needs a
shell (I think JM has one on the box where spamc runs), let me know and
I'll get you one without question.
>> [pyzor, razor, dcc]
>>
>>>> I could see trouble with any ONE of these things -- but all three?
>>>> Something feels odd.
>>>
>>> Do you have a firewall blocking those services from working?
>>
>> No firewall at all, and those processes hit (as in, generate scores) most
>> of the time. I'm under the impression if they were going to be killed by a
>> firewall config, it would be a very all or nothing type thing.
>>
>> It's a realtek NIC, but that shouldn't matter at all since ALL this box
>> does is mysql and spam processes (less than a meg). Duplex is 100/full
>> (auto) on a known good (but unmanaged) switch.
>
> Well, if the timeouts (and they are just timeouts) are congestion related, it
> would make sense that all three would timeout.
True, of course. I don't think it's congestion either -- although at some
point we may start running our own (public) DCC server in-house. Need to
speak to Vernon about that.
>> And there was the mention I saw earlier of still-escaping alarms. However,
>> I haven't had a true lockup since making all these changes. Usually those
>> happen late at night when I say "they musta fixed it, things haven't locked
>> up in a while..." :)
>
> Mine only hangs up when I think to myself, "gee it's been awhile since it
> last hung up" and then proceed to travel at least three hours away form the
> server, often far from connectivity or into exceptionally slow or unreliable
> dial-up land.
It's at the point where I've thought about having a process tail -F the
log for the occurence of BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB, and take action
on it. There's only been once or twice where it's hung up AGAIN after
restarting -- usually on a ton of messages for one user. This is usually
the point at which I send a message to the mailing list about "can we
please get a way to dump messages-in-process to a single file, ala the
apache server-status screen" so I can see WHY it's hanging up, and on
which rule.
On the thought that it's congestion, maybe I should start running
smokeping against the various servers utilized by dcc/pyzor/razor --
would anyone else be interested in this data?
>> I'll let you guys know as soon as I do (it does), however. __alarm__
>> messages without a corresponding freeze CONCERN me, but not NEARLY as much
>> as when the system is "of a down".
>
> "__alarm__" without another warning explaining it immediately afterwards are
> bad.
>
> "__alarm__ignore__" are just timeouts that can be, well, ignored.
I await your reply :)
-Dan
--
"The first annual 5th of July party...have you been invited?"
"It's a Jack Party."
"Okay, so Long Island's been invited."
--Cali and Gushi, 6/23/02
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
Re: [sa-list] Re: [sa-list] Re: Spamd keeps getting hung up!
Posted by "Daryl C. W. O'Shea" <sp...@dostech.ca>.
On 08/03/06 09:17 PM, Dan Mahoney, System Admin wrote:
> On Tue, 7 Mar 2006, Theo Van Dinter wrote:
>
>> On Tue, Mar 07, 2006 at 08:14:38PM -0500, Dan Mahoney, System Admin
>> wrote:
> [Odd SQL Error]
>
>>
>>> The only VERY ODD error I am seeing now (and only for ONE user) is this:
>>>
>>> Mar 7 20:19:12 quark spamd[83564]: config: failed to parse line,
>>> skipping: rewrite_subject_0
>>> Mar 7 20:19:12 quark spamd[83564]: config: failed to parse line,
>>> skipping: report_header_1
>>> Mar 7 20:19:12 quark spamd[83564]: config: failed to parse line,
>>> skipping: use_terse_report_1
>>> Mar 7 20:19:12 quark spamd[83564]: config: failed to parse line,
>>> skipping: defang_mime_0
>>> Mar 7 20:19:12 quark spamd[83564]: config: failed to parse line,
>>> skipping: auto_learn_0
>>
>> Why are these odd? None of these are valid config lines for 3.1.
>
> Because my configs are in a SQL database, and for that user, the config
> lines look totally normal. For this user only, the SQL interpreter is
> adding an underscore where there shouldn't be one. The user doesn't
> even have a homedir on the system where spamd runs -- and spamd runs as
> user "spamd" who has no prefs of its own.
Just because the configs are in a SQL database, doesn't mean that
they'll work even though they are invalid. ;)
Underscore or not, they're not valid.
> [pyzor, razor, dcc]
>
>>> I could see trouble with any ONE of these things -- but all three?
>>> Something feels odd.
>>
>> Do you have a firewall blocking those services from working?
>
> No firewall at all, and those processes hit (as in, generate scores)
> most of the time. I'm under the impression if they were going to be
> killed by a firewall config, it would be a very all or nothing type thing.
>
> It's a realtek NIC, but that shouldn't matter at all since ALL this box
> does is mysql and spam processes (less than a meg). Duplex is 100/full
> (auto) on a known good (but unmanaged) switch.
Well, if the timeouts (and they are just timeouts) are congestion
related, it would make sense that all three would timeout.
> And there was the mention I saw earlier of still-escaping alarms.
> However, I haven't had a true lockup since making all these changes.
> Usually those happen late at night when I say "they musta fixed it,
> things haven't locked up in a while..." :)
Mine only hangs up when I think to myself, "gee it's been awhile since
it last hung up" and then proceed to travel at least three hours away
form the server, often far from connectivity or into exceptionally slow
or unreliable dial-up land.
> I'll let you guys know as soon as I do (it does), however. __alarm__
> messages without a corresponding freeze CONCERN me, but not NEARLY as
> much as when the system is "of a down".
"__alarm__" without another warning explaining it immediately afterwards
are bad.
"__alarm__ignore__" are just timeouts that can be, well, ignored.
Daryl
Re: [sa-list] Re: [sa-list] Re: Spamd keeps getting hung up!
Posted by Michael Parker <pa...@pobox.com>.
Theo Van Dinter wrote:
> On Wed, Mar 08, 2006 at 09:17:52PM -0500, Dan Mahoney, System Admin wrote:
>>>> Mar 7 20:19:12 quark spamd[83564]: config: failed to parse line,
>>>> skipping: rewrite_subject_0
>>> Why are these odd? None of these are valid config lines for 3.1.
>> Because my configs are in a SQL database, and for that user, the config
>> lines look totally normal. For this user only, the SQL interpreter is
>> adding an underscore where there shouldn't be one.
>
> I have a feeling this is a display issue with the error message and not
> when reading in the options (or else every option would throw an error).
> Regardless of that, the options are invalid so the errors are valid.
>
The logger module replaces spaces with underscores.
Michael
Re: Spamd keeps getting hung up!
Posted by "Dan Mahoney, System Admin" <da...@prime.gushi.org>.
On Wed, 8 Mar 2006, Theo Van Dinter wrote:
>> match (m//) at /usr/local/lib/perl5/site_perl/5.8.6/Mail/S
>> pamAssassin/Conf/Parser.pm line 547, <GEN2284> line 2.
>
> $set_score = -$set_score if ( $conf->{tflags}->{$k} =~ /\bnice\b/ );
>
> $conf->{tflags}->{$k} is undefined, which shouldn't be possible with the
> standard code (tflags are set to '' whenever a test is added).
So what should I do about it? Apply the line above?
>> No firewall at all, and those processes hit (as in, generate scores) most
>> of the time. I'm under the impression if they were going to be killed by
>> a firewall config, it would be a very all or nothing type thing.
>
> Sure. If it's intermittant, and not a local system issue, it could just be
> timeouts on the Internet connect, at the remote end, etc.
Which I guess begs the question -- is there a way to get more verbose
logging from those three services? Either related to connection times on
average, or at least spewing timeout/connect failures in the log.
Usualy on nights I'm having connectivity issues, other "odd" things
happen. IRC connections bounce. My IM connection (which runs from the
server) hangs up, and of course, people complain.
I'm not saying this is NOT it, but I haven't noticed a correlation between
SA hangups and this.
--
"You're a daddy. I'm a mommy. She's our baby. Deal with it."
-Cali, 11/7/02, about 1:35 AM
--------Dan Mahoney--------
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---------------------------
Re: [sa-list] Re: [sa-list] Re: Spamd keeps getting hung up!
Posted by Theo Van Dinter <fe...@apache.org>.
On Wed, Mar 08, 2006 at 09:17:52PM -0500, Dan Mahoney, System Admin wrote:
> >>Mar 7 20:19:12 quark spamd[83564]: config: failed to parse line,
> >>skipping: rewrite_subject_0
> >Why are these odd? None of these are valid config lines for 3.1.
>
> Because my configs are in a SQL database, and for that user, the config
> lines look totally normal. For this user only, the SQL interpreter is
> adding an underscore where there shouldn't be one.
I have a feeling this is a display issue with the error message and not
when reading in the options (or else every option would throw an error).
Regardless of that, the options are invalid so the errors are valid.
> Mar 8 00:01:57 quark spamd[83570]: config: failed to parse line,
> skipping: report_header_1
> Mar 8 00:01:57 quark spamd[83570]: config: failed to parse line,
> skipping: use_terse_report_0
> Mar 8 00:01:57 quark spamd[83570]: config: failed to parse line,
> skipping: defang_mime_1
> Mar 8 00:01:57 quark spamd[83570]: config: failed to parse line,
> skipping: bayes_auto_learn_threshold_spa_6.6
> Mar 8 00:01:57 quark spamd[83570]: config: failed to parse line,
> skipping: bayes_auto_learn_threshold_non_2.0
these are all invalid options as well.
> Mar 8 00:01:57 quark spamd[83570]: Use of uninitialized value in pattern
> match (m//) at /usr/local/lib/perl5/site_perl/5.8.6/Mail/S
> pamAssassin/Conf/Parser.pm line 547, <GEN2284> line 2.
$set_score = -$set_score if ( $conf->{tflags}->{$k} =~ /\bnice\b/ );
$conf->{tflags}->{$k} is undefined, which shouldn't be possible with the
standard code (tflags are set to '' whenever a test is added).
> >Do you have a firewall blocking those services from working?
>
> No firewall at all, and those processes hit (as in, generate scores) most
> of the time. I'm under the impression if they were going to be killed by
> a firewall config, it would be a very all or nothing type thing.
Sure. If it's intermittant, and not a local system issue, it could just be
timeouts on the Internet connect, at the remote end, etc.
--
Randomly Generated Tagline:
"The quickest way to double your money is to fold it in half and put it
back in your pocket." - Zen Musings