You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Matt Doran <ma...@papercut.com> on 2010/01/08 03:14:40 UTC
Warnings in mail.log: "no meta_dependencies defined", "Use of uninitialized
value in split", "Use of uninitialized value in concatenation"
Hi guys,
I was recently doing some reconfigurating/optimization of our
SpamAssassin setup. And I've started seeing the following entries in
the mail.log. They don't appear for every mail processed, but it does
happen multiple times a day.
I'm running Debian stable, SpamAssassin 3.2.5 in daemon mode, and using
sa-update to keep the rules up-to-date. I enable the shortcircuiting
plugin to implement some of these optimizations, and I'm wondering
whether this is what introduced these problems.
There seems to be 2 classes of warning. The first is this:
no meta_dependencies defined for SUBJ_RE_NUM at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 414.
Use of uninitialized value in split at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 417.
These 2 lines are repeated over and over for about 280 different rules.
Then next warning is about an "uninitialized value in concatenation".
The warning can look a bit different but it's always at the same line.
Here's some examples:
Use of uninitialized value in concatenation (.) or string at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028.
last message repeated 1088 times
and
Use of uninitialized value in concatenation (.) or string at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028,
<$tmpfile> line 2290.
Use of uninitialized value in concatenation (.) or string at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028.
Use of uninitialized value in concatenation (.) or string at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028,
<$tmpfile> line 2290.
Use of uninitialized value in concatenation (.) or string at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028.
Use of uninitialized value in concatenation (.) or string at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028,
<$tmpfile> line 2290.
and
Use of uninitialized value in concatenation (.) or string at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028,
<GEN41> line 1294.
last message repeated 682 times
Use of uninitialized value in concatenation (.) or string at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028.
Use of uninitialized value in concatenation (.) or string at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028,
<GEN41> line 1294.
Use of uninitialized value in concatenation (.) or string at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028,
<GEN41> line 1294.
Use of uninitialized value in concatenation (.) or string at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028.
Use of uninitialized value in concatenation (.) or string at
/usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028,
<GEN41> line 1294.
I didn't see these warnings before my changes. Any ideas on the cause
of these and how to resolve them?
Regards,
--
Matt Doran
PaperCut Software International Pty. Ltd.
http://www.papercut.com/
Phone: +61 3 9809 5194
Re: Why no relays?
Posted by Matt Doran <ma...@papercut.com>.
Matt Doran wrote:
> OK, I think I'm onto something here. The warnings would appear for
> the first message processed by that process. I was able to reproduce
> by restarting the daemon and sending in a new message.
>>>
>>> >From my experimenting with the shortcircuit plugin I know that it
>>> has the ability to change the order rules are evaluated (based on
>>> the "priority"). So I suspected that the order that rules are
>>> evaluated introduced this problem. I added a short-circuit rule for
>>> NO_RELAYS, i.e.
>>>
>>> shortcircuit NO_RELAYS ham
>>>
>>>
>>> If I remove this rule, I no longer see the warnings. Hmmmm.
>>>
>>> So now I need a SA/shortcircuit guru to help me understand what I
>>> did wrong. Is there a way for me to short-circuit when there are
>>> no relays without introducing these problems?
>>
>> Well, let's back up a bit.
>>
>> NO_RELAYS is an *error*. This should never happen. If it is
>> happening, you very likely have a problem that needs fixing for SA to
>> work accurately.
>>
>> Usually if this is firing off, you've integrated SA in a way that it
>> never sees Recieved: headers generated by your local MTA, which is
>> bad. (all mail should have at least one Received: header before it
>> gets to SA, even if it's just saying "received from localhost by
>> localhost".) Usually this is from a borked MTA layer integration.
>>
>> So, my personal recomendation would be to stop trying to shortcircuit
>> this rule, and figure out why there are no Received: headers in the
>> message.
>>
>>
> I was just investigating this myself (actually looking at the code
> Received.pm but not getting very far). Maybe you have an idea? I
> found an email with NO_RELAYS, but it had the following Received headers:
>
> Received: from localhost
> ([127.0.0.1] helo=smaug.papercutsoftware.com ident=matt)
> by smaug.papercutsoftware.com with esmtp (Exim 4.69)
> (envelope-from <on...@papercut.com>)
> id 1NT3bj-0006Zd-Se
> for matt@localhost; Fri, 08 Jan 2010 12:25:44 +1100
> Received: from papercut.com [216.92.193.84]
> by smaug.papercutsoftware.com with IMAP (fetchmail-6.3.9-rc2)
> for <ma...@localhost> (single-drop); Fri, 08 Jan 2010 12:25:43 +1100 (EST)
> Received: (qmail 67842 invoked by uid 3020); 8 Jan 2010 01:25:38 -0000
> Received: (qmail 67839 invoked by uid 65534); 8 Jan 2010 01:25:38 -0000
>
> but when I run this through: spamassassin -D I only see the following
> entries about received headers.
>
> [28013] dbg: received-header: parsed as [ ip=127.0.0.1
> rdns=localhost helo=smaug.papercutsoftware.com
> by=smaug.papercutsoftware.com ident=matt
> envfrom=online-orders@papercut.com intl=0 id=1NT3bj-0006Zd-Se
> auth= msa=0 ]
> [28013] dbg: received-header: relay 127.0.0.1 trusted? yes
> internal? yes msa? no
> [28013] dbg: received-header: found fetchmail marker, restarting parse
> [28013] dbg: metadata: X-Spam-Relays-Trusted:
> [28013] dbg: metadata: X-Spam-Relays-Untrusted:
> [28013] dbg: metadata: X-Spam-Relays-Internal:
> [28013] dbg: metadata: X-Spam-Relays-External:
>
>
> Why wouldn't it be parsing/recognizing the 2nd "Received"
> header? Is there anything rule/config that makes SA ignore a
> received header/relay?
>
>
OK, I just noticed the line "received-header: found fetchmail marker,
restarting parse". We are using fetchmail .... so maybe this isn't
working as expected.
This email is one that is generated by out web-server on an ISP. So I
guess there sort of isn't any relays in a way. It seems that
SpamAssassin ignores the fetchmail Received header and also any later
headers (e.g. local delivery), and this leaves only these headers:
Received: (qmail 67842 invoked by uid 3020); 8 Jan 2010 01:25:38 -0000
Received: (qmail 67839 invoked by uid 65534); 8 Jan 2010 01:25:38 -0000
And I guess that SpamAssassin's relay parsing is ignoring these as
"local" (i.e. it's not a "relay"). If that's the case, then it is OK
to have NO_RELAYS, particularly if mail was generated on the same
machine as the mail server (e.g. cron jobs). So isn't this a legitmate
rule to use?
So then I come back to the question, can I use NO_RELAYS in a
short-circuit without causing issues?
Regards,
Matt
Why no relays (was Re: Warnings in mail.log: "no meta_dependencies
defined", "Use of uninitialized value in split", "Use of uninitialized value
in concatenation")
Posted by Matt Doran <ma...@papercut.com>.
OK, I think I'm onto something here. The warnings would appear for the
first message processed by that process. I was able to reproduce by
restarting the daemon and sending in a new message.
>>
>> >From my experimenting with the shortcircuit plugin I know that it
>> has the ability to change the order rules are evaluated (based on the
>> "priority"). So I suspected that the order that rules are
>> evaluated introduced this problem. I added a short-circuit rule for
>> NO_RELAYS, i.e.
>>
>> shortcircuit NO_RELAYS ham
>>
>>
>> If I remove this rule, I no longer see the warnings. Hmmmm.
>>
>> So now I need a SA/shortcircuit guru to help me understand what I did
>> wrong. Is there a way for me to short-circuit when there are no
>> relays without introducing these problems?
>
> Well, let's back up a bit.
>
> NO_RELAYS is an *error*. This should never happen. If it is happening,
> you very likely have a problem that needs fixing for SA to work
> accurately.
>
> Usually if this is firing off, you've integrated SA in a way that it
> never sees Recieved: headers generated by your local MTA, which is
> bad. (all mail should have at least one Received: header before it
> gets to SA, even if it's just saying "received from localhost by
> localhost".) Usually this is from a borked MTA layer integration.
>
> So, my personal recomendation would be to stop trying to shortcircuit
> this rule, and figure out why there are no Received: headers in the
> message.
>
>
I was just investigating this myself (actually looking at the code
Received.pm but not getting very far). Maybe you have an idea? I
found an email with NO_RELAYS, but it had the following Received headers:
Received: from localhost
([127.0.0.1] helo=smaug.papercutsoftware.com ident=matt)
by smaug.papercutsoftware.com with esmtp (Exim 4.69)
(envelope-from <on...@papercut.com>)
id 1NT3bj-0006Zd-Se
for matt@localhost; Fri, 08 Jan 2010 12:25:44 +1100
Received: from papercut.com [216.92.193.84]
by smaug.papercutsoftware.com with IMAP (fetchmail-6.3.9-rc2)
for <ma...@localhost> (single-drop); Fri, 08 Jan 2010 12:25:43 +1100 (EST)
Received: (qmail 67842 invoked by uid 3020); 8 Jan 2010 01:25:38 -0000
Received: (qmail 67839 invoked by uid 65534); 8 Jan 2010 01:25:38 -0000
but when I run this through: spamassassin -D I only see the following
entries about received headers.
[28013] dbg: received-header: parsed as [ ip=127.0.0.1
rdns=localhost helo=smaug.papercutsoftware.com
by=smaug.papercutsoftware.com ident=matt
envfrom=online-orders@papercut.com intl=0 id=1NT3bj-0006Zd-Se auth=
msa=0 ]
[28013] dbg: received-header: relay 127.0.0.1 trusted? yes internal?
yes msa? no
[28013] dbg: received-header: found fetchmail marker, restarting parse
[28013] dbg: metadata: X-Spam-Relays-Trusted:
[28013] dbg: metadata: X-Spam-Relays-Untrusted:
[28013] dbg: metadata: X-Spam-Relays-Internal:
[28013] dbg: metadata: X-Spam-Relays-External:
Why wouldn't it be parsing/recognizing the 2nd "Received" header?
Is there anything rule/config that makes SA ignore a received header/relay?
Thanks,
Matt
Re: Warnings in mail.log: "no meta_dependencies defined",
"Use of uninitialized value in split",
"Use of uninitialized value in concatenation"
Posted by Matt Kettler <mk...@verizon.net>.
>
> Matt Doran wrote:
>> Matt Kettler wrote:
>>> On 1/7/2010 9:14 PM, Matt Doran wrote:
>>>> Hi guys,
>>>>
>>>> I was recently doing some reconfigurating/optimization of our
>>>> SpamAssassin setup. And I've started seeing the following entries
>>>> in the mail.log. They don't appear for every mail processed, but
>>>> it does happen multiple times a day.
>>>>
>>>> I'm running Debian stable, SpamAssassin 3.2.5 in daemon mode, and
>>>> using sa-update to keep the rules up-to-date. I enable the
>>>> shortcircuiting plugin to implement some of these optimizations,
>>>> and I'm wondering whether this is what introduced these problems.
>>>>
>>>>
>>>> There seems to be 2 classes of warning. The first is this:
>>>>
>>>> no meta_dependencies defined for SUBJ_RE_NUM at
>>>> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 414.
>>>> Use of uninitialized value in split at
>>>> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 417.
>>>>
>>>> These 2 lines are repeated over and over for about 280 different rules.
>>>>
>>>>
>>>> Then next warning is about an "uninitialized value in
>>>> concatenation". The warning can look a bit different but it's
>>>> always at the same line. Here's some examples:
>>>>
>>>> Use of uninitialized value in concatenation (.) or string at
>>>> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028.
>>>> last message repeated 1088 times
>>>>
>>>> and
>>>>
>>>> Use of uninitialized value in concatenation (.) or string at
>>>> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028,
>>>> <$tmpfile> line 2290.
>>>>
>>> <snip>
>>>>
>>>> I didn't see these warnings before my changes. Any ideas on the
>>>> cause of these and how to resolve them?
>>>
> OK, I think I'm onto something here. The warnings would appear for
> the first message processed by that process. I was able to reproduce
> by restarting the daemon and sending in a new message.
>
> >From my experimenting with the shortcircuit plugin I know that it has
> the ability to change the order rules are evaluated (based on the
> "priority"). So I suspected that the order that rules are evaluated
> introduced this problem. I added a short-circuit rule for NO_RELAYS, i.e.
>
> shortcircuit NO_RELAYS ham
>
>
> If I remove this rule, I no longer see the warnings. Hmmmm.
>
> So now I need a SA/shortcircuit guru to help me understand what I did
> wrong. Is there a way for me to short-circuit when there are no
> relays without introducing these problems?
Well, let's back up a bit.
NO_RELAYS is an *error*. This should never happen. If it is happening,
you very likely have a problem that needs fixing for SA to work accurately.
Usually if this is firing off, you've integrated SA in a way that it
never sees Recieved: headers generated by your local MTA, which is bad.
(all mail should have at least one Received: header before it gets to
SA, even if it's just saying "received from localhost by localhost".)
Usually this is from a borked MTA layer integration.
So, my personal recomendation would be to stop trying to shortcircuit
this rule, and figure out why there are no Received: headers in the message.
Re: Warnings in mail.log: "no meta_dependencies defined", "Use of
uninitialized value in split", "Use of uninitialized value in concatenation"
Posted by Matt Doran <ma...@papercut.com>.
Matt Doran wrote:
> Matt Kettler wrote:
>> On 1/7/2010 9:14 PM, Matt Doran wrote:
>>> Hi guys,
>>>
>>> I was recently doing some reconfigurating/optimization of our
>>> SpamAssassin setup. And I've started seeing the following entries
>>> in the mail.log. They don't appear for every mail processed, but
>>> it does happen multiple times a day.
>>>
>>> I'm running Debian stable, SpamAssassin 3.2.5 in daemon mode, and
>>> using sa-update to keep the rules up-to-date. I enable the
>>> shortcircuiting plugin to implement some of these optimizations, and
>>> I'm wondering whether this is what introduced these problems.
>>>
>>>
>>> There seems to be 2 classes of warning. The first is this:
>>>
>>> no meta_dependencies defined for SUBJ_RE_NUM at
>>> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 414.
>>> Use of uninitialized value in split at
>>> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 417.
>>>
>>> These 2 lines are repeated over and over for about 280 different rules.
>>>
>>>
>>> Then next warning is about an "uninitialized value in
>>> concatenation". The warning can look a bit different but it's
>>> always at the same line. Here's some examples:
>>>
>>> Use of uninitialized value in concatenation (.) or string at
>>> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028.
>>> last message repeated 1088 times
>>>
>>> and
>>>
>>> Use of uninitialized value in concatenation (.) or string at
>>> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028,
>>> <$tmpfile> line 2290.
>>>
>> <snip>
>>>
>>> I didn't see these warnings before my changes. Any ideas on the
>>> cause of these and how to resolve them?
>>
OK, I think I'm onto something here. The warnings would appear for the
first message processed by that process. I was able to reproduce by
restarting the daemon and sending in a new message.
From my experimenting with the shortcircuit plugin I know that it has
the ability to change the order rules are evaluated (based on the
"priority"). So I suspected that the order that rules are evaluated
introduced this problem. I added a short-circuit rule for NO_RELAYS, i.e.
shortcircuit NO_RELAYS ham
If I remove this rule, I no longer see the warnings. Hmmmm.
So now I need a SA/shortcircuit guru to help me understand what I did
wrong. Is there a way for me to short-circuit when there are no relays
without introducing these problems?
Thanks in advance,
Matt
Re: Warnings in mail.log: "no meta_dependencies defined", "Use of
uninitialized value in split", "Use of uninitialized value in concatenation"
Posted by Matt Doran <ma...@papercut.com>.
Matt Kettler wrote:
> On 1/7/2010 9:14 PM, Matt Doran wrote:
>> Hi guys,
>>
>> I was recently doing some reconfigurating/optimization of our
>> SpamAssassin setup. And I've started seeing the following entries
>> in the mail.log. They don't appear for every mail processed, but it
>> does happen multiple times a day.
>>
>> I'm running Debian stable, SpamAssassin 3.2.5 in daemon mode, and
>> using sa-update to keep the rules up-to-date. I enable the
>> shortcircuiting plugin to implement some of these optimizations, and
>> I'm wondering whether this is what introduced these problems.
>>
>>
>> There seems to be 2 classes of warning. The first is this:
>>
>> no meta_dependencies defined for SUBJ_RE_NUM at
>> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 414.
>> Use of uninitialized value in split at
>> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 417.
>>
>> These 2 lines are repeated over and over for about 280 different rules.
>>
>>
>> Then next warning is about an "uninitialized value in
>> concatenation". The warning can look a bit different but it's
>> always at the same line. Here's some examples:
>>
>> Use of uninitialized value in concatenation (.) or string at
>> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028.
>> last message repeated 1088 times
>>
>> and
>>
>> Use of uninitialized value in concatenation (.) or string at
>> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028,
>> <$tmpfile> line 2290.
>>
> <snip>
>>
>> I didn't see these warnings before my changes. Any ideas on the
>> cause of these and how to resolve them?
>
> Sounds like you've got major garbage in your config files, and it's
> causing spamd to freak out every time it reloads. (it looks like it is
> badly failing to parse your rules).
>
> What happens if you run this command:
>
> spamassassin --lint
>
Nothing :)
smaug:~# spamassassin --lint
smaug:~#
Re: Warnings in mail.log: "no meta_dependencies defined",
"Use of uninitialized value in split",
"Use of uninitialized value in concatenation"
Posted by Matt Kettler <mk...@verizon.net>.
On 1/7/2010 9:14 PM, Matt Doran wrote:
> Hi guys,
>
> I was recently doing some reconfigurating/optimization of our
> SpamAssassin setup. And I've started seeing the following entries in
> the mail.log. They don't appear for every mail processed, but it
> does happen multiple times a day.
>
> I'm running Debian stable, SpamAssassin 3.2.5 in daemon mode, and
> using sa-update to keep the rules up-to-date. I enable the
> shortcircuiting plugin to implement some of these optimizations, and
> I'm wondering whether this is what introduced these problems.
>
>
> There seems to be 2 classes of warning. The first is this:
>
> no meta_dependencies defined for SUBJ_RE_NUM at
> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 414.
> Use of uninitialized value in split at
> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 417.
>
> These 2 lines are repeated over and over for about 280 different rules.
>
>
> Then next warning is about an "uninitialized value in
> concatenation". The warning can look a bit different but it's always
> at the same line. Here's some examples:
>
> Use of uninitialized value in concatenation (.) or string at
> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028.
> last message repeated 1088 times
>
> and
>
> Use of uninitialized value in concatenation (.) or string at
> /usr/share/perl5/Mail/SpamAssassin/Plugin/Check.pm line 1028,
> <$tmpfile> line 2290.
>
<snip>
>
> I didn't see these warnings before my changes. Any ideas on the
> cause of these and how to resolve them?
Sounds like you've got major garbage in your config files, and it's
causing spamd to freak out every time it reloads. (it looks like it is
badly failing to parse your rules).
What happens if you run this command:
spamassassin --lint