You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Bob McClure Jr <bo...@bobcatos.com> on 2007/01/02 22:54:11 UTC
sa-stats.pl blows up on maillog covering turn of year
sa-stats.pl as distributed with SA v3.1.7 blows out a ton of
WARNING: ignoring future date in syslog line: Dec 31 20:26:56 bubba spamd[7149]: prefork: child states: II
and the like, and ends up reporting zeros for results. Another
machine with the same sa-stats.pl (and an earlier version as well)
works just fine. Both machines are running Fedora Core 4 with Perl
v5.8.6, but the one difference I found is in Parse::Syslog. The
machine that works has v1.03. The one that blows up has v1.09.
I can't tell if it's Parse::Syslog that is broken, or sa-stats.pl is
failing to take advantage of a new feature of Parse::Syslog.
On both machines, sa-stats.pl is called from a script in
/etc/cron.daily/sa-stats thusly:
#!/bin/sh
# Set a 24-year period.
start=`date -d yesterday`
# to today
end=`date`
/usr/local/sbin/sa-stats -s "$start" -e "$end"
Has anyone else experienced or fixed this?
Cheers,
--
Bob McClure, Jr. Bobcat Open Systems, Inc.
bob@bobcatos.com http://www.bobcatos.com
Whatever you have learned or received or heard from me, or seen in me
-- put it into practice. And the God of peace will be with you.
Philippians 4:9 (NIV)
Re: sa-stats.pl blows up on maillog covering turn of year
Posted by Ray Anderson <rs...@rb-com.com>.
Bob McClure Jr wrote:
> sa-stats.pl as distributed with SA v3.1.7 blows out a ton of
>
> WARNING: ignoring future date in syslog line: Dec 31 20:26:56 bubba spamd[7149]: prefork: child states: II
>
> and the like, and ends up reporting zeros for results. Another
> machine with the same sa-stats.pl (and an earlier version as well)
> works just fine. Both machines are running Fedora Core 4 with Perl
> v5.8.6, but the one difference I found is in Parse::Syslog. The
> machine that works has v1.03. The one that blows up has v1.09.
>
> I can't tell if it's Parse::Syslog that is broken, or sa-stats.pl is
> failing to take advantage of a new feature of Parse::Syslog.
>
> On both machines, sa-stats.pl is called from a script in
> /etc/cron.daily/sa-stats thusly:
>
> #!/bin/sh
> # Set a 24-year period.
> start=`date -d yesterday`
> # to today
> end=`date`
> /usr/local/sbin/sa-stats -s "$start" -e "$end"
>
> Has anyone else experienced or fixed this?
>
> Cheers,
>
Same problem here.
-=R
Re: sa-stats.pl blows up on maillog covering turn of year
Posted by Chris <cp...@earthlink.net>.
On Thursday 04 January 2007 10:03 pm, Bob McClure Jr wrote:
>
> That's not the one I'm having trouble with. Both of Dallas' scripts
> work fine. It's the sa-stats.pl script that is bundled with SA
> (produces a quite different report) that does unpleasant things when
> used with the current (v1.09) Parse::Syslog module on a maillog that
> crosses the year boundary.
>
> Cheers,
My apologies, I misunderstood which sa-stats you were having problems with.
--
Chris
KeyID 0xE372A7DA98E6705C
http://learn.to/quote
Re: sa-stats.pl blows up on maillog covering turn of year
Posted by Bob McClure Jr <bo...@bobcatos.com>.
On Thu, Jan 04, 2007 at 07:09:54PM -0600, Chris wrote:
> On Wednesday 03 January 2007 10:18 pm, Bob McClure Jr wrote:
> > > I run sa-stats.pl written by Dallas:
> > >
> > > # file: sa-stats.pl
> > > # date: 2005-07-27
> > > # version: 0.9
> > > # author: Dallas Engelken <da...@nmgi.com>
> > > # desc: SA 3.x log parser
> > >
> > > on my 3.1.7 SA version with no problems.
> > > --
> > > Chris
> > > KeyID 0xE372A7DA98E6705C
> > > http://learn.to/quote
> >
> > So do I. In fact I use the new and improved v1.02 for SA v3.1.x. For
> > those just tuning in, it's here:
> >
> > http://www.rulesemporium.com/programs/sa-stats-1.0.txt
> >
> > It and the sa-stats.pl included with SA produce very different
> > reports, and I find both reports useful. Pity they have the same
> > name. I renamed Dallas' script sa-stats-sare.pl to keep them
> > straight. Hmm. I shoulda called it sare-stats.pl.
> >
> You didn't say if you're still having problems or not.
I probably am, but have disabled the scripts until the logs with Dec
2006 data are rotated out.
> I just downloaded,
> configured and ran the new version as root from the cli and had no problems:
>
> [root@cpollock SAStuff]# ./sa-stats-1.0.pl
>
>
> Email: 148 Autolearn: 0 AvgScore: 35.43 AvgScanTime: 8.13 sec
> Spam: 113 Autolearn: 0 AvgScore: 48.18 AvgScanTime: 8.28 sec
> Ham: 35 Autolearn: 0 AvgScore: -5.74 AvgScanTime: 7.66 sec
>
> Time Spent Running SA: 0.33 hours
> Time Spent Processing Spam: 0.26 hours
> Time Spent Processing Ham: 0.07 hours
>
> TOP SPAM RULES FIRED
> ----------------------------------------------------------------------
> RANK RULE NAME COUNT %OFMAIL %OFSPAM %OFHAM
> ----------------------------------------------------------------------
> 1 SAGREY 110 74.32 97.35 0.00
> 2 BOTNET 100 68.92 88.50 5.71
> 3 BAYES_99 96 64.86 84.96 0.00
> 4 RAZOR2_CHECK 80 54.05 70.80 0.00
> 5 RAZOR2_CF_RANGE_51_100 79 53.38 69.91 0.00
> <remainder snipped>
>
> --
> Chris
> KeyID 0xE372A7DA98E6705C
> http://learn.to/quote
That's not the one I'm having trouble with. Both of Dallas' scripts
work fine. It's the sa-stats.pl script that is bundled with SA
(produces a quite different report) that does unpleasant things when
used with the current (v1.09) Parse::Syslog module on a maillog that
crosses the year boundary.
Cheers,
--
Bob McClure, Jr. Bobcat Open Systems, Inc.
bob@bobcatos.com http://www.bobcatos.com
Whatever you have learned or received or heard from me, or seen in me
-- put it into practice. And the God of peace will be with you.
Philippians 4:9 (NIV)
Re: sa-stats.pl blows up on maillog covering turn of year
Posted by Chris <cp...@earthlink.net>.
On Wednesday 03 January 2007 10:18 pm, Bob McClure Jr wrote:
> > I run sa-stats.pl written by Dallas:
> >
> > # file: sa-stats.pl
> > # date: 2005-07-27
> > # version: 0.9
> > # author: Dallas Engelken <da...@nmgi.com>
> > # desc: SA 3.x log parser
> >
> > on my 3.1.7 SA version with no problems.
> > --
> > Chris
> > KeyID 0xE372A7DA98E6705C
> > http://learn.to/quote
>
> So do I. In fact I use the new and improved v1.02 for SA v3.1.x. For
> those just tuning in, it's here:
>
> http://www.rulesemporium.com/programs/sa-stats-1.0.txt
>
> It and the sa-stats.pl included with SA produce very different
> reports, and I find both reports useful. Pity they have the same
> name. I renamed Dallas' script sa-stats-sare.pl to keep them
> straight. Hmm. I shoulda called it sare-stats.pl.
>
You didn't say if you're still having problems or not. I just downloaded,
configured and ran the new version as root from the cli and had no problems:
[root@cpollock SAStuff]# ./sa-stats-1.0.pl
Email: 148 Autolearn: 0 AvgScore: 35.43 AvgScanTime: 8.13 sec
Spam: 113 Autolearn: 0 AvgScore: 48.18 AvgScanTime: 8.28 sec
Ham: 35 Autolearn: 0 AvgScore: -5.74 AvgScanTime: 7.66 sec
Time Spent Running SA: 0.33 hours
Time Spent Processing Spam: 0.26 hours
Time Spent Processing Ham: 0.07 hours
TOP SPAM RULES FIRED
----------------------------------------------------------------------
RANK RULE NAME COUNT %OFMAIL %OFSPAM %OFHAM
----------------------------------------------------------------------
1 SAGREY 110 74.32 97.35 0.00
2 BOTNET 100 68.92 88.50 5.71
3 BAYES_99 96 64.86 84.96 0.00
4 RAZOR2_CHECK 80 54.05 70.80 0.00
5 RAZOR2_CF_RANGE_51_100 79 53.38 69.91 0.00
6 RCVD_IN_XBL 76 51.35 67.26 0.00
7 DIGEST_MULTIPLE 70 47.97 61.95 2.86
8 URIBL_BLACK 70 47.30 61.95 0.00
9 PYZOR_CHECK 68 46.62 60.18 2.86
10 URIBL_WS_SURBL 67 45.27 59.29 0.00
11 RAZOR2_CF_RANGE_E8_51_100 66 44.59 58.41 0.00
12 URIBL_JP_SURBL 64 43.24 56.64 0.00
13 URIBL_OB_SURBL 64 43.24 56.64 0.00
14 UNPARSEABLE_RELAY 64 45.27 56.64 8.57
15 CLAMAV 55 37.16 48.67 0.00
16 URIBL_AB_SURBL 55 37.16 48.67 0.00
17 DCC_CHECK 43 40.54 38.05 48.57
18 SARE_CSNUMTAG 42 28.38 37.17 0.00
19 RCVD_IN_BL_SPAMCOP_NET 31 20.95 27.43 0.00
20 DRUGS_ERECTILE 29 19.59 25.66 0.00
----------------------------------------------------------------------
TOP HAM RULES FIRED
----------------------------------------------------------------------
RANK RULE NAME COUNT %OFMAIL %OFSPAM %OFHAM
----------------------------------------------------------------------
1 BAYES_00 35 23.65 0.00 100.00
2 AWL 30 22.30 2.65 85.71
3 DCC_CHECK 17 40.54 38.05 48.57
4 HTML_MESSAGE 8 23.65 23.89 22.86
5 NO_REAL_NAME 8 6.76 1.77 22.86
6 FORGED_RCVD_HELO 6 14.19 13.27 17.14
7 SPF_PASS 6 4.73 0.88 17.14
8 SARE_HEAD_HDR_XSEQ 4 2.70 0.00 11.43
9 FM_MULTI_ODD2 4 6.76 5.31 11.43
10 DK_SIGNED 3 2.03 0.00 8.57
11 MIME_HTML_ONLY 3 11.49 12.39 8.57
12 DBL_12_LETTER_FLDR 3 3.38 1.77 8.57
13 HTML_90_100 3 2.70 0.88 8.57
14 SARE_UNI 3 2.03 0.00 8.57
15 UNPARSEABLE_RELAY 3 45.27 56.64 8.57
16 BOTNET 2 68.92 88.50 5.71
17 DNS_FROM_RFC_POST 2 7.43 7.96 5.71
18 DNS_FROM_RFC_WHOIS 2 5.41 5.31 5.71
19 MIME_HEADER_CTYPE_ONLY 2 2.70 1.77 5.71
20 DK_POLICY_SIGNSOME 2 6.08 6.19 5.71
----------------------------------------------------------------------
--
Chris
KeyID 0xE372A7DA98E6705C
http://learn.to/quote
Re: sa-stats.pl blows up on maillog covering turn of year
Posted by Bob McClure Jr <bo...@bobcatos.com>.
On Wed, Jan 03, 2007 at 05:44:34PM -0600, Chris wrote:
> On Tuesday 02 January 2007 3:54 pm, Bob McClure Jr wrote:
> > sa-stats.pl as distributed with SA v3.1.7 blows out a ton of
> >
> > WARNING: ignoring future date in syslog line: Dec 31 20:26:56 bubba
> > spamd[7149]: prefork: child states: II
> >
> > and the like, and ends up reporting zeros for results. Another
> > machine with the same sa-stats.pl (and an earlier version as well)
> > works just fine. Both machines are running Fedora Core 4 with Perl
> > v5.8.6, but the one difference I found is in Parse::Syslog. The
> > machine that works has v1.03. The one that blows up has v1.09.
> >
> I run sa-stats.pl written by Dallas:
>
> # file: sa-stats.pl
> # date: 2005-07-27
> # version: 0.9
> # author: Dallas Engelken <da...@nmgi.com>
> # desc: SA 3.x log parser
>
> on my 3.1.7 SA version with no problems.
> --
> Chris
> KeyID 0xE372A7DA98E6705C
> http://learn.to/quote
So do I. In fact I use the new and improved v1.02 for SA v3.1.x. For
those just tuning in, it's here:
http://www.rulesemporium.com/programs/sa-stats-1.0.txt
It and the sa-stats.pl included with SA produce very different
reports, and I find both reports useful. Pity they have the same
name. I renamed Dallas' script sa-stats-sare.pl to keep them
straight. Hmm. I shoulda called it sare-stats.pl.
Cheers,
--
Bob McClure, Jr. Bobcat Open Systems, Inc.
bob@bobcatos.com http://www.bobcatos.com
Whatever you have learned or received or heard from me, or seen in me
-- put it into practice. And the God of peace will be with you.
Philippians 4:9 (NIV)
Re: sa-stats.pl blows up on maillog covering turn of year
Posted by Chris <cp...@earthlink.net>.
On Tuesday 02 January 2007 3:54 pm, Bob McClure Jr wrote:
> sa-stats.pl as distributed with SA v3.1.7 blows out a ton of
>
> WARNING: ignoring future date in syslog line: Dec 31 20:26:56 bubba
> spamd[7149]: prefork: child states: II
>
> and the like, and ends up reporting zeros for results. Another
> machine with the same sa-stats.pl (and an earlier version as well)
> works just fine. Both machines are running Fedora Core 4 with Perl
> v5.8.6, but the one difference I found is in Parse::Syslog. The
> machine that works has v1.03. The one that blows up has v1.09.
>
I run sa-stats.pl written by Dallas:
# file: sa-stats.pl
# date: 2005-07-27
# version: 0.9
# author: Dallas Engelken <da...@nmgi.com>
# desc: SA 3.x log parser
on my 3.1.7 SA version with no problems.
--
Chris
KeyID 0xE372A7DA98E6705C
http://learn.to/quote