You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Bill Landry <bi...@pointshare.com> on 2004/08/22 22:31:00 UTC
AWL question
Is there a way to generate a report from the AWL database that will show senders and the score applied to their messages? When running SA via amavisd-new, all I see in the headers is that the AWL test was run, but not the actual score that was applies.
Bill
Re: AWL question
Posted by Bill Landry <bi...@pointshare.com>.
----- Original Message -----
From: "Ryan Thompson" <ry...@sasknow.com>
> Bill Landry wrote to Ryan Thompson:
>
> > No documentation found for "check_whitelist".
> >
> > I'm running SA 2.64.
>
> Ahh. That explains it. check_whitelist is quite a bit better in 3.0. :-)
> 2.64's doesn't have perldoc, for one thing. For another thing, I just
> tried running it, and I was able to reproduce your results from a copy
> of a known-good database that shows proper scores with 3.0's
> check_whitelist.
>
> > Anyway, I figured that this was the column
> > discriptions, however, I was confused as to why my first and second
columns
> > showed the same score, instead of an average in the first column. From
my
> > previous post:
> >
> > 35.2 (35.2/4) -- user@MUNGE|ip=216.93
> > 16.9 (16.9/2) -- user@MUNGE|ip=209.206
> > 32.0 (32.0/2) -- user@MUNGE|ip=69.6
> > 30.3 (30.3/9) -- user@MUNGE|ip=64.28
> > 22.3 (22.3/4) -- user@MUNGE|ip=216.21
>
> It's the second column (TOTSCORE) that's out. See this:
>
> $t/$v, (sprintf "(%.1f/%d)",$t/$v,$v),
>
> This is quite an obvious bug in check_whitelist; $t/$v is used for both
> the average score and the total score. The second $t/$v should be just
> $t. That line should read:
>
> $t/$v, (sprintf "(%.1f/%d)",$t,$v),
>
> To verify, I ran 2.64's check_whitelist against 3.00's check_whitelist,
> on the same auto-whitelist file, and:
>
> # diff 2.64.txt 3.00.txt
> #
>
> So, with the fix, above, the results are identical.
>
> That also means you could just grab check_whitelist from 3.0's tools/
> directory. It's better documented, more robust, and it actually works.
>
> Still, I smell a bugzilla report coming on... :-/
Thanks Ryan, I grabbed a copy of check_whitelist from my 3.0.0.rc1 test
server and now have perldoc info and the output from the dump that I had
expected.
Thanks again for you help!
Bill
Re: AWL question
Posted by Ryan Thompson <ry...@sasknow.com>.
Bill Landry wrote to Ryan Thompson:
> No documentation found for "check_whitelist".
>
> I'm running SA 2.64.
Ahh. That explains it. check_whitelist is quite a bit better in 3.0. :-)
2.64's doesn't have perldoc, for one thing. For another thing, I just
tried running it, and I was able to reproduce your results from a copy
of a known-good database that shows proper scores with 3.0's
check_whitelist.
> Anyway, I figured that this was the column
> discriptions, however, I was confused as to why my first and second columns
> showed the same score, instead of an average in the first column. From my
> previous post:
>
> 35.2 (35.2/4) -- user@MUNGE|ip=216.93
> 16.9 (16.9/2) -- user@MUNGE|ip=209.206
> 32.0 (32.0/2) -- user@MUNGE|ip=69.6
> 30.3 (30.3/9) -- user@MUNGE|ip=64.28
> 22.3 (22.3/4) -- user@MUNGE|ip=216.21
It's the second column (TOTSCORE) that's out. See this:
$t/$v, (sprintf "(%.1f/%d)",$t/$v,$v),
This is quite an obvious bug in check_whitelist; $t/$v is used for both
the average score and the total score. The second $t/$v should be just
$t. That line should read:
$t/$v, (sprintf "(%.1f/%d)",$t,$v),
To verify, I ran 2.64's check_whitelist against 3.00's check_whitelist,
on the same auto-whitelist file, and:
# diff 2.64.txt 3.00.txt
#
So, with the fix, above, the results are identical.
That also means you could just grab check_whitelist from 3.0's tools/
directory. It's better documented, more robust, and it actually works.
Still, I smell a bugzilla report coming on... :-/
- Ryan
--
Ryan Thompson <ry...@sasknow.com>
SaskNow Technologies - http://www.sasknow.com
901-1st Avenue North - Saskatoon, SK - S7K 1Y4
Tel: 306-664-3600 Fax: 306-244-7037 Saskatoon
Toll-Free: 877-727-5669 (877-SASKNOW) North America
Re: AWL question
Posted by Bill Landry <bi...@pointshare.com>.
----- Original Message -----
From: Ryan Thompson
> Sure. From the internal (perldoc) documentation:
>
> OUTPUT
> The output looks like this:
>
> AVG (TOTSCORE/COUNT) -- EMAIL|ip=IPBASE
Thanks for the explanation, Ryan. I'm not sure how you came up with the
perldoc output. When I ran:
perldoc check_whitelist
I came up with:
No documentation found for "check_whitelist".
I'm running SA 2.64. Anyway, I figured that this was the column
discriptions, however, I was confused as to why my first and second columns
showed the same score, instead of an average in the first column. From my
previous post:
35.2 (35.2/4) -- user@MUNGE|ip=216.93
16.9 (16.9/2) -- user@MUNGE|ip=209.206
32.0 (32.0/2) -- user@MUNGE|ip=69.6
30.3 (30.3/9) -- user@MUNGE|ip=64.28
22.3 (22.3/4) -- user@MUNGE|ip=216.21
If we take the first entry above, shouldn't the first column (the avg
column) be 35.2 divided by 4 equal 8.8? Why would the average in the first
column show 35.2 instead of 8.8? Could it mean that all four messages from
this user totaled 35.2, so the average would also be 35.2?
I've only been running AWL for a few hours, so it's possible that I do not
have enough entries to show something different in the average column than
what's shown in totscore/count column.
Bill
Re: AWL question
Posted by Loren Wilton <lw...@earthlink.net>.
> > 35.2 (35.2/4) -- user@MUNGE|ip=216.93
>
> Sure. From the internal (perldoc) documentation:
>
> OUTPUT
> The output looks like this:
>
> AVG (TOTSCORE/COUNT) -- EMAIL|ip=IPBASE
>
> For example:
>
> 0.0 (0.0/7) -- dawson@example.com|ip=208.192
> 21.8 (43.7/2) --
mcdaniel_2s2000@example.com|ip=200.106
>
> AVG is the average score; TOTSCORE is the total score of all
mails
And interestingly in the current examples AVG happens to be the same as
TOTSCORE rather than TOTSCORE/COUNT as shown in the docs.
I wonder if this is a bug in the code or the docs?
Loren
Re: AWL question
Posted by Ryan Thompson <ry...@sasknow.com>.
Bill Landry wrote to Ryan Thompson:
> ----- Original Message -----
> From: Ryan Thompson
>
>>> Is there a way to generate a report from the AWL database that will
>>> show senders and the score applied to their messages? When running SA
>>> via amavisd-new, all I see in the headers is that the AWL test was
>>> run, but not the actual score that was applies.
>>
>> I used to use _TESTSSCORES_ in the Status header, but with
>> amavisd-new, I think you're in the same boat as MIMEDefang,
>> in that you have to generate your own headers. If you can
>> generate headers with scores, you'd be on your way.
>>
>> However, there's also a little-known tool available in the tools/
>> directory of the SA distribution called check_whitelist, that will
>> do a dump of your AWL database. That may be all you need.
>
> Thanks Ryan, that works. However, I have one more question about the AWL
> dump output. Can someone explain what the first two columns means in this
> output:
>
> 35.2 (35.2/4) -- user@MUNGE|ip=216.93
Sure. From the internal (perldoc) documentation:
OUTPUT
The output looks like this:
AVG (TOTSCORE/COUNT) -- EMAIL|ip=IPBASE
For example:
0.0 (0.0/7) -- dawson@example.com|ip=208.192
21.8 (43.7/2) -- mcdaniel_2s2000@example.com|ip=200.106
AVG is the average score; TOTSCORE is the total score of all mails
seen so far; COUNT is the number of messages seen from that sender;
EMAIL is the sender's email address, and IPBASE is the AWL base IP
address.
AWL base IP address is a way to identify the sender's IP address they
frequently send from, in an approximate way, but remaining hard for
spammers to spoof. The algorithm is as follows:
- take the last Received header that contains a public IP address -- namely
one which is not in private, unrouted IP space.
- chop off the last two octets, assuming that the user may be in an ISP's
dynamic address pool.
Cheers,
- Ryan
--
Ryan Thompson <ry...@sasknow.com>
SaskNow Technologies - http://www.sasknow.com
901-1st Avenue North - Saskatoon, SK - S7K 1Y4
Tel: 306-664-3600 Fax: 306-244-7037 Saskatoon
Toll-Free: 877-727-5669 (877-SASKNOW) North America
Re: AWL question
Posted by Bill Landry <bi...@pointshare.com>.
----- Original Message -----
From: Ryan Thompson
>> Is there a way to generate a report from the AWL database that will
>> show senders and the score applied to their messages? When running SA
>> via amavisd-new, all I see in the headers is that the AWL test was
>> run, but not the actual score that was applies.
>
> I used to use _TESTSSCORES_ in the Status header, but with
> amavisd-new, I think you're in the same boat as MIMEDefang,
> in that you have to generate your own headers. If you can
> generate headers with scores, you'd be on your way.
>
> However, there's also a little-known tool available in the tools/
> directory of the SA distribution called check_whitelist, that will
> do a dump of your AWL database. That may be all you need.
Thanks Ryan, that works. However, I have one more question about the AWL
dump output. Can someone explain what the first two columns means in this
output:
35.2 (35.2/4) -- user@MUNGE|ip=216.93
16.9 (16.9/2) -- user@MUNGE|ip=209.206
32.0 (32.0/2) -- user@MUNGE|ip=69.6
30.3 (30.3/9) -- user@MUNGE|ip=64.28
22.3 (22.3/4) -- user@MUNGE|ip=216.21
Thanks,
Bill
Re: AWL question
Posted by Ryan Thompson <ry...@sasknow.com>.
Bill Landry wrote to users@spamassassin.apache.org:
> Is there a way to generate a report from the AWL database that will
> show senders and the score applied to their messages? When running SA
> via amavisd-new, all I see in the headers is that the AWL test was
> run, but not the actual score that was applies.
I used to use _TESTSSCORES_ in the Status header, but with amavisd-new,
I think you're in the same boat as MIMEDefang, in that you have to
generate your own headers. If you can generate headers with scores,
you'd be on your way.
However, there's also a little-known tool available in the tools/
directory of the SA distribution called check_whitelist, that will do a
dump of your AWL database. That may be all you need.
- Ryan
--
Ryan Thompson <ry...@sasknow.com>
SaskNow Technologies - http://www.sasknow.com
901-1st Avenue North - Saskatoon, SK - S7K 1Y4
Tel: 306-664-3600 Fax: 306-244-7037 Saskatoon
Toll-Free: 877-727-5669 (877-SASKNOW) North America