You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Arvinn Løkkebakken <ar...@whitebird.no> on 2005/05/11 11:22:35 UTC
my internal server is making records in the AWL
I was looking around in my awl table (MySQL) and was surprised to find
that the /16 network that my trusted and internal servers is inside made
it to the second place when doing the following query:
SELECT ip, count(ip) AS rows FROM awl GROUP BY ip ORDER BY rows DESC
LIMIT 10
After doing a little more research I found that the scores
(totscore/count) of those rows were often very high. A lot in the range
10-30 points. The number of rows containg my network was about 4% of the
total number of rows in the table
This really did worry so I decided to put on some debugging.
This is what I found..
May 8 00:26:44 mailscan3 spamd[3013]: debug: received-header: relay
212.71.66.104 trusted? yes internal? yes
[... and then a little later ...]
May 8 00:26:46 mailscan3 spamd[3013]: debug: SQL Based AWL: Connected
to DBI:mysql:spamassassin:localhost
May 8 00:26:46 mailscan3 spamd[3013]: debug: auto-whitelist (sql-based)
get_addr_entry: No entry found for lavernel@llandudno.com|ip=212.71
May 8 00:26:46 mailscan3 spamd[3013]: debug: auto-whitelist
(sql-based): lavernel@llandudno.com|ip=212.71 scores 0/0
May 8 00:26:46 mailscan3 spamd[3013]: debug: auto-whitelist (sql-based)
get_addr_entry: No entry found for lavernel@llandudno.com|ip=none
May 8 00:26:46 mailscan3 spamd[3013]: debug: auto-whitelist
(sql-based): lavernel@llandudno.com|ip=none scores 0/0
May 8 00:26:46 mailscan3 spamd[3013]: debug: AWL active, pre-score:
23.178, autolearn score: 23.178, mean: undef, IP: 212.71.66.104
May 8 00:26:46 mailscan3 spamd[3013]: debug: auto-whitelist (sql-based)
add_score: Created new entry for lavernel@llandudno.com|ip=212.71 with
totscore: 23.178
I found several similar cases in my debug-logs:
# zcat mail-20050509.gz | grep -c 'AWL active.*undef, IP: 212.71.66.104'
255
# zcat mail-20050508.gz | grep -c 'AWL active.*undef, IP: 212.71'
203
# zcat mail-20050509.gz | grep -c 'AWL active.*undef, IP:'
13487
How can that happen? Anybody else here with the same experience?
Arvinn
Re: my internal server is making records in the AWL
Posted by Arvinn Løkkebakken <ar...@whitebird.no>.
James R wrote:
> Arvinn Løkkebakken wrote:
>
>>
>>
>> James R wrote:
>>
>>> Arvinn Løkkebakken wrote:
>>>
>>>>
>>>>
>>>> Arvinn Løkkebakken wrote:
>>>>
>>>>> How can that happen? Anybody else here with the same experience?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Are we talking about a bug here? I would really like to know if
>>>> this is a problem in my setup or if others are experiencing the same..
>>>>
>>>> Arvinn
>>>>
>>>>
>>> What's the problem? Looks like, in your example, the user wasn't
>>> found in the AWL table, and was added. The mail scored some 23 pts,
>>> and was added to the awl table with that score. AWL isn't a
>>> whitelist nor a black list.
>>> http://wiki.apache.org/spamassassin/AwlWrongWay
>>> http://wiki.apache.org/spamassassin/AutoWhitelist
>>>
>> I know perfectly well what AWL is. My question doesn't have anything
>> to do with the score.
>> It's not right behaviour. Read subject and logs again.
>>
>> The mail was relayed to my scanner through my relay wich is internal.
>> The log says so too. It's NOT right behaviour to then make a record
>> in AWL with the /16 network that my internal server belongs to,
>> instead of the /16 network, which of the ip that sent the mail to my
>> relay, belongs to.
>>
>> If this was right behaviour, all records in AWL would have been from
>> the same network. Get it?
>>
>> Arvinn
>>
>>
> Sorry, with out all of the information you'll find it hard for anyone
> to help you. What version of SA are you using? What is calling spamd?
> What mail software?
>
This information did not really felt necessary as long as the debug told
what was happning. I'm running Qmail-scanner 1.25st which calls spamc.
SpamAssassin 3.0.3. All the internal servers are Qmail. As long as SA
gets the mail with the Received headers this would be a non-issue afaik.
> I've looked at 3 other systems, and none have the internal private ip
> address in the AWL. I'm using the 192.168 range of IPS locally, and on
> the other systems. Your subject was also vague, and a bunch of logs
> with out all of the info is also very vague. I'm running 3.0.3 btw,
> MySQL, AWL, Bayes, user_prefs.
>
Ok. Then I will trim it down to the most important two lines:
May 8 00:26:44 mailscan3 spamd[3013]: debug: received-header: relay
212.71.66.104 trusted? yes internal? yes
May 8 00:26:46 mailscan3 spamd[3013]: debug: auto-whitelist (sql-based)
add_score: Created new entry for lavernel@llandudno.com|ip=212.71 with
totscore: 23.178
The case is that this mail was relayed through on of my front-end MX
just like all the other ten thousands of messages per day. Somehow
though, this time (and a couple of hundred time more per per-day) SA
decides to use the ip of my front-end MX as the "from ip".
> However, I do see my *public* ip address in the AWL, your ip address
> in the logs you gave, if i'm not mistaken, is a public ip address.
> Even with my trusted networks set, i still see those trusted server's
> ip addresses end up in the AWL, which to me, isn't a bug.
>
> tho, I could be completely wrong.
I'm not sure what you mean by your public ip-address. Is this the
ip-adresse of your front-end MX MTA? Then it should be in your
internal_networks as well.
My front-end MX is included in both my trusted_networks and
internal_networks.
Consider this setup:
(Some SMTP client on the Internet), possibly a relay, shouldn't matter IMO.
---smtp-->
(front-end MX)
---smtp-->
(MTA calling SA)
--smtp-->
(final destination.)
In more the 95% of the times in my case spams and hams are checked
against AWL with the ip of the "(Some SMTP client on the Internet)". If
the combination from-adressess and IP does not already exists, SA makes
the new a record. I believe you would agree with me that this is the
expected behaviour?
It simply wouldn't make sense if the ip of the "(front-end MX)" was used
for this IMHO. The statistical value of such data would be much less
meaningfull.
Therefore I need to investigate why my implementation doesn't always
work in the expected (at least my expected) fashion.
Maybe it's expected behaviour as long "(Some SMTP client on the
Internet)" is the originator of the mail (represents the last Received
header) instead of a relay? If so I would disagree with this policy and
disable it if possible.
Arviinn
Re: my internal server is making records in the AWL
Posted by James R <ja...@trusswood.dyndns.org>.
Arvinn Løkkebakken wrote:
>
>
> James R wrote:
>
>> Arvinn Løkkebakken wrote:
>>
>>>
>>>
>>> Arvinn Løkkebakken wrote:
>>>
>>>> How can that happen? Anybody else here with the same experience?
>>>
>>>
>>>
>>>
>>>
>>> Are we talking about a bug here? I would really like to know if this
>>> is a problem in my setup or if others are experiencing the same..
>>>
>>> Arvinn
>>>
>>>
>> What's the problem? Looks like, in your example, the user wasn't found
>> in the AWL table, and was added. The mail scored some 23 pts, and was
>> added to the awl table with that score. AWL isn't a whitelist nor a
>> black list.
>> http://wiki.apache.org/spamassassin/AwlWrongWay
>> http://wiki.apache.org/spamassassin/AutoWhitelist
>>
> I know perfectly well what AWL is. My question doesn't have anything to
> do with the score.
> It's not right behaviour. Read subject and logs again.
>
> The mail was relayed to my scanner through my relay wich is internal.
> The log says so too. It's NOT right behaviour to then make a record in
> AWL with the /16 network that my internal server belongs to, instead of
> the /16 network, which of the ip that sent the mail to my relay, belongs
> to.
>
> If this was right behaviour, all records in AWL would have been from the
> same network. Get it?
>
> Arvinn
>
>
Sorry, with out all of the information you'll find it hard for anyone to
help you. What version of SA are you using? What is calling spamd? What
mail software?
I've looked at 3 other systems, and none have the internal private ip
address in the AWL. I'm using the 192.168 range of IPS locally, and on
the other systems. Your subject was also vague, and a bunch of logs with
out all of the info is also very vague. I'm running 3.0.3 btw, MySQL,
AWL, Bayes, user_prefs.
However, I do see my *public* ip address in the AWL, your ip address in
the logs you gave, if i'm not mistaken, is a public ip address. Even
with my trusted networks set, i still see those trusted server's ip
addresses end up in the AWL, which to me, isn't a bug.
tho, I could be completely wrong.
--
Thanks,
James
Re: my internal server is making records in the AWL
Posted by Arvinn Løkkebakken <ar...@whitebird.no>.
James R wrote:
> Arvinn Løkkebakken wrote:
>
>>
>>
>> Arvinn Løkkebakken wrote:
>>
>>> How can that happen? Anybody else here with the same experience?
>>
>>
>>
>>
>> Are we talking about a bug here? I would really like to know if this
>> is a problem in my setup or if others are experiencing the same..
>>
>> Arvinn
>>
>>
> What's the problem? Looks like, in your example, the user wasn't found
> in the AWL table, and was added. The mail scored some 23 pts, and was
> added to the awl table with that score. AWL isn't a whitelist nor a
> black list.
> http://wiki.apache.org/spamassassin/AwlWrongWay
> http://wiki.apache.org/spamassassin/AutoWhitelist
>
I know perfectly well what AWL is. My question doesn't have anything to
do with the score.
It's not right behaviour. Read subject and logs again.
The mail was relayed to my scanner through my relay wich is internal.
The log says so too. It's NOT right behaviour to then make a record in
AWL with the /16 network that my internal server belongs to, instead of
the /16 network, which of the ip that sent the mail to my relay, belongs to.
If this was right behaviour, all records in AWL would have been from the
same network. Get it?
Arvinn
Re: my internal server is making records in the AWL
Posted by James R <ja...@trusswood.dyndns.org>.
Arvinn Løkkebakken wrote:
>
>
> Arvinn Løkkebakken wrote:
>
>> How can that happen? Anybody else here with the same experience?
>
>
>
> Are we talking about a bug here? I would really like to know if this is
> a problem in my setup or if others are experiencing the same..
>
> Arvinn
>
>
What's the problem? Looks like, in your example, the user wasn't found
in the AWL table, and was added. The mail scored some 23 pts, and was
added to the awl table with that score. AWL isn't a whitelist nor a
black list.
http://wiki.apache.org/spamassassin/AwlWrongWay
http://wiki.apache.org/spamassassin/AutoWhitelist
--
Thanks,
James Rallo
Trusswood Inc.
James@Trusswood.DynDns.org
www.Trusswood.DynDns.org
Tele: (321) 383-0366
Fax: (321) 383-0362
Re: my internal server is making records in the AWL
Posted by Arvinn Løkkebakken <ar...@whitebird.no>.
Arvinn Løkkebakken wrote:
> How can that happen? Anybody else here with the same experience?
Are we talking about a bug here? I would really like to know if this is
a problem in my setup or if others are experiencing the same..
Arvinn