You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Jo Rhett <jr...@netconsonance.com> on 2008/05/03 08:42:37 UTC
Re: can we make AWL ignore mail from self to self?
On Apr 29, 2008, at 7:40 PM, Matt Kettler wrote:
>> I'm not repeating for the 5th time that there are no trusted
>> mailservers. Only this host.
> That's a contradiction, because "this host" is a mailserver.
> Clearly you have a trusted mailserver.
> However, in the interest of moving the discussion forward, you have
> exactly one trusted mailserver, your MX, which is perfectly valid.
Yes. I'm sorry but this is obvious. I don't know how to pick the
words exactly as you want them, but most people understood what I
meant 5 or 6 replies ago ;-)
> The question lies in why does the AWL seem to be confusing forged
> email with your own email. That's generally quite critically
> dependent on the trust path.
No, that's not the question at all. (more below)
> Have you tried running one of the forged messages, and an actual
> legitimate message through SA manually with the -D flag to see what
> the trusted and untrusted hosts are, as SA sees it?
Yes. Many times. That's not the point of this thread.
The point of this thread is the obvious ease of forging e-mail from
recipient to (same) recipient. It's one situation where the AWL
wouldn't work very well. It would be fairly easy to forge, and
worthwhile enough for botnets to just do this (which they are, in
force, for the last month)
I personally see no value in applying AWL to messages from self to
self. I may be wrong, and I'm open to arguements against this, but I
am suggesting that the AWL module should skip over self->self
messages. It seems too easy to forge, and no gain in doing so.
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: can we make AWL ignore mail from self to self?
Posted by Jo Rhett <jr...@netconsonance.com>.
On May 22, 2008, at 1:23 PM, Dave Funk wrote:
>> Lots of users of this host have Windows PCs, and running SA on all
>> outbound mail has both alerted them quickly to the problem and
>> avoided nailing other people with spam and/or virus runs.
>
> Genuine curiosity Jo, have you seen instances of viruses/trojans
> sending
> -authenticated- mail? Have they learned how to read users'
> passwords, etc?
>
> We require our PC users to authenticate when sending and I had
> assumed that would stop viruses/trojans. Am I being naive?
Yes, you are. Most of the viri use the existing Outlook
configuration, which includes the user's saved SMTP AUTH passwords.
Like I said, SA has saved our butt each time it happened. I wouldn't
say that without it having happened multiple times...
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: can we make AWL ignore mail from self to self?
Posted by SM <sm...@resistor.net>.
At 13:23 22-05-2008, Dave Funk wrote:
>We require our PC users to authenticate when sending and I had
>assumed that would stop viruses/trojans. Am I being naive?
No. But it's only one extra step for malware to capture SMTP
authentication information.
Regards,
-sm
Re: can we make AWL ignore mail from self to self?
Posted by Dave Funk <db...@engineering.uiowa.edu>.
On Thu, 22 May 2008, Jo Rhett wrote:
>> Then I guess you use authenticated SMTP for that.
>> The easiest way to handle this probably is to simply avoid calling SA for
>> authenticated mail.
>
> That's a hack with consequences. Like "just disable the firewall". Uh, no
> ;-)
>
> Lots of users of this host have Windows PCs, and running SA on all outbound
> mail has both alerted them quickly to the problem and avoided nailing other
> people with spam and/or virus runs.
Genuine curiosity Jo, have you seen instances of viruses/trojans sending
-authenticated- mail? Have they learned how to read users' passwords, etc?
We require our PC users to authenticate when sending and I had assumed
that would stop viruses/trojans. Am I being naive?
--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{
Re: can we make AWL ignore mail from self to self?
Posted by Jo Rhett <jr...@netconsonance.com>.
>> You've presented good logic for acceping mail from self to self.
>> But you haven't explained by using the AWL for mail from self to
>> self is better than not having it.
On Jun 2, 2008, at 4:02 AM, Jonas Eckerman wrote:
> Because it can help discriminate between spam and ham addressed from
> self to self. Heres an example:
>
> StupidWebService send self->self addressed ham from relay 1.2.3.4
>
> EvilSpammer send self->self addressed spam from relay 5.6.7.8 (wich,
> unfortunately, belongs to a big ISP so the relay doesn'ät get
> blocked).
>
> One day StupidWebService send a ham that triggered a bunch of
> positive hits (including BAYES_99). Since mail from self@1.2 has a
> negative score in the AWL, the mail gets though all right.
>
> One day EvilSpammer manages to send a mail that doesnät hit any
> positive rules, but does hit BAYES_00. Since self@5.6 has a high
> positive score in the AWL, the mail still gets flagged as spam.
>
> If the AWL ignore mail from self->self, the two mails in the above
> example would have been misclassified.
Indeed. I submit you are right.
FYI: I still haven't had another misclassification since the first, so
I'm beginning to think that this was a lark.
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: can we make AWL ignore mail from self to self?
Posted by Jonas Eckerman <jo...@frukt.org>.
Jo Rhett wrote:
>> And considering that SpamAssassin doesn't (in many configurations)
>> even know what recipient address a message has, it might actually be
>> easier than having the AWL ignore mail from self->self.
> It has to, for the AWL to work.
No, it hasn't. The AWL only uses the *senders* address and the IP
address of the client. It doesn't use the recipients address.
The AWL helps discriminate between senders. Not sender->recipient
pairs.
>> As long as the MSA adds authentication info in it's received header,
>> this could be fetched from "X-Spam-Relays-Trusted" pseudo header. The
>> changes to do this would not be more difficult or invlolved than the
>> changes necessary to exempt self->self mail from the AWL AFAICS.
> Easy or not, I don't see the value just yet.
Including the authentication state in the AWL key would
1: Fix the problem you reported (unless I misunderstood you)
2: Fit with the current function of the AWL (discriminating
between senders with no regard for recipient addresses).
> The AWL wouldn't work if it didn't know the recipient. Since this is
> something it stores in the AWL database we know that the recipient
> information is there.
That's strange, considering that the AWL does work now, and it
doesn't know the recipient.
Also, the AWL doesn't store the recipient address in the database.
If you use SQL base AWL, Mail::SpamAssassin::SQLBasedAddrList
will store a username in the database, but neither
Mail::SpamAssassin::Plugin::AWL nor
Mail::SpamAssassin::AutoWhitelist knows anything about that AFAICS.
Also, the username in the database might or might not be the
recipients address or username. This depends entirely on how the
system is setup. Here it is either "mdf" or "spamd", and never
the recipients address or local username (the local users aren't
on the same machine as SA, so it knows nothing about them).
> You've presented good logic for acceping mail from self to self. But
> you haven't explained by using the AWL for mail from self to self is
> better than not having it.
Because it can help discriminate between spam and ham addressed
from self to self. Heres an example:
StupidWebService send self->self addressed ham from relay 1.2.3.4
EvilSpammer send self->self addressed spam from relay 5.6.7.8
(wich, unfortunately, belongs to a big ISP so the relay doesn'ät
get blocked).
One day StupidWebService send a ham that triggered a bunch of
positive hits (including BAYES_99). Since mail from self@1.2 has
a negative score in the AWL, the mail gets though all right.
One day EvilSpammer manages to send a mail that doesnät hit any
positive rules, but does hit BAYES_00. Since self@5.6 has a high
positive score in the AWL, the mail still gets flagged as spam.
If the AWL ignore mail from self->self, the two mails in the
above example would have been misclassified.
Regards
/Jonas
--
Jonas Eckerman, FSDB & Fruktträdet
http://whatever.frukt.org/
http://www.fsdb.org/
http://www.frukt.org/
Re: can we make AWL ignore mail from self to self?
Posted by Jo Rhett <jr...@netconsonance.com>.
On May 29, 2008, at 4:18 AM, Jonas Eckerman wrote:
> Please do remember that I am in no way trying to stop or hinder you
> in implementing your fix. The fact that I have other suggestions
> does not mean that I'm opposing you.
Of course. This is normal discussion.
>> A lot of work to hack around a simple problem. The AWL works just
>> fine for mail from "my users" to other "my users". In fact, it
>> works exceedingly well for that. What value is there in separating
>> them?
>
> It would create a difference (a regards the AWL) between self->self
> addressed mail sent from authenticated/local users ans similar mail
> from other systems.
I understand the concept, I don't see the value.
> And considering that SpamAssassin doesn't (in many configurations)
> even know what recipient address a message has, it might actually be
> easier than having the AWL ignore mail from self->self.
It has to, for the AWL to work.
> As long as the MSA adds authentication info in it's received header,
> this could be fetched from "X-Spam-Relays-Trusted" pseudo header.
> The changes to do this would not be more difficult or invlolved than
> the changes necessary to exempt self->self mail from the AWL AFAICS.
Easy or not, I don't see the value just yet.
> Also, while the adressee of a mail is often available with
> PerMsgStatus all_to_addrs, this function is not very reliable. It
> actually extracts a whole bunch of addresses that might be the
> recipient from the mail header. There is no guarantee that any of
> the returned addresses really are the recipient of the mail.
>
> So, to implement exemption of self->self-mail you first have to
> implement a way for SpamAssassin to know what the recipient address
> is in order to know if a mail is self->self-addressed.
The AWL wouldn't work if it didn't know the recipient. Since this is
something it stores in the AWL database we know that the recipient
information is there.
> I want the AWL to apply to mail that is addressed from self->self.
>
> Since the AWL also takes the IP address into account and since all
> mail from authenticated/local users here comes from 127.0.0.1 to the
> software calling SpamAssassin, I do not have your problem here and
> would not benefit from your fix.
>
> While most mail addressed self->self that comes from external
> systems is spam, every now and then ham addressed from self->self do
> come in from idiotic systems and sometimes from users who for some
> reason is not using our servers when sending mail.
>
> The AWL as it is now does distinguish between "good" and "bad" mail
> that are or pretends to be from our users, and I see no reason to
> remove possible benefits of that distinction for mail that happens
> to be addressed to the same user as it's addressed from.
You've presented good logic for acceping mail from self to self. But
you haven't explained by using the AWL for mail from self to self is
better than not having it.
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: can we make AWL ignore mail from self to self?
Posted by Jonas Eckerman <jo...@frukt.org>.
Please do remember that I am in no way trying to stop or hinder
you in implementing your fix. The fact that I have other
suggestions does not mean that I'm opposing you.
Jo Rhett wrote:
> I don't trust "my users" in this context.
Nothing I said implied or required trust in your users.
> A lot of work to hack around a simple problem. The AWL works just fine
> for mail from "my users" to other "my users". In fact, it works
> exceedingly well for that. What value is there in separating them?
It would create a difference (a regards the AWL) between
self->self addressed mail sent from authenticated/local users ans
similar mail from other systems.
And considering that SpamAssassin doesn't (in many
configurations) even know what recipient address a message has,
it might actually be easier than having the AWL ignore mail from
self->self.
It also might (depedning on configuration) not require any
changes at all to SpamAssassin.
> What alternatives? So far I've only heard (a) disable the AWL (b) don't
> use AWL it sucks and (c) hack the system to use different AWLs. None of
> which really make any logical sense to solve the problem.
I also mentioned the having the AWL include the authentication
state in AWL data key.
As long as the MSA adds authentication info in it's received
header, this could be fetched from "X-Spam-Relays-Trusted" pseudo
header. The changes to do this would not be more difficult or
invlolved than the changes necessary to exempt self->self mail
from the AWL AFAICS.
Also, while the adressee of a mail is often available with
PerMsgStatus all_to_addrs, this function is not very reliable. It
actually extracts a whole bunch of addresses that might be the
recipient from the mail header. There is no guarantee that any of
the returned addresses really are the recipient of the mail.
So, to implement exemption of self->self-mail you first have to
implement a way for SpamAssassin to know what the recipient
address is in order to know if a mail is self->self-addressed.
>> If you do implement your fix and submit it, please make it an option.
>> I for one would turn it off since it would not improve things here.
> You are the first person to say so. Can you explain why?
I want the AWL to apply to mail that is addressed from self->self.
Since the AWL also takes the IP address into account and since
all mail from authenticated/local users here comes from 127.0.0.1
to the software calling SpamAssassin, I do not have your problem
here and would not benefit from your fix.
While most mail addressed self->self that comes from external
systems is spam, every now and then ham addressed from self->self
do come in from idiotic systems and sometimes from users who for
some reason is not using our servers when sending mail.
The AWL as it is now does distinguish between "good" and "bad"
mail that are or pretends to be from our users, and I see no
reason to remove possible benefits of that distinction for mail
that happens to be addressed to the same user as it's addressed from.
Regards
/Jonas
--
Jonas Eckerman, FSDB & Fruktträdet
http://whatever.frukt.org/
http://www.fsdb.org/
http://www.frukt.org/
Re: can we make AWL ignore mail from self to self?
Posted by Jo Rhett <jr...@netconsonance.com>.
On May 23, 2008, at 3:45 AM, Jonas Eckerman wrote:
> 1: Just read it as of when I said "your own users" I meant the users
> of the host in question (the ones you mention above). More
> specifically, the users using your host as a MSA (authenticated or
> locally).
I don't trust "my users" in this context.
> 2: I never suggested disabling the AWL entirely. I suggested
> disabling it for the above mentioned users.
>
> I also suggested (and this is prefferable to disabling it in my
> opinion) to separate the AWL so that you use one AWL for mail from
> the above mentioned users and another for unathenticated mail from
> external relays.
>
> Is there any specific reason you do not want to use two different
> AWLs for those two different types of traffic?
Non-standard configuration/setup I would have to maintain
*AND*
A lot of work to hack around a simple problem. The AWL works just
fine for mail from "my users" to other "my users". In fact, it works
exceedingly well for that. What value is there in separating them?
>>> A more involved change would be to have the AWL store the
>>> authentication state as well as mail address and relay IP/16. When
>>> scanning mail from your own users using the same AWL database as
>>> for for mail to your users, this seems necessary to me.
>
>> Again, this seems to be a lot of work for no real gain. What I
>> have proposed makes sense for widespread use. Why hack/slash/burn
>> when a good fix would improve it for everyone?
>
> In case you haven't noticed it, your suggestion is not seen as a
> "good fix" for the problem by everyone. I was merely suggesting
> other ways to go about this.
Actually, that's not true. Nobody has suggested that this fix would
be bad. Matt was querying me thinking I had screwed up my trusted
hosts, but not a single person has suggested that this change would be
bad.
> If you wish other peoiple to implement/accept something that fixes
> your problem and you can't convince them that your own ideas are
> good, it may be that alternative means of fixing the problem are
> seen as better and therefore stand a bigger chance of being
> implemented/eccepted.
What alternatives? So far I've only heard (a) disable the AWL (b)
don't use AWL it sucks and (c) hack the system to use different AWLs.
None of which really make any logical sense to solve the problem.
> If you do implement your fix and submit it, please make it an
> option. I for one would turn it off since it would not improve
> things here.
You are the first person to say so. Can you explain why?
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: can we make AWL ignore mail from self to self?
Posted by Jonas Eckerman <jo...@frukt.org>.
Jo Rhett wrote:
> Lots of users of this host have Windows PCs,
>> Another way to do it would be to use different AWLs, or disabling AWL,
>> for mail from your own users (either authenticated or locally
>> submitted). This makes a lot of sense to me.
> Have no "my own users" except me ;-) And disabling AWL entirely is
> again a hack. Let's focus on a fix.
1: Just read it as of when I said "your own users" I meant the
users of the host in question (the ones you mention above). More
specifically, the users using your host as a MSA (authenticated
or locally).
2: I never suggested disabling the AWL entirely. I suggested
disabling it for the above mentioned users.
I also suggested (and this is prefferable to disabling it in my
opinion) to separate the AWL so that you use one AWL for mail
from the above mentioned users and another for unathenticated
mail from external relays.
Is there any specific reason you do not want to use two different
AWLs for those two different types of traffic?
>> A more involved change would be to have the AWL store the
>> authentication state as well as mail address and relay IP/16. When
>> scanning mail from your own users using the same AWL database as for
>> for mail to your users, this seems necessary to me.
> Again, this seems to be a lot of work for no real gain. What I have
> proposed makes sense for widespread use. Why hack/slash/burn when a
> good fix would improve it for everyone?
In case you haven't noticed it, your suggestion is not seen as a
"good fix" for the problem by everyone. I was merely suggesting
other ways to go about this.
If you wish other peoiple to implement/accept something that
fixes your problem and you can't convince them that your own
ideas are good, it may be that alternative means of fixing the
problem are seen as better and therefore stand a bigger chance of
being implemented/eccepted.
I am not, however, trying to stop you from implementing ignoring
self->self mail by the AWL.
If you do implement your fix and submit it, please make it an
option. I for one would turn it off since it would not improve
things here.
Regards
/Jonas
--
Jonas Eckerman, FSDB & Fruktträdet
http://whatever.frukt.org/
http://www.fsdb.org/
http://www.frukt.org/
Re: can we make AWL ignore mail from self to self?
Posted by Jo Rhett <jr...@netconsonance.com>.
On May 22, 2008, at 12:42 PM, Rob McEwen wrote:
> First, even if this isn't what you meant, I must set the record
> straight... requiring SMTP password-authentication is NOT a hack.
> Instead, that is a security feature. I'm not sure if you meant that
> differently, but I state this just to be on the safe side.
>
> Second, you do require SMTP authentication, right? Because not doing
> so would likely open up your server as an "open relay".
Rob, please read what you reply to. I've been doing SMTP AUTH since
before we got it standardized.
I said that disabling running SA for SMTP-AUTH users is a hack much
like disabling a firewall and I won't do it.
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: can we make AWL ignore mail from self to self?
Posted by Rob McEwen <ro...@invaluement.com>.
Jo Rhett wrote:
> That's a hack with consequences. Like "just disable the firewall".
> Uh, no ;-)
>
> Lots of users of this host have Windows PCs, and running SA on all
> outbound mail has both alerted them quickly to the problem and avoided
> nailing other people with spam and/or virus runs.
Something seems out of order here.
First, even if this isn't what you meant, I must set the record
straight... requiring SMTP password-authentication is NOT a hack.
Instead, that is a security feature. I'm not sure if you meant that
differently, but I state this just to be on the safe side.
Second, you do require SMTP authentication, right? Because not doing so
would likely open up your server as an "open relay". Additionally, the
vast majority of the spams and viruses that you referred to would not
have a chance of using your server to nail "other people" with spams or
viruses if you required SMTP authentication.
Most not-large-isp mail servers do just fine NOT spam filtering SMTP
password-authenticated messages with many years going by between any
single incident of a spam or virus being sent from that server.
The main reason larger ISPs must do some spam filtering on their
outbound mail sent from members of that ISP is because
(a) they do NOT use SMTP password-authentication and, instead, allow
relaying simply based on the message originating from a particular block
of IPs (very bad form... but the large ISPs can't find an easy way to
convert millions of users over to SMTP authentication). If that is your
situation, then I probably stand corrected as far as your situation is
concerned.
..OR..
(b) they are a heavily abused service.. such as freemail providers where
criminals sign up to try to send spam. Therefore, they should do
outbound filtering even on authenticated mail.
Otherwise, SMTP password-authenticated e-mail should almost always not
be filtered, or be minimally filtered.
Rob McEwen
Re: can we make AWL ignore mail from self to self?
Posted by Jo Rhett <jr...@netconsonance.com>.
On May 22, 2008, at 7:29 AM, Jonas Eckerman wrote:
> Jo Rhett wrote:
>
>> I'm not -- my Treo delivers mail directly to my mail server. From
>> DHCP-assigned addresses all over the world. I enjoy travel ;-)
>
> Then I guess you use authenticated SMTP for that.
> The easiest way to handle this probably is to simply avoid calling
> SA for authenticated mail.
That's a hack with consequences. Like "just disable the firewall".
Uh, no ;-)
Lots of users of this host have Windows PCs, and running SA on all
outbound mail has both alerted them quickly to the problem and avoided
nailing other people with spam and/or virus runs.
> Another way to do it would be to use different AWLs, or disabling
> AWL, for mail from your own users (either authenticated or locally
> submitted). This makes a lot of sense to me.
Have no "my own users" except me ;-) And disabling AWL entirely is
again a hack. Let's focus on a fix.
> A more involved change would be to have the AWL store the
> authentication state as well as mail address and relay IP/16. When
> scanning mail from your own users using the same AWL database as for
> for mail to your users, this seems necessary to me.
Again, this seems to be a lot of work for no real gain. What I have
proposed makes sense for widespread use. Why hack/slash/burn when a
good fix would improve it for everyone?
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: can we make AWL ignore mail from self to self?
Posted by Jonas Eckerman <jo...@frukt.org>.
Jo Rhett wrote:
> I'm not -- my Treo delivers mail directly to my mail server. From
> DHCP-assigned addresses all over the world. I enjoy travel ;-)
Then I guess you use authenticated SMTP for that.
The easiest way to handle this probably is to simply avoid
calling SA for authenticated mail.
Another way to do it would be to use different AWLs, or disabling
AWL, for mail from your own users (either authenticated or
locally submitted). This makes a lot of sense to me.
A more involved change would be to have the AWL store the
authentication state as well as mail address and relay IP/16.
When scanning mail from your own users using the same AWL
database as for for mail to your users, this seems necessary to me.
Regards
/Jonas
--
Jonas Eckerman, FSDB & Fruktträdet
http://whatever.frukt.org/
http://www.fsdb.org/
http://www.frukt.org/
Re: can we make AWL ignore mail from self to self?
Posted by Jo Rhett <jr...@netconsonance.com>.
> Jo Rhett wrote:
>> Matt, how can I possibly get you to move past this unfounded
>> assumption that my trust path is broken and focus on the real
>> problem? The trust path is not broken, it's just fine.
On May 20, 2008, at 5:47 PM, Matt Kettler wrote:
> Ok, then the AWL code is *SEVERELY* bugged. The question then
> becomes why isn't the source address part of the AWL working properly.
I'm not sure I know this or can agree. I'm fairly sure its
orthagonal, but I may be wrong.
> That IP range is what would detect the forgeries, or at least give
> the forgeries a different AWL entry than email you really sent
> yourself.
I only send mail to myself from my wireless provider or open WiFi
networks. e.g. "note to self" while I am on the road.
> The source IPs are different, so your real self-to-self should be
> handled independently, with a completely separate AWL entry, from
> the spammer forged self-to-self.
You're assuming I use the same source IP when I send myself mail, and
that just isn't true.
>> Or that you receive e-mail from the very same public wireless and/
>> or phone providers as everyone else does. My trust path doesn't
>> have to be broken if the networks used to send the e-mail are
>> public networks.
>>
>> (if you can laugh == "welcome to the 21st century and the
>> Crackberry/Treo/iPhone") Not trying to be snide.
> If you're using any kind of forwarder, including crackberry, their
> servers should be trusted by you so that SA's checks get applied to
> the mailserver that dropped mail off at them. That's the purpose of
> the trust path, to allow you to trust the headers of those systems
> receiving mail on your behalf and forwarding it to you.
I'm not -- my Treo delivers mail directly to my mail server. From
DHCP-assigned addresses all over the world. I enjoy travel ;-)
I'd also like to point out that no provider is willing to share their
server lists openly and consistently enough for this to occur. We
have to put crackberry users in their own domain because we use SPF on
the main domains and crackberry keeps changing their servers.
"no provider" == crackberry, verizon, sprint, etc... the wireless
providers who intercept outbound mail.
--
Jo Rhett
Net Consonance : consonant endings by net philanthropy, open source
and other randomness
Re: can we make AWL ignore mail from self to self?
Posted by Matt Kettler <mk...@verizon.net>.
Jo Rhett wrote:
> On May 3, 2008, at 7:59 PM, Matt Kettler wrote:
>>>> Have you tried running one of the forged messages, and an actual
>>>> legitimate message through SA manually with the -D flag to see what
>>>> the trusted and untrusted hosts are, as SA sees it?
>>>
>>> Yes. Many times. That's not the point of this thread.
>> I still think it is.
>
> Matt, how can I possibly get you to move past this unfounded
> assumption that my trust path is broken and focus on the real
> problem? The trust path is not broken, it's just fine.
Ok, then the AWL code is *SEVERELY* bugged. The question then becomes
why isn't the source address part of the AWL working properly.
>
>> If your AWL is applying the same history data to forged email as
>> unforged email, either there's a *major* bug in the AWL code, or your
>> trust path is broken. Period.
>>
>> The AWL is designed to be able to distinguish forged mail from
>> nonforged mail. If that's not working, that's a major problem.
>
> I've read the code and I see nothing designed to determine forgeries.
> There is code to save data with an IP range, but that's not relevant
> to this issue.
That's entirely relevant. That IP range is what would detect the
forgeries, or at least give the forgeries a different AWL entry than
email you really sent yourself.
The source IPs are different, so your real self-to-self should be
handled independently, with a completely separate AWL entry, from the
spammer forged self-to-self.
>
>>> The point of this thread is the obvious ease of forging e-mail from
>>> recipient to (same) recipient. It's one situation where the AWL
>>> wouldn't work very well.
>
>>
>> Actually, it's very difficult to forge in a way that will confuse the
>> AWL, if your trust path and the AWL code is working properly. After
>> all, it looks at the combination of email address and first untrusted
>> IP. Forged email will not be from the same IP as legitimate email,
>> unless your trust path is broken and SA always sees all mail as
>> entering your network from the same IP.
>
> Or that you receive e-mail from the very same public wireless and/or
> phone providers as everyone else does. My trust path doesn't have to
> be broken if the networks used to send the e-mail are public networks.
>
> (if you can laugh == "welcome to the 21st century and the
> Crackberry/Treo/iPhone") Not trying to be snide.
If you're using any kind of forwarder, including crackberry, their
servers should be trusted by you so that SA's checks get applied to the
mailserver that dropped mail off at them. That's the purpose of the
trust path, to allow you to trust the headers of those systems receiving
mail on your behalf and forwarding it to you.
>>> It would be fairly easy to forge, and worthwhile enough for botnets
>>> to just do this (which they are, in force, for the last month)
>>>
>>> I personally see no value in applying AWL to messages from self to
>>> self.
>> I agree, but I see no value in applying the exception. I'd rather try
>> to fix the more general problem of your AWL not distinguishing
>> message sources properly.
>
> I see no evidence of this. My trust path is just fine (ie
> "nonexistent" == all mail not from localhost isn't trusted)
Agreed that's probably reasonable in many networks.
>
>>> I may be wrong, and I'm open to arguements against this, but I am
>>> suggesting that the AWL module should skip over self->self
>>> messages. It seems too easy to forge, and no gain in doing so.
>
>> You're overlooking how the AWL works. It's actually really hard to
>> forge.
>>
>> However, I will agree with you there's limited value in self-to-self
>> AWL records.. but there's also no harm in them if the AWL is working
>> properly.
>
> Instead of making statements like this, please explain how the AWL
> deals the forgery. Because I have the code right in front of me and I
> see absolutely nothing in the AWL code that tries to identify
> forgeries. Instead of making unfounded statements, can you be
> specific about the issues?
>
Re: can we make AWL ignore mail from self to self?
Posted by Jo Rhett <jr...@netconsonance.com>.
On May 3, 2008, at 7:59 PM, Matt Kettler wrote:
>>> Have you tried running one of the forged messages, and an actual
>>> legitimate message through SA manually with the -D flag to see
>>> what the trusted and untrusted hosts are, as SA sees it?
>>
>> Yes. Many times. That's not the point of this thread.
> I still think it is.
Matt, how can I possibly get you to move past this unfounded
assumption that my trust path is broken and focus on the real
problem? The trust path is not broken, it's just fine.
> If your AWL is applying the same history data to forged email as
> unforged email, either there's a *major* bug in the AWL code, or
> your trust path is broken. Period.
>
> The AWL is designed to be able to distinguish forged mail from
> nonforged mail. If that's not working, that's a major problem.
I've read the code and I see nothing designed to determine forgeries.
There is code to save data with an IP range, but that's not relevant
to this issue.
>> The point of this thread is the obvious ease of forging e-mail from
>> recipient to (same) recipient. It's one situation where the AWL
>> wouldn't work very well.
>
> Actually, it's very difficult to forge in a way that will confuse
> the AWL, if your trust path and the AWL code is working properly.
> After all, it looks at the combination of email address and first
> untrusted IP. Forged email will not be from the same IP as
> legitimate email, unless your trust path is broken and SA always
> sees all mail as entering your network from the same IP.
Or that you receive e-mail from the very same public wireless and/or
phone providers as everyone else does. My trust path doesn't have to
be broken if the networks used to send the e-mail are public networks.
(if you can laugh == "welcome to the 21st century and the Crackberry/
Treo/iPhone") Not trying to be snide.
>> It would be fairly easy to forge, and worthwhile enough for botnets
>> to just do this (which they are, in force, for the last month)
>>
>> I personally see no value in applying AWL to messages from self to
>> self.
> I agree, but I see no value in applying the exception. I'd rather
> try to fix the more general problem of your AWL not distinguishing
> message sources properly.
I see no evidence of this. My trust path is just fine (ie
"nonexistent" == all mail not from localhost isn't trusted)
>> I may be wrong, and I'm open to arguements against this, but I am
>> suggesting that the AWL module should skip over self->self
>> messages. It seems too easy to forge, and no gain in doing so.
> You're overlooking how the AWL works. It's actually really hard to
> forge.
>
> However, I will agree with you there's limited value in self-to-self
> AWL records.. but there's also no harm in them if the AWL is working
> properly.
Instead of making statements like this, please explain how the AWL
deals the forgery. Because I have the code right in front of me and I
see absolutely nothing in the AWL code that tries to identify
forgeries. Instead of making unfounded statements, can you be
specific about the issues?
Re: can we make AWL ignore mail from self to self?
Posted by Jo Rhett <jr...@netconsonance.com>.
Let's focus this on specific technical details:
1. How does AWL deal with forgery (other than by saving a /16 of the
source IP)
2. How can I easily see the AWL database for a given destination
address?
Re: can we make AWL ignore mail from self to self?
Posted by Matt Kettler <mk...@verizon.net>.
Jo Rhett wrote:
> On Apr 29, 2008, at 7:40 PM, Matt Kettler wrote:
>>> I'm not repeating for the 5th time that there are no trusted
>>> mailservers. Only this host.
>> That's a contradiction, because "this host" is a mailserver. Clearly
>> you have a trusted mailserver.
>> However, in the interest of moving the discussion forward, you have
>> exactly one trusted mailserver, your MX, which is perfectly valid.
>
> Yes. I'm sorry but this is obvious. I don't know how to pick the
> words exactly as you want them, but most people understood what I
> meant 5 or 6 replies ago ;-)
>
>> The question lies in why does the AWL seem to be confusing forged
>> email with your own email. That's generally quite critically
>> dependent on the trust path.
>
> No, that's not the question at all. (more below)
>
>> Have you tried running one of the forged messages, and an actual
>> legitimate message through SA manually with the -D flag to see what
>> the trusted and untrusted hosts are, as SA sees it?
>
> Yes. Many times. That's not the point of this thread.
I still think it is.
If your AWL is applying the same history data to forged email as
unforged email, either there's a *major* bug in the AWL code, or your
trust path is broken. Period.
The AWL is designed to be able to distinguish forged mail from nonforged
mail. If that's not working, that's a major problem.
>
> The point of this thread is the obvious ease of forging e-mail from
> recipient to (same) recipient. It's one situation where the AWL
> wouldn't work very well.
Actually, it's very difficult to forge in a way that will confuse the
AWL, if your trust path and the AWL code is working properly. After all,
it looks at the combination of email address and first untrusted IP.
Forged email will not be from the same IP as legitimate email, unless
your trust path is broken and SA always sees all mail as entering your
network from the same IP.
> It would be fairly easy to forge, and worthwhile enough for botnets to
> just do this (which they are, in force, for the last month)
>
> I personally see no value in applying AWL to messages from self to self.
I agree, but I see no value in applying the exception. I'd rather try to
fix the more general problem of your AWL not distinguishing message
sources properly.
> I may be wrong, and I'm open to arguements against this, but I am
> suggesting that the AWL module should skip over self->self messages.
> It seems too easy to forge, and no gain in doing so.
>
You're overlooking how the AWL works. It's actually really hard to forge.
However, I will agree with you there's limited value in self-to-self AWL
records.. but there's also no harm in them if the AWL is working properly.