Re: RFC's suck

[John GALLET <sp...@saphirtech.fr>]

Hi,

[repost from yesterday, I was not using the correct From address for this 
list...]

> Yes, it means that every Received: header in an email is valid with a 
> valid IP, valid configuration (whatever that is deemed to be), and valid 
> DNS. Only servers that were correctly classified as mailservers would 
> even be able to be verified. Mailservers that were spam sources would be 
> easily identified and blacklisted across the board. Or something.

I am a bit lost here. Are you saying that right now the *main* problem 
with spam is "source spoofing" and that just by having a strict format for 
emails in the protocole we would turn the whole spam fighting industry 
into a single huge database of "known spammers" ?

If it were true, I think it still raises a few issues. First, not all 
providers agree to implement SPF currently. One of the reasons being that 
I am [EDIT: usually !!] using a totally legitimate @wanadoo.fr address but 
I am not currently connected through their network. So basically all 
providers would have to issue SSL certificates for all their clients that 
could be of course stolen by malware etc. and become "legit". What I mean 
is that the very first step of email sending between my favorite mail 
reader and the SMTP/IMAP server would still be a weak point (Like we say 
in French "l'enemi, c'est l'utilisateur" i.e. "the end-used is the 
enemy/the source of all evil"). And I am not even talking about all the 
email domains that are not ISPs such as hotmail or yahoo and the like: you 
can secure all the way from their server to yours, it will never prevent 
the "garbage in - garbage out" approach.

Second, I am not aware of any lawsuits yet against RBLs but I am quite 
surprised no "official spammer" has already done that, or tried direct 
attacks targeted at the RBLs servers: they have enough zombies to spam, so 
they could.

Furthermore, let me be the devil's advocate for a second: would not you 
agree that many a rule in SA can actually catch spam because they are RFC 
compliant but stupid enough to add fancy headers or fancy header 
formatting ?

> > Putting this on a distinct port seems more a marketing thing. Why not 
> > add it as a capability in a normal SMTP server?
>
> Because the idea is to be able to simply retire the current SMTP and 
> that will be a lot simpler if the new service is on a new port.

I would agree to that. Http already does (i.e. port 8080 vs port 80), it 
would facilitate migration.

> A secure verifiable delivery chain from server to server would almost 
> completely eliminate the need for SA.

I can not agree to that. The point of entry has to be secured and I am 
afraid it will be a pain to do so.

> And I'm not saying it would be easy, or happen over-night.  I figure if 
> people started working on an RFC right now we might see the end of the 
> current SMTP in 15-20 years unless there was a huge push in which case 
> it could maybe happen in 10-15 years.

Might be. We have been talking over and over about IPV6 for about 15 years 
now, and currently it only incurs problems between compliant and 
non-compliant equipmentswith zero gain.

> I'm not saying it would ELIMINATE Spam, but it would certainly reduce it 
> to a manageable level.

Having an authenticated chain can only help if it is not broken or if we 
can detect it was broken, otherwise it will have the reverse effect of 
spammers injecting massive spam into "trusted" network chains that can not 
be banned for fear of hitting legit users.

> Nothing we're doing now is reducing it at all, the amount of spam has 
> been increasing steadily every year since the very first Green Card 
> posting to USENET.

Amen to that.

To come back (a little) to the original post, IMHO we can not and should 
not do without specs i.e. RFCs. THe existing ones are not that bad, I sent 
my firts emails back in 1992 with my mom's address, so these RFCs have 
made the world communicate (for better and worse) for 20 years. Back at 
the time, the bandwith was so low and email access so controlled (add to 
that a tiny bit of optimism about the kind human nature) that spam was not 
an issue. A new RFC can be needed, but I really can not believe its main 
improvement would be protocole formatting...

HTH
JG

Re: RFC's suck

[Kenneth Porter <sh...@sewingwitch.com>]

On Thursday, April 02, 2009 12:13 PM -0600 LuKreme <kr...@kreme.com> 
wrote:

> You should be sending mail out through your ISP which should be accepting
> your outbound mail as from you since they know who you are.  Once your
> ISP (with their correctly configured SASL enabled mailserver) passes it
> along to the next server, you have a valid chain that goes from your
> connection to your ISP to the destination.

How does one join this "clique" of trusted senders? Right now I can buy a 
static IP and set up a mail server that's relatively trusted. In your 
scheme, what additional steps do I need to take to avoid submitting my mail 
to the wiretap-ridden server run by my ISP?

I'm a fan of "web of trust" designs so I'd favor registering a potential 
sender with some kind of distributed reputation system. Given that, you 
just need the sender to supply authentication credentials to test against 
its trusted web. You could either sign the message, authenticate the 
transport, or both.

Re: RFC's suck

[LuKreme <kr...@kreme.com>]

On 1-Apr-2009, at 01:55, John GALLET wrote:
> [repost from yesterday, I was not using the correct From address for  
> this list...]
>
>> Yes, it means that every Received: header in an email is valid with  
>> a valid IP, valid configuration (whatever that is deemed to be),  
>> and valid DNS. Only servers that were correctly classified as  
>> mailservers would even be able to be verified. Mailservers that  
>> were spam sources would be easily identified and blacklisted across  
>> the board. Or something.
>
> I am a bit lost here. Are you saying that right now the *main*  
> problem with spam is "source spoofing"

That is >50% of our connection attempts on our server. And a good  
percentage of the 'accepted' spam as well.

> and that just by having a strict format for emails in the protocole  
> we would turn the whole spam fighting industry into a single huge  
> database of "known spammers" ?

Essentially? Well, sorta. Really what I am saying is that a strict  
format and protocol on servers would eliminate most of the current  
spam since so much of it comes from zombied machines. It would not  
eliminate all spam; it will eliminate FORGED spam. Spam claiming to be  
from paypal.com or amazon.com, for example. It would not eliminate the  
spam from pay.pal.ru assuming that is a valid domain.

> If it were true, I think it still raises a few issues. First, not  
> all providers agree to implement SPF currently. One of the reasons  
> being that I am [EDIT: usually !!] using a totally legitimate  
> @wanadoo.fr address but I am not currently connected through their  
> network.

So?  You should be sending mail out through your ISP which should be  
accepting your outbound mail as from you since they know who you are.   
Once your ISP (with their correctly configured SASL enabled  
mailserver) passes it along to the next server, you have a valid chain  
that goes from your connection to your ISP to the destination.   
Presumably your ISP has some way (Message-ID? embedded hash? Server- 
side logging?) of knowing that mail came from YOU and being able to  
track that back to you if you start flooding the network with spam.   
YOU do not need a cert, you just need to connect securely to your ISP  
or to the outbound mailserver you are going to use.  Your secure  
connection to that server is enough to autheticate you as a valid  
sender.  The zombie-malware will not be able to establish a SASL  
connection to your ISP or to your Submission port, so it cant send  
mail that meets the new protocol.

> So basically all providers would have to issue SSL certificates for  
> all their clients that could be of course stolen by malware etc. and  
> become "legit". What I mean is that the very first step of email  
> sending between my favorite mail reader and the SMTP/IMAP server  
> would still be a weak point (Like we say in French "l'enemi, c'est  
> l'utilisateur" i.e. "the end-used is the enemy/the source of all  
> evil").

The end users are always the weak point. But this is manageable. It  
could be as simple as requiring you to send mail from your wanadoo.fr  
email address to wanadoo.fr's submission port (which is, once again, a  
secure connection), but this is by no means the only possible  
solution. I think it is probably the most workable one, but there are  
certainly other options (greylisting new From addresses on the sender  
server, for example.  So if you've send mail via myisp.tld as From: user@wanadoo.fr 
  in the past, it passes.  But if you suddenly start sending from pay@pal.ru 
  it holds up and waits for some other verification).

> And I am not even talking about all the email domains that are not  
> ISPs such as hotmail or yahoo and the like: you can secure all the  
> way from their server to yours, it will never prevent the "garbage  
> in - garbage out" approach.

Where most spam is coming from is a zombied Windows machine that has  
some malware that creates a fake SMTP server that starts sending out  
hundreds of thousands of mails, all with multiple forged headers. If  
you make forged headers essentially impossible to be valid, then you  
eliminate a large chunk of the current spam problem; not all of it,  
obviously, but a lot of it.

> Second, I am not aware of any lawsuits yet against RBLs but I am  
> quite surprised no "official spammer" has already done that, or  
> tried direct attacks targeted at the RBLs servers: they have enough  
> zombies to spam, so they could.

Many RBLs have had very serious problems over the years, and many have  
vanished because of this.  However, since no RBL *blocks* any message,  
it is more of a problem where simply defending yourself from spurious  
legal action is expensive.  This has nothing to do with spam, per se.

> Furthermore, let me be the devil's advocate for a second: would not  
> you agree that many a rule in SA can actually catch spam because  
> they are RFC compliant but stupid enough to add fancy headers or  
> fancy header formatting ?

Oh, absolutely.  Heck, simply adding blocks to my helo for obviously  
dynamic addresses eliminates a crapload of spam from my server.   
Probably 50-60%. But I think the next step is to get beyond the  
current SMTP standard where the only headers you can trust in an email  
is the ones your own server adds.

>> > Putting this on a distinct port seems more a marketing thing. Why  
>> not > add it as a capability in a normal SMTP server?
>>
>> Because the idea is to be able to simply retire the current SMTP  
>> and that will be a lot simpler if the new service is on a new port.
>
> I would agree to that. Http already does (i.e. port 8080 vs port  
> 80), it would facilitate migration.
>
>> A secure verifiable delivery chain from server to server would  
>> almost completely eliminate the need for SA.
>
> I can not agree to that. The point of entry has to be secured and I  
> am afraid it will be a pain to do so.

The point of entry to the first server can be secured without forcing  
end users to get certificates. Require SASL connections with  
authentication. Obviously, if an OS allows any old program to access  
authentication data you're going to have a problem, but I don't think  
even Windows is THAT broken.  I could be wrong, I don't run it.

>> I'm not saying it would ELIMINATE Spam, but it would certainly  
>> reduce it to a manageable level.
>
> Having an authenticated chain can only help if it is not broken or  
> if we can detect it was broken, otherwise it will have the reverse  
> effect of spammers injecting massive spam into "trusted" network  
> chains that can not be banned for fear of hitting legit users.

Yep, this is why it will require new standards.  The current system is  
so broken that it cannot be repaired, IMO.  We are going to need a new  
system.

> To come back (a little) to the original post, IMHO we can not and  
> should not do without specs i.e. RFCs.

Absolutely.

> THe existing ones are not that bad, I sent my firts emails back in  
> 1992 with my mom's address, so these RFCs have made the world  
> communicate (for better and worse) for 20 years. Back at the time,  
> the bandwith was so low and email access so controlled (add to that  
> a tiny bit of optimism about the kind human nature) that spam was  
> not an issue. A new RFC can be needed, but I really can not believe  
> its main improvement would be protocole formatting...

The other option is to require some sort of asinine 'sender  
verification' BS where everyone has to have an ID certificate to even  
send mail. that's never going to happen.

Here's how I'm envisioning a chain working:

You ==> Submission ==> Your Server ==> Destination Server ==> IMAP ==>  
Destination user

Where ==> means a secured connection with a valid and verifiable  
header showing the path.  Destination server should be able to know  
that the connection from You to Submission was secure and that the  
header added for that connection is valid.  And I do mean *know* with  
certitude. This does not require Destination Server to know WHO you  
are, just that your connection was secure, valid, and verifiable.  
Something based on a hash of the originating IP, the server IP, a  
token the server ads to the header, the IP of the next-hop, and the  
time stamp, perhaps.

Received: From 12.34.56.789 by 98.76.54.321 [EFF54G2A5] at 123456789  
for 123.456.7.89 {1cdaaa4cf2b17082d029a8ba0376b2c1}

Where both the random token and the epoch time are added by the server  
and the md5 hash is calculated off the string "From 12.34.56.789 by  
98.76.54.321 [EFF54G2A5] at 123456789 for 123.456.7.89" (which is in  
that exact format). The accepting server calculates the hash, verifies  
it, and then adds its own header.

Received: From 98.76.54.321 by 123.456.7.89 [F123679afqaz.wanadoo.fr]  
at 123456791 for user@mylocaldomain.tld  
{397c272bc1068b2fa5eae417d25596a0}

Any mail that arrives at a Destination server where any one of those  
==> is actually a --> (unsecured or invalid or non-verifiable  
connection) is rejected.

Obvisuly there is more to this, because one needs to be able to verify  
with 98.76.54.321 and 123.456.7.89 that those headers are actually  
valid, but there has to be a way to do that.

Obviously this is going to take people smarter than I, but I do think  
this is possible.

-- 
Incredible! One of the worst performances of my career and they
	never doubted it for a second.