You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Kelly Jones <ke...@gmail.com> on 2007/03/10 20:27:55 UTC
Tool for validating sender address as spam-fighting technique?
To fight spam, I want to validate the address (not necessarily in
real-time) of the a given email sender. Is there a Unix tool that does
this?
The basics are simple: to validate "kmnyqi@wnonline.net", I connect to
the MX record of wnonline.net and go as far as "RCPT TO" as follows:
> host -t mx wnonline.net
wnonline.net mail is handled by 5 wnspf.bayou.com.
> telnet wnspf.bayou.com. 25
Trying 209.209.192.75...
Connected to wnspf.bayou.com..
Escape character is '^]'.
220 Welcome to Bayou mxfilter
HELO domaintester.com
250 mxfilter.bayou.com
MAIL FROM: <te...@ignoreme.com>
250 Ok
RCPT TO: <km...@wnonline.net>
550 <km...@wnonline.net>: Recipient address rejected: 5.1.1
<km...@wnonline.net>... User unknown
QUIT
221 Bye
Connection closed by foreign host.
This tells me kmnyqi@wnonline.net is an invalid address and that mail
from that address is probably bogus.
A more sophisticated tool would cache results, handle temporary
failures (eg, inability to connect to the MX server), handle multiple
MX records, perhaps even publish results [carefully, to avoid giving
spammers a source of legit email addresses!], etc. Plus, I'd prefer to
use a tested tool vs hacking something up myself.
I realize this technique is far from perfect:
Spammers spoof legit addresses
Bounces/Mailing lists/etc legitimately use "do not reply" addresses
It could be considered unfriendly to the target MX servers
Some mail servers incorrectly say "user unknown" when they see spam,
figuring it's more of a deterrent than saying "you're a spammer"
Some mail servers inefficiently accept mail for "foo@xxx.com" (where
xxx.com is one of their domains), figure out if foo exists later, and
send a bounce back to the envelope sender, instead of rejecting email
at the SMTP level (a really good tool would create throwaway addresses
to catch these cases too)
... but I still think it might help.
--
We're just a Bunch Of Regular Guys, a collective group that's trying
to understand and assimilate technology. We feel that resistance to
new ideas and technology is unwise and ultimately futile.
Re: Tool for validating sender address as spam-fighting technique?
Posted by Jonas Eckerman <jo...@frukt.org>.
Kelly Jones wrote:
> To fight spam, I want to validate the address (not necessarily in
> real-time) of the a given email sender.
1: SMTP sender validation is controversial as it increases load
on innocent mail servers. (This can result in effective DDoS
attacks against non-large systems.)
2: Using this type of sender validation can make *your* server
look like an address probe. It can also make your server connect
to spam traps. This means that the validation can get *your*
server blacklisted.
2: Don't verify at RCPT TO as this could create a loop condition
between two SMTP servers.
3: Since this is a SpamAssassin list you really should ask if it
can be done in SpamAssassin. It can. I've got an experimental
SpamAssassin plugin for this.
> A more sophisticated tool would cache results,
My plugin does this. It also excempts addresses looking like
mailing list bounce addresses from the checks. And it only does
the checks if the result has the possibility opf changing wether
SA considers the message as spam or not.
> handle temporary failures
And this.
> Plus, I'd prefer to
> use a tested tool vs hacking something up myself.
My plugin works here, but I consider the whole concept as
experimental, controversial and higly questionable.
I have no idea wether the plugin works in any setup except ours
and I will not give any support to anyone using it. This does
make use of it almost the same as hacking something up yourself.
My SA plugins are at <http://whatever.frukt.org/>.
Regards
/Jonas
--
Jonas Eckerman, FSDB & Fruktträdet
http://whatever.frukt.org/
http://www.fsdb.org/
http://www.frukt.org/
RE: Tool for validating sender address as spam-fighting technique?
Posted by "Randal, Phil" <pr...@herefordshire.gov.uk>.
smf-sav is one sendmail milter which does this:
http://smfs.sourceforge.net/smf-sav.html
SAV v1.3.0 - console utility for e-Mail Sender Address Verification
(also at http://smfs.sf.net/ )
Cheers,
Phil
-----Original Message-----
From: Kelly Jones [mailto:kelly.terry.jones@gmail.com]
Sent: 10 March 2007 19:28
To: freebsd-questions@freebsd.org; users@spamassassin.apache.org;
linuxusersgroup@googlegroups.com; nmlug@nmlug.org;
nmosug-l@mailman.swcp.com
Subject: Tool for validating sender address as spam-fighting technique?
To fight spam, I want to validate the address (not necessarily in
real-time) of the a given email sender. Is there a Unix tool that does
this?
The basics are simple: to validate "kmnyqi@wnonline.net", I connect to
the MX record of wnonline.net and go as far as "RCPT TO" as follows:
> host -t mx wnonline.net
wnonline.net mail is handled by 5 wnspf.bayou.com.
> telnet wnspf.bayou.com. 25
Trying 209.209.192.75...
Connected to wnspf.bayou.com..
Escape character is '^]'.
220 Welcome to Bayou mxfilter
HELO domaintester.com
250 mxfilter.bayou.com
MAIL FROM: <te...@ignoreme.com>
250 Ok
RCPT TO: <km...@wnonline.net>
550 <km...@wnonline.net>: Recipient address rejected: 5.1.1
<km...@wnonline.net>... User unknown
QUIT
221 Bye
Connection closed by foreign host.
This tells me kmnyqi@wnonline.net is an invalid address and that mail
from that address is probably bogus.
A more sophisticated tool would cache results, handle temporary
failures (eg, inability to connect to the MX server), handle multiple
MX records, perhaps even publish results [carefully, to avoid giving
spammers a source of legit email addresses!], etc. Plus, I'd prefer to
use a tested tool vs hacking something up myself.
I realize this technique is far from perfect:
Spammers spoof legit addresses
Bounces/Mailing lists/etc legitimately use "do not reply" addresses
It could be considered unfriendly to the target MX servers
Some mail servers incorrectly say "user unknown" when they see spam,
figuring it's more of a deterrent than saying "you're a spammer"
Some mail servers inefficiently accept mail for "foo@xxx.com" (where
xxx.com is one of their domains), figure out if foo exists later, and
send a bounce back to the envelope sender, instead of rejecting email
at the SMTP level (a really good tool would create throwaway addresses
to catch these cases too)
... but I still think it might help.
--
We're just a Bunch Of Regular Guys, a collective group that's trying
to understand and assimilate technology. We feel that resistance to
new ideas and technology is unwise and ultimately futile.
Re: Tool for validating sender address as spam-fighting technique?
Posted by John Rudd <jr...@ucsc.edu>.
Bob Proulx wrote:
> Sietse van Zanen wrote:
>> Ralf Hildebrandt wrote:
>>> Kelly Jones wrote:
>>>> To fight spam, I want to validate the address (not necessarily in
>>>> real-time) of the a given email sender. Is there a Unix tool that
>>>> does this?
>>> Postfix has exactly this built in. It's the
>>> "reject_unverified_sender" restriction.
>> Yes, but you don't always want to reject such mails. NDR's,
>> automated mails etc are often send from empty or non-existent e-mail
>> addresses.
>
> Any legitimate email will have a valid sender address. That includes
> DSNs and automated messages.
Yes, and that address is, by RFC mandate, empty. An empty (meaning
"<>") address is a valid sender address, and must be accepted.
Re: Tool for validating sender address as spam-fighting technique?
Posted by Bob Proulx <bo...@proulx.com>.
Sietse van Zanen wrote:
> Ralf Hildebrandt wrote:
> > Kelly Jones wrote:
> > > To fight spam, I want to validate the address (not necessarily in
> > > real-time) of the a given email sender. Is there a Unix tool that
> > > does this?
> >
> > Postfix has exactly this built in. It's the
> > "reject_unverified_sender" restriction.
>
> Yes, but you don't always want to reject such mails. NDR's,
> automated mails etc are often send from empty or non-existent e-mail
> addresses.
Any legitimate email will have a valid sender address. That includes
DSNs and automated messages. Within a small network it may be okay to
use an invalid sending address locally. But for any mail across a wan
the sending address must be able to receive bounces.
So yes I do want to always reject mail with invalid sending addresses.
The postfix and exim implementations are great. But I also see where
this would be very useful to do off-on-the-side of the mta for a
corpus of mail. I do spam filtering for a number of mailman hosted
mailing lists. For various reasons outside my control this is not
done on the front end machine. It would be useful to be able to use
verified senders to catagorize mail as it is processed through the
pipeline. Having a tool outside of the mta to provide this data would
be very useful to me anyway.
Bob
RE: Tool for validating sender address as spam-fighting technique?
Posted by Sietse van Zanen <si...@wizdom.nu>.
Yes, but you don't always want to reject such mails. NDR's, automated mails etc are often send from empty or non-existent e-mail addresses.
You will want to score points, like other SA tests. Maybe a good idea to write such a test, as it doesn't exist yet.
I know nagios has some tools that can automate the TCP part, it would just need a shell around it.
-Sietse
-----Original Message-----
From: Ralf Hildebrandt [mailto:Ralf.Hildebrandt@charite.de]
Sent: Saturday, March 10, 2007 20:32
To: users@spamassassin.apache.org
Subject: Re: Tool for validating sender address as spam-fighting technique?
* Kelly Jones <ke...@gmail.com>:
> To fight spam, I want to validate the address (not necessarily in
> real-time) of the a given email sender. Is there a Unix tool that does
> this?
Postfix has exactly this built in. It's the
"reject_unverified_sender" restriction.
--
Ralf Hildebrandt (i.A. des IT-Zentrums) Ralf.Hildebrandt@charite.de
Charite - Universitätsmedizin Berlin Tel. +49 (0)30-450 570-155
Gemeinsame Einrichtung von FU- und HU-Berlin Fax. +49 (0)30-450 570-962
IT-Zentrum Standort CBF send no mail to plonk@charite.de
Re: Tool for validating sender address as spam-fighting technique?
Posted by Ralf Hildebrandt <Ra...@charite.de>.
* Kelly Jones <ke...@gmail.com>:
> To fight spam, I want to validate the address (not necessarily in
> real-time) of the a given email sender. Is there a Unix tool that does
> this?
Postfix has exactly this built in. It's the
"reject_unverified_sender" restriction.
--
Ralf Hildebrandt (i.A. des IT-Zentrums) Ralf.Hildebrandt@charite.de
Charite - Universitätsmedizin Berlin Tel. +49 (0)30-450 570-155
Gemeinsame Einrichtung von FU- und HU-Berlin Fax. +49 (0)30-450 570-962
IT-Zentrum Standort CBF send no mail to plonk@charite.de
Re: Tool for validating sender address as spam-fighting technique?
Posted by Jeffrey Goldberg <je...@goldmark.org>.
[mailed and posted]
On Mar 10, 2007, at 1:27 PM, Kelly Jones wrote:
> To fight spam, I want to validate the address (not necessarily in
> real-time) of the a given email sender. Is there a Unix tool that does
> this?
>
> The basics are simple: to validate "kmnyqi@wnonline.net", I connect to
> the MX record of wnonline.net and go as far as "RCPT TO" as follows:
>
[...]
> RCPT TO: <km...@wnonline.net>
> 550 <km...@wnonline.net>: Recipient address rejected: 5.1.1
> <km...@wnonline.net>... User unknown
>
>
> This tells me kmnyqi@wnonline.net is an invalid address and that mail
> from that address is probably bogus.
>
> A more sophisticated tool would cache results, handle temporary
> failures [...]
In some anti-spam discussions what you are proposing is referred to
as "call-backs".
> I realize this technique is far from perfect:
>
> Spammers spoof legit addresses
Indeed they do. As someone who has at times received more than 100
bounces per minute over the span of a week because some spammer
decided that my address would be a good one to forge, I am well aware
of that.
In general, we have a question of how well any spamming counter
measure scales. If most SMTP servers did the kind of verification
you wish to do, than most spam would be sent with forged genuine
addresses. So when considering using such a system, consider the
overall cost to legitimate users vs the counter counter measures
spammers will take.
In this case the counter counter measures available to spammers is so
much easier and cheaper than the verification system itself, that
it's not really a good idea to try such verification.
> Bounces/Mailing lists/etc legitimately use "do not reply" addresses
>
> It could be considered unfriendly to the target MX servers
As you've already seen, some managers of MXes complain, others find
the added load negligible.
> Some mail servers incorrectly say "user unknown" when they see spam,
> figuring it's more of a deterrent than saying "you're a spammer"
Yes. An unknown user response is more likely to get the address
removed from a list than a policy bounce.
> Some mail servers inefficiently accept mail for "foo@xxx.com" (where
> xxx.com is one of their domains), figure out if foo exists later, and
> send a bounce back to the envelope sender, instead of rejecting email
> at the SMTP level
Yes. While I'm a believer in "reject early, reject often" lots of
sites don't or can't say that an address is bad at SMTP RCPT TO:
time. Often these are situations where an MS-Exchange server is the
"real" mailhub for an organization, but it is being protected from
direct access from the Internet by having a Unix box stand between it
and the network.
Other MTAs just accept and then bounce (instead of rejected at SMTP
time) as a matter of (mis)design.
Some very deliberately don't provide verification at STMP time as a
means to make it harder for spammers to collect email addresses.
Indeed, this is why the VRFY command is almost never implemented by
STMP servers these days.
Anyway, others have pointed you to tools for doing what you want.
The newsgroups or mailing lists (and FAQs) for the particular MTA you
use will point you to how to plug in such tools into their MTA.
As I've suggested, I'm not a particular fan of this particular
counter measure, but you will have seen that opinions differ.
Cheers,
-j
--
Jeffrey Goldberg http://www.goldmark.org/jeff/
Re: Tool for validating sender address as spam-fighting technique?
Posted by John Rudd <jr...@ucsc.edu>.
Randal, Phil wrote:
> You're going to get bounce blowback anyhow, whether you use SAV or not.
>
> Using Recipient Address Validation (or any kind of reject at the gateway
> level without first scanning for spam) would also increase blowback if
> junk mail is being sent via relays.
>
> No Address validation at the gateway - this seems to result in the
> fewest bounces:
>
SAV also creates an unreasonable burden on other people's mail servers
(in the same way that challenge response systems do, except challenge
response burdens both other people's mail servers AND other people).
Imagine 1,000,000 people all getting a spam that claims to be from you.
Now imagine if all of them are using SAV. You're about to get
1,000,000 connections to your mail server, all trying to validate your
address. For a message you didn't actually send. Meanwhile, you can't
receive legitimate mail during that window of time, because all of those
jerks using SAV are saturating your mail server.
Like I said, these types of call-back/SAV probes are not new. Verizon
has been doing it for years. If it was such a great anti-spam trick,
you'd think people would be marveling about how spam-free their Verizon
accounts are. I can't recall EVER having heard that.
SAV is one shade less polluting than Challenge-Response. (and that's
being polite... I consider people with Challenge-Response systems to be
total shit-bags, for example ... imagine what nice labels I might apply
to people using SAV systems)
RE: Tool for validating sender address as spam-fighting technique?
Posted by "Randal, Phil" <pr...@herefordshire.gov.uk>.
You're going to get bounce blowback anyhow, whether you use SAV or not.
Using Recipient Address Validation (or any kind of reject at the gateway
level without first scanning for spam) would also increase blowback if
junk mail is being sent via relays.
No Address validation at the gateway - this seems to result in the
fewest bounces:
Spammer -> relay (accepted)
Relay -> you (accepted)
<spam filtering swallows most of the junk>
Backend server -> bounce if invalid recipient (bad but most of the junk
has been eliminated by this stage)
Spammer -> you (accepted)
<spam filtering swallows most of the junk>
Backend server -> bounce if invalid recipient (bad but most of the junk
has been eliminated by this stage)
----------
Recipient Address Validation at the gateway:
Spammer -> relay (accepted)
Relay -> you (denied because of invalid recipient)
Relay to forged sender -> bounce (bad)
Spammer -> you (denied because of invalid recipient)
---------
Sender address validation at the gateway:
Spammer -> relay (accepted)
Relay -> you (denied because of invalid sender)
Relay to forged sender -> bounce
Spammer -> you (denied because of invalid sender)
Looks like the best (most ethical?) strategy is to swallow all email and
never bounce or reject anything. That could put a huge load on your
internet pipe and spam filtering boxes.
But rejecting / bouncing at the earliest possible stage is, from the
viewpoint of the person doing it, a very sane and rational strategy. It
saves them bandwidth and resources on their gateway boxes.
Note also that my comments on relays also apply to the use of DNSBLs at
the gateway.
So, mail relays are bad, ISP's backup MX servers are bad, etc, etc.
It's a no-win situation, alas.
Phil
-----Original Message-----
From: jm@jmason.org [mailto:jm@jmason.org]
Sent: 11 March 2007 12:32
To: Kelly Jones
Cc: freebsd-questions@freebsd.org; users@spamassassin.apache.org;
linuxusersgroup@googlegroups.com; nmlug@nmlug.org;
nmosug-l@mailman.swcp.com
Subject: Re: Tool for validating sender address as spam-fighting
technique?
for what it's worth, I would suggest *not* adopting this
as an anti-spam technique.
Sender-address verification is _bad_ as an anti-spam technique, in my
opinion. Basically, there's one obvious response for spammers looking
to
evade it -- use "real" sender addresses. Where's an easy place to find
real addresses? On the list of target addresses they're spamming!
Hence, the spam recipients now get twice as much mail from each spam run
-- spam aimed at them, *and* bounce blowback from hundreds of spams
aimed
at others, forged to appear to be from them. It's the obvious response
to
SAV, which is one reason why we never implemented something like that in
SpamAssassin.
--j.
Kelly Jones writes:
> To fight spam, I want to validate the address (not necessarily in
> real-time) of the a given email sender. Is there a Unix tool that does
> this?
>
> The basics are simple: to validate "kmnyqi@wnonline.net", I connect to
> the MX record of wnonline.net and go as far as "RCPT TO" as follows:
>
> > host -t mx wnonline.net
> wnonline.net mail is handled by 5 wnspf.bayou.com.
>
> > telnet wnspf.bayou.com. 25
> Trying 209.209.192.75...
> Connected to wnspf.bayou.com..
> Escape character is '^]'.
> 220 Welcome to Bayou mxfilter
> HELO domaintester.com
> 250 mxfilter.bayou.com
> MAIL FROM: <te...@ignoreme.com>
> 250 Ok
> RCPT TO: <km...@wnonline.net>
> 550 <km...@wnonline.net>: Recipient address rejected: 5.1.1
> <km...@wnonline.net>... User unknown
> QUIT
> 221 Bye
> Connection closed by foreign host.
>
> This tells me kmnyqi@wnonline.net is an invalid address and that mail
> from that address is probably bogus.
>
> A more sophisticated tool would cache results, handle temporary
> failures (eg, inability to connect to the MX server), handle multiple
> MX records, perhaps even publish results [carefully, to avoid giving
> spammers a source of legit email addresses!], etc. Plus, I'd prefer to
> use a tested tool vs hacking something up myself.
>
> I realize this technique is far from perfect:
>
> Spammers spoof legit addresses
>
> Bounces/Mailing lists/etc legitimately use "do not reply" addresses
>
> It could be considered unfriendly to the target MX servers
>
> Some mail servers incorrectly say "user unknown" when they see spam,
> figuring it's more of a deterrent than saying "you're a spammer"
>
> Some mail servers inefficiently accept mail for "foo@xxx.com" (where
> xxx.com is one of their domains), figure out if foo exists later, and
> send a bounce back to the envelope sender, instead of rejecting email
> at the SMTP level (a really good tool would create throwaway addresses
> to catch these cases too)
>
> ... but I still think it might help.
>
> --
> We're just a Bunch Of Regular Guys, a collective group that's trying
> to understand and assimilate technology. We feel that resistance to
> new ideas and technology is unwise and ultimately futile.