You are viewing a plain text version of this content. The canonical link for it is here.
Posted to community@apache.org by Ceki Gulcu <no...@qos.ch> on 2004/04/13 21:00:40 UTC
Mailing lists hiding sender's address?
Hello all,
In recent months, the ratio of spam to legitimate messages ratio has
plummeted to about 1 in 20 for my main mailbox. Consequently, I am
now very aggressively filtering out incoming traffic.
One of the downsides of participating in mailing lists is that your
email address becomes easy prey for spammers. Do we have any mechanism
for hiding or otherwise obfuscating the email address of senders to
our list? Has this topic been discussed previously?
Any pointers would be much appreciated.
--
Ceki Gülcü
For log4j documentation consider "The complete log4j manual"
ISBN: 2970036908 http://www.qos.ch/shop/products/clm_t.jsp
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Mailing lists hiding sender's address?
Posted by Leo Simons <ls...@jicarilla.org>.
Ceki Gülcü wrote:
> Your comments are welcome.
a continuously tweaked spamassassin has about a 99.99% spam catching
success for me and no false positives.
The main pain is when viruses are sent to autoresponders or swamped
relays or whatever, which respond, mailboxes fill up, and in the end I
recieve like 150 "mail delivery failed", "warning, your e-mail contained
a virus", "this is an autoresponder. Please authorize yourself if you
sent me an e-mail titled 'hi' the other day" messages a day which are
near-impossible to seperate from the legitimate mail delivery failures,
which of course happen all the time too, when people's mailboxes fill up
with spam.
It seems to me that all programs that send out automated messages create
more problems than they solve.
--
cheers,
- Leo Simons
-----------------------------------------------------------------------
Weblog -- http://leosimons.com/
Component Community -- http://componentplanet.org/
Component Glue -- http://jicarilla.org/
-----------------------------------------------------------------------
"We started off trying to set up a small anarchist community, but
people wouldn't obey the rules."
-- Alan Bennett
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: Mailing lists hiding sender's address?
Posted by Brian Behlendorf <br...@collab.net>.
On Fri, 16 Apr 2004, Noel J. Bergman wrote:
> > The next step would be to only allow mail from people who are in the
> > ASF "web-of-trust". :-)
>
> Thereby eliminating 99% of the user base? ;-)
It would sure cut down on the bug reports!
Brian
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: Mailing lists hiding sender's address?
Posted by "Noel J. Bergman" <no...@devtech.com>.
> The next step would be to only allow mail from people who are in the
> ASF "web-of-trust". :-)
Thereby eliminating 99% of the user base? ;-)
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Mailing lists hiding sender's address?
Posted by Ask Bjørn Hansen <ah...@gmail.com>.
On Fri, 16 Apr 2004 19:11:54 -0400, Noel J. Bergman <no...@devtech.com> wrote:
> The real problem would be if they started to use widespread spoofing of
> valid subscriber addresses, e.g., sending lots of e-mail as you to lists
> on which you are subscribed.
FWIW, viruses do occasionally mail the lists from a subscribed address...
> The solution is to require digital certificates to authenticate identity.
> All e-mail would need to carry them or be considered spam a priori.
> Stefano uses a digital signature on every message, but since he uses PGP
> MIME instead of S/MIME, my MUA treats it a bit oddly. I rarely bother to
> sign my messages, but signed messages could be verified by the server as
> part of its filtering.
The next step would be to only allow mail from people who are in the
ASF "web-of-trust". :-)
- ask
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: Mailing lists hiding sender's address?
Posted by "Noel J. Bergman" <no...@devtech.com>.
> > I rarely bother to sign my messages
> I notice you signed that one. ;-)
On purpose. :-)
> Signed messages cause problems on gmane
> The problem is that gmane obfuscates the sender itself,
> so that the sender and signer don't match, and you get
> a "Security Warning"
It is an entirely proper warning. If they are going to corrupt the message,
they ought to remove the key, IMO. But I think that using a VERP-like value
for the Sender: header is ill-conceived.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: Mailing lists hiding sender's address?
Posted by Martin Cooper <ma...@apache.org>.
> -----Original Message-----
> From: Noel J. Bergman [mailto:noel@devtech.com]
> Sent: Friday, April 16, 2004 4:12 PM
> To: community@apache.org
> Subject: RE: Mailing lists hiding sender's address?
>
>
> > A little obfuscation goes a long way. Look at our subscription mechanism
> to
> > mailing lists. It is trivially easy to defeat, but as far as I know no
> one
> > has yet subscribed to one of our lists to spam us.
>
> Actually, that part is not so easy because it requires them to have a
> valid mailbox in order to complete the process. Spammers don't like to be
> seen.
>
> The real problem would be if they started to use widespread spoofing of
> valid subscriber addresses, e.g., sending lots of e-mail as you to lists
> on which you are subscribed.
>
> The solution is to require digital certificates to authenticate identity.
> All e-mail would need to carry them or be considered spam a priori.
> Stefano uses a digital signature on every message, but since he uses PGP
> MIME instead of S/MIME, my MUA treats it a bit oddly. I rarely bother to
> sign my messages, but signed messages could be verified by the server as
> part of its filtering.
I notice you signed that one. ;-)
Signed messages cause problems on gmane, at least when using Outlook Express
as a newsreader, and I would guess with other newsreaders as well. The
problem is that gmane obfuscates the sender itself, so that the sender and
signer don't match, and you get a "Security Warning", with a message like
this:
The digital ID's e-mail address does not match sender's
Signer: noel@devtech.com
Sender: noel-3FWFAxrxNblBDgjK7y7TUQ@public.gmane.org
You can still read the message, but the value of the signature is negated.
--
Martin Cooper
> --- Noel
>
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: Mailing lists hiding sender's address?
Posted by "Noel J. Bergman" <no...@devtech.com>.
> Stefano uses a digital signature on every message, but since he uses PGP
> MIME instead of S/MIME, my MUA treats it a bit oddly.
Hmm ... apparently not anymore. Thawte Freemail cert now. :-)
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: Mailing lists hiding sender's address?
Posted by "Noel J. Bergman" <no...@devtech.com>.
> A little obfuscation goes a long way. Look at our subscription mechanism
to
> mailing lists. It is trivially easy to defeat, but as far as I know no
one
> has yet subscribed to one of our lists to spam us.
Actually, that part is not so easy because it requires them to have a
valid mailbox in order to complete the process. Spammers don't like to be
seen.
The real problem would be if they started to use widespread spoofing of
valid subscriber addresses, e.g., sending lots of e-mail as you to lists
on which you are subscribed.
The solution is to require digital certificates to authenticate identity.
All e-mail would need to carry them or be considered spam a priori.
Stefano uses a digital signature on every message, but since he uses PGP
MIME instead of S/MIME, my MUA treats it a bit oddly. I rarely bother to
sign my messages, but signed messages could be verified by the server as
part of its filtering.
--- Noel
Re: Mailing lists hiding sender's address?
Posted by Ceki Gülcü <ce...@qos.ch>.
At 10:24 PM 4/14/2004, you wrote:
>Ceki Gülcü <no...@qos.ch> wrote:
>
>>Your comments are welcome.
>
>If you pick 10 people and ask them, you come up with at least 20 solutions
>for spam and 100 ideas.
>
>I think email obfuscation is just as useless yet appealing as security
>thru obscurity if the amount of email obfuscated is high enough (and
>apache produces tons of it). We can pick n random obfucation methods, but
>this will make the job just n times harder and I don't think we can come
>up with more than, say, 20 meaningful obfuscation methods.
A little obfuscation goes a long way. Look at our subscription mechanism to
mailing lists. It is trivially easy to defeat, but as far as I know no one
has yet subscribed to one of our lists to spam us. Following through with
this logic, we should either abandon mailing lists altogether because they
can be defeated or accept that address obfuscation can help to reduce spam.
>Another idea is to remove the From: header entirely. This way, you don't
>know who sent the email, nor spammers can. But this will totally destroy
>the mail list ecosystem.
>
>Your proposal of using bogus address looks appealing at first, but it
>potentially makes the problem even worse: it might be a worm sending you
>that message, pretending to be me.
>
>Net result: Ceki is now (involuntarely) spamming Stefano.
>
>Note that if this method was institutionalized, you need rebouncing
>prevention (my bogus address spamming your bogus address and ping-pong
>forever).
Infinite loop prevention is already built in the script I sent to list.
>I am strongly against any system that bounces email and my 300
>spam/bounces messages a day prove why [ and I can't estimate how many
>don't even reach my inbox due to @apache.org prefiltering ]
The procmail script I sent can be used without the collaboration of list
moderators let alone apmail.
>--
>Stefano.
--
Ceki Gülcü
For log4j documentation consider "The complete log4j manual"
ISBN: 2970036908 http://www.qos.ch/shop/products/clm_t.jsp
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Mailing lists hiding sender's address?
Posted by Stefano Mazzocchi <st...@apache.org>.
Ceki Gülcü <no...@qos.ch> wrote:
> Your comments are welcome.
If you pick 10 people and ask them, you come up with at least 20
solutions for spam and 100 ideas.
I think email obfuscation is just as useless yet appealing as security
thru obscurity if the amount of email obfuscated is high enough (and
apache produces tons of it). We can pick n random obfucation methods,
but this will make the job just n times harder and I don't think we can
come up with more than, say, 20 meaningful obfuscation methods.
Another idea is to remove the From: header entirely. This way, you don't
know who sent the email, nor spammers can. But this will totally destroy
the mail list ecosystem.
Your proposal of using bogus address looks appealing at first, but it
potentially makes the problem even worse: it might be a worm sending you
that message, pretending to be me.
Net result: Ceki is now (involuntarely) spamming Stefano.
Note that if this method was institutionalized, you need rebouncing
prevention (my bogus address spamming your bogus address and ping-pong
forever).
I am strongly against any system that bounces email and my 300
spam/bounces messages a day prove why [ and I can't estimate how many
don't even reach my inbox due to @apache.org prefiltering ]
--
Stefano.
RE: Mailing lists hiding sender's address?
Posted by Martin Cooper <ma...@apache.org>.
On Wed, 14 Apr 2004, Noel J. Bergman wrote:
> > By obfuscation I obviously mean a transformation which is easy for a
> > human to decipher but difficult for a machine.
>
> I understood.
>
> Based upon testimony from Ron Scelson, I believe that the spam industry has
> grown to about US$1 billion per year. How much effort do you think is being
> invested if there is that much money at stake? FWIW, the anti-SPAM industry
> also huge. There is a 2 - 3 US$Billon vested interest in spam.
>
> > So instead of writing an email address such as jones@foo.bar.com as
> > is, one can write
>
> > 1) j o n e s @ f o o . b a r . c o m (note the extra spaces)
> > 2) jones at foo . bar . com (read 'at' as '@')
> > 3) jones AT foo DOT bar DOT com (read 'AT' as '@' and DOT as '.')
>
> This are commonly done and trivially recognizable by a spider.
>
> Remember the amount of money at stake. If you had no morals, how much would
> you need to be paid to write a parser that would parse text and pick out
> potentially valid e-mail addresses? There is so much money involved in spam
> that it now seems to have evolved into organized crime.
I have to agree with Noel here. I used to work for a company that sells
anti-spam (or more generally, anti-mail-problem) software. They have a
team of people in Eastern Europe who spend their lives analysing all of
the latest spam, using many different techniques, and providing
sometimes-daily updates to the software, with new schemes to catch the new
schemes the spammers come up with. It seems clear to me that the spammers
must have parallel teams of people who spend their lives coming up with
new ways to circumvent the mechanisms that foiled their old schemes, and
so on and so on.
I'd say that any obfuscation scheme that would be easily interpreted by a
human will not take long for spammers to work around, as soon as it
becomes worth their while. This is especially true for text-based
obfuscation, but even the use of images (e.g. GoDaddy's mechanism to
protect domain registration details) is no longer enough to foil the
determined spammers out there.
--
Martin Cooper
> see: http://spam.weblogsinc.com/entry/5067632429786165/
> http://www.accountancyage.com/News/1135797
> http://www.detnews.com/2004/technology/0402/14/technology-63815.htm
>
> > 4) user=jones, domain=foo.bar.com
> > 5) j|o|n|e|s||foo|bar|com (read '||' as '@', remove '|' in the user
> > part and replace '|' as '.' in the domain part)
>
> Those would be less likely to be recognized, unless they became common. Any
> commonly used pattern would be added to spiders.
>
> > Thus, we can allow users to respond directly to the sender of a post
> > but still make it hard for spammers to mass-collect poster email
> > addresses.
>
> You mean that the user will have to correct every reply address by hand?
> That would just annoy people, and encourage them to participate elsewhere.
> And any pattern that is common enough to be supported by an MUA would be
> added to the next generation of spiders.
>
> > In a more elaborate variation of this theme, one subscribes to a
> > mailing list under the regular email address but post under a fake
> > address such as "jones-autoreply@foo.bar.com".
>
> > Now, if any person tries to write to the visible address
> > "autoreply-jones@foo.bar.com", they get a reply such as
> > ...
> > please redirect your message to:
> > j o n e s @ f o o . b a r . c o m (note the extra spaces)
> > ...
>
> See www.tmda.net. There is already a TMDA mechanism that works by bouncing
> the firsrt post from a sender and requiring manual response. Spammers can
> handle them, too, if they want, although the theory is that spammers don't
> leave mailboxes around for long to receive and process such bounces. On the
> other hand, a lot of people receive TMDA bounces and just say that it isn't
> worth communicating with the person.
>
> Such strategies are more bothersome for legitimate mail users than spammers.
>
> --- Noel
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: community-unsubscribe@apache.org
> For additional commands, e-mail: community-help@apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: Mailing lists hiding sender's address?
Posted by "Noel J. Bergman" <no...@devtech.com>.
> By obfuscation I obviously mean a transformation which is easy for a
> human to decipher but difficult for a machine.
I understood.
Based upon testimony from Ron Scelson, I believe that the spam industry has
grown to about US$1 billion per year. How much effort do you think is being
invested if there is that much money at stake? FWIW, the anti-SPAM industry
also huge. There is a 2 - 3 US$Billon vested interest in spam.
> So instead of writing an email address such as jones@foo.bar.com as
> is, one can write
> 1) j o n e s @ f o o . b a r . c o m (note the extra spaces)
> 2) jones at foo . bar . com (read 'at' as '@')
> 3) jones AT foo DOT bar DOT com (read 'AT' as '@' and DOT as '.')
This are commonly done and trivially recognizable by a spider.
Remember the amount of money at stake. If you had no morals, how much would
you need to be paid to write a parser that would parse text and pick out
potentially valid e-mail addresses? There is so much money involved in spam
that it now seems to have evolved into organized crime.
see: http://spam.weblogsinc.com/entry/5067632429786165/
http://www.accountancyage.com/News/1135797
http://www.detnews.com/2004/technology/0402/14/technology-63815.htm
> 4) user=jones, domain=foo.bar.com
> 5) j|o|n|e|s||foo|bar|com (read '||' as '@', remove '|' in the user
> part and replace '|' as '.' in the domain part)
Those would be less likely to be recognized, unless they became common. Any
commonly used pattern would be added to spiders.
> Thus, we can allow users to respond directly to the sender of a post
> but still make it hard for spammers to mass-collect poster email
> addresses.
You mean that the user will have to correct every reply address by hand?
That would just annoy people, and encourage them to participate elsewhere.
And any pattern that is common enough to be supported by an MUA would be
added to the next generation of spiders.
> In a more elaborate variation of this theme, one subscribes to a
> mailing list under the regular email address but post under a fake
> address such as "jones-autoreply@foo.bar.com".
> Now, if any person tries to write to the visible address
> "autoreply-jones@foo.bar.com", they get a reply such as
> ...
> please redirect your message to:
> j o n e s @ f o o . b a r . c o m (note the extra spaces)
> ...
See www.tmda.net. There is already a TMDA mechanism that works by bouncing
the firsrt post from a sender and requiring manual response. Spammers can
handle them, too, if they want, although the theory is that spammers don't
leave mailboxes around for long to receive and process such bounces. On the
other hand, a lot of people receive TMDA bounces and just say that it isn't
worth communicating with the person.
Such strategies are more bothersome for legitimate mail users than spammers.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: Mailing lists hiding sender's address?
Posted by Ceki Gülcü <no...@qos.ch>.
At 05:25 PM 4/14/2004, you wrote:
> > Do we have any mechanism for hiding or otherwise obfuscating
> > the email address of senders to our list?
>
>No. The raw mbox archives show everything, as do the eyebrowse archives.
Thanks for the info.
>So why not obfuscate? Even if we were to do so, any spammer could subscribe
>to our lists, and simply harvest sender addresses, unless we hide them and
>enforce reply-to-list behavior. So at least one question is whether the
>effort is worth the payoff.
>
> --- Noel
By obfuscation I obviously mean a transformation which is easy for a
human to decipher but difficult for a machine.
Two things that machines are unable to do as well as human beings are
image recognition and language understanding.
So instead of writing an email address such as jones@foo.bar.com as
is, one can write
1) j o n e s @ f o o . b a r . c o m (note the extra spaces)
2) jones at foo . bar . com (read 'at' as '@')
3) jones AT foo DOT bar DOT com (read 'AT' as '@' and DOT as '.')
4) user=jones, domain=foo.bar.com
5) j|o|n|e|s||foo|bar|com (read '||' as '@', remove '|' in the user
part and replace '|' as '.' in the domain part)
The number of variations are quite high.
Thus, we can allow users to respond directly to the sender of a post
but still make it hard for spammers to mass-collect poster email
addresses.
In a more elaborate variation of this theme, one subscribes to a
mailing list under the regular email address but post under a fake
address such as "jones-autoreply@foo.bar.com". This is possible if
the autoreply address is added as an allowed alias to the list. The
list moderator can do this, so can the user herself! See 2.4 Adding
subscriber aliases in the EZLM manual at
http://www.ezmlm.org/ezman-0.32/ezman2.html
Now, if any person tries to write to the visible address
"autoreply-jones@foo.bar.com", they get a reply such as
Hello,
This is an automatic email responder acting on behalf of Mr. Jones.
You have recently sent an email to one of his autoreplying email
addresses. If you are a real person trying to reach Mr. Jones, then
please redirect your message to:
j o n e s @ f o o . b a r . c o m (note the extra spaces)
The above address is the only one that is valid. His other email
addresses receive way too much rubbish forcing him to resort to this
rather aggressive filtering method. If you are a real person trying to
conduct legitimate correspondence, please accept our apologies for
this impersonal response. Thank you for your understanding.
A copy of your original message is included below:
*********************************************************************
COPY OF ORIGINAL MESSAGE
I have implemented such an autoresponder using procmail. I am
including it for your convenience.
# Start .procmail file for jones-autoreply@foo.bar.com
# ****** WARNING * WARNING * WARNING * WARNING *
## I am not really experienced with procmail. This works for me
## but maybe buggy, or even very buggy.
PMDIR=$HOME/Procmail
LOGFILE=$PMDIR/pmlog
# To insert a blank line between each message's log entry,
# uncomment next two lines (this is helpful for debugging)
LOG="
"
VERBOSE=yes
DATE=`date +%Y-%m-%d_%H_%M_%S`
TMP_RESP=$PMDIR/temp/response-$DATE
# Reply to sender of message to "autoreply" stating that
# Include original message watch for user loops
:0 w
* ^TO_.*jones-autoreply@foo.bar.com
* !^X-Loop: qos.ch
* !^FROM_DAEMON
{
# Start a Redirection lock
LOCKFILE = $PMDIR/redirect.lock
:0 chw
| formail -r -A"Precedence: junk" -A"X-Loop: foo.bar" -A"From:
bounce@foo.bar.com"> $TMP_RESP; cat $HOME/.redirect.msg >> $TMP_RESP;
:0 cw
| formail -k -X"To:" -X"Subject:" -X"From:" -X"Date:"|head -512 >>
$TMP_RESP
:0 hw
| cat > /dev/null; /usr/sbin/sendmail -oi -t < $TMP_RESP
# Release the lock
LOCKFILE
}
# On mails coming from the mail daemon, check if it
# is a failed reply to a previous redirect message
# Most spammers use invalid email addresses...
:0
* ^FROM_DAEMON
{
# Check the body
:0B
* ^X-Loop: foo\.bar\.com
* ^Precedence: junk
* ^A copy of your original message is included below:
/dev/null
}
# End of .procmail file
In case you cannot add a fake Unix user, ask your system admin to
create an alias that redirects to your real email address. In your
.procmailrc you can check whether the message is for you or for your
autoreply alias. If it is for your autoreply alias, just reply as
shown above.
In summary, the above technique should work provided that you can use
procmail and have an autoreply alias redirected to your real account.
There are redirection services which do just that. Thus, I think all
you need is procmail.
I am using the technique described here to send this message.
Using the autoresponder technique described here, one can remain
invisible to spammers while still be able to conduct legitimate
correspondence.
Your comments are welcome.
--
Ceki Gülcü
For log4j documentation consider "The complete log4j manual"
ISBN: 2970036908 http://www.qos.ch/shop/products/clm_t.jsp
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
Re: Mailing lists hiding sender's address?
Posted by Aaron Bannert <aa...@clove.org>.
I agree with what Noel's saying, and I think it is probably not
worth the effort. Also consider that our mailing lists are being
freely archived and web-served from independent offsite
services, like MARC, and we'd have no control over any
obfuscation that these services may or may not provide.
-aaron
On Apr 14, 2004, at 8:25 AM, Noel J. Bergman wrote:
>> Do we have any mechanism for hiding or otherwise obfuscating
>> the email address of senders to our list?
>
> No. The raw mbox archives show everything, as do the eyebrowse
> archives.
>
> So why not obfuscate? Even if we were to do so, any spammer could
> subscribe
> to our lists, and simply harvest sender addresses, unless we hide them
> and
> enforce reply-to-list behavior. So at least one question is whether
> the
> effort is worth the payoff.
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org
RE: Mailing lists hiding sender's address?
Posted by "Noel J. Bergman" <no...@devtech.com>.
> Do we have any mechanism for hiding or otherwise obfuscating
> the email address of senders to our list?
No. The raw mbox archives show everything, as do the eyebrowse archives.
So why not obfuscate? Even if we were to do so, any spammer could subscribe
to our lists, and simply harvest sender addresses, unless we hide them and
enforce reply-to-list behavior. So at least one question is whether the
effort is worth the payoff.
--- Noel
---------------------------------------------------------------------
To unsubscribe, e-mail: community-unsubscribe@apache.org
For additional commands, e-mail: community-help@apache.org