You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Christian Grunfeld <ch...@gmail.com> on 2011/11/23 18:55:46 UTC
new paradigm
Hi,
I have an idea to discuss here with experts !
What is the main MAIN difference between spam and ham ?
...
...
Answer: spam is "one way ticket" and ham is 99.99% "round trip" !
(legit notifications can be "one way ticket" but you can mark them as
ham later)
What do I mean? you never never answer (or it is really strange) a
spam message. Average users, who someone said that are stupid and more
stupid when they are in front of a machine, also dont respond to a
spammy message. At least if they are marked as spam.
So the idea is...in this days where the ratio of spam/ham is about 80%
(put the ratio you want but be sure it is high enough) lets start with
marking all incomings as spam !
Past days when the ratio of spam/ham was 5% or 10% it was quite logic
that the reverse was true. That is, all incomings were ham and we
tried with a lot of methods to extract or mark the bad emails!
We spent 15 years (up to now) with the Presumption of innocence
analogy of "Everyone charged with a criminal offence shall be presumed
innocent until proved guilty according to law". This approach is
wasting a lot of resources because of the high spam/ham ratio!
Nowdays its easier to invert the logic!
*mark all incomings as spam the first time
*check spam folder always
*mark as ham....or (here is the relationship with the first question)
...just answer emails to the people you allways comunicates as you
always did. Here you round the trip and legitimate the sender !
For this we need a modified version of SA autowhitelist not based on
scores but on trusted or answered emails !
Flaws ?
False positives....yes, ONLY the first time for each sender! just
answer your good mails and they´ll become ham next time. Mails not
answered (spam) remains as spam next and next and next !
False negatives...yes, if someone impersonates in the From: as someone
trusted by you (phising). But this could be reduced using the same
methods as autowhitelist uses keeping in a DB pairs of senders - IPs.
Greylists also uses DBs like this.
So, what do we have to waste resources on tons of rules, tons of perl
code, tons of regex if we know that 80% is spam? lets mark all of them
as spam and let this method work!
Time to think in a new antispam paradigm !
Cheers
Re: new paradigm
Posted by da...@chaosreigns.com.
Many people have spent many nights lying awake trying to figure out what
to do about spam. The the extent that when a person believes they have
come up with an idea that is both new and useful, they are usually wrong.
This results in some hostile attitudes toward new ideas - I have certainly
felt that hostility myself enough times.
You may find this entertaining:
http://www.rhyolite.com/anti-spam/you-might-be.html
This mailing list exists for discussing the overall problem of spam:
http://irtf.org/mailman/listinfo/asrg
On 11/23, Christian Grunfeld wrote:
> *check spam folder always
This is your biggest problem. Most people aren't willing to spend their
time reading all their spam to make sure it's all actually spam. That's
kind of the whole point of spam filters.
> False positives....yes, ONLY the first time for each sender! just
I've spent years helping with dnswl.org, trying to list all legitimate
sender IPs, and it still gets false positives (and false negatives).
I've recently started a personal project to assemble similar data in a
different fashion,
http://www.chaosreigns.com/iprep/
The problem is, while most people might get email from most spammers, most
people only get mail from a small percentage of non-spammer mail servers.
There are many of them. Tiny companies and personal servers all over the
world, that you will *never* get an email from.
So even if you have a large group of people trying to collectively come up
with a list of places that don't send spam, it's still hard, you will still
get some false positives.
Do some analysis on your own email to see it for yourself.
> So, what do we have to waste resources on tons of rules, tons of perl
> code, tons of regex if we know that 80% is spam? lets mark all of them
> as spam and let this method work!
I'd love to develop a public whitelist thorough enough that that could
become even a remote consideration. It's not going to work on an
individual user level.
By all means, please prove me wrong. Come up with a clean implementation
for spamassassin, submit it as a patch (against trunk) via
https://issues.apache.org/SpamAssassin/
On 11/23, Christian Grunfeld wrote:
> > Well, if I have to do *that*, I might as well not do any filtering at all.
> > The whole purpose of anti-spam software is to shield me from spam.
>
> Not 100% correct. Now I always check spam folder, dont you?
You and I and David are not the issue. Our use is not representative of a
statistically significant percentage of users.
> Do you advise your people not to check spam folders? Are you 100% sure
> that machines can sort 100% efectively what is spam and what is not?
What is advised and what can be expected are not the same.
On 11/23, Christian Grunfeld wrote:
> let people who wants spam to answer spam ! if you dont want spam dont
> reply. Easy !
No. The problem is everyone else who is still providing a profit to
spammers. Not easy to fix.
> There are a lot of people who wants to sell viagra and send
> spam....but I dont answer to them, dou you? :p
Again, the behavior of people on this list is insignificant compared to the
behavior of average users.
> > Many users do this only sporadically, if they do. Some users don't know
> > where to find the spam folder. Some organizations do not deploy per-user
> > quarantine area or spam folders. Etc. etc.
>
> bad thing !
Sure. Fix it.
> Just answering with your MUA. The email goes back through the MTA.
> Then it can put the sender in the whitelist.
That's not something that's particularly well implemented.
--
"Every man, woman and child on the face of this earth is at the mercy
of chaos." - a maxwell smart movie
http://www.ChaosReigns.com
Re: new paradigm
Posted by Michael Scheidell <mi...@secnap.com>.
On 11/24/11 3:16 AM, Martin Gregorie wrote:
> - you need to maintain a database containing every address
> you ever received mail from and have sent mail to. All addresses
> must be recorded as you receive mail from them and updated to record
> when you send mail to them. You could delete addresses that you
> haven't replied to for, say, a month but that is about all you can
> delete.
>
sounds like amavisd-new 'penpals'.
(sliding credit score starting at -100, counting down to 0 for your time
period..).
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation
* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.spammertrap.com/
______________________________________________________________________
Re: new paradigm
Posted by Martin Gregorie <ma...@gregorie.org>.
On Wed, 2011-11-23 at 14:55 -0300, Christian Grunfeld wrote:
> For this we need a modified version of SA autowhitelist not based on
> scores but on trusted or answered emails !
>
This can work well, BUT:
- you need to maintain a database containing every address
you ever received mail from and have sent mail to. All addresses
must be recorded as you receive mail from them and updated to record
when you send mail to them. You could delete addresses that you
haven't replied to for, say, a month but that is about all you can
delete.
- marking first-time messages as spam won't work for a business and
probably won't work for a private mail user.
- this doesn't work at all for mailing lists that get spammed from time
to time, especially if the spam comes from an online forum (hint:
the same sender address tends to be used for all forum posters).
How do I know? I run a mail archiving system which provides exactly this
service (automatic whitelisting of addresses I've sent mail to) to my
copy of SA via a plug-in. The points I listed reflect my experience.
Martin
Re: new paradigm
Posted by RW <rw...@googlemail.com>.
On Wed, 23 Nov 2011 14:55:46 -0300
Christian Grunfeld wrote:
> Nowdays its easier to invert the logic!
> *mark all incomings as spam the first time
> *check spam folder always
> *mark as ham....or (here is the relationship with the first question)
> ...just answer emails to the people you allways comunicates as you
> always did. Here you round the trip and legitimate the sender !
This has (more or less) been available from Hotmail and some other
ESPs for years, so it's scarcely a new paradigm. That's really where it
belongs because it's only really suitable for someone that keeps an
account for communicating with a stable list of friends and family.
Nancy McGough (www.ii.com) has a much more practical version of this.
She has an inbox for whitelisted mail, and a second SA filtered inbox
which she examines for things to whitelist. This seems to be a sensible
approach for people that get a huge amount of spam.
Re: new paradigm
Posted by Bernd Petrovitsch <be...@petrovitsch.priv.at>.
On Mit, 2011-11-23 at 14:55 -0300, Christian Grunfeld wrote:
[....]
> Flaws ?
> False positives....yes, ONLY the first time for each sender! just
> answer your good mails and they´ll become ham next time. Mails not
> answered (spam) remains as spam next and next and next !
1) That might look negligible for quite closed groups of people but if
you are on e.g. the Linux-kernel-mailinglist and some others near it,
it's quite different. After all, Cc: the LKML is mostly for the ML
archive and that Google can find it.
2) The next drawback is that some widely used MUAs do not show the email
address per default but only the comment near it (which should be the
real name but is technically even more comment than rest of the From:
field).
3) And I skim over the spam-folder once a week or so. Skimming spam (at
least) daily to actively look for ham is much more effort than just
making sure that an important (business or private) mail didn't got in
there (because of the sum all the usual little sins of the typical
Outlook-user).
4) And how does that - done in my MUA - integrate with SpamAssassin and
the like on the mail server?
evolution has nice buttons since ages. Does anyone know how "export" the
junk - aka spam - (and explicit not-junk - aka ham -) automatically to
the BayesDB for SA?
[....]
> Time to think in a new antispam paradigm !
The other problem is that probably many people don't like that everyone
thinks that they are spamming (even if it is true) unless the receiver
declares it ham.
I have no problem with that - actually it the way in real life with the
paper ads at the door and in the snailmailbox.
But I'm sure you are already trying and living by that witz email, no?
Bernd
--
Bernd Petrovitsch Email : bernd@petrovitsch.priv.at
LUGA : http://www.luga.at
Re: new paradigm
Posted by sp...@lists.grepular.com.
On 24/11/11 13:18, Lucio Chiappetti wrote:
>> If a message comes in to my MTA with one of those Message-Id's in the
>> "In-Reply-To" header, it bypasses the spam filtering because it is a
>> response to a message that I sent
>
> what about if your message was stored in a folder of your correspondent,
> his machine is infected by a virus, and this virus sends fake replies
> using your message id ? I've seen cases like that in the past.
That has never happened to me.
--
Mike Cardwell https://grepular.com/ https://twitter.com/mickeyc
Professional http://cardwellit.com/ http://linkedin.com/in/mikecardwell
PGP.mit.edu 0018461F/35BC AF1D 3AA2 1F84 3DC3 B0CF 70A5 F512 0018 461F
Re: new paradigm
Posted by Michael Scheidell <mi...@secnap.com>.
On 11/24/11 8:18 AM, Lucio Chiappetti wrote:
> On Wed, 23 Nov 2011, spamassassin@lists.grepular.com wrote:
>
>> If a message comes in to my MTA with one of those Message-Id's in the
>> "In-Reply-To" header, it bypasses the spam filtering because it is a
>> response to a message that I sent
again, sounds like amavisd-new penpals.
>
> what about if your message was stored in a folder of your
> correspondent, his machine is infected by a virus, and this virus
> sends fake replies using your message id ? I've seen cases like that
> in the past.
you can't whitelist a virus in amavisd-new.
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation
* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.spammertrap.com/
______________________________________________________________________
Re: new paradigm
Posted by Lucio Chiappetti <lu...@lambrate.inaf.it>.
On Wed, 23 Nov 2011, spamassassin@lists.grepular.com wrote:
> If a message comes in to my MTA with one of those Message-Id's in the
> "In-Reply-To" header, it bypasses the spam filtering because it is a
> response to a message that I sent
what about if your message was stored in a folder of your correspondent,
his machine is infected by a virus, and this virus sends fake replies
using your message id ? I've seen cases like that in the past.
Re: new paradigm
Posted by sp...@lists.grepular.com.
On 23/11/11 17:55, Christian Grunfeld wrote:
> What do I mean? you never never answer (or it is really strange) a
> spam message.
On my personal email system, my MSA records Message-Id's of outgoing
mail into a database. If a message comes in to my MTA with one of those
Message-Id's in the "In-Reply-To" header, it bypasses the spam filtering
because it is a response to a message that I sent, so clearly shouldn't
be filtered.
--
Mike Cardwell https://grepular.com/ https://twitter.com/mickeyc
Professional http://cardwellit.com/ http://linkedin.com/in/mikecardwell
PGP.mit.edu 0018461F/35BC AF1D 3AA2 1F84 3DC3 B0CF 70A5 F512 0018 461F
Re: new paradigm
Posted by Dave Warren <li...@hireahit.com>.
On 11/24/2011 6:04 AM, Christian Grunfeld wrote:
>> So you're suggesting that users review 2700-3000 spam messages messages/day
>> (depending on how many were already whitelisted) to look for some of those
>> 300?
> may be you are thinking about that volume per user, not the case!
> I have 200-300 users so...1 over 10 ham/spam per user!
I understand that you're not suggesting that each user filters 3000 spam
messages per day, but across your entire user base, you are. That's a
huge burden when competently run spam filters shouldn't be leaving you
with 10 spam for every 1 ham (and yes, I realize that you're planning on
delivering more of that ham to the user; that arguably makes it worse
since users have to review more spam for each remaining ham)
Human brain power costs a ton more than computing power, and is a heck
of a lot more error-prone when skimming over sender and subject lines.
--
Dave Warren, CEO
Hire A Hit Consulting Services
http://ca.linkedin.com/in/davejwarren
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
> So you're suggesting that users review 2700-3000 spam messages messages/day
> (depending on how many were already whitelisted) to look for some of those
> 300?
may be you are thinking about that volume per user, not the case!
I have 200-300 users so...1 over 10 ham/spam per user!
Re: new paradigm
Posted by Dave Warren <li...@hireahit.com>.
On 11/23/2011 7:01 PM, Christian Grunfeld wrote:
>> Define "bypass first level"? Are you suggesting that for every 1 ham you
>> deliver, you deliver 10 spams into user's mailboxes? Or do you do further
>> filtering?
> I defined it in the part you did not quote!
> First level, MTA level: check helo, sender domain, IP<-> name maps
> and also greylists !
It wasn't clear if that's what you're referring to as "first level",
personally I'd throw DNSBLs and just about anything else that happens
pre-DATA into "first level"
I run various content scanning (including SpamAssasssin) at the MTA
level myself.
> These reduce 30000 incomings to 3000. Then these 3000 goes trough
> other filters: SA, antivirus, etc. 300 of these 3000 are really ham,
> the other 2700 are spam. I did not say that SA efectively selects the
> 300 and the 2700 ! if that was true we are not discussing in this list
> !
So you're suggesting that users review 2700-3000 spam messages
messages/day (depending on how many were already whitelisted) to look
for some of those 300?
--
Dave Warren, CEO
Hire A Hit Consulting Services
http://ca.linkedin.com/in/davejwarren
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
> Define "bypass first level"? Are you suggesting that for every 1 ham you
> deliver, you deliver 10 spams into user's mailboxes? Or do you do further
> filtering?
I defined it in the part you did not quote!
First level, MTA level: check helo, sender domain, IP <-> name maps
and also greylists !
These reduce 30000 incomings to 3000. Then these 3000 goes trough
other filters: SA, antivirus, etc. 300 of these 3000 are really ham,
the other 2700 are spam. I did not say that SA efectively selects the
300 and the 2700 ! if that was true we are not discussing in this list
!
what I tried to say is that after the first MTA checks I get the % I
told from the begining. 300 is 10% (ham) of 3000 and 2700 is 90%
(spam) of 3000. That is the volume that really goes to filtering with
SA. So if I have 90% chance of spam why not inverting the logic to
solve the puzzle instead of trying to solve it with the same logic
when the %s were the oposite?
This is the idea I brought to the list. It is just an idea. I ve never
said I will solve the FUSSP !
I feel attacked when you say that in a quite pedantic/ironic way!
> I suspect that for the type of filtering you're talking about, the only
> numbers that matter are the amount of ham you currently deliver into user
> mailboxes and the amount of spam you currently deliver. What stage in your
> spam filtering catches messages (if you have multiple stages) isn't
> particularly relevant, all that matters is what the user sees.
answered
Re: new paradigm
Posted by Dave Warren <li...@hireahit.com>.
On 11/23/2011 4:56 PM, Christian Grunfeld wrote:
> I still have much more spam than ham after all first checks.
> I have something like 30000 total per day. 3000 bypass first level
> tests and of those 3000, 300 are ham.
Define "bypass first level"? Are you suggesting that for every 1 ham you
deliver, you deliver 10 spams into user's mailboxes? Or do you do
further filtering?
I suspect that for the type of filtering you're talking about, the only
numbers that matter are the amount of ham you currently deliver into
user mailboxes and the amount of spam you currently deliver. What stage
in your spam filtering catches messages (if you have multiple stages)
isn't particularly relevant, all that matters is what the user sees.
--
Dave Warren, CEO
Hire A Hit Consulting Services
http://ca.linkedin.com/in/davejwarren
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
2011/11/23 Henrik K <he...@hege.li>:
> 85% of incoming is extremely simple to block with MTA rules (zen, helo,
> dynamic etc). And no FPs to mention. You don't need to count this crap in
> anything.
completely agree on that! I check helo, sender domains, IP <-> names
maps and greylists
> 12% of incoming is real ham.
>
> 3% of incoming is harder to catch spam.
disagree on that! I still have much more spam than ham after all first checks.
I have something like 30000 total per day. 3000 bypass first level
tests and of those 3000, 300 are ham.
Re: new paradigm
Posted by Henrik K <he...@hege.li>.
On Wed, Nov 23, 2011 at 02:55:46PM -0300, Christian Grunfeld wrote:
>
> So the idea is...in this days where the ratio of spam/ham is about 80%
> (put the ratio you want but be sure it is high enough) lets start with
> marking all incomings as spam !
Maybe you are trolling but whatever..
85% of incoming is extremely simple to block with MTA rules (zen, helo,
dynamic etc). And no FPs to mention. You don't need to count this crap in
anything.
12% of incoming is real ham.
3% of incoming is harder to catch spam.
Adjust the ratio as you please, but here that's about right.
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
>> If your assumption was true, there was no spam today. If nobody would ever
>> answer to spam messages, there was no reason for spammers to keep spamming.
your assumption is not correct ! Spammers are not there because all
the people answer them ! They are there and send HUGE volumes of mails
because only a few % of people fall into the trap and answer !
The other big % of people can be benefited of a method like this
Re: new paradigm
Posted by David B Funk <db...@engineering.uiowa.edu>.
On Wed, 23 Nov 2011, Christian Grunfeld wrote:
>> If your assumption was true, there was no spam today. If nobody would ever
>> answer to spam messages, there was no reason for spammers to keep spamming.
>
> let people who wants spam to answer spam ! if you dont want spam dont
> reply. Easy !
> There are a lot of people who wants to sell viagra and send
> spam....but I dont answer to them, dou you? :p
Significant number of non-savvy users who reply: "Take me off your D*MN
list" will defeat your paradigm
>>> *check spam folder always
>>
>> Many users do this only sporadically, if they do. Some users don't know
>> where to find the spam folder. Some organizations do not deploy per-user
>> quarantine area or spam folders. Etc. etc.
>
> bad thing !
True, but if I need to read thru -all- my spam to make sure I don't miss
that one robo-mail from the airline about a problem with my on-line ticket
purchase, what's the point?
>>> *mark as ham....
>>
>> What mechanism do you propose to have the MUA tell the MTA/MDA that
>> something is not spam? Also take into account the tens or maybe even
>> hundreds of different MUA's around (thick clients, webmail clients,
>> applications etc. etc.) which need to be modified to support your idea...
>
> Just answering with your MUA. The email goes back through the MTA.
> Then it can put the sender in the whitelist.
OK, and how do you deal with the situation where the reply comes from a
different address than the original message? (EG: I send a message to
"inqueries@company.com" and the reply comes from "customer-support@company.com").
>>> Flaws ?
>>
>> Yes, many. Think of the automatic out-of-office replies, think of all
>> messages that are sent from noreply@ addresses these days (where the
>> originating organization tries to make clear by naming it 'noreply@' that
>> replies are not welcome), think of (solicited) newsletters, mailing lists
>> etc. etc.
>
> automatic replies are those ! REPLIES to a mail sent by you ! thats
> round trip ! this work best with people you used to comunicate with !
> that is the idea !
What about all those replies that result from some activity -other- than
an initial e-mail? (EG I use a webform to sign up for a mail-list and they
send me a reply that contains a confirmation link).
--
Dave Funk University of Iowa
<dbfunk (at) engineering.uiowa.edu> College of Engineering
319/335-5751 FAX: 319/384-0549 1256 Seamans Center
Sys_admin/Postmaster/cell_admin Iowa City, IA 52242-1527
#include <std_disclaimer.h>
Better is not better, 'standard' is better. B{
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
> If your assumption was true, there was no spam today. If nobody would ever
> answer to spam messages, there was no reason for spammers to keep spamming.
let people who wants spam to answer spam ! if you dont want spam dont
reply. Easy !
There are a lot of people who wants to sell viagra and send
spam....but I dont answer to them, dou you? :p
>> *check spam folder always
>
> Many users do this only sporadically, if they do. Some users don't know
> where to find the spam folder. Some organizations do not deploy per-user
> quarantine area or spam folders. Etc. etc.
bad thing !
>> *mark as ham....
>
> What mechanism do you propose to have the MUA tell the MTA/MDA that
> something is not spam? Also take into account the tens or maybe even
> hundreds of different MUA's around (thick clients, webmail clients,
> applications etc. etc.) which need to be modified to support your idea...
Just answering with your MUA. The email goes back through the MTA.
Then it can put the sender in the whitelist.
>> Flaws ?
>
> Yes, many. Think of the automatic out-of-office replies, think of all
> messages that are sent from noreply@ addresses these days (where the
> originating organization tries to make clear by naming it 'noreply@' that
> replies are not welcome), think of (solicited) newsletters, mailing lists
> etc. etc.
automatic replies are those ! REPLIES to a mail sent by you ! thats
round trip ! this work best with people you used to comunicate with !
that is the idea !
> Yes, why do you think the world spends so much resources on the spam problem
> if the solution would be so easy to implement...?
the world is using the same solution for a problem which changed a lot
in a decade! I dont know...just wondering/asking
Re: new paradigm
Posted by "Rolf E. Sonneveld" <R....@sonnection.nl>.
On 11/23/11 6:55 PM, Christian Grunfeld wrote:
> Hi,
>
> I have an idea to discuss here with experts !
>
> What is the main MAIN difference between spam and ham ?
> ...
> ...
> Answer: spam is "one way ticket" and ham is 99.99% "round trip" !
What research can you cite for these figures? I beg to differ. Think of
all the ticketing systems, confirmation tickets by e-mail,
invoices-by-email, mail to info@ addresses etc. Do you really, really
propose to blacklist / mark as spam them all, first time?
> (legit notifications can be "one way ticket" but you can mark them as
> ham later)
>
> What do I mean? you never never answer (or it is really strange) a
> spam message. Average users, who someone said that are stupid and more
> stupid when they are in front of a machine, also dont respond to a
> spammy message. At least if they are marked as spam.
If your assumption was true, there was no spam today. If nobody would
ever answer to spam messages, there was no reason for spammers to keep
spamming.
> So the idea is...in this days where the ratio of spam/ham is about 80%
> (put the ratio you want but be sure it is high enough) lets start with
> marking all incomings as spam !
>
> Past days when the ratio of spam/ham was 5% or 10% it was quite logic
> that the reverse was true. That is, all incomings were ham and we
> tried with a lot of methods to extract or mark the bad emails!
> We spent 15 years (up to now) with the Presumption of innocence
> analogy of "Everyone charged with a criminal offence shall be presumed
> innocent until proved guilty according to law". This approach is
> wasting a lot of resources because of the high spam/ham ratio!
>
> Nowdays its easier to invert the logic!
> *mark all incomings as spam the first time
> *check spam folder always
Many users do this only sporadically, if they do. Some users don't know
where to find the spam folder. Some organizations do not deploy per-user
quarantine area or spam folders. Etc. etc.
> *mark as ham....
What mechanism do you propose to have the MUA tell the MTA/MDA that
something is not spam? Also take into account the tens or maybe even
hundreds of different MUA's around (thick clients, webmail clients,
applications etc. etc.) which need to be modified to support your idea...
> or (here is the relationship with the first question)
> ...just answer emails to the people you allways comunicates as you
> always did. Here you round the trip and legitimate the sender !
>
> For this we need a modified version of SA autowhitelist not based on
> scores but on trusted or answered emails !
>
> Flaws ?
Yes, many. Think of the automatic out-of-office replies, think of all
messages that are sent from noreply@ addresses these days (where the
originating organization tries to make clear by naming it 'noreply@'
that replies are not welcome), think of (solicited) newsletters, mailing
lists etc. etc.
> False positives....yes, ONLY the first time for each sender! just
> answer your good mails and they´ll become ham next time. Mails not
> answered (spam) remains as spam next and next and next !
> False negatives...yes, if someone impersonates in the From: as someone
> trusted by you (phising). But this could be reduced using the same
> methods as autowhitelist uses keeping in a DB pairs of senders - IPs.
> Greylists also uses DBs like this.
>
> So, what do we have to waste resources on tons of rules, tons of perl
> code, tons of regex if we know that 80% is spam?
Yes, why do you think the world spends so much resources on the spam
problem if the solution would be so easy to implement...?
/rolf
Re: new paradigm
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
On Wed, 23 Nov 2011 16:22:38 -0300
Christian Grunfeld <ch...@gmail.com> wrote:
> Do not assume by default that people want spam !
But your proposal *ensures* that people will have to wade through
huge quantities of spam to pull out the non-spam they want. That's
going backwards.
Regards,
David.
Re: new paradigm
Posted by Adam Moffett <ad...@plexicomm.net>.
On 11/23/2011 02:22 PM, Christian Grunfeld wrote:
>> Undoubtedly it is *easier*, just as I can easily eliminate all my spam by
>> unplugging the ethernet cable. Just keep in mind this method would only be
>> useful for people who already know who they want to talk to.
> And that is the big % of what people do or want to do ! most people
> wants to comunicate with who they want to talk to !
> I think you are defining people in the wrong way. Do not assume by
> default that people want spam !
>
If you described your idea to bunch of average internet users and
surveyed them about it, you'll find that a big % of them probably think
they do agree with you. When you go to implement it for real you'll
find that the percentage of users who actually stick with the idea in
the long run will get smaller and smaller. It's hard for an idea that
requires users to change their behavior to gain traction and keep it.
Please prove me wrong. I would be super happy if you're right because
it would make my job tremendously simpler.
I do like the idea, and I would try it for my own personal email because
it would be easy to do and there truly aren't many people I want to talk
to once I get home from the office and I think I'd be done whitelisting
people by the second day.
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
> Undoubtedly it is *easier*, just as I can easily eliminate all my spam by
> unplugging the ethernet cable. Just keep in mind this method would only be
> useful for people who already know who they want to talk to.
And that is the big % of what people do or want to do ! most people
wants to comunicate with who they want to talk to !
I think you are defining people in the wrong way. Do not assume by
default that people want spam !
Re: new paradigm
Posted by Adam Moffett <ad...@plexicomm.net>.
Undoubtedly it is *easier*, just as I can easily eliminate all my spam
by unplugging the ethernet cable. Just keep in mind this method would
only be useful for people who already know who they want to talk to.
> The idea is as simple as: past days was easier to blacklist...nowdays
> is easier to whitelist !
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
> I don't think AWL does with the original poster is describing, but
> implementation would be trivial in the MTA without spamassassin involved at
> all.
>
> If the user expects to receive mail from a limited number of people like
> only their relatives (me@myhome.com) then this actually might make sense for
> them, but if they expect to receive email from any random person who might
> be a potential customer (sales@mybusiness.com) then they would have a
> problem with this.
>
> I might try this or something like it for my own use. I would simply tag as
> [spam] any message whose From:, Reply-To:, or envelope sender didn't match
> my whitelist. Then I would populate the whitelist with the envelope
> recipient on any message sent by an authenticated user. You could do the
> whole thing in the Exim config file without invoking spamassassin at all.
> In fact I don't think it would be hard to keep a separate whitelist file
> for each user. If I'm going to get a confirmation email or some such from
> some random address then I can look in my spam folder. If I expect to get
> future emails from the same sender....I'll just reply to their message. It
> doesn't matter if it's a DoNotReply@ address because they'd still be added
> to the whitelist when I hit send. The fact that they blackhole or bounce my
> reply won't affect anything.
you catch the point !
You dont need anything but and auto white list ! but not the SA AWL
because its like an average filter based on scores. This method does
not need that.
The idea is as simple as: past days was easier to blacklist...nowdays
is easier to whitelist !
Re: new paradigm
Posted by Adam Moffett <ad...@plexicomm.net>.
>
> An interesting idea. Sort of a challenge and response with the onus
> on the recipient. But I think this is handled by auto whitelist which
> SpamAssassin was one of the first to implement.
>
> Regards,
> KAM
I don't think AWL does with the original poster is describing, but
implementation would be trivial in the MTA without spamassassin involved
at all.
If the user expects to receive mail from a limited number of people like
only their relatives (me@myhome.com) then this actually might make sense
for them, but if they expect to receive email from any random person who
might be a potential customer (sales@mybusiness.com) then they would
have a problem with this.
I might try this or something like it for my own use. I would simply
tag as [spam] any message whose From:, Reply-To:, or envelope sender
didn't match my whitelist. Then I would populate the whitelist with the
envelope recipient on any message sent by an authenticated user. You
could do the whole thing in the Exim config file without invoking
spamassassin at all. In fact I don't think it would be hard to keep a
separate whitelist file for each user. If I'm going to get a
confirmation email or some such from some random address then I can look
in my spam folder. If I expect to get future emails from the same
sender....I'll just reply to their message. It doesn't matter if it's a
DoNotReply@ address because they'd still be added to the whitelist when
I hit send. The fact that they blackhole or bounce my reply won't
affect anything.
I have worked at ISP's for the past 12 years and I 100% whole heartedly
agree with David Skoll's observations about the general mass of users,
but I think there are still a subset of people who would benefit from
doing it this way.
Re: new paradigm
Posted by Benny Pedersen <me...@junc.org>.
On Wed, 23 Nov 2011 13:05:40 -0500, Kevin A. McGrail wrote:
> An interesting idea. Sort of a challenge and response with the onus
> on the recipient. But I think this is handled by auto whitelist
> which
> SpamAssassin was one of the first to implement.
SAGREY plugin is wonderfull with /32 in AWL and DKIM tracking
here i have spammers send spammails to my whois account, shame on me
using blacklist_to whois email addr :=)
from and to header is equal, but from and envelope sender is diff
my own from.pm checking this it hits spams that is not spf_pass or
dkim_verifyed
Re: new paradigm
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 11/23/2011 12:55 PM, Christian Grunfeld wrote:
> Hi,
>
> I have an idea to discuss here with experts !
>
> What is the main MAIN difference between spam and ham ?
> ...
> ...
> Answer: spam is "one way ticket" and ham is 99.99% "round trip" !
> (legit notifications can be "one way ticket" but you can mark them as
> ham later)
>
> What do I mean? you never never answer (or it is really strange) a
> spam message. Average users, who someone said that are stupid and more
> stupid when they are in front of a machine, also dont respond to a
> spammy message. At least if they are marked as spam.
>
> So the idea is...in this days where the ratio of spam/ham is about 80%
> (put the ratio you want but be sure it is high enough) lets start with
> marking all incomings as spam !
>
> Past days when the ratio of spam/ham was 5% or 10% it was quite logic
> that the reverse was true. That is, all incomings were ham and we
> tried with a lot of methods to extract or mark the bad emails!
> We spent 15 years (up to now) with the Presumption of innocence
> analogy of "Everyone charged with a criminal offence shall be presumed
> innocent until proved guilty according to law". This approach is
> wasting a lot of resources because of the high spam/ham ratio!
>
> Nowdays its easier to invert the logic!
> *mark all incomings as spam the first time
> *check spam folder always
> *mark as ham....or (here is the relationship with the first question)
> ...just answer emails to the people you allways comunicates as you
> always did. Here you round the trip and legitimate the sender !
>
> For this we need a modified version of SA autowhitelist not based on
> scores but on trusted or answered emails !
>
> Flaws ?
> False positives....yes, ONLY the first time for each sender! just
> answer your good mails and they´ll become ham next time. Mails not
> answered (spam) remains as spam next and next and next !
> False negatives...yes, if someone impersonates in the From: as someone
> trusted by you (phising). But this could be reduced using the same
> methods as autowhitelist uses keeping in a DB pairs of senders - IPs.
> Greylists also uses DBs like this.
>
> So, what do we have to waste resources on tons of rules, tons of perl
> code, tons of regex if we know that 80% is spam? lets mark all of them
> as spam and let this method work!
>
> Time to think in a new antispam paradigm !
>
> Cheers
An interesting idea. Sort of a challenge and response with the onus on
the recipient. But I think this is handled by auto whitelist which
SpamAssassin was one of the first to implement.
Regards,
KAM
Re: new paradigm
Posted by Dave Warren <li...@hireahit.com>.
On 11/23/2011 4:41 PM, Christian Grunfeld wrote:
>> Our (commercial) software has a similar feature, not quite as fancy as
>> amavisd's, but still pretty useful.
>
> Many things become clear to me now ! Are you an antispam vendor?
>
> No offence but ...now I understand why a "simple" solution makes no
> sense to you ! You need a big thing wich wastes a lot of resources in
> order to get people pay for that (commercial) package !
> You can not also tell people who pay you that has to mark or reply
> emails a few days in order to train the whitelist.
I have mucked with default-deny policies upon user requests, even had a
handful of users using it. To date, 0 have stuck with it.
--
Dave Warren, CEO
Hire A Hit Consulting Services
http://ca.linkedin.com/in/davejwarren
Re: new paradigm
Posted by Emmanuel Seyman <es...@edd.fr>.
* Benny Pedersen [25/11/2011 17:54] :
>
> maillists often not remove originating sender addr, if thay did how
> can i get all that private emails orinating from maillists ?
I believe that rh is suggesting not to put email adresses in the body of
your mail if you're replying to a mailing-list.
Most of the times, these have public archives and while the archiving
software changes the addresses in the header to make them spambot-proof,
it doesn't do this for addresses in the body.
Emmanuel
Re: new paradigm
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
On Thu, 24 Nov 2011 19:51:58 -0300
Christian Grunfeld <ch...@gmail.com> wrote:
> Maths language is unique so if someone of you dont agree with me in
> what follows I can give you lectures out of the list.
Converting real-world problems to purely mathematical expressions is
not always helpful.
"Assume a cow may be approximated by a sphere..."
> You say: lets assume all non spam
No, that is not at all what I am saying. I'm saying: "Assume that
we do not know whether or not incoming mail is spam." Or even: "Assume
that incoming mail has a 75% likelihood of being spam."
Regards,
David.
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
I messed up with english :p
direct and contrapositive and I miss the negation af all
contrapositive is negation and switch the hypothesis and the conclusion
2011/11/24 Christian Grunfeld <ch...@gmail.com>:
> 2011/11/24 R - elists <li...@abbacomm.net>:
>> i think you are realistically confused about truly "negating something"
>>
>> english is not your native language is it?
>
> No, it is not ! I am not as good in english as you but I am very good
> with maths and logic!
> (I want someone jumps over R-elists who tried to discredited me based
> on a language barrier like Karsten Bräckelmann jumped over me before!)
>
> Maths language is unique so if someone of you dont agree with me in
> what follows I can give you lectures out of the list.
> I think YOU are confused about "negating something" because negating
> is not the only thing and it is NOT what I am doing !
>
> given A -> B called direct problem or direct statment
> there exists:
>
> converse: B -> A
> inverse: ~A -> ~B
> contrapositive: ~B -> ~A
>
> The direct and the inverse problem are logicaly equivalent. The same
> between converse and contrapositive !
>
> Here we are dealing with direct problem and inverse problem !
>
> You say: lets assume all non spam
> A (is spam)
> B (mark as spam)
> if an email is spam then mark it as spam (A->B)
>
> I say: lets assume all spam
> ~A(is ham)
> ~B(mark as ham)
> if an email is ham then mark it as ham (~A -> ~B)
>
> both are logically the same and gives same results but with the second
> aproach you need to train the system with a few mails (your contacts)
> with the other you need tons of mails in Bayes DB and tons of running
> code.
>
RE: new paradigm
Posted by Benny Pedersen <me...@junc.org>.
On Fri, 25 Nov 2011 00:03:20 -0800, R - elists wrote:
> when you reply to people, dont put their email address in the post.
maillists often not remove originating sender addr, if thay did how can
i get all that private emails orinating from maillists ?
RE: new paradigm
Posted by R - elists <li...@abbacomm.net>.
Christian,
when you reply to people, dont put their email address in the post.
please stop that.
again, if you would read the posts slowly and correctly, i was not attacking
you or your ideas.
see the word "not" there...
this is a discussion list, not a discrediting list.
in terms of negation, i was thinking one should ponder something more along
the lines of
"NOT truth"
but what do i know ;->
- rh
Re: new paradigm
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Thu, 2011-11-24 at 19:51 -0300, Christian Grunfeld wrote:
> (I want someone jumps over R-elists who tried to discredited me based
> on a language barrier like Karsten Bräckelmann jumped over me before!)
You are discrediting yourself, dude.
I slapped you on the wrist for being prejudiced, accusatory, and attack
actively contributing members. You resorted long ago to fighting people
disagreeing with you, instead of focusing on your arguments.
I did not jump on you -- I encouraged you (in an ironical way) to come
forward and implement your superior, groundbreaking idea and change in
paradigm. Well, if it doesn't irritate you that others actually have
done this before, and found it to not work...
So, go for it! You don't want us to do it for you, do you?
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
2011/11/24 R - elists <li...@abbacomm.net>:
> i think you are realistically confused about truly "negating something"
>
> english is not your native language is it?
No, it is not ! I am not as good in english as you but I am very good
with maths and logic!
(I want someone jumps over R-elists who tried to discredited me based
on a language barrier like Karsten Bräckelmann jumped over me before!)
Maths language is unique so if someone of you dont agree with me in
what follows I can give you lectures out of the list.
I think YOU are confused about "negating something" because negating
is not the only thing and it is NOT what I am doing !
given A -> B called direct problem or direct statment
there exists:
converse: B -> A
inverse: ~A -> ~B
contrapositive: ~B -> ~A
The direct and the inverse problem are logicaly equivalent. The same
between converse and contrapositive !
Here we are dealing with direct problem and inverse problem !
You say: lets assume all non spam
A (is spam)
B (mark as spam)
if an email is spam then mark it as spam (A->B)
I say: lets assume all spam
~A(is ham)
~B(mark as ham)
if an email is ham then mark it as ham (~A -> ~B)
both are logically the same and gives same results but with the second
aproach you need to train the system with a few mails (your contacts)
with the other you need tons of mails in Bayes DB and tons of running
code.
RE: new paradigm
Posted by R - elists <li...@abbacomm.net>.
christian
i wasnt picking on you or your ideas
locks are not a good anology unless you unplug or close port 25
those were mentioned on the list
you are possibly on to some things, yet part of what you are on to is
already late to the table
i think you are realistically confused about truly "negating something"
english is not your native language is it?
- rh
Re: new paradigm
Posted by Noel Butler <no...@ausics.net>.
On Thu, 2011-11-24 at 15:04 -0500, David F. Skoll wrote:
>
> How is it less effort to be forced to check every incoming email than to
> allow your computer to do some or most of that work? You are not making
> any sense.
Yes it does, if he's actually a spammer, he seems to arguing all users
must read every mail, and thats what those tossers want.
I smell a troll
Re: new paradigm
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
On Thu, 24 Nov 2011 16:56:38 -0300
Christian Grunfeld <ch...@gmail.com> wrote:
> with your logic.... why do you have door locks in your house?
1) What I'm protecting [my family and my posessions] is a lot more valuable
to me than a few seconds wasted by a spam that slips through.
2) I don't have hundreds of people a day coming to my house and trying
to get in. So the cost of checking who is at the door each time
someone knocks is manageable, but the cost of checking every incoming
email manually is not.
[...]
> Good example but bad knowledge about maths. They are NOT there only to
> come up with alternative reasoning solutions. They are there to give
> the SAME solution with less effort !
How is it less effort to be forced to check every incoming email than to
allow your computer to do some or most of that work? You are not making
any sense. You are advocating a system that is guaranteed to produce
false-positives and that avoids losing mail to FPs by requiring manual
sorting of email.
Regards,
David.
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
> pardon me for my ignorance, yet if you think about it, the OP's idea is why
> some royalty had food and drink tester / tasters centuries ago
>
> assume all food and drink is poisoned
>
> problem is, if the poison wasnt fast acting, the royalty would ingest it and
> die anyways.
with your logic.... why do you have door locks in your house?
if you do not assume all others can get in your house, why do you use
them? leave doors open as you are the only who will ever get into !
leave doors open as other people are "essentially say "neutral" until
proven otherwise"
> not or negating theory in math and other methodologies is really only there
> to come up with alternative reasoning for solutions
Good example but bad knowledge about maths. They are NOT there only to
come up with alternative reasoning solutions. They are there to give
the SAME solution with less effort !
RE: new paradigm
Posted by R - elists <li...@abbacomm.net>.
pardon me for my ignorance, yet if you think about it, the OP's idea is why
some royalty had food and drink tester / tasters centuries ago
assume all food and drink is poisoned
problem is, if the poison wasnt fast acting, the royalty would ingest it and
die anyways.
eh?
not or negating theory in math and other methodologies is really only there
to come up with alternative reasoning for solutions
realistically it is not the big picture solution...
correct? :-)
so, to clarify, does spamassassin theory essentially say "neutral" until
proven otherwise?
or?
- rh
Re: new paradigm
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
On Thu, 24 Nov 2011 14:36:53 -0300
Christian Grunfeld <ch...@gmail.com> wrote:
> * a lot of people on this list would never change their minds. That is
> why spammers beat us....they change their minds in all possible ways !
Spammers are not beating us. For the most part, anti-spam systems work
pretty well to keep email useful and usable.
If we adopted your suggestion, then spammers *would* win because we would
have no computerized shielding against spam --- we'd have to glance at
every single message subject, whether we wanted to or not.
Regards,
David.
Re: new paradigm
Posted by Benny Pedersen <me...@junc.org>.
On Thu, 24 Nov 2011 15:00:10 -0300, Christian Grunfeld wrote:
> I said what i ve said ! The idea could be good, bad, not so bad,
> idiot...but more serious things came to light in the list !
in general, general rules is not usefull in anyway, in danish wording:
morale er godt, dobbelt morale er dobbelt så godt
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
2011/11/24 Benny Pedersen <me...@junc.org>:
> On Thu, 24 Nov 2011 14:36:53 -0300, Christian Grunfeld wrote:
>>
>> what I can summarize reading past 40 emails is:
>
> the world is full of idiots, including me, thats what you say ?
No. I do not treat any people by idiot !
I said what i ve said ! The idea could be good, bad, not so bad,
idiot...but more serious things came to light in the list !
Re: new paradigm
Posted by Benny Pedersen <me...@junc.org>.
On Thu, 24 Nov 2011 14:36:53 -0300, Christian Grunfeld wrote:
> what I can summarize reading past 40 emails is:
the world is full of idiots, including me, thats what you say ?
Re: new paradigm
Posted by Axb <ax...@gmail.com>.
On 2011-11-24 18:36, Christian Grunfeld wrote:
> what I can summarize reading past 40 emails is:
>
> * a lot of people on this list would never change their minds. That is
> why spammers beat us....they change their minds in all possible ways !
> * a lot of people on this list do not tell their users that antispam
> systems can fail and they can lose emails (spam or not) and are proud
> of it !
> * a lot of people on this list are silently discarding those emails
> and users do not even know !
> * a lot of people on this list are violating RFCs doing the previous thing !
>
> C
...and all of this has nothing to do with SpamAssassin
You may find way smarter ppl in some other list/forum.
HELO checking (was Re: new paradigm)
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
On Thu, 24 Nov 2011 15:31:59 -0500
Michael Scheidell <mi...@secnap.com> wrote:
> I wonder what the rfc's say about helo line not matching dns:
> Received: from mail.apache.org (hermes.apache.org
> [140.211.11.3])
RFC 5321 strongly hints that that is no reason to reject mail.
An SMTP server MAY verify that the domain name argument in the EHLO
command actually corresponds to the IP address of the client.
However, if the verification fails, the server MUST NOT refuse to
accept a message on that basis.
This doesn't exactly cover your situation... in your situation, the
machine calls itself mail.apache.org but 140.211.11.3 reverse-resolves
to hermes.apache.org. mail.apache.org, however, resolves to
140.211.11.3. So I would say rejecting mail because of this type of
mismatch is against the spirit of the RFC.
Regards,
David.
Re: new paradigm
Posted by Michael Scheidell <mi...@secnap.com>.
On 11/24/11 3:30 PM, Martin Hepworth wrote
> Rfc 5321 says I can discard if I have high confidence it's rubbish !
> --
> Martin
I wonder what the rfc's say about helo line not matching dns:
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
--
Michael Scheidell, CTO
o: 561-999-5000
d: 561-948-2259
>*| *SECNAP Network Security Corporation
* Best Mobile Solutions Product of 2011
* Best Intrusion Prevention Product
* Hot Company Finalist 2011
* Best Email Security Product
* Certified SNORT Integrator
______________________________________________________________________
This email has been scanned and certified safe by SpammerTrap(r).
For Information please see http://www.spammertrap.com/
______________________________________________________________________
Re: new paradigm
Posted by Noel Butler <no...@ausics.net>.
On Thu, 2011-11-24 at 20:30 +0000, Martin Hepworth wrote:
>
> > * a lot of people on this list are violating RFCs doing the previous
> thing !
> >
> > C
> >
> Rfc 5321 says I can discard if I have high confidence it's rubbish !
> --
> Martin
>
Indeed, that RFC was introduced a few years back, late 08 IIRC, when
2821 was transformed into modern days, it only reflects what admins have
been doling for over a decade anyway, it was discussed in the sendmail
group for some time prior, so i would have thought everyone would be
aware of it if they did any such google fu'ing.
Re: new paradigm
Posted by Martin Hepworth <ma...@gmail.com>.
On Thursday, 24 November 2011, Christian Grunfeld <
christian.grunfeld@gmail.com> wrote:
> what I can summarize reading past 40 emails is:
>
> * a lot of people on this list would never change their minds. That is
> why spammers beat us....they change their minds in all possible ways !
> * a lot of people on this list do not tell their users that antispam
> systems can fail and they can lose emails (spam or not) and are proud
> of it !
> * a lot of people on this list are silently discarding those emails
> and users do not even know !
> * a lot of people on this list are violating RFCs doing the previous
thing !
>
> C
>
Rfc 5321 says I can discard if I have high confidence it's rubbish !
--
Martin
--
--
Martin Hepworth
Oxford, UK
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
what I can summarize reading past 40 emails is:
* a lot of people on this list would never change their minds. That is
why spammers beat us....they change their minds in all possible ways !
* a lot of people on this list do not tell their users that antispam
systems can fail and they can lose emails (spam or not) and are proud
of it !
* a lot of people on this list are silently discarding those emails
and users do not even know !
* a lot of people on this list are violating RFCs doing the previous thing !
C
Re: new paradigm
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
On Thu, 24 Nov 2011 11:07:42 -0300
Christian Grunfeld <ch...@gmail.com> wrote:
> > Sorry to follow up on myself.
> > I should mention that our product can operate in a mode whereby it
> > holds all mail in the quarantine except from whitelisted senders.
> > We also have a "whitelist-people-I-write-to" mechanism, so I guess
> > we anticipated the OP's "new paradigm" by a few years.
> So the idea is yours ! and you were questioning me !
> people on the list, attack him from now on!
Yes, I think it's a bad idea based on several years experience with
hundreds of thousands of end-users.
As I recall, one of our customers asked for this feature and it was easy
to implement. I believe that customer used it for a couple of weeks and
then realized it wasn't such a good idea after all. :)
Regards,
David.
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
2011/11/24 David F. Skoll <df...@roaringpenguin.com>:
> Sorry to follow up on myself.
>
> I should mention that our product can operate in a mode whereby it
> holds all mail in the quarantine except from whitelisted senders. We
> also have a "whitelist-people-I-write-to" mechanism, so I guess we
> anticipated the OP's "new paradigm" by a few years.
So the idea is yours ! and you were questioning me !
people on the list, attack him from now on!
Re: new paradigm
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
Sorry to follow up on myself.
I should mention that our product can operate in a mode whereby it
holds all mail in the quarantine except from whitelisted senders. We
also have a "whitelist-people-I-write-to" mechanism, so I guess we
anticipated the OP's "new paradigm" by a few years.
I estimate that fewer than one in 10,000 people uses the
quarantine-everything mode. I happen to use it on a honeypot address
that only ever gets spam (for Bayes training purposes), but I'm
personally unaware of a single customer who actually uses it for a
live email address.
Regards,
David.
Re: new paradigm
Posted by Benny Pedersen <me...@junc.org>.
On Thu, 24 Nov 2011 02:30:19 +0100, Karsten Bräckelmann wrote:
> Go for it, Christian. Prove us all wrong, and finally develop the
> long
> awaited FUSSP.
>
> (No, I am in no way affiliated with Roaring Penguin. I am a SA dev.)
>
> Fuck, now I did reply to this thread. I tried hard not to. I tried...
+2
when antispam solutions begin to ask recipient if they want email from
sender in there addressbooks then most spam problems are gone, spam is
not just I block 0.0.0.0/0 in mta so i receive no spam :-)
if sender is not in addressbook, why not tempfail sender while mta ask
receipient if email is wanted from sender ? maybe provide whois info to
recipient might help ?
i dont understand why so many dynamic ips is missing in
pbl.spamhaus.org does client expect there ip to be unlisted as dynamic
to get smtp auth to work ? :-)
imho there is alot of places to stop more spam, but none do it :(
fuck i replyed :-)
Re: new paradigm
Posted by Walter Hurry <wa...@lavabit.com>.
On Thu, 24 Nov 2011 19:58:55 +0100, Karsten Bräckelmann wrote:
> On Wed, 2011-11-23 at 22:36 -0500, Kevin A. McGrail wrote:
>> "Karsten Bräckelmann" <gu...@rudersport.de> wrote:
>> > On Wed, 2011-11-23 at 20:06 -0500, David F. Skoll wrote:
>> > > On Wed, 23 Nov 2011 21:41:43 -0300 Christian Grunfeld wrote:
>
>> > > > Many things become clear to me now ! Are you an antispam vendor?
>
>> > > I welcome competition. If your idea is that good, you will easily
>> > > take away our market share. So go for it.
>
>> > Go for it, Christian. Prove us all wrong, and finally develop the
>> > long awaited FUSSP.
>
>> Well, as an American about to celebrate Thanksgiving, I must say I am
>> thankful for healthy debate. And Perl, apache, linux, mysql,
>> spamassassin and mimedefang plus lots of other projects that I use
>> every day and should thank more often.
>
> While there's no Thanksgiving here, I too enjoy healthy debates.
>
> What I do not enjoy, and what triggered my reaction, was the instant,
> outright rejection and questioning of an educated opinion, entirely
> based on the fact the poster works for a company selling anti-spam
> services.
>
> At that point, the discussion degraded from healthy debate to prejudice
> and accusation.
+1
Re: new paradigm
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Wed, 2011-11-23 at 22:36 -0500, Kevin A. McGrail wrote:
> "Karsten Bräckelmann" <gu...@rudersport.de> wrote:
> > On Wed, 2011-11-23 at 20:06 -0500, David F. Skoll wrote:
> > > On Wed, 23 Nov 2011 21:41:43 -0300 Christian Grunfeld wrote:
> > > > Many things become clear to me now ! Are you an antispam vendor?
> > > I welcome competition. If your idea is that good, you will easily
> > > take away our market share. So go for it.
> > Go for it, Christian. Prove us all wrong, and finally develop the long
> > awaited FUSSP.
> Well, as an American about to celebrate Thanksgiving, I must say I am
> thankful for healthy debate. And Perl, apache, linux, mysql,
> spamassassin and mimedefang plus lots of other projects that I use
> every day and should thank more often.
While there's no Thanksgiving here, I too enjoy healthy debates.
What I do not enjoy, and what triggered my reaction, was the instant,
outright rejection and questioning of an educated opinion, entirely
based on the fact the poster works for a company selling anti-spam
services.
At that point, the discussion degraded from healthy debate to prejudice
and accusation.
> Someday, the drinks are definitely on me! And yes, that includes
> Christian.
>
> And thanks especially for companies like Roaring Penguin that are
> spearheading the tough areas of open-sourcing things while still
> feeding their families.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: new paradigm
Posted by "Kevin A. McGrail" <km...@pccc.com>.
"Karsten Bräckelmann" <gu...@rudersport.de> wrote:
>On Wed, 2011-11-23 at 20:06 -0500, David F. Skoll wrote:
>> On Wed, 23 Nov 2011 21:41:43 -0300 Christian Grunfeld wrote:
>>
>> > Many things become clear to me now ! Are you an antispam vendor?
>
>> > No offence but ...now I understand why a "simple" solution makes no
>> > sense to you ! You need a big thing wich wastes a lot of resources
>in
>> > order to get people pay for that (commercial) package !
>> > You can not also tell people who pay you that has to mark or reply
>> > emails a few days in order to train the whitelist.
>>
>> I welcome competition. If your idea is that good, you will easily
>> take away our market share. So go for it.
>
>+1
>
>Go for it, Christian. Prove us all wrong, and finally develop the long
>awaited FUSSP.
>
>(No, I am in no way affiliated with Roaring Penguin. I am a SA dev.)
>
>Fuck, now I did reply to this thread. I tried hard not to. I tried...
Well, as an American about to celebrate Thanksgiving, I must say I am thankful for healthy debate. And Perl, apache, linux, mysql, spamassassin and mimedefang plus lots of other projects that I use every day and should thank more often.
Someday, the drinks are definitely on me! And yes, that includes Christian.
And thanks especially for companies like Roaring Penguin that are spearheading the tough areas of open-sourcing things while still feeding their families.
Though I could do without the anti-spam songs :-)
Happy Thanksgiving,
KAM
Re: new paradigm
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Wed, 2011-11-23 at 20:06 -0500, David F. Skoll wrote:
> On Wed, 23 Nov 2011 21:41:43 -0300 Christian Grunfeld wrote:
>
> > Many things become clear to me now ! Are you an antispam vendor?
> > No offence but ...now I understand why a "simple" solution makes no
> > sense to you ! You need a big thing wich wastes a lot of resources in
> > order to get people pay for that (commercial) package !
> > You can not also tell people who pay you that has to mark or reply
> > emails a few days in order to train the whitelist.
>
> I welcome competition. If your idea is that good, you will easily
> take away our market share. So go for it.
+1
Go for it, Christian. Prove us all wrong, and finally develop the long
awaited FUSSP.
(No, I am in no way affiliated with Roaring Penguin. I am a SA dev.)
Fuck, now I did reply to this thread. I tried hard not to. I tried...
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: new paradigm
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
On Wed, 23 Nov 2011 21:41:43 -0300
Christian Grunfeld <ch...@gmail.com> wrote:
> Many things become clear to me now ! Are you an antispam vendor?
My company is, yes.
> No offence but ...now I understand why a "simple" solution makes no
> sense to you ! You need a big thing wich wastes a lot of resources in
> order to get people pay for that (commercial) package !
> You can not also tell people who pay you that has to mark or reply
> emails a few days in order to train the whitelist.
I welcome competition. If your idea is that good, you will easily
take away our market share. So go for it.
Regards,
David.
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
> Our (commercial) software has a similar feature, not quite as fancy as
> amavisd's, but still pretty useful.
Many things become clear to me now ! Are you an antispam vendor?
No offence but ...now I understand why a "simple" solution makes no
sense to you ! You need a big thing wich wastes a lot of resources in
order to get people pay for that (commercial) package !
You can not also tell people who pay you that has to mark or reply
emails a few days in order to train the whitelist.
Re: new paradigm
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
On Wed, 23 Nov 2011 20:16:06 +0100
Mark Martinec <Ma...@ijs.si> wrote:
> A concept of 'ongoing conversation' or 'replied to' is implemented as
> a 'pen pals' feature in amavisd,
Our (commercial) software has a similar feature, not quite as fancy as
amavisd's, but still pretty useful.
To be clear: The concept of whitelisting people you email is a good one.
What I object to in the OP's proposal is considering all incoming email
as spam unless the user is whitelisted.
Regards,
David.
Re: new paradigm
Posted by Lucio Chiappetti <lu...@lambrate.inaf.it>.
On Wed, 23 Nov 2011, Christian Grunfeld wrote:
> Greylists do great job stoping robots but there are spammers with well
> configured MTAs who tries and tries and tries and bypass greylists.
Since the frequency of users checking quarantine has also been mentioned:
We've been running spamassassin for about 5 years (we used plain DNSBL
before and lately some people were complaining about FPs), with quarantine
in a daily global folder for all the institute (not per user), with a
crontab which sends to each user a list ("spam report") of apparent
originator and subject of the quarantined potential spam.
A few users did check this daily report, and very rarely (once per month
?) asked to release an odd FP. Other users (like me) felt the number of
information messages was excessive, I had a further personal filter which
scanned the spam report (which is anyhow archived for 7 weeks, but I
almost never check), counted the number of occurrences of the same subject
(high = potential spam, single = maybe FP) and told me of suspicious FPs.
They were so few I usually did not check the report or the condensed
report, but only checked the quarantine in the rare cases I did not
receive a reply I was expecting.
On the contrary the spam still passing through spamassassin was becoming
more and more (our fault, we do not update the server very often) for all
our users.
Since about 6 months we implemented greylists, with an initial whitelist
of several academic domain MXs which are our regular correspondent, and
that cut the amount of spam severely and very satisfactorily.
We still run spamassassin downstream of the greylisting, and the
information in the reports is now reduced to manageable size (but I've
taken the habit of not checking it), and the surviving spam is almost nil.
We run a crontab which reports (to me) the origin and destination of
messages which are autowhitelisted by the greylist after more than 30 min.
I scan those reports, and pick up the odd academic domain which requires
to be permanently whitelisted (I wait until I have a dozen of those to
tell the system manager to actually whitelist them). I notice that the
majority of the cases which pass through graylist after such a long delay
are (but for a few mail exploders) spammers, of the sort of bank or credit
card phishing I guess.
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
2011/11/23 Mark Martinec <Ma...@ijs.si>:
> A concept of 'ongoing conversation' or 'replied to' is implemented as
> a 'pen pals' feature in amavisd, when it is used in place of spamd
> to call SpamAssassin. The idea is to automatically contribute some negative
> spam score points to ongoing conversations - based on envelope sender
> and recipients, as well as Message-ID and References or In-Reply-To
> mail header fields.
I use amavis and some time ago I talked with you about a patch for
virtual users in amavis !
but I missed that ! good point !
The idea works like greylists but greylist just whitelists when a
sender-IP-recipient hits more than a threshold. Greylists do great job
stoping robots but there are spammers with well configured MTAs who
tries and tries and tries and bypass greylists. For that situation
this solution can aid greylists in the way that conversations also
need to be replied in order to get whitelisted
Cheers
Re: new paradigm
Posted by Mark Martinec <Ma...@ijs.si>.
A concept of 'ongoing conversation' or 'replied to' is implemented as
a 'pen pals' feature in amavisd, when it is used in place of spamd
to call SpamAssassin. The idea is to automatically contribute some negative
spam score points to ongoing conversations - based on envelope sender
and recipients, as well as Message-ID and References or In-Reply-To
mail header fields.
Amavisd-new release notes:
new in amavisd-new-2.4.2, June 27, 2006
- new feature: "pen pals soft-whitelisting" lowers spam score of received
replies (or followup correspondence) to a message previously sent by a
local user to this address;
Pre-requisites:
* both the outgoing and the incoming mail must pass through amavisd
(although outgoing mail may have checks disabled or made more permissive
if desired);
* SQL logging must be enabled (@storage_sql_dsn) and records should
be kept for at least several days (some statistics (2006-11 update):
90% of replied mail (or followups) is sent within 2 weeks since
previous correspondence, 40% within 24 hours, 20% within 3 hours,
10% within 30 minutes, 5% within 12 minutes);
* @mynetworks and @local_domains_maps must reflect reality, allowing amavisd
to distinguish between outgoing, incoming and internal-to-internal mail;
* the information about client IP address must be available to amavisd,
i.e. Postfix XFORWARD protocol extension must be enabled, or AM.PDP+milter;
* configuration variable $penpals_bonus_score must be set to a positive
value (such as 1.0, increase to perhaps 5 or 8 after seeing that it works),
zero disables the feature and is a default;
* $sql_clause{'sel_penpals'} must contain a SELECT clause (which by
default it does, unless overridden by an old assignment to %sql_clause
in amavisd.conf);
* sender/recipient address pair must exactly match recipient/sender pair of
previous correspondence (except for allowed case-changes in domain part),
which means that care must be taken when canonical and/or virtual mapping
is performed by MTA (such as mapping between internal and external address
forms) - if external address forms of local addresses are to be seen by
a content filter then canonical mapping (int->ext) must be done *before*
filtering and virtual mapping (ext->int) *after*; alternatively, if
internal address forms are to be seen by a content filter, then canonical
mapping should be done after filtering, and virtual mapping before;
see README.postfix, section "TO DO 'VIRTUAL ALIAS' MAPPING AND OTHER
POSTFIX CLEANUP PROCESSING BEFORE OR AFTER CONTENT FILTERING?"
(P.S. later renamed to 'Advanced Postfix and amavisd-new configuration');
How it works:
* SQL logging stores records about all mail messages processed by amavisd,
their sender, recipients, delivery status, mail contents type (no changes
there, this feature was introduced with amavisd-new-2.3.0); for the
purpose of pen pals scheme only records with local-domain senders matter;
* when a message is received, an SQL lookup against an SQL logging database
is performed, looking for previous messages sent in reverse direction,
i.e. from a local user (which is now a recipient of the current mail)
to the address that is now the sender of the message being processed;
A SELECT clause in $sql_clause{'sel_penpals'} is used, which by default
only considers records of previous messages that were actually
delivered (not rejected, discarded or bounced), and were not infected.
SQL lookup returns a timestamp of the most recent such message (if any),
the difference (in seconds) between the current time and the timestamp
is an 'age' as used in the following formula;
* an exponential decay formula calculates score points to be deducted
from the SA score:
weight = 1 / 2^(age/penpals_halflife)
score_boost = -penpals_bonus_score * weight
i.e. penpals_bonus_score is multiplied by 1, 1/2, 1/4, 1/8, 1/16, ...
at age 0, 1*halflife, 2*halflife, 3*halflife, 4*halflife ...
weight is a continuous function of age (actually, in steps of one second);
[...] (more in release notes)
new in: amavisd-new-2.5.0, April 23, 2007
- formerly penpals could only match replies to previous outgoing mail
where envelope sender and recipient addresses are exactly reversed.
Now, in addition to this, penpals can also match replies which reference
previous outgoing mail by its 'Message-ID' (taking into account the
'References' or 'In-Reply-To' header fields), even if the envelope
sender address of the reply is null or does not match a recipient address
of a previous outgoing mail. This covers for incoming replies to mailing
list postings, incoming message disposition notifications (MDN, RFC 3798)
and incoming replies from alias or role addresses. A query on a
message-id is fast compared to matching on recipient id, and if it
succeeds, the later one is skipped.
Mark
Re: new paradigm
Posted by Noel Butler <no...@ausics.net>.
On Sun, 2011-11-27 at 16:48 +0000, RW wrote:
> On Sun, 27 Nov 2011 16:43:04 +0000
> RW wrote:
>
> > On Fri, 25 Nov 2011 10:06:44 +1000
> > Noel Butler wrote:
> >
> >
> > > its up to them if they want to or not, the spam folders have very
> > > little in them here because of our approach, and in our tests we
> > > have had 0.00000001% of FP's in that, which is really good.
> >
> > At 1.7 million email a day that's at very most 1 FP in 16 years, which
> > would suggest you have had less than 1 FP since SA was released.
>
> OK I missed "12 front ends", but even so, it's still not credible.
You people need to get out more, I see I included one (1) (uno) extra
zero, but the care factor is still just as low.
I couldnt give a rats if the FP rate was a full 1.0%, that's far better
than them having the suspected gutter trash in their inbox where they
must open the message in full, the lack of complaints we get proves to
me our system works, so sorry if YOU (and others) cant live with that,
it simply just aint my problem.
Re: new paradigm
Posted by RW <rw...@googlemail.com>.
On Sun, 27 Nov 2011 16:43:04 +0000
RW wrote:
> On Fri, 25 Nov 2011 10:06:44 +1000
> Noel Butler wrote:
>
>
> > its up to them if they want to or not, the spam folders have very
> > little in them here because of our approach, and in our tests we
> > have had 0.00000001% of FP's in that, which is really good.
>
> At 1.7 million email a day that's at very most 1 FP in 16 years, which
> would suggest you have had less than 1 FP since SA was released.
OK I missed "12 front ends", but even so, it's still not credible.
Re: new paradigm
Posted by RW <rw...@googlemail.com>.
On Fri, 25 Nov 2011 10:06:44 +1000
Noel Butler wrote:
> its up to them if they want to or not, the spam folders have very
> little in them here because of our approach, and in our tests we have
> had 0.00000001% of FP's in that, which is really good.
At 1.7 million email a day that's at very most 1 FP in 16 years, which
would suggest you have had less than 1 FP since SA was released.
> I guess those that actually do check spam folders only look at the
> displayed name in the run down list (without going into any of the
> spams), we also replace the subject with just "[spam] attached"
I think it would be very difficult to spot an FP that way. The FPs I've
seen are mostly generated by web-servers, and they are mostly
"first-contact". And even if the displayed name is recognisable it's
usually the subject that makes the FP stand-out.
Re: new paradigm
Posted by Noel Butler <no...@ausics.net>.
your opinion means less than that to me, since for some unknown reason,
for some time you have taken an extreme hatred of me, but hey what ever
floats your boat I dont know you so I dont give a fuck about your
reasons or your rants.
On Sun, 2011-11-27 at 13:05 -0800, jdow wrote:
> Whereas my concerns for your mathematical nonsense is zip, nada, zero, nothing,
> goawayyoubothermechild.
>
> Seriously, your claim is patent nonsense yet you expect people to listen to
> you. That IS rather childish behavior, you know. You can't have been running
> anti-spam tools long enough to reach your number given the email volumes you
> cite. That would take close to 100 years to get a meaningful enough count of
> spam to allow you to make the claim you made. IMAO that makes you the spammer/
> troll not Christian.
>
Re: new paradigm
Posted by jdow <jd...@earthlink.net>.
Whereas my concerns for your mathematical nonsense is zip, nada, zero, nothing,
goawayyoubothermechild.
Seriously, your claim is patent nonsense yet you expect people to listen to
you. That IS rather childish behavior, you know. You can't have been running
anti-spam tools long enough to reach your number given the email volumes you
cite. That would take close to 100 years to get a meaningful enough count of
spam to allow you to make the claim you made. IMAO that makes you the spammer/
troll not Christian.
{^_-}
On 2011/11/27 04:04, Noel Butler wrote:
> yaaaaaaaaaaaaaaaaaaaawwwwwwwwwwwwwwwwwwwwwwwwnnnnnnnnnnnnnnnnnnn
>
> my care factor about what some spammy troll like yourself has to say, is,
> well... in the words of Elton John - too low for zero
>
>
>
> On Sun, 2011-11-27 at 00:25 -0300, Christian Grunfeld wrote:
>> > 2011/11/24 Noel Butler<noel.butler@ausics.net <ma...@ausics.net>>:
>> >> its up to them if they want to or not, the spam folders have very little
>> >> in
>> >> them here because of our approach, and in our tests we have had
>> >> 0.00000001%
>> >> of FP's in that, which is really good.
>> >
>> > 0.00000001% is 1 FP over 10.000.000.000 !!!!!! 1 over 10 billion mails
>> > !!!!!!
>> >
>> > I really love people on this list !!!
>> >
>> > Your maths are flawed, you assume too much and wrongly.
>> > We have 12 front ends, each process about 1.5 - 1.7 million ACCEPTED
>> > messages a day, hence why we are so happy with the way things are. But hey,
>> > imagine the resources we'd recover if we allowed little twirp spammers like
>> > yourself the luxury of turning off anti spam measures and letting the users
>> > decide.... Wait... Nah... I don't think so some how.
>>
>> My maths are flawed? I think yours are !
>>
>> 1% FP is 1 FP over 100 emails
>> 0.1% is 1 over 1,000
>> 0.01% is 1 over 10,000
>> 0.001% is 1 over 100,.000
>> ....
>> I think you can continue
>> ....
>> 0.000,000,01% is 1 FP over 10,000,000,000 !!
>>
>> I'm not scared about your email volume...I doubt about your FP ratio !!!
>
FP rate (was Re: new paradigm_
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
On Sun, 27 Nov 2011 22:04:25 +1000
Noel Butler <no...@ausics.net> wrote:
> my care factor about what some spammy troll like yourself has to say,
> is, well... in the words of Elton John - too low for zero
With all due respect, a reported FP of 0.00000001% is simply not believable.
Regards,
David.
Re: new paradigm
Posted by Noel Butler <no...@ausics.net>.
yaaaaaaaaaaaaaaaaaaaawwwwwwwwwwwwwwwwwwwwwwwwnnnnnnnnnnnnnnnnnnn
my care factor about what some spammy troll like yourself has to say,
is, well... in the words of Elton John - too low for zero
On Sun, 2011-11-27 at 00:25 -0300, Christian Grunfeld wrote:
> > 2011/11/24 Noel Butler <no...@ausics.net>:
> >> its up to them if they want to or not, the spam folders have very little
> >> in
> >> them here because of our approach, and in our tests we have had
> >> 0.00000001%
> >> of FP's in that, which is really good.
> >
> > 0.00000001% is 1 FP over 10.000.000.000 !!!!!! 1 over 10 billion mails
> > !!!!!!
> >
> > I really love people on this list !!!
> >
> > Your maths are flawed, you assume too much and wrongly.
> > We have 12 front ends, each process about 1.5 - 1.7 million ACCEPTED
> > messages a day, hence why we are so happy with the way things are. But hey,
> > imagine the resources we'd recover if we allowed little twirp spammers like
> > yourself the luxury of turning off anti spam measures and letting the users
> > decide.... Wait... Nah... I don't think so some how.
>
> My maths are flawed? I think yours are !
>
> 1% FP is 1 FP over 100 emails
> 0.1% is 1 over 1,000
> 0.01% is 1 over 10,000
> 0.001% is 1 over 100,.000
> ....
> I think you can continue
> ....
> 0.000,000,01% is 1 FP over 10,000,000,000 !!
>
> I'm not scared about your email volume...I doubt about your FP ratio !!!
Re: new paradigm
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
On Sun, 27 Nov 2011 00:25:59 -0300
Christian Grunfeld <ch...@gmail.com> wrote:
> 0.000,000,01% is 1 FP over 10,000,000,000 !!
> I'm not scared about your email volume...I doubt about your FP
> ratio !!!
I agree. I don't believe that FP ratio either.
Regards,
David.
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
> 2011/11/24 Noel Butler <no...@ausics.net>:
>> its up to them if they want to or not, the spam folders have very little
>> in
>> them here because of our approach, and in our tests we have had
>> 0.00000001%
>> of FP's in that, which is really good.
>
> 0.00000001% is 1 FP over 10.000.000.000 !!!!!! 1 over 10 billion mails
> !!!!!!
>
> I really love people on this list !!!
>
> Your maths are flawed, you assume too much and wrongly.
> We have 12 front ends, each process about 1.5 - 1.7 million ACCEPTED
> messages a day, hence why we are so happy with the way things are. But hey,
> imagine the resources we'd recover if we allowed little twirp spammers like
> yourself the luxury of turning off anti spam measures and letting the users
> decide.... Wait... Nah... I don't think so some how.
My maths are flawed? I think yours are !
1% FP is 1 FP over 100 emails
0.1% is 1 over 1,000
0.01% is 1 over 10,000
0.001% is 1 over 100,.000
....
I think you can continue
....
0.000,000,01% is 1 FP over 10,000,000,000 !!
I'm not scared about your email volume...I doubt about your FP ratio !!!
Re: new paradigm
Posted by Noel Butler <no...@ausics.net>.
On Sat, 2011-11-26 at 12:21 -0300, Christian Grunfeld wrote:
> 2011/11/24 Noel Butler <no...@ausics.net>:
> > its up to them if they want to or not, the spam folders have very little in
> > them here because of our approach, and in our tests we have had 0.00000001%
> > of FP's in that, which is really good.
>
> 0.00000001% is 1 FP over 10.000.000.000 !!!!!! 1 over 10 billion mails !!!!!!
>
> I really love people on this list !!!
Your maths are flawed, you assume too much and wrongly.
We have 12 front ends, each process about 1.5 - 1.7 million ACCEPTED
messages a day, hence why we are so happy with the way things are. But
hey, imagine the resources we'd recover if we allowed little twirp
spammers like yourself the luxury of turning off anti spam measures and
letting the users decide.... Wait... Nah... I don't think so some how.
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
2011/11/24 Noel Butler <no...@ausics.net>:
> its up to them if they want to or not, the spam folders have very little in
> them here because of our approach, and in our tests we have had 0.00000001%
> of FP's in that, which is really good.
0.00000001% is 1 FP over 10.000.000.000 !!!!!! 1 over 10 billion mails !!!!!!
I really love people on this list !!!
Re: new paradigm
Posted by Noel Butler <no...@ausics.net>.
On Wed, 2011-11-23 at 15:30 -0300, Christian Grunfeld wrote:
> >> *check spam folder always
> >
> > Well, if I have to do *that*, I might as well not do any filtering at all.
> > The whole purpose of anti-spam software is to shield me from spam.
>
> Not 100% correct. Now I always check spam folder, dont you?
never, because I trust our SA and local rules
> Do you advise your people not to check spam folders? Are you 100% sure
> that machines can sort 100% efectively what is spam and what is not?
>
its up to them if they want to or not, the spam folders have very little
in them here because of our approach, and in our tests we have had
0.00000001% of FP's in that, which is really good.
> >> Here you round the trip and legitimate the sender !
> >
> > Which sender? The envelope sender? The From: header sender? The
> > Reply-To: sender? Or the Sender: sender?
>
> well the From: header is what users see and trust. They do not know
> about any other sender information
I guess those that actually do check spam folders only look at the
displayed name in the run down list (without going into any of the
spams), we also replace the subject with just "[spam] attached" meaning
all those dropkick deadhead spammers who put their spam in subject line
waste their time as it never gets seen, so on whole I guess the users
ignore any names they dont recognise, our system auto deletes spam after
7 days, the spam folders never contribute to any quotas so its a win-win
for our users.
Re: new paradigm
Posted by Kris Deugau <kd...@vianet.ca>.
Christian Grunfeld wrote:
> Not 100% correct. Now I always check spam folder, dont you?
> Do you advise your people not to check spam folders? Are you 100% sure
> that machines can sort 100% efectively what is spam and what is not?
SpamAssassin, in the installations I maintain, is accurate *enough* that
I only poke into my spam folders now and then to see if there's anything
entertaining.
On my personal account, I haven't looked at anything tagged by SA for
quite a while - probably since the last major version update.
-kgd
Re: new paradigm
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
On Wed, 23 Nov 2011 15:30:08 -0300
Christian Grunfeld <ch...@gmail.com> wrote:
> > Well, if I have to do *that*, I might as well not do any filtering
> > at all. The whole purpose of anti-spam software is to shield me
> > from spam.
> Not 100% correct. Now I always check spam folder, dont you?
I do have a quarantine. However, some of my spam doesn't even go
into the quarantine. It just gets outright rejected without my seeing it.
We do have thousands of end-users on our hosted system, and I can tell
you this: About 90% of them *never* check their quarantine. If you're
not a technical person, it's not a priority.
> Do you advise your people not to check spam folders? Are you 100% sure
> that machines can sort 100% efectively what is spam and what is not?
I'm confident enough in our system's abilities to trust it to auto-reject
a good percentage of my spam. Let me see... I have 1316 rejected messages
in my recent history. 831 were auto-rejected. So more than 63%.
We auto-reject with a 5xx SMTP failure code, so in the unlikely event
of a FP, the sender will know about it.
[...]
> > Which sender? The envelope sender? The From: header sender? The
> > Reply-To: sender? Or the Sender: sender?
> well the From: header is what users see and trust. They do not know
> about any other sender information
A user who replies to a message with a Reply-To: header won't reply to
the From: address. So users will need education and this is a
non-starter.
Regards,
David.
Re: new paradigm
Posted by Christian Grunfeld <ch...@gmail.com>.
>> *check spam folder always
>
> Well, if I have to do *that*, I might as well not do any filtering at all.
> The whole purpose of anti-spam software is to shield me from spam.
Not 100% correct. Now I always check spam folder, dont you?
Do you advise your people not to check spam folders? Are you 100% sure
that machines can sort 100% efectively what is spam and what is not?
>> Here you round the trip and legitimate the sender !
>
> Which sender? The envelope sender? The From: header sender? The
> Reply-To: sender? Or the Sender: sender?
well the From: header is what users see and trust. They do not know
about any other sender information
Re: new paradigm
Posted by "David F. Skoll" <df...@roaringpenguin.com>.
On Wed, 23 Nov 2011 14:55:46 -0300
Christian Grunfeld <ch...@gmail.com> wrote:
> So the idea is...in this days where the ratio of spam/ham is about 80%
> (put the ratio you want but be sure it is high enough) lets start with
> marking all incomings as spam !
The cure is worse than the disease, because:
> *check spam folder always
Well, if I have to do *that*, I might as well not do any filtering at all.
The whole purpose of anti-spam software is to shield me from spam.
> Here you round the trip and legitimate the sender !
Which sender? The envelope sender? The From: header sender? The
Reply-To: sender? Or the Sender: sender?
Email is messy, alas.
> False positives....yes, ONLY the first time for each sender!
This pretty much kills the idea for most businesses. It could work for
personal email.
Regards,
David.