You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Dianne Skoll <df...@roaringpenguin.com> on 2016/07/29 20:48:30 UTC
Is greylisting effective? (was Re: Using Postfix and Postgrey - not
scanning after hold)
On Fri, 29 Jul 2016 22:39:15 +0200
Robert Schetterer <rs...@sys4.de> wrote:
> > I don't use postfix or postscreen.
> hm.. that does not fit the subject..why did you involved yourself ?
I am sorry. I should have changed the thread subject.
> you may get that quite better, i see
> a lot of server greylisting useless ,only filling up others queues
> waiting for a second slot ,so it may only cheap for you but not for
> your partners
> Dont slow down communication if you dont need to
So what I didn't mention is that in our implementation, once an IP
address successully passes greylisting, we no longer greylist it for
the next 45 days. (It would probably be pointless... if an IP passes
greylisting once, it probably will keep passing it.)
The impact on legitimate mail senders is therefore very minimal.
> its up to you using more up2date tec stuff, be sure postscreen
> is used in equal setups then yours and is well tested
I don't quite understand that last sentence...
Regards,
Dianne.
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Reindl Harald <h....@thelounge.net>.
Am 01.08.2016 um 19:02 schrieb Matus UHLAR - fantomas:
>> On 31 Jul 2016, at 22:12, Benny Pedersen <me...@junc.eu> wrote:
>>> i bet greylist is cough invalid mailservers at the doorstep, it could
>>> be that postscreen is bad aswell ?
>
> On 01.08.16 07:46, @lbutlr wrote:
>> Sure, if by “invalid” you mean Amazon, most banks, several airlines,
>> large
>> mail services, and many many others.
>>
>> Nearly any company with multiple mail servers will send mail from any of
>> their servers, and may retry from a different server than the initial
>> attempt, thus resetting the greylist.
>
> while we're at it, I really don't understand why they do it like this.
> what's the point behind changing IP address after each delivery attempt?
load balancing of the outgoing mailqueue
these are not single instance servers
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Reindl Harald <h....@thelounge.net>.
Am 02.08.2016 um 00:05 schrieb Benny Pedersen:
> On 2016-08-01 19:02, Matus UHLAR - fantomas wrote:
>
>> while we're at it, I really don't understand why they do it like this.
>> what's the point behind changing IP address after each delivery attempt?
>
> goal is to expose more networks ips to be blocked at the recipient
> server for abuse, ironical :)
fascinating how a single human can talk that much nonsense from the
viewpoint of his micro-world of a simple inbound server for himself and
his brother....
go ahadea and block the wolrd excpect your worlds whitelist
that will stop to scale frm the moment you are responsible for others
mails where the RCPT don't care about yoor narrow-sighted viewpoint and
hire some other mailadmin which touch of the reality
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Benny Pedersen <me...@junc.eu>.
On 2016-08-01 19:02, Matus UHLAR - fantomas wrote:
> while we're at it, I really don't understand why they do it like this.
> what's the point behind changing IP address after each delivery
> attempt?
goal is to expose more networks ips to be blocked at the recipient
server for abuse, ironical :)
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by "@lbutlr" <kr...@kreme.com>.
On 01 Aug 2016, at 11:02, Matus UHLAR - fantomas <uh...@fantomas.sk> wrote:
>> On 31 Jul 2016, at 22:12, Benny Pedersen <me...@junc.eu> wrote:
>>> i bet greylist is cough invalid mailservers at the doorstep, it could be that postscreen is bad aswell ?
>
> On 01.08.16 07:46, @lbutlr wrote:
>> Sure, if by “invalid” you mean Amazon, most banks, several airlines, large
>> mail services, and many many others.
>>
>> Nearly any company with multiple mail servers will send mail from any of
>> their servers, and may retry from a different server than the initial
>> attempt, thus resetting the greylist.
>
> while we're at it, I really don't understand why they do it like this.
> what's the point behind changing IP address after each delivery attempt?
It’s not necessarily intentional.
I have 100 mail servers. I send a mail to someone. It goes to one of the 100 mail servers to get sent out. It has a temp fail. I queue the mail up send again in 15 minutes. It goes to one of the 100 mail server to get sent out.
Lather, rinse, repeat.
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Benny Pedersen <me...@junc.eu>.
On 2016-08-01 19:18, Larry Rosenman wrote:
> Shared outbound spool, and the next available host sends it.
next host start a new greylist time frame to delay again
> It's not nefarious, just load balancing.
yes misunderstanding what not to do
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Larry Rosenman <le...@lerctr.org>.
On 2016-08-01 12:02, Matus UHLAR - fantomas wrote:
>> On 31 Jul 2016, at 22:12, Benny Pedersen <me...@junc.eu> wrote:
>>> i bet greylist is cough invalid mailservers at the doorstep, it could
>>> be that postscreen is bad aswell ?
>
> On 01.08.16 07:46, @lbutlr wrote:
>> Sure, if by \u201cinvalid\u201d you mean Amazon, most banks, several airlines,
>> large
>> mail services, and many many others.
>>
>> Nearly any company with multiple mail servers will send mail from any
>> of
>> their servers, and may retry from a different server than the initial
>> attempt, thus resetting the greylist.
>
> while we're at it, I really don't understand why they do it like this.
> what's the point behind changing IP address after each delivery
> attempt?
Shared outbound spool, and the next available host sends it.
It's not nefarious, just load balancing.
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 E-Mail: ler@lerctr.org
US Mail: 17716 Limpia Crk, Round Rock, TX 78664-7281
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
>On 31 Jul 2016, at 22:12, Benny Pedersen <me...@junc.eu> wrote:
>> i bet greylist is cough invalid mailservers at the doorstep, it could be that postscreen is bad aswell ?
On 01.08.16 07:46, @lbutlr wrote:
>Sure, if by \u201cinvalid\u201d you mean Amazon, most banks, several airlines, large
> mail services, and many many others.
>
>Nearly any company with multiple mail servers will send mail from any of
> their servers, and may retry from a different server than the initial
> attempt, thus resetting the greylist.
while we're at it, I really don't understand why they do it like this.
what's the point behind changing IP address after each delivery attempt?
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
A day without sunshine is like, night.
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Robert Schetterer <rs...@sys4.de>.
Am 04.08.2016 um 22:30 schrieb Chris:
> Greylisting is just one of several tools available to a system
> administrator for filtering out spam
as multiple described it does not
Best Regards
MfG Robert Schetterer
--
[*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64
Schlei�heimer Stra�e 26/MG, 80333 M�nchen
Sitz der Gesellschaft: M�nchen, Amtsgericht M�nchen: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Chris <ch...@gmail.com>.
I do not use postfix but I do greylist so I thought I would chime in
with my opinion.
Greylisting is just one of several tools available to a system
administrator for filtering out spam, like any of the other tools if
used incorrectly it will be problematic.
I do much cheaper filtering first before the server even considers
greylisting, filtering which does not require calling external tools,
does not require dns lookups. That kind of filtering should be done
on messages first as it is cheaper. Next I do not greylist every
single email, that would be excessive, I know some sysadmins do this,
but it would cause too much delays causing complaints and increase
server load with more retries. Instead I greylist email's that come
from server's that fail basic 101 configuration checks, such as a
mismatched/missing rdns record, or failed spf check. Whilst running
in this configuration for a number of years I have had zilch
complaints of missing emails, only the occasional moan about delayed
emails. I also configure my server's so that end users who decide to
opt out can opt out, I have a whitelist file with target domain's that
will allow these failed rdns/spf emails to be delivered immediately
although they will still be subject to other checks unless whitelisted
in other checks also.
Regarding RBL lists, this one is perhaps not so simple, I do outright
block email's from certian lists I consider to be very reliable, aware
that occasionally the likes of gmail may find themselves on such a
list I exclude the major email providers from RBL checks, this of
course also reduces queries sent to those providers. Plus as with the
greylisting, customers of mine can opt out of these checks.
List providers with history of false positives I tend to not use or I
may use them when they have a record of expiring senders quickly but
only using defer instead of deny, which should make the sender
reattempt delivery later. I do not yet have a internal scoring
system, the only scoring system I use is spamassassin which is ok but
I found over the years it is definitely becoming less effective.
My plan is to combine RBL providers alongside some other spam
networking communities and use a scoring system, so I can do away with
the outright blocking, as although I do not get complaints, I respect
there is always the possibility of false positives.
There is also the option of delaying the incoming server for a few
seconds before allowing it to proceed, this can weed out spammers as
well who dont like been slowed down so may skip over it.
Chris
On 2 August 2016 at 22:18, John Hardin <jh...@impsec.org> wrote:
> On Tue, 2 Aug 2016, Benny Pedersen wrote:
>
>> On 2016-08-02 20:00, John Hardin wrote:
>>
>>> Is there any way to use postscreen as a frontend filter for a sendmail
>>> MTA?
>>
>>
>> content-filter works nicely in postfix, but that postscreen will not use
>> content-filter to help on its problem
>>
>> postfix can use sendmail as a content-filter
>
>
> Guffaw.
>
>> what goal ?
>
>
> To get the benefits of postscreen without replacing my working sendmail
> install.
>
>
> --
> John Hardin KA7OHZ http://www.impsec.org/~jhardin/
> jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
> key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
> -----------------------------------------------------------------------
> Vista "security improvements" consist of attempting to shift blame
> onto the user when things go wrong.
>
> -----------------------------------------------------------------------
> 3 days until the 281st anniversary of John Peter Zenger's acquittal
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey
- not scanning after hold)
Posted by John Hardin <jh...@impsec.org>.
On Tue, 2 Aug 2016, Benny Pedersen wrote:
> On 2016-08-02 20:00, John Hardin wrote:
>
>> Is there any way to use postscreen as a frontend filter for a sendmail
>> MTA?
>
> content-filter works nicely in postfix, but that postscreen will not use
> content-filter to help on its problem
>
> postfix can use sendmail as a content-filter
Guffaw.
> what goal ?
To get the benefits of postscreen without replacing my working sendmail
install.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Vista "security improvements" consist of attempting to shift blame
onto the user when things go wrong.
-----------------------------------------------------------------------
3 days until the 281st anniversary of John Peter Zenger's acquittal
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Reindl Harald <h....@thelounge.net>.
Am 02.08.2016 um 23:02 schrieb Benny Pedersen:
> On 2016-08-02 20:00, John Hardin wrote:
>
>> Is there any way to use postscreen as a frontend filter for a sendmail
>> MTA?
>
> content-filter works nicely in postfix
which is not the topic
> but that postscreen will not use
> content-filter to help on its problem
that's why it's not the topic
> postfix can use sendmail as a content-filter
> what goal?
*postscreen as it is* in front of something else
there is nothing complex in the question you quoted above
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Benny Pedersen <me...@junc.eu>.
On 2016-08-02 21:27, Robert Schetterer wrote:
> you may use a complete postfix server including postscreen etc "before"
> sendmail....but then it might better to simply change to postfix in
> total, but such setups are often use with MS exchange
if that can serve as a content-filter it could be used in postfix
standard smtp is nice imho :=)
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Robert Schetterer <rs...@sys4.de>.
Am 02.08.2016 um 20:04 schrieb Reindl Harald:
>
>
> Am 02.08.2016 um 20:00 schrieb John Hardin:
>> On Tue, 2 Aug 2016, Bill Cole wrote:
>>
>>> What's special about the postscreen delay is:
>>>
>>> 1. It delays only the last line of a multi-line greeting, so it
>>> catches MANY more bots than a simple delay.
>>>
>>> 2. It caches PASS results so even the very short (6s by default) delay
>>> that it imposes only hits the first encounter with a client that
>>> connects frequently. This is critically important in high-volume
>>> situations where the difference between mean session lengths of 0.5s
>>> and 6s is the difference between 2 and 12 MX boxes in a cluster.
>>>
>>> Combined, this is why Sendmail and other MTA greeting delays are less
>>> spectacularly effective than they used to be and less effective than
>>> postscreen. The resource cost of prolonging every session to 6s is
>>> untenable for busy machines, so bots that have adapted can get
>>> through. Back in the early days of Sendmail's GreetPause a value of 3s
>>> would catch most bots but over the years some bots have adapted by
>>> doing their own hard delays and others have learned to wait for
>>> anything from the server. Few (if any) have adapted by actually
>>> parsing the greeting and making sure that they've seen the end of a
>>> multi-line greeting before talking.
>>
>> That all sounds great.
>>
>> Is there any way to use postscreen as a frontend filter for a sendmail
>> MTA?
>
> no - postscreen is not a smtp proxy
>
> in fact the connection is handed over from postcreen to the smtpd
> process after a client has passed the tests
>
you may use a complete postfix server including postscreen etc "before"
sendmail....but then it might better to simply change to postfix in
total, but such setups are often use with MS exchange
Best Regards
MfG Robert Schetterer
--
[*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64
Schleiheimer Strae 26/MG, 80333 Mnchen
Sitz der Gesellschaft: Mnchen, Amtsgericht Mnchen: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Reindl Harald <h....@thelounge.net>.
Am 01.08.2016 um 23:36 schrieb shanew@shanew.net:
> Others could probably add to that list, but that's just off the top of
> my head. But, even if a spam source retries and successfully makes it
> past the greylisting, the greylisting still provides potential
> benefits, like:
>
> - While it was waiting to retry, its IP has been added to BLs, which
> my other filters will score appropriately
> - While it was waiting to retry, the phishing URL in it has been
> reported and taken down (or the URL shortener link it used has been
> removed)
> - While it was waiting to retry, the virus it carries has been
> identified and pushed out to my virus definitions
> - While it was waiting to retry, its registered domain has been
> removed
> - While it was waiting to retry, others who received the spam have
> reported it to services like Razor and DCC, which other filters will
> act on
excatly that's the point
whe had last month 1212 greylistings
a majority was blacklisted later, bet it RBL or URIBL
well, 1212 is not much of the overall mailflow, the reason is just
"knowing what you are doing" and have greylisting only as last resort
before contentfllters and skip it if the sender matches SPF or is on any
known DSNWL
finally that means zero bad impact for regular mail
if only *one* phishing mail which would have trapped a user was rejected
with *zero* costs on a wise setup it has done it's job
again: the point is not to delay everybody, it's about to delay pure
junk senders wihouth touching anything which has a single reputation
sign and that goal won't change at any point of time - just becasue of
the zero-cost and zero-impact for any regular mailflow
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey
- not scanning after hold)
Posted by sh...@shanew.net.
On Sun, 31 Jul 2016, Robert Schetterer wrote:
> Greylisting was invented as an idea against bots. Its based on the idea
> that bots "fire and forget" when they see a tmp error and dont get back.
>
> But thats historic, bots are recoded, better antibot tecs were invented.
> The only problem now is people still believe in historic stuff.
This argument ignores two important facts.
First, even if 98% of bots and viruses (and that number is pure
conjecture on my part) are now smart enough to retry, that doesn't
change that greylisting is a just about the lowest "cost" way of
preventing the ones that aren't smart enough (or aren't designed to
retry because they want to push the most amount of junk at the
lowest-hanging fruit).
Second, the ability of a bot, virus, server or any other spam source
to retry delivery after a temp failure is not the only "weakness"
greylisting takes advantage of. A spam source might not get past my
greylist for any number of reasons, including the classic case of poor
coding/design, but also:
- It is detected and blocked (or taken offline) by the source network
before its greylist period is up
- It make use of a compromised account, and that account is disabled
or secured before its greylist period is up
- It is part of a distributed botnet, so subsequent attempts come from
a different IP/network
- It sends a high volume of spam, so it doesn't come back around to
retry again until after its entry has been removed, requiring
a whole new greylisting period
Others could probably add to that list, but that's just off the top of
my head. But, even if a spam source retries and successfully makes it
past the greylisting, the greylisting still provides potential
benefits, like:
- While it was waiting to retry, its IP has been added to BLs, which
my other filters will score appropriately
- While it was waiting to retry, the phishing URL in it has been
reported and taken down (or the URL shortener link it used has been
removed)
- While it was waiting to retry, the virus it carries has been
identified and pushed out to my virus definitions
- While it was waiting to retry, its registered domain has been
removed
- While it was waiting to retry, others who received the spam have
reported it to services like Razor and DCC, which other filters will
act on
- If it has to keep retrying addresses to my server, I'm consuming
resources (however minimally) that could be used to send their junk
to others
Again, I'm sure others could add more based on their experiences.
I'm not saying greylisting is without problems, that it just works out
of the box (initial and ongoing configuration is critical), or that
everyone should be using it, but there's a lot more going on here than
just outwitting poorly written bots.
--
Public key #7BBC68D9 at | Shane Williams
http://pgp.mit.edu/ | System Admin - UT CompSci
=----------------------------------+-------------------------------
All syllogisms contain three lines | shanew@shanew.net
Therefore this is not a syllogism | www.ischool.utexas.edu/~shanew
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Reindl Harald <h....@thelounge.net>.
Am 02.08.2016 um 18:55 schrieb Bill Cole:
> Combined, this is why Sendmail and other MTA greeting delays are less
> spectacularly effective than they used to be and less effective than
> postscreen. The resource cost of prolonging every session to 6s is
> untenable for busy machines, so bots that have adapted can get through.
> Back in the early days of Sendmail's GreetPause a value of 3s would
> catch most bots but over the years some bots have adapted by doing their
> own hard delays and others have learned to wait for anything from the
> server. Few (if any) have adapted by actually parsing the greeting and
> making sure that they've seen the end of a multi-line greeting before
> talking
in fact most bots have a timeout around 10 seconds
postscreen_greet_wait = ${stress?3}${stress:11}s
and you will see a massive drop down of rejected mails in mailgraph
because they all hang up after 10 seconds and since this result in
postscreen is cached a legit client must only pass it one time while the
bot not passing the tests hang forever there
well, and that greet_wait is also a good thing for all the scored
dnsbl/dnswl because if there are some slow repsonding they don't get
skipped and so the bot sits there useless waiting 11 seconds to get a
"blocked using highest scored DNSBl"
postscreen other than smtpd is a single process, dnsbl/dnswl requests
are done by extra, shared processes and so postscreen is unbeatable for
years now if it comes to a inbound MX
after that you have 200-800 rejected mails in your contentfilters and
smtpd-restricitions compared ot many thousands without
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Reindl Harald <h....@thelounge.net>.
Am 02.08.2016 um 20:00 schrieb John Hardin:
> On Tue, 2 Aug 2016, Bill Cole wrote:
>
>> What's special about the postscreen delay is:
>>
>> 1. It delays only the last line of a multi-line greeting, so it
>> catches MANY more bots than a simple delay.
>>
>> 2. It caches PASS results so even the very short (6s by default) delay
>> that it imposes only hits the first encounter with a client that
>> connects frequently. This is critically important in high-volume
>> situations where the difference between mean session lengths of 0.5s
>> and 6s is the difference between 2 and 12 MX boxes in a cluster.
>>
>> Combined, this is why Sendmail and other MTA greeting delays are less
>> spectacularly effective than they used to be and less effective than
>> postscreen. The resource cost of prolonging every session to 6s is
>> untenable for busy machines, so bots that have adapted can get
>> through. Back in the early days of Sendmail's GreetPause a value of 3s
>> would catch most bots but over the years some bots have adapted by
>> doing their own hard delays and others have learned to wait for
>> anything from the server. Few (if any) have adapted by actually
>> parsing the greeting and making sure that they've seen the end of a
>> multi-line greeting before talking.
>
> That all sounds great.
>
> Is there any way to use postscreen as a frontend filter for a sendmail MTA?
no - postscreen is not a smtp proxy
in fact the connection is handed over from postcreen to the smtpd
process after a client has passed the tests
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 2 Aug 2016, at 14:00, John Hardin wrote:
> On Tue, 2 Aug 2016, Bill Cole wrote:
>
>> What's special about the postscreen delay is:
>>
>> 1. It delays only the last line of a multi-line greeting, so it
>> catches MANY more bots than a simple delay.
>>
>> 2. It caches PASS results so even the very short (6s by default)
>> delay that it imposes only hits the first encounter with a client
>> that connects frequently. This is critically important in high-volume
>> situations where the difference between mean session lengths of 0.5s
>> and 6s is the difference between 2 and 12 MX boxes in a cluster.
>>
>> Combined, this is why Sendmail and other MTA greeting delays are less
>> spectacularly effective than they used to be and less effective than
>> postscreen. The resource cost of prolonging every session to 6s is
>> untenable for busy machines, so bots that have adapted can get
>> through. Back in the early days of Sendmail's GreetPause a value of
>> 3s would catch most bots but over the years some bots have adapted by
>> doing their own hard delays and others have learned to wait for
>> anything from the server. Few (if any) have adapted by actually
>> parsing the greeting and making sure that they've seen the end of a
>> multi-line greeting before talking.
>
> That all sounds great.
>
> Is there any way to use postscreen as a frontend filter for a sendmail
> MTA?
Not currently.
Architecturally, Postfix and Sendmail are so different that it isn't
immediately obvious how one would hook postscreen to a sendmail process.
A postscreen process accepts connections over the net and hands off the
file descriptor for a "passing" connection to a smtpd process via a unix
domain socket, after which it is not involved in that session. It's not
a proxy or a pass-through filter, it's a connection handler.
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Benny Pedersen <me...@junc.eu>.
On 2016-08-02 20:00, John Hardin wrote:
> Is there any way to use postscreen as a frontend filter for a sendmail
> MTA?
content-filter works nicely in postfix, but that postscreen will not use
content-filter to help on its problem
postfix can use sendmail as a content-filter
what goal ?
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey
- not scanning after hold)
Posted by John Hardin <jh...@impsec.org>.
On Tue, 2 Aug 2016, Bill Cole wrote:
> What's special about the postscreen delay is:
>
> 1. It delays only the last line of a multi-line greeting, so it catches MANY
> more bots than a simple delay.
>
> 2. It caches PASS results so even the very short (6s by default) delay that
> it imposes only hits the first encounter with a client that connects
> frequently. This is critically important in high-volume situations where the
> difference between mean session lengths of 0.5s and 6s is the difference
> between 2 and 12 MX boxes in a cluster.
>
> Combined, this is why Sendmail and other MTA greeting delays are less
> spectacularly effective than they used to be and less effective than
> postscreen. The resource cost of prolonging every session to 6s is untenable
> for busy machines, so bots that have adapted can get through. Back in the
> early days of Sendmail's GreetPause a value of 3s would catch most bots but
> over the years some bots have adapted by doing their own hard delays and
> others have learned to wait for anything from the server. Few (if any) have
> adapted by actually parsing the greeting and making sure that they've seen
> the end of a multi-line greeting before talking.
That all sounds great.
Is there any way to use postscreen as a frontend filter for a sendmail
MTA?
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
From the Liberty perspective, it doesn't matter if it's a
jackboot or a Birkenstock smashing your face. -- Robb Allen
-----------------------------------------------------------------------
3 days until the 281st anniversary of John Peter Zenger's acquittal
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey - not scanning after hold)
Posted by Ryan Coleman <ry...@cwis.biz>.
> On Aug 1, 2016, at 10:15 AM, Benny Pedersen <me...@junc.eu> wrote:
>
>>>
>>> i bet greylist is cough invalid mailservers at the doorstep, it could be that postscreen is bad aswell ?
>> Sure, if by “invalid” you mean Amazon, most banks, several airlines,
>> large mail services, and many many others.
>
> basicly badly marketing
>
>> Nearly any company with multiple mail servers will send mail from any
>> of their servers, and may retry from a different server than the
>> initial attempt, thus resetting the greylist.
>
> marketing trys to fuck how smtp works have always being a fail with greylist
What on earth are you trying to say here? Because after my 20 years in the industry I haven’t a clue what you’re saying.
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Benny Pedersen <me...@junc.eu>.
On 2016-08-01 15:46, @lbutlr wrote:
> Where did you get the idea that postfix will not deliver later?
i did not say that
>> i bet greylist is cough invalid mailservers at the doorstep, it could
>> be that postscreen is bad aswell ?
>
> Sure, if by \u201cinvalid\u201d you mean Amazon, most banks, several airlines,
> large mail services, and many many others.
basicly badly marketing
> Nearly any company with multiple mail servers will send mail from any
> of their servers, and may retry from a different server than the
> initial attempt, thus resetting the greylist.
marketing trys to fuck how smtp works have always being a fail with
greylist
> There is a reason that greylist software comes with default
> exclusions, because greylisting is known to cause missed email if used
> as designed.
postscreen dont care at all
take that
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by "@lbutlr" <kr...@kreme.com>.
On 31 Jul 2016, at 22:12, Benny Pedersen <me...@junc.eu> wrote:
> On 2016-08-01 05:55, @lbutlr wrote:
>> On 31 Jul 2016, at 01:06, Robert Schetterer <rs...@sys4.de> wrote:
>>> But thats historic, bots are recoded, better antibot tecs were invented.
>>> The only problem now is people still believe in historic stuff.
>> Yeah, that about sums it up. Greylisting never worked well, always
>> caused problems with lost email, and in 2016 is simply a bad idea. Not
>> just a not good idea, but a bad idea.
>
> back to basic then, why would a mta like postfix not deliver later when it get a tempfail ?
Where did you get the idea that postfix will not deliver later?
> i bet greylist is cough invalid mailservers at the doorstep, it could be that postscreen is bad aswell ?
Sure, if by “invalid” you mean Amazon, most banks, several airlines, large mail services, and many many others.
Nearly any company with multiple mail servers will send mail from any of their servers, and may retry from a different server than the initial attempt, thus resetting the greylist.
There is a reason that greylist software comes with default exclusions, because greylisting is known to cause missed email if used as designed.
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Benny Pedersen <me...@junc.eu>.
On 2016-08-01 05:55, @lbutlr wrote:
> On 31 Jul 2016, at 01:06, Robert Schetterer <rs...@sys4.de> wrote:
>> But thats historic, bots are recoded, better antibot tecs were
>> invented.
>> The only problem now is people still believe in historic stuff.
>
> Yeah, that about sums it up. Greylisting never worked well, always
> caused problems with lost email, and in 2016 is simply a bad idea. Not
> just a not good idea, but a bad idea.
back to basic then, why would a mta like postfix not deliver later when
it get a tempfail ?
i bet greylist is cough invalid mailservers at the doorstep, it could be
that postscreen is bad aswell ?
see in sqlgrey whitelist how many old ips that do not retry, shurg
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 1 Aug 2016, at 15:53, Axb wrote:
> On 01.08.2016 21:30, Vincent Fox wrote:
>> I keep seeing people say "well if you have postscreen, greylisting is
>> just dumb".
I think that's a bit too strong. Robust greylisting that accommodates
the reality of mail systems that share one spool across many outbound
machines is not "dumb" but it is substantially less useful if you have
any spambot protection before it. In the case of the original question,
the context was specific to Postfix, where postscreen is a highly
effective optional tool operating well ahead of any full greylisting
implementation and taking out most of the bots that greylisting would
with less risk and resource cost.
>> Well what is the equivalent for other MTA?
There is no complete equivalent for any other MTA that I am aware of.
Postscreen has a unique (as far as I know) greeting delay implementation
(see below) as well as scored DNSBL checking, which makes it more
feasible to use some hair-trigger DNSBLs safely than it is if you use
them as hard tests (by requiring multiple hits on iffy lists.) There are
also optional after-greeting behavioral checks which amount to
simplistic zero-minimum-time greylisting because a "passing" client gets
a 4xx from postscreen itself at RCPT, after which it must retry within
the cache lifetime (30d by default) to get normal handling. Most sites
don't enable the after-greeting tests because it has that greylist-like
effect without the protections of a good greylist implementation.
> google for "Greet pause" and "Early talker"
>
> afaik there's implementations for Sendmail
Which had GreetPause well before postscreen or even the weaker
pre-postscreen 'sleep' trick that predated postscreen.
> and Haraka. There may be something similar for EXIM , QMAIL may have
> some obscure patch..
CGP also implements a greeting delay.
What's special about the postscreen delay is:
1. It delays only the last line of a multi-line greeting, so it catches
MANY more bots than a simple delay.
2. It caches PASS results so even the very short (6s by default) delay
that it imposes only hits the first encounter with a client that
connects frequently. This is critically important in high-volume
situations where the difference between mean session lengths of 0.5s and
6s is the difference between 2 and 12 MX boxes in a cluster.
Combined, this is why Sendmail and other MTA greeting delays are less
spectacularly effective than they used to be and less effective than
postscreen. The resource cost of prolonging every session to 6s is
untenable for busy machines, so bots that have adapted can get through.
Back in the early days of Sendmail's GreetPause a value of 3s would
catch most bots but over the years some bots have adapted by doing their
own hard delays and others have learned to wait for anything from the
server. Few (if any) have adapted by actually parsing the greeting and
making sure that they've seen the end of a multi-line greeting before
talking.
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Vincent Fox <vb...@ucdavis.edu>.
I have greet_pause long ago enabled in sendmail.
Out of 1,112,871 messages yesterday only 1096 tripped greet_pause.
If it occasionally trips up a miscoded client I tell them to fix it or stop using
and am happy that this tiny effort made the internet a slightly saner place.
But it's not in the same league at all.
Between milter-greylist, and several measures, it covers quite a bit
at the first layer, before we have to get to the expensive SpamAssassin stuff.
The hate on greylisting seems a bit off to me. I realize it's not the most
effective tool in my toolbox, but it helps. There have been this year malware
blast campaigns, where I traced that the IP involved were blocked by the
MX servers with greylisting, and examples that got through came through
department MTA that did not.
On the whole I'd rather have postscreen, but I don't make those choices.
Thus we patch together a simulacrum.
________________________________
From: Axb <ax...@gmail.com>
Sent: Monday, August 1, 2016 12:53:27 PM
To: users@spamassassin.apache.org
Subject: Re: Is greylisting effective? (was Re: Using Postfix and Postgrey - not scanning after hold)
On 01.08.2016 21:30, Vincent Fox wrote:
> I keep seeing people say "well if you have postscreen, greylisting is just dumb".
>
> Well what is the equivalent for other MTA?
google for "Greet pause" and "Early talker"
afaik there's implementations for Sendmail and Haraka. There may be
something similar for EXIM , QMAIL may have some obscure patch..
Axb
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Axb <ax...@gmail.com>.
On 01.08.2016 21:30, Vincent Fox wrote:
> I keep seeing people say "well if you have postscreen, greylisting is just dumb".
>
> Well what is the equivalent for other MTA?
google for "Greet pause" and "Early talker"
afaik there's implementations for Sendmail and Haraka. There may be
something similar for EXIM , QMAIL may have some obscure patch..
Axb
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Vincent Fox <vb...@ucdavis.edu>.
I keep seeing people say "well if you have postscreen, greylisting is just dumb".
Well what is the equivalent for other MTA?
I still see a lot of spambots on PBL hosts, that never contact again. So the
blanket statement "bots are recoded" just doesn't jibe with what I see.
Maybe you could many bots, or newer bots, but not all of them in current
usage recognize the 4xx, wait and retry.
________________________________
From: @lbutlr <kr...@kreme.com>
Sent: Sunday, July 31, 2016 8:55 PM
To: users@spamassassin.apache.org
Subject: Re: Is greylisting effective? (was Re: Using Postfix and Postgrey - not scanning after hold)
On 31 Jul 2016, at 01:06, Robert Schetterer <rs...@sys4.de> wrote:
> But thats historic, bots are recoded, better antibot tecs were invented.
> The only problem now is people still believe in historic stuff.
Yeah, that about sums it up. Greylisting never worked well, always caused problems with lost email, and in 2016 is simply a bad idea. Not just a not good idea, but a bad idea.
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by "@lbutlr" <kr...@kreme.com>.
On 31 Jul 2016, at 01:06, Robert Schetterer <rs...@sys4.de> wrote:
> But thats historic, bots are recoded, better antibot tecs were invented.
> The only problem now is people still believe in historic stuff.
Yeah, that about sums it up. Greylisting never worked well, always caused problems with lost email, and in 2016 is simply a bad idea. Not just a not good idea, but a bad idea.
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Robert Schetterer <rs...@sys4.de>.
Am 30.07.2016 um 13:10 schrieb Kim Roar Foldy Hauge:
> On Sat, 30 Jul 2016, Robert Schetterer wrote:
>
>> Am 30.07.2016 um 03:34 schrieb Reindl Harald:
>>>
>>>
>>> Am 29.07.2016 um 22:48 schrieb Dianne Skoll:
>>>> On Fri, 29 Jul 2016 22:39:15 +0200
>>>> Robert Schetterer <rs...@sys4.de> wrote:
>>>>
>>>>>> I don't use postfix or postscreen.
>>>>> hm.. that does not fit the subject..why did you involved yourself ?
>>>>
>>>> I am sorry. I should have changed the thread subject.
>>>>
>>>>> you may get that quite better, i see
>>>>> a lot of server greylisting useless ,only filling up others queues
>>>>> waiting for a second slot ,so it may only cheap for you but not for
>>>>> your partners
>>>>> Dont slow down communication if you dont need to
>>>>
>>>> So what I didn't mention is that in our implementation, once an IP
>>>> address successully passes greylisting, we no longer greylist it for
>>>> the next 45 days. (It would probably be pointless... if an IP passes
>>>> greylisting once, it probably will keep passing it.)
>>>
>>> that's nothing special and postgrey does the same, the whole point of
>>> greylisting is that badly written bots don't try again (the same happens
>>> if they connect to a backup-MX responding with 4xx)
>>>
>>> also it don't help for clients which *do not* pass like large senders
>>> with outbound clusters coming each time from a different IP
>>>
>>> hence you skip greylisting based on DNSWL and spf-policyd because that
>>> big legit senders hit DNSWL or have a proper SPF while random bots of
>>> infected machines don't and this ones are your target for greylisting
>>>
>>>
>>>
>>
>> Harald is right, the goal has to be "reject" spam asap, not to tell
>> "come again later", i.e i had 4 bot cons per second, this will run out
>> the system of smtp slots rapidly which means any good sender isnt able
>> to sent mail too, greylisting makes such situations more worst.
>>
>
> I'm no expert here, but postgrey is usually a purely local test. It
> should terminate with a "currently busy, try again later" message very
> quickly. SPF checks and white listing require dns lookups that can
> potentially take much longer. Several orders of magnitude longer.
>
> Efficient handling of spam is all about doing the least expensive tests
> first in terms of cpu/time. Caching DNS can probably help a bit, but it
> will still require the occasional lookup now and then that take a lot
> longer than a good greylisting implementation should ever do.
>
> Doing an expensive test on every mail when it's not needed is badly
> designed setup.
>
> Many of the dns based lists also limit the amount of checks per day.
> Worst case scenario, you stop getting results from lists due to over
> use. If you use google's 8.8.8.8 servers for dns lookups one can quickly
> run into that problem, I did. A high volume of dns checks could force
> you into having to pay for the amount of traffic you cause.
>
> Many expensive network (takes a long time) checks will probably make you
> run out of slots a lot faster than the reconnects due to greylisting
> will do due to the time spent waiting for the lookups to finish.
>
> If speed of delivery is important, you could lower the amount of time
> mail stays greylisted. Ideally you'd like the mail delivered the first
> time a server tries to send it again. If a server tries to resend once,
> it will most likely try more than once anyway. Having a minimum time of
> 300 seconds, the default of postgrey, is probably a bit excessive.
>
Greylisting was invented as an idea against bots. Its based on the idea
that bots "fire and forget" when they see a tmp error and dont get back.
This idea was criticized for design failures since it exists ( Harald
and me explained it in detail ), but was acceptable in lack of better
ideas that time.
But thats historic, bots are recoded, better antibot tecs were invented.
The only problem now is people still believe in historic stuff.
Best Regards
MfG Robert Schetterer
--
[*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64
Schleiheimer Strae 26/MG, 80333 Mnchen
Sitz der Gesellschaft: Mnchen, Amtsgericht Mnchen: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Reindl Harald <h....@thelounge.net>.
Am 30.07.2016 um 23:10 schrieb Bill Cole:
> On 30 Jul 2016, at 7:10, Kim Roar Foldøy Hauge wrote:
>
>> I'm no expert here, but postgrey is usually a purely local test. It
>> should terminate with a "currently busy, try again later" message very
>> quickly.
>
> Unless your database is very large, yes.
>
>> SPF checks and white listing require dns lookups that can potentially
>> take much longer. Several orders of magnitude longer.
>
> Occasionally, yes, no matter how you do DNS. However, Postfix smtpd will
> do multiple DNS lookups that have a strong chance of being slow before
> getting to any policy daemon like Postgrey. Alternatively, postscreen is
> a much simpler process than smtpd and in its usual config does nearly no
> input processing while doing DNSBL lookups in parallel, time-limited to
> 10s. This is usually much more efficient for dealing with spambots than
> going through everything that smtpd does before calling any policy
> daemon. Most DNSBLs worth using have robust authoritative DNS, so
> (unlike SPF or IP->Hostname->IP checking, both of which can require many
> queries) it is rarely slow to get DNSBL results
and it don't eat any smtpd process, responses shoudl be cached by a
local, rescursing resolver anyways and finally postfix-smtpd/postrey
have only to deal with a very low volume and the very expensive content
filter sees mostly ham
*if* a message makes it through postscreen and up to greylisting the
DNSWL and SPF results are cached anyways and re-used by spamasssassin
which can even re-use the spf header of the python policyd
our ould MX had peaks with 3000 MHz on the ESXi cluster, the current one
after switch to postscreen and configure things proper including
shortcircuit is running most of the time with 60-250 Mhz by a amount of
15000 destination addresses spammers try to deliver crap
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 30 Jul 2016, at 7:10, Kim Roar Fold�y Hauge wrote:
> I'm no expert here, but postgrey is usually a purely local test. It
> should terminate with a "currently busy, try again later" message very
> quickly.
Unless your database is very large, yes.
> SPF checks and white listing require dns lookups that can potentially
> take much longer. Several orders of magnitude longer.
Occasionally, yes, no matter how you do DNS. However, Postfix smtpd will
do multiple DNS lookups that have a strong chance of being slow before
getting to any policy daemon like Postgrey. Alternatively, postscreen is
a much simpler process than smtpd and in its usual config does nearly no
input processing while doing DNSBL lookups in parallel, time-limited to
10s. This is usually much more efficient for dealing with spambots than
going through everything that smtpd does before calling any policy
daemon. Most DNSBLs worth using have robust authoritative DNS, so
(unlike SPF or IP->Hostname->IP checking, both of which can require many
queries) it is rarely slow to get DNSBL results.
> Efficient handling of spam is all about doing the least expensive
> tests first in terms of cpu/time. Caching DNS can probably help a bit,
> but it will still require the occasional lookup now and then that take
> a lot longer than a good greylisting implementation should ever do.
Actually, having a local (same machine or same LAN) caching DNS resolver
is extremely helpful relative to using a resolver on the other side of a
router (i.e. an ISP's resolver) or worse, somewhere on the other side of
2-20 routers (i.e. OpenDNS, Google Public DNS, etc.) A lookup from a
local cache typically takes <10ms, while a lookup from a public DNS
server routinely will have 30-100ms of network latency built in and even
one's connectivity provider's DNS will probably have at least 10ms. In
any case, a DNS lookup that isn't in a resolver's cache is likely to
take on the order of 100ms (but potentially multiple seconds to timeout
on lookups that lead nowhere, indirectly) on top of the network latency.
This leads to another reason to have a local resolver dedicated to mail
servers: you can tune reply timeouts to assure that you never suffer
those really long waits.
> Doing an expensive test on every mail when it's not needed is badly
> designed setup.
Which is not really a strong argument for Postgrey or any other
greylisting tool that works as as a Postfix policy daemon. The only way
to spare a Postfix server from the potentially very slow client hostname
DNS queries is to have postscreen in front of smtpd, at least to do
pre-greet enforcement.
> Many of the dns based lists also limit the amount of checks per day.
> Worst case scenario, you stop getting results from lists due to over
> use.
Yes, but those query limits are designed to make sure that commercial
entities that get value from those DNSBLs and are large enough to afford
participation in keeping them in operation pay for that value they
receive.
> If you use google's 8.8.8.8 servers for dns lookups one can quickly
> run into that problem, I did. A high volume of dns checks could force
> you into having to pay for the amount of traffic you cause.
Using Google's public DNS or anything like it assures that a mail system
is low-performing and unreliable. There's nothing wrong with running an
amateur mail system but one should understand that using free distant
DNS run by unaccountable entities relegates a mail system to "hobbyist"
class.
> Many expensive network (takes a long time) checks will probably make
> you run out of slots a lot faster than the reconnects due to
> greylisting will do due to the time spent waiting for the lookups to
> finish.
I can only imagine that being a problem, and only on a very busy and
badly misconfigured mail server. I've run systems using multiple
machines each handling over a million SMTP connections per day and never
had a substantial problem with doing multiple DNSBL lookups on every
connection that passes a short greeting delay and SPF checks on
everything that gets to the point of body filtering (i.e. SPF within
SpamAssassin.)
> If speed of delivery is important, you could lower the amount of time
> mail stays greylisted. Ideally you'd like the mail delivered the first
> time a server tries to send it again. If a server tries to resend
> once, it will most likely try more than once anyway. Having a minimum
> time of 300 seconds, the default of postgrey, is probably a bit
> excessive.
Reducing that is unlikely to reduce delays, because a lot of systems use
300s as a minimum delay between retries; for a long time the standard
queue run time for Sendmail was 300s and that has been the default for
Postfix since v2.4. The main exceptions using shorter retry times would
be high-volume mailing lists systems (legit or otherwise) that cannot
afford to let their deferral queues grow large.
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Reindl Harald <h....@thelounge.net>.
Am 30.07.2016 um 13:10 schrieb Kim Roar Foldøy Hauge:
> On Sat, 30 Jul 2016, Robert Schetterer wrote:
>
>> Am 30.07.2016 um 03:34 schrieb Reindl Harald:
>>>
>>>
>>> Am 29.07.2016 um 22:48 schrieb Dianne Skoll:
>>>> On Fri, 29 Jul 2016 22:39:15 +0200
>>>> Robert Schetterer <rs...@sys4.de> wrote:
>>>>
>>>>>> I don't use postfix or postscreen.
>>>>> hm.. that does not fit the subject..why did you involved yourself ?
>>>>
>>>> I am sorry. I should have changed the thread subject.
>>>>
>>>>> you may get that quite better, i see
>>>>> a lot of server greylisting useless ,only filling up others queues
>>>>> waiting for a second slot ,so it may only cheap for you but not for
>>>>> your partners
>>>>> Dont slow down communication if you dont need to
>>>>
>>>> So what I didn't mention is that in our implementation, once an IP
>>>> address successully passes greylisting, we no longer greylist it for
>>>> the next 45 days. (It would probably be pointless... if an IP passes
>>>> greylisting once, it probably will keep passing it.)
>>>
>>> that's nothing special and postgrey does the same, the whole point of
>>> greylisting is that badly written bots don't try again (the same happens
>>> if they connect to a backup-MX responding with 4xx)
>>>
>>> also it don't help for clients which *do not* pass like large senders
>>> with outbound clusters coming each time from a different IP
>>>
>>> hence you skip greylisting based on DNSWL and spf-policyd because that
>>> big legit senders hit DNSWL or have a proper SPF while random bots of
>>> infected machines don't and this ones are your target for greylisting
>>>
>>
>> Harald is right, the goal has to be "reject" spam asap, not to tell
>> "come again later", i.e i had 4 bot cons per second, this will run out
>> the system of smtp slots rapidly which means any good sender isnt able
>> to sent mail too, greylisting makes such situations more worst.
>
> I'm no expert here, but postgrey is usually a purely local test. It
> should terminate with a "currently busy, try again later" message very
> quickly
yes, but when the total amount reaches your maximum of smtpd processes
because 4 bots per second there are no longer slots für legit clients
and if you then greylist a large amount fo legit clients which are all
coming back (in case of high legit traffic) things get much worser
in times of postscreen (and "Using Postfix and Postgrey" with current
software implies that it is available) that all is not mucha problem
because most crap don't make it to smtpd
well, and finally limit the impact by just use iptables on the server
ctstate NEW recent: UPDATE seconds: 2 hit_count: 5 name: DEFAULT side:
source mask: 255.255.255.255
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey
- not scanning after hold)
Posted by Kim Roar Foldøy Hauge <ki...@samfunnet.no>.
On Sat, 30 Jul 2016, Robert Schetterer wrote:
> Am 30.07.2016 um 03:34 schrieb Reindl Harald:
>>
>>
>> Am 29.07.2016 um 22:48 schrieb Dianne Skoll:
>>> On Fri, 29 Jul 2016 22:39:15 +0200
>>> Robert Schetterer <rs...@sys4.de> wrote:
>>>
>>>>> I don't use postfix or postscreen.
>>>> hm.. that does not fit the subject..why did you involved yourself ?
>>>
>>> I am sorry. I should have changed the thread subject.
>>>
>>>> you may get that quite better, i see
>>>> a lot of server greylisting useless ,only filling up others queues
>>>> waiting for a second slot ,so it may only cheap for you but not for
>>>> your partners
>>>> Dont slow down communication if you dont need to
>>>
>>> So what I didn't mention is that in our implementation, once an IP
>>> address successully passes greylisting, we no longer greylist it for
>>> the next 45 days. (It would probably be pointless... if an IP passes
>>> greylisting once, it probably will keep passing it.)
>>
>> that's nothing special and postgrey does the same, the whole point of
>> greylisting is that badly written bots don't try again (the same happens
>> if they connect to a backup-MX responding with 4xx)
>>
>> also it don't help for clients which *do not* pass like large senders
>> with outbound clusters coming each time from a different IP
>>
>> hence you skip greylisting based on DNSWL and spf-policyd because that
>> big legit senders hit DNSWL or have a proper SPF while random bots of
>> infected machines don't and this ones are your target for greylisting
>>
>>
>>
>
> Harald is right, the goal has to be "reject" spam asap, not to tell
> "come again later", i.e i had 4 bot cons per second, this will run out
> the system of smtp slots rapidly which means any good sender isnt able
> to sent mail too, greylisting makes such situations more worst.
>
I'm no expert here, but postgrey is usually a purely local test. It should
terminate with a "currently busy, try again later" message very quickly.
SPF checks and white listing require dns lookups that can potentially take
much longer. Several orders of magnitude longer.
Efficient handling of spam is all about doing the least expensive tests
first in terms of cpu/time. Caching DNS can probably help a bit, but it
will still require the occasional lookup now and then that take a lot
longer than a good greylisting implementation should ever do.
Doing an expensive test on every mail when it's not needed is badly
designed setup.
Many of the dns based lists also limit the amount of checks per day. Worst
case scenario, you stop getting results from lists due to over use. If you
use google's 8.8.8.8 servers for dns lookups one can quickly run into that
problem, I did. A high volume of dns checks could force you into having to
pay for the amount of traffic you cause.
Many expensive network (takes a long time) checks will probably make you
run out of slots a lot faster than the reconnects due to greylisting will
do due to the time spent waiting for the lookups to finish.
If speed of delivery is important, you could lower the amount of time mail
stays greylisted. Ideally you'd like the mail delivered the first time
a server tries to send it again. If a server tries to resend once, it will
most likely try more than once anyway. Having a minimum time of 300
seconds, the default of postgrey, is probably a bit excessive.
--
Kim Roar Foldy Hauge
Event:Presse - The Gathering 2016
webmaster@samfunnet.no
Root@HC,HX,JH,LZ,OT,P,VH
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Robert Schetterer <rs...@sys4.de>.
Am 30.07.2016 um 03:34 schrieb Reindl Harald:
>
>
> Am 29.07.2016 um 22:48 schrieb Dianne Skoll:
>> On Fri, 29 Jul 2016 22:39:15 +0200
>> Robert Schetterer <rs...@sys4.de> wrote:
>>
>>>> I don't use postfix or postscreen.
>>> hm.. that does not fit the subject..why did you involved yourself ?
>>
>> I am sorry. I should have changed the thread subject.
>>
>>> you may get that quite better, i see
>>> a lot of server greylisting useless ,only filling up others queues
>>> waiting for a second slot ,so it may only cheap for you but not for
>>> your partners
>>> Dont slow down communication if you dont need to
>>
>> So what I didn't mention is that in our implementation, once an IP
>> address successully passes greylisting, we no longer greylist it for
>> the next 45 days. (It would probably be pointless... if an IP passes
>> greylisting once, it probably will keep passing it.)
>
> that's nothing special and postgrey does the same, the whole point of
> greylisting is that badly written bots don't try again (the same happens
> if they connect to a backup-MX responding with 4xx)
>
> also it don't help for clients which *do not* pass like large senders
> with outbound clusters coming each time from a different IP
>
> hence you skip greylisting based on DNSWL and spf-policyd because that
> big legit senders hit DNSWL or have a proper SPF while random bots of
> infected machines don't and this ones are your target for greylisting
>
>
>
Harald is right, the goal has to be "reject" spam asap, not to tell
"come again later", i.e i had 4 bot cons per second, this will run out
the system of smtp slots rapidly which means any good sender isnt able
to sent mail too, greylisting makes such situations more worst.
Best Regards
MfG Robert Schetterer
--
[*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64
Schleiheimer Strae 26/MG, 80333 Mnchen
Sitz der Gesellschaft: Mnchen, Amtsgericht Mnchen: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
Re: Is greylisting effective? (was Re: Using Postfix and Postgrey -
not scanning after hold)
Posted by Reindl Harald <h....@thelounge.net>.
Am 29.07.2016 um 22:48 schrieb Dianne Skoll:
> On Fri, 29 Jul 2016 22:39:15 +0200
> Robert Schetterer <rs...@sys4.de> wrote:
>
>>> I don't use postfix or postscreen.
>> hm.. that does not fit the subject..why did you involved yourself ?
>
> I am sorry. I should have changed the thread subject.
>
>> you may get that quite better, i see
>> a lot of server greylisting useless ,only filling up others queues
>> waiting for a second slot ,so it may only cheap for you but not for
>> your partners
>> Dont slow down communication if you dont need to
>
> So what I didn't mention is that in our implementation, once an IP
> address successully passes greylisting, we no longer greylist it for
> the next 45 days. (It would probably be pointless... if an IP passes
> greylisting once, it probably will keep passing it.)
that's nothing special and postgrey does the same, the whole point of
greylisting is that badly written bots don't try again (the same happens
if they connect to a backup-MX responding with 4xx)
also it don't help for clients which *do not* pass like large senders
with outbound clusters coming each time from a different IP
hence you skip greylisting based on DNSWL and spf-policyd because that
big legit senders hit DNSWL or have a proper SPF while random bots of
infected machines don't and this ones are your target for greylisting