You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Eric Cunningham <er...@whoi.edu> on 2014/10/06 18:39:21 UTC
recent channel update woes
Hello, has anyone else experienced an HUGE uptick in the number of
rejected legitimate emails following an sa-update run over this past
weekend (possibly yesterday, Oct 5)? It looks like something caused our
once-adequate-and-happy required_hits value of 7.0 to be way too
restrictive suddenly blocking nearly every inbound email that wasn't
previously whitelisted. For the moment, I've had to raise required_hits
to 25.0 to quell the torrent of rejected emails. Any ideas,
explanations or, more importantly, help to remedy this are appreciated.
Thank you.
Re: Administrivia
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 10/6/2014 2:50 PM, Karsten Bräckelmann wrote:
> Just to give some answers. This issue should further be handled
> off-list.
Thanks for your $0.02. I hate being accused of spamming...
Administrivia (was: Re: recent channel update woes)
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Mon, 2014-10-06 at 13:36 -0400, Kevin A. McGrail wrote:
> On 10/6/2014 1:23 PM, Kevin A. McGrail wrote:
> > On 10/6/2014 1:11 PM, Jason Goldberg wrote:
> > > How to i get removed from this stupid list.
> > >
> > > I love begin spammed by a list about spam which i did not signup for.
> >
> > Email users-help@spamassassin.apache.org and the system will mail you
> > instructions.
> >
> > If you did not sign up for the list, that is very troublesome and we
> > can ask infrastructure to research but I believe we have a
> > confirmation email requirement to get on the list.
First of all: Jason's posts are stuck in moderation. The sender address
he uses is not the one he subscribed with.
Sidney and I (both list moderators) have been contacting Jason off-list
with detailed instructions how to find the subscribed address and
offering further help.
> Obviously we take this very seriously as anti-spammers because the
> definition I follow for spam is it's about consent not content. If you
> didn't consent to receive these emails, we have a major issue.
The list server requires clear and active confirmation of the
subscription request by mail, validating both the address as well as
consent.
> I've confirmed we have a confirmation email process in place that
> requires the subscribee to confirm the subscription request. And I
> believe this has been in place for many years. So if you did not
> subscribe to the list or confirm the subscription, you may need to check
> if your email address credentials have been compromised as that's the
> second most likely scenario for the cause beyond an administrator adding
> you directly.
>
> Karsten, any thoughts other than if a list administrator added them
> directly? Have infrastructure check the records for when and how the
> subscriber was added? Open a ticket with Google?
He has not been added by a list administrator.
Without the subscribed address, there is absolutely nothing we can do. I
grepped the subscription list and transaction logs for parts of Jason's
name and company. The address in question is entirely different.
Just to give some answers. This issue should further be handled
off-list.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: recent channel update woes
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 10/6/2014 1:23 PM, Kevin A. McGrail wrote:
> On 10/6/2014 1:11 PM, Jason Goldberg wrote:
>> How to i get removed from this stupid list.
>>
>> I love begin spammed by a list about spam which i did not signup for.
>
> Email users-help@spamassassin.apache.org and the system will mail you
> instructions.
>
> If you did not sign up for the list, that is very troublesome and we
> can ask infrastructure to research but I believe we have a
> confirmation email requirement to get on the list.
Obviously we take this very seriously as anti-spammers because the
definition I follow for spam is it's about consent not content. If you
didn't consent to receive these emails, we have a major issue.
I've confirmed we have a confirmation email process in place that
requires the subscribee to confirm the subscription request. And I
believe this has been in place for many years. So if you did not
subscribe to the list or confirm the subscription, you may need to check
if your email address credentials have been compromised as that's the
second most likely scenario for the cause beyond an administrator adding
you directly.
Karsten, any thoughts other than if a list administrator added them
directly? Have infrastructure check the records for when and how the
subscriber was added? Open a ticket with Google?
Regards,
KAM
Re: recent channel update woes
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 10/6/2014 1:11 PM, Jason Goldberg wrote:
> How to i get removed from this stupid list.
>
> I love begin spammed by a list about spam which i did not signup for.
Email users-help@spamassassin.apache.org and the system will mail you
instructions.
If you did not sign up for the list, that is very troublesome and we can
ask infrastructure to research but I believe we have a confirmation
email requirement to get on the list.
Regards,
KAM
Re: recent channel update woes
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 10/6/2014 1:00 PM, Eric Cunningham wrote:
> No, I did not see anything about an RHS_URIBL_DOB issue. Could you,
> as you say, offer some data points on this?
http://spamassassin.1065346.n5.nabble.com/URIBL-RHS-DOB-high-hits-td112138.html
And being discussed on users list right now...
Regards,
KAM
Re: recent channel update woes
Posted by Eric Cunningham <er...@whoi.edu>.
On 10/06/2014 12:51 PM, Kevin A. McGrail wrote:
> On 10/6/2014 12:39 PM, Eric Cunningham wrote:
>> Hello, has anyone else experienced an HUGE uptick in the number of
>> rejected legitimate emails following an sa-update run over this past
>> weekend (possibly yesterday, Oct 5)? It looks like something caused
>> our once-adequate-and-happy required_hits value of 7.0 to be way too
>> restrictive suddenly blocking nearly every inbound email that wasn't
>> previously whitelisted. For the moment, I've had to raise
>> required_hits to 25.0 to quell the torrent of rejected emails. Any
>> ideas, explanations or, more importantly, help to remedy this are
>> appreciated. Thank you.
> Did you see the RHS_URIBL_DOB issue?
>
> Further, I would look at one specific email and find out why it got over
> the threshold. Repeat for a few emails until a pattern or a lack of
> pattern emerges.
>
> Making systemic statements without any individual data points just leads
> to chicken little scenarios.
>
> regards,
> KAM
>
>
No, I did not see anything about an RHS_URIBL_DOB issue. Could you, as
you say, offer some data points on this?
Re: recent channel update woes
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 10/6/2014 12:39 PM, Eric Cunningham wrote:
> Hello, has anyone else experienced an HUGE uptick in the number of
> rejected legitimate emails following an sa-update run over this past
> weekend (possibly yesterday, Oct 5)? It looks like something caused
> our once-adequate-and-happy required_hits value of 7.0 to be way too
> restrictive suddenly blocking nearly every inbound email that wasn't
> previously whitelisted. For the moment, I've had to raise
> required_hits to 25.0 to quell the torrent of rejected emails. Any
> ideas, explanations or, more importantly, help to remedy this are
> appreciated. Thank you.
Did you see the RHS_URIBL_DOB issue?
Further, I would look at one specific email and find out why it got over
the threshold. Repeat for a few emails until a pattern or a lack of
pattern emerges.
Making systemic statements without any individual data points just leads
to chicken little scenarios.
regards,
KAM
Re: recent channel update woes
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2014-10-07 at 18:49 -0400, Eric Cunningham wrote:
> Is there a way to configure URIBL_RHS_DOB conditionally such that if
> there are issues with dob.sibl.support-intelligence.net like we're
> seeing, that associated scoring remains neutral rather than increasing
> (or decreasing)?
No. As-is, a correct DNSxL listing is indistinguishable from a false
positive listing.
One possible strategy to detect FP listings would be an additional DNSxL
query of a test-point or known-to-be not listed value. This comes at the
cost of increased load both for the DNSxL as well as SA instance, and
will lag behind due to TTL and DNS caching. The lower the lag, the lower
the caching, the higher the additional load.
By doing such tests not on a per message basis but per spamd child. or
even having the parent process monitor for possible world-listed
situations, the additional overhead and load could be massively reduced.
Simply monitoring real results (without test queries) likely would not
work. It is entirely possible that really large chunks of the mail
stream continuously result in positive DNSxL listings. Prime candidates
would be PBL hitting botnet spew, or exclusively DNSWL trusted messages
during otherwise low traffic conditions. Distinguishing lots of
consecutive correct listings from false positives would be really hard
and prone to errors.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: recent channel update woes
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Wed, 2014-10-08 at 01:18 +0200, Reindl Harald wrote:
> Am 08.10.2014 um 00:49 schrieb Eric Cunningham:
> > Is there a way to configure URIBL_RHS_DOB conditionally such that if
> > there are issues with dob.sibl.support-intelligence.net like we're
> > seeing, that associated scoring remains neutral rather than increasing
> > (or decreasing)?
>
> not really - if you get the response from the DNS - well, you are done
>
> the only exception are dnslists which stop to answer if you excedd the
> free limit but in that case they answer with a different response what
> is caught by the rules
Exceeding free usage limit is totally different from the recent DOB
"listing the world" issue.
Also, exceeding limit is handled differently in lots of ways. It ranges
from specific "limit exceeded" results, up to "listing the world" at the
hostile end or in extreme situations to finally get the admin's
attention. It also includes simply no results other than NXDOMAIN, which
is hard to distinguish from proper operation in certain low-listing
conditions.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: recent channel update woes
Posted by Dave Warren <da...@hireahit.com>.
On 2014-10-07 16:58, Karsten Bräckelmann wrote:
>> I monitor positive and negative responses, for IP based DNS BLs, I use
>> >the following by default:
>> >
>> >127.0.0.1 should not be listed.
>> >127.0.0.2 should be listed.
> Depending on how the DNSBL implements such static test-points, they
> might not be affected by the issue causing the false listings.
> Similarly, domains likely to appear on exonerate lists (compare
> uridnsbl_skip_domain e.g.) might also not be affected.
>
> For paranoid monitoring, low-profile domains that definitely do not and
> will not match the listing criteria might be better suited for the task.
I included: $MYIP for that reason; If I'm listed, either the world is
being listed, or I have a problem. Either way, I want to know about it, now.
>> >$MYIP should not be listed.
In the event that I'm blocked from querying the DNSBL, that a DNSBL is
offline, under attack or whatever, odds are that 127.0.0.2 (or whatever
is applicable) will disappear.
--
Dave Warren
http://www.hireahit.com/
http://ca.linkedin.com/in/davejwarren
Re: recent channel update woes
Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Tue, 2014-10-07 at 16:37 -0700, Dave Warren wrote:
> If you're paranoid, you can monitor the DNSBLs that you use via script
> (externally from SpamAssassin) and generate something that reports to
> you when there's a possible issue. If you're really paranoid, you can
> have it write a .cf that would 0 out the scores, but I assure you that
> you'll spend more time building, testing and maintaining such a system
> than it's worth in the long run, in my experience it's better to just
> page an admin.
>
> I monitor positive and negative responses, for IP based DNS BLs, I use
> the following by default:
>
> 127.0.0.1 should not be listed.
> 127.0.0.2 should be listed.
Depending on how the DNSBL implements such static test-points, they
might not be affected by the issue causing the false listings.
Similarly, domains likely to appear on exonerate lists (compare
uridnsbl_skip_domain e.g.) might also not be affected.
For paranoid monitoring, low-profile domains that definitely do not and
will not match the listing criteria might be better suited for the task.
> $MYIP should not be listed.
>
> Obviously these need to be tweaked and configured per-list, not all
> lists list 127.0.0.2, and some lists use status codes, so "should not be
> listed" and "should be listed" are really "match/do-not-match some
> condition"
>
> In the case of DNSWL, $MYIP should be listed, if I get de-listed, I want
> to know about that too.
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}
Re: recent channel update woes
Posted by Dave Warren <da...@hireahit.com>.
On 2014-10-07 16:18, Reindl Harald wrote:
> what happens here is unintentional and so you can't say if the
> response is wrong - if you would know the answer you would not ask the
> server
If you're paranoid, you can monitor the DNSBLs that you use via script
(externally from SpamAssassin) and generate something that reports to
you when there's a possible issue. If you're really paranoid, you can
have it write a .cf that would 0 out the scores, but I assure you that
you'll spend more time building, testing and maintaining such a system
than it's worth in the long run, in my experience it's better to just
page an admin.
I monitor positive and negative responses, for IP based DNS BLs, I use
the following by default:
127.0.0.1 should not be listed.
127.0.0.2 should be listed.
$MYIP should not be listed.
Obviously these need to be tweaked and configured per-list, not all
lists list 127.0.0.2, and some lists use status codes, so "should not be
listed" and "should be listed" are really "match/do-not-match some
condition"
In the case of DNSWL, $MYIP should be listed, if I get de-listed, I want
to know about that too.
--
Dave Warren
http://www.hireahit.com/
http://ca.linkedin.com/in/davejwarren
Re: recent channel update woes
Posted by Reindl Harald <h....@thelounge.net>.
Am 08.10.2014 um 00:49 schrieb Eric Cunningham:
>> Am 06.10.2014 um 19:22 schrieb Benny Pedersen:
>>> On October 6, 2014 6:39:21 PM Eric Cunningham <er...@whoi.edu> wrote:
>>>
>>>> Hello, has anyone else experienced an HUGE uptick in the number of
>>>> rejected legitimate emails following an sa-update run over this past
>>>
>>> And spammassin only tags mail, it does not reject, so stop saying it an
>>> sa issue when its not
>>
>> on a sane setup it is part of a milter and rejects above a specific
>> level because it makes little sense to accept high score spam and only
>> move it in a different folder
>>
>> frankly 3 weeks ago we had about 30000 junk attemps per day and now we
>> have the same per week - guess why - because delayes, postscreen and
>> reject highscore spam instead sign "250 OK" to the bot client
>>
>> X-Spam-Status: No, score=-106.2, tag-level=4.5, block-level=8.0
>>
>
> Is there a way to configure URIBL_RHS_DOB conditionally such that if
> there are issues with dob.sibl.support-intelligence.net like we're
> seeing, that associated scoring remains neutral rather than increasing
> (or decreasing)?
not really - if you get the response from the DNS - well, you are done
the only exception are dnslists which stop to answer if you excedd the
free limit but in that case they answer with a different response what
is caught by the rules
what happens here is unintentional and so you can't say if the response
is wrong - if you would know the answer you would not ask the server
Re: recent channel update woes
Posted by Eric Cunningham <er...@whoi.edu>.
> Am 06.10.2014 um 19:22 schrieb Benny Pedersen:
>> On October 6, 2014 6:39:21 PM Eric Cunningham <er...@whoi.edu> wrote:
>>
>>> Hello, has anyone else experienced an HUGE uptick in the number of
>>> rejected legitimate emails following an sa-update run over this past
>>
>> And spammassin only tags mail, it does not reject, so stop saying it an
>> sa issue when its not
>
> on a sane setup it is part of a milter and rejects above a specific
> level because it makes little sense to accept high score spam and only
> move it in a different folder
>
> frankly 3 weeks ago we had about 30000 junk attemps per day and now we
> have the same per week - guess why - because delayes, postscreen and
> reject highscore spam instead sign "250 OK" to the bot client
>
> X-Spam-Status: No, score=-106.2, tag-level=4.5, block-level=8.0
>
Is there a way to configure URIBL_RHS_DOB conditionally such that if
there are issues with dob.sibl.support-intelligence.net like we're
seeing, that associated scoring remains neutral rather than increasing
(or decreasing)?
Re: recent channel update woes
Posted by Reindl Harald <h....@thelounge.net>.
Am 06.10.2014 um 19:22 schrieb Benny Pedersen:
> On October 6, 2014 6:39:21 PM Eric Cunningham <er...@whoi.edu> wrote:
>
>> Hello, has anyone else experienced an HUGE uptick in the number of
>> rejected legitimate emails following an sa-update run over this past
>
> And spammassin only tags mail, it does not reject, so stop saying it an
> sa issue when its not
on a sane setup it is part of a milter and rejects above a specific
level because it makes little sense to accept high score spam and only
move it in a different folder
frankly 3 weeks ago we had about 30000 junk attemps per day and now we
have the same per week - guess why - because delayes, postscreen and
reject highscore spam instead sign "250 OK" to the bot client
X-Spam-Status: No, score=-106.2, tag-level=4.5, block-level=8.0
Re: recent channel update woes
Posted by Benny Pedersen <me...@junc.eu>.
On October 6, 2014 6:39:21 PM Eric Cunningham <er...@whoi.edu> wrote:
> Hello, has anyone else experienced an HUGE uptick in the number of
> rejected legitimate emails following an sa-update run over this past
And spammassin only tags mail, it does not reject, so stop saying it an sa
issue when its not