You are viewing a plain text version of this content. The canonical link for it is here.
Posted to sysadmins@spamassassin.apache.org by "Kevin A. McGrail" <ke...@mcgrail.com> on 2017/11/07 13:11:11 UTC
Re: Mailserver at 52.169.9.191
+sysadmins@s.a.o
-Microsoft
Hi Matthias,
It's the only time I have ever seen it abused to be honest. I added the
Azure abuse team so it should get resolved and they responded that they
are escalating it to their CERT team. Plus I am curious what the hell is
going on so I'll ask them to follow-up
How did you figure out it was the brain dev company Btw? I didn't see
the connection. Also, I think they had a webmail interface up at
52.169.9.191 and it appears to be offline now. Guessing the abuse team
cut the machine off.
I did an outright drop on the IP. I just removed it and appear to no
longer have the 3 second monster. Thanks for noticing it.
One thing that would be cool is a heat-map/aggregation of the sa-update
data which might also find issues like this but also show useful
information like where our sa-update mirrors are getting used most,
identify the actual aggregated load, etc. Thoughts?
Regards,
KAM
On November 6, 2017 10:51:10 PM PST, Matthias Leisi <ma...@dnswl.org>
wrote:
Btw., I 403’d this IP in my local config.
Maybe we could distribute a .htaccess file with the update files as
a workaround for such issues?
— Matthias
<https://www.dnswl.org/>
Matthias Leisi, Project Leader dnswl.org <https://www.dnswl.org/>
Mail reputation – Protect against false positives
matthias@dnswl.org <ma...@dnswl.org> | Twitter: @dnswlorg
<https://twitter.com/dnswlorg>
> Am 07.11.2017 um 04:35 schrieb Kevin A. McGrail
> <kevin.mcgrail@mcgrail.com <ma...@mcgrail.com>>:
>
> +Microsoft Abuse:
>
> After further research the machine at 52.169.9.191 is causing
> 2/3's of our SpamAssassin Update server traffic for the last
> month. Please rectify this immediately.
>
> Regards
> KAM
>
> On 11/5/2017 3:30 PM, Matthias Leisi wrote:
>> Hello,
>>
>> We run one of the mirrors used by sa-update. From our logs, we
>> see that the IP address 52.169.9.191 (which seems to be
>> mail.brainloopdevops.com <http://mail.brainloopdevops.com/>, and
>> for which whois shows your email address) runs sa-update about
>> once every three seconds. Generally, once a day is the suggested
>> update frequency (https://wiki.apache.org/spamassassin/RuleUpdates).
>>
>> Please change the update frequency to an acceptable level.
>>
>> Regards,
>> — Matthias, for the dnswl.org <http://dnswl.org/> project
>>
>>
>>
>> <dnswlorg_logo.png> <https://www.dnswl.org/>
>> Matthias Leisi, Project Leader dnswl.org <https://www.dnswl.org/>
>> Mail reputation – Protect against false positives
>>
>> matthias@dnswl.org <ma...@dnswl.org> | Twitter:
>> @dnswlorg <https://twitter.com/dnswlorg>
>>
>>
>
Re: Mailserver at 52.169.9.191
Posted by "Kevin A. McGrail" <ke...@mcgrail.com>.
On 11/8/2017 3:36 PM, Dave Jones wrote:
> I think there are old "zombie" scripts out there trying to curl fetch
> updates outside of using sa-update. Maybe they are trying to fetch
> them into their own copy/repo or something? I don't know for sure.
>
> Should we remove DNS entries for all "ancient" versions. What
> versions do we officially support currently with ruleset generation?
> We are only testing new rulesets against 3.4.x. Should we remove the
> DNS records for anything older than 3.3.0? That is the oldest TXT
> record for modern rulesets in 2017.
For now, I vote no but if any mirror operators feel otherwise, I would
be willing to change to a neutral. It's just interesting to me but not
causing any resource issues for me. Especially now the 3 second guy is
gone thanks!
Regards,
KAM
Re: Mailserver at 52.169.9.191
Posted by Dave Jones <da...@apache.org>.
On 11/08/2017 01:45 PM, Kevin A. McGrail wrote:
> On 11/8/2017 10:02 AM, Dave Jones wrote:
>>>
>>> Dave, with rules how many versions do we publish now? Just one with
>>> a cname for a few versions? Which versions? Sorry, I can't figure
>>> out how to get into PowerDNS to check!
>>>
>>
>> http://svn.apache.org/viewvc/spamassassin/dns/spamassassin.org?view=markup
>>
>>
>> It appears that sa-update versions 3.4.1 and above support the CNAME.
>
> Thanks, that's what I thought. Everything older hasn't had new versions
> in years. I wonder why sa-updates are still being downloaded for them.
> We might consider removing the DNS entries. Or just not care as it's so
> minor.
>
>
> Regards,
> KAM
>
I think there are old "zombie" scripts out there trying to curl fetch
updates outside of using sa-update. Maybe they are trying to fetch them
into their own copy/repo or something? I don't know for sure.
Should we remove DNS entries for all "ancient" versions. What versions
do we officially support currently with ruleset generation? We are only
testing new rulesets against 3.4.x. Should we remove the DNS records
for anything older than 3.3.0? That is the oldest TXT record for modern
rulesets in 2017.
Dave
Re: Mailserver at 52.169.9.191
Posted by "Kevin A. McGrail" <ke...@mcgrail.com>.
On 11/8/2017 10:02 AM, Dave Jones wrote:
>>
>> Dave, with rules how many versions do we publish now? Just one with
>> a cname for a few versions? Which versions? Sorry, I can't figure
>> out how to get into PowerDNS to check!
>>
>
> http://svn.apache.org/viewvc/spamassassin/dns/spamassassin.org?view=markup
>
>
> It appears that sa-update versions 3.4.1 and above support the CNAME.
Thanks, that's what I thought. Everything older hasn't had new versions
in years. I wonder why sa-updates are still being downloaded for them.
We might consider removing the DNS entries. Or just not care as it's so
minor.
Regards,
KAM
Re: Mailserver at 52.169.9.191
Posted by Dave Jones <da...@apache.org>.
On 11/08/2017 08:45 AM, Kevin A. McGrail wrote:
>
>>
>> I’m very much in favor of aggregating and analyzing data, that’s
>> bascially what we do at dnswl.org <http://dnswl.org> :) Having said
>> that, I usually don’t see that much load on our sa-update mirror, just
>> a bit of bandwidth being used.
>>
>
> Well nice sleuthing to figure out the culprits company. I am
> considering the issue closed with MS and they will never let us know the
> outcome due to privacy
>
> How can we aggregate the logs? We have a free Apache G Suite instance
> I'll mention if that sparks some ideas. But perhaps even if I just
> agreed to send my logs to you, it would be cool to see some stats on it.
>
> To compare, here's the same commands on a few weeks of logs on my
> server. Really surprising to see those old versions:
>
> 26592 3.3.2"
> 12120 3.3.1"
> 4531 3.1.8"
> 3073 3.4.1"
> 1506 3.2.5"
> 561 3.4.0"
> 470 3.2.1"
> 464 3.2.4"
> 107 3.2.3"
> 104 3.1.7"
> 53 3.3.0"
> 19 3.2.0"
> 18 3.2.2"
> 2 3.1.9"
>
> Dave, with rules how many versions do we publish now? Just one with a
> cname for a few versions? Which versions? Sorry, I can't figure out
> how to get into PowerDNS to check!
>
http://svn.apache.org/viewvc/spamassassin/dns/spamassassin.org?view=markup
It appears that sa-update versions 3.4.1 and above support the CNAME.
> BTW, Matthias, can you subscribe to this list as a mirror operator,
> please? Just email sysadmins-subscribe@spamassassin.apache.org, please?
>
>
> Regards,
>
> KAM
>
>
Re: Mailserver at 52.169.9.191
Posted by "Kevin A. McGrail" <ke...@mcgrail.com>.
>
> I’m very much in favor of aggregating and analyzing data, that’s
> bascially what we do at dnswl.org <http://dnswl.org> :) Having said
> that, I usually don’t see that much load on our sa-update mirror, just
> a bit of bandwidth being used.
>
Well nice sleuthing to figure out the culprits company. I am
considering the issue closed with MS and they will never let us know the
outcome due to privacy
How can we aggregate the logs? We have a free Apache G Suite instance
I'll mention if that sparks some ideas. But perhaps even if I just
agreed to send my logs to you, it would be cool to see some stats on it.
To compare, here's the same commands on a few weeks of logs on my
server. Really surprising to see those old versions:
26592 3.3.2"
12120 3.3.1"
4531 3.1.8"
3073 3.4.1"
1506 3.2.5"
561 3.4.0"
470 3.2.1"
464 3.2.4"
107 3.2.3"
104 3.1.7"
53 3.3.0"
19 3.2.0"
18 3.2.2"
2 3.1.9"
Dave, with rules how many versions do we publish now? Just one with a
cname for a few versions? Which versions? Sorry, I can't figure out
how to get into PowerDNS to check!
BTW, Matthias, can you subscribe to this list as a mirror operator,
please? Just email sysadmins-subscribe@spamassassin.apache.org, please?
Regards,
KAM
Re: Mailserver at 52.169.9.191
Posted by Matthias Leisi <ma...@dnswl.org>.
> It's the only time I have ever seen it abused to be honest. I added the Azure abuse team so it should get resolved and they responded that they are escalating it to their CERT team. Plus I am curious what the hell is going on so I'll ask them to follow-up
Indeed, I should have done that already with the first mail. Thanks for jumping in.
> How did you figure out it was the brain dev company Btw? I didn't see the connection. Also, I think they had a webmail interface up at 52.169.9.191 and it appears to be offline now. Guessing the abuse team cut the machine off.
Good question, I’ll try to remember :) dig -x was non-conclusive, the website just showed a roundcube login. But somehow I got to a „settings“ page which showed IMAP, POP and SMTP servers to configure, with the right hostname, which resolved back to the very IP.
> One thing that would be cool is a heat-map/aggregation of the sa-update data which might also find issues like this but also show useful information like where our sa-update mirrors are getting used most, identify the actual aggregated load, etc. Thoughts?
I’m very much in favor of aggregating and analyzing data, that’s bascially what we do at dnswl.org <http://dnswl.org/> :) Having said that, I usually don’t see that much load on our sa-update mirror, just a bit of bandwidth being used.
It could give us an overview of how / where SpamAssassin is being used, and maybe see how the usage changes over time? Example below (would be sufficient to run on a sample of the logs, when we’re only interested in relative size):
# grep "sa-update\/svn" access_log | grep -v "\.tar\.gz " > /tmp/x
# cat /tmp/x | cut -d "/" -f 1,7 | sort | uniq | cut -d "/" -f 2 | sort | uniq -c | sort -rn | less
176 3.3.2"
93 3.3.1"
15 3.4.1"
15 3.1.8"
3 3.2.4"
2 3.4.0"
2 3.2.1"
1 3.2.3“
That’s about half a day worth of logs. 3.2.x seems basically dead, but there are still 15 sites using 3.1.8 (oh my). But there are quite a few with user agent <> „sa-update“ (but with curl/wget).
— Matthias