You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Olivier Coutu <ol...@zerospam.ca> on 2018/03/13 18:13:39 UTC
Dealing with links to malicious documents
In the last few months, we have seen an increase of generic emails (e.g.
regarding unpaid invoices) being sent with links to infected legitimate
websites hosting malware. This malware often comes in the form of docs
with macros e.g. https://pastebin.com/VHz41RUL
In a lot of cases, neither the sender nor the URL are listed in any
blacklists at send time, and we are looking into ways to deal with these
links. We have developed some heuristics based on the text but this is
more reactive than proactive and the spams often are very similar to
legitimate emails. Ideally we would be able to see what is /really/
behind these links.
The technologies we know exist are:
a) Link following
Whether it is only for url shorteners or for all links, simulating a
click could give us info on what will happen, but has implications when
the website interprets that like a click from the user and updates their
database in some way such as unsubscribing a user.
b) Link rewriting
Rewrite the link so that it is analysed by the anti-spam provider at
click-time. Costly to implement and breaks message integrity/DKIM. Even
after 24h, a lot of these infected websites are not listed on
blacklists. This method also has privacy implications.
c) DNS-based approaches
Similar to link rewriting, use a dns-firewall such as Cisco Umbrella to
block queries to malicious websites. Our tests indicate that this does
not work very well for the aforementioned infected websites. It might
work well for C&C servers but we feel like that is a bit late to avoid
an infection.
Are there other solutions that we have not thought of? Are any of you
having trouble with these types of links?
Thanks.
- Olivier
Re: Dealing with links to malicious documents
Posted by Axb <ax...@gmail.com>.
On 03/13/2018 07:13 PM, Olivier Coutu wrote:
> In the last few months, we have seen an increase of generic emails (e.g.
> regarding unpaid invoices) being sent with links to infected legitimate
> websites hosting malware. This malware often comes in the form of docs
> with macros e.g. https://pastebin.com/VHz41RUL
>
> In a lot of cases, neither the sender nor the URL are listed in any
> blacklists at send time, and we are looking into ways to deal with these
> links. We have developed some heuristics based on the text but this is
> more reactive than proactive and the spams often are very similar to
> legitimate emails. Ideally we would be able to see what is /really/
> behind these links.
>
> The technologies we know exist are:
>
> a) Link following
> Whether it is only for url shorteners or for all links, simulating a
> click could give us info on what will happen, but has implications when
> the website interprets that like a click from the user and updates their
> database in some way such as unsubscribing a user.
>
> b) Link rewriting
> Rewrite the link so that it is analysed by the anti-spam provider at
> click-time. Costly to implement and breaks message integrity/DKIM. Even
> after 24h, a lot of these infected websites are not listed on
> blacklists. This method also has privacy implications.
>
> c) DNS-based approaches
> Similar to link rewriting, use a dns-firewall such as Cisco Umbrella to
> block queries to malicious websites. Our tests indicate that this does
> not work very well for the aforementioned infected websites. It might
> work well for C&C servers but we feel like that is a bit late to avoid
> an infection.
>
> Are there other solutions that we have not thought of? Are any of you
> having trouble with these types of links?
Why not write a bunch of SA URI rules and/or ClamAV sigs to handle them.
Re: Dealing with links to malicious documents
Posted by Rupert Gallagher <ru...@protonmail.com>.
Hardin has it right, especially for invoices.
"The document needs to be attached, so that it can be scanned."
Re: Dealing with links to malicious documents
Posted by Markus Clardy <ma...@clardy.eu>.
I had created a plugin to read the headers (just make a HEAD request) of
all URIs in an email, and then you can make tests based off them. An
example of this would be to look for specific mimetypes at the end of the
link, so for example, see if the mimetype is Application/msword.
You can find it here: https://github.com/m50/spamassassin_uriheader
I will warn you, this may not necessarily be a great idea, as you may get
abuse notifications. I also need to make a few changes to it, but I haven't
touched it in a while.
First thing I want to do is whitelist certain domains from being header
checked.
Secondly, is replace the User Agent with one from a legitimate browser
(spoofing a browser), just in case.
But if you are interested in it, you can take a look.
On Tue, Mar 13, 2018 at 10:03 PM, John Hardin <jh...@impsec.org> wrote:
> On Tue, 13 Mar 2018, Bill Cole wrote:
>
> On 13 Mar 2018, at 14:21 (-0400), John Hardin wrote:
>>
>> d) Don't accept emails from outside your organization that link to hosted
>>> documents. The document needs to be attached, so that it can be scanned.
>>> Unfortunately this is not feasible if you're not a (at least
>>> semi-)monolithic organization where you can apply such policies.
>>>
>>
>> Also not feasible if any users subscribe to this list or most technical
>> discussion mailing lists. For example, here you are likely to get links
>> into the SA Wiki or to KAM's rules. On the Postfix list it is a rare week
>> that does not have multiple links to the DEBUG_README file posted.
>>
>
> I don't count a plain text file as a "document" in this context.
>
> The example provided was apparently to a directory (URL ending in '/') but
>> redirected to a .doc.
>>
>
> This of course is the weakness with that option.
>
>
> --
> John Hardin KA7OHZ http://www.impsec.org/~jhardin/
> jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
> key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
> -----------------------------------------------------------------------
> The problem is when people look at Yahoo, slashdot, or groklaw and
> jump from obvious and correct observations like "Oh my God, this
> place is teeming with utter morons" to incorrect conclusions like
> "there's nothing of value here". -- Al Petrofsky, in Y! SCOX
>
> -----------------------------------------------------------------------
> Tomorrow: Albert Einstein's 139th Birthday
>
--
- Markus
Re: Dealing with links to malicious documents
Posted by John Hardin <jh...@impsec.org>.
On Tue, 13 Mar 2018, Bill Cole wrote:
> On 13 Mar 2018, at 14:21 (-0400), John Hardin wrote:
>
>> d) Don't accept emails from outside your organization that link to hosted
>> documents. The document needs to be attached, so that it can be scanned.
>> Unfortunately this is not feasible if you're not a (at least
>> semi-)monolithic organization where you can apply such policies.
>
> Also not feasible if any users subscribe to this list or most technical
> discussion mailing lists. For example, here you are likely to get links into
> the SA Wiki or to KAM's rules. On the Postfix list it is a rare week that
> does not have multiple links to the DEBUG_README file posted.
I don't count a plain text file as a "document" in this context.
> The example provided was apparently to a directory (URL ending in '/')
> but redirected to a .doc.
This of course is the weakness with that option.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
The problem is when people look at Yahoo, slashdot, or groklaw and
jump from obvious and correct observations like "Oh my God, this
place is teeming with utter morons" to incorrect conclusions like
"there's nothing of value here". -- Al Petrofsky, in Y! SCOX
-----------------------------------------------------------------------
Tomorrow: Albert Einstein's 139th Birthday
Re: Dealing with links to malicious documents
Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 13 Mar 2018, at 14:21 (-0400), John Hardin wrote:
> d) Don't accept emails from outside your organization that link to
> hosted documents. The document needs to be attached, so that it can be
> scanned. Unfortunately this is not feasible if you're not a (at least
> semi-)monolithic organization where you can apply such policies.
Also not feasible if any users subscribe to this list or most technical
discussion mailing lists. For example, here you are likely to get links
into the SA Wiki or to KAM's rules. On the Postfix list it is a rare
week that does not have multiple links to the DEBUG_README file posted.
The example provided was apparently to a directory (URL ending in '/')
but redirected to a .doc.
--
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Currently Seeking Steady Work: https://linkedin.com/in/billcole
Re: Dealing with links to malicious documents
Posted by John Hardin <jh...@impsec.org>.
On Tue, 13 Mar 2018, Alex wrote:
> Hi,
>
> On Tue, Mar 13, 2018 at 2:21 PM, John Hardin <jh...@impsec.org> wrote:
>> On Tue, 13 Mar 2018, Olivier Coutu wrote:
>>
>>> In the last few months, we have seen an increase of generic emails (e.g.
>>> regarding unpaid invoices) being sent with links to infected legitimate
>>> websites hosting malware. This malware often comes in the form of docs with
>>> macros e.g. https://pastebin.com/VHz41RUL
>>>
>>> In a lot of cases, neither the sender nor the URL are listed in any
>>> blacklists at send time, and we are looking into ways to deal with these
>>> links. We have developed some heuristics based on the text but this is more
>>> reactive than proactive and the spams often are very similar to legitimate
>>> emails. Ideally we would be able to see what is /really/ behind these links.
>>>
>>> The technologies we know exist are:
>>>
>>> a) Link following
>>> Whether it is only for url shorteners or for all links, simulating a click
>>> could give us info on what will happen, but has implications when the
>>> website interprets that like a click from the user and updates their
>>> database in some way such as unsubscribing a user.
>>>
>>> b) Link rewriting
>>> Rewrite the link so that it is analysed by the anti-spam provider at
>>> click-time. Costly to implement and breaks message integrity/DKIM. Even
>>> after 24h, a lot of these infected websites are not listed on blacklists.
>>> This method also has privacy implications.
>>>
>>> c) DNS-based approaches
>>> Similar to link rewriting, use a dns-firewall such as Cisco Umbrella to
>>> block queries to malicious websites. Our tests indicate that this does not
>>> work very well for the aforementioned infected websites. It might work well
>>> for C&C servers but we feel like that is a bit late to avoid an infection.
>>>
>>> Are there other solutions that we have not thought of? Are any of you
>>> having trouble with these types of links?
>>
>>
>> d) Don't accept emails from outside your organization that link to hosted
>> documents. The document needs to be attached, so that it can be scanned.
>> Unfortunately this is not feasible if you're not a (at least
>> semi-)monolithic organization where you can apply such policies.
>
> I don't think denying access to direct downloads in email is really a
> policy that would work in any moderately sized organization. I tried
> looking at this in the recent past, and it would block a ton of
> legitimate email.
Really? That surprises me a bit. PDFs I could see, but document files?
> Also, in this particular case, you don't know that the link downloads
> something until you actually click on it or follow it.
Ah, yes, that wouldn't work.
>> e) if you are a (semi-)monolithic organization that uses a boundary proxy to
>> control web access, then add such URLs to that proxy's blocklist until the
>> contents can be scanned, or so that the proxy does the redirect-through-AV
>> automatically (not sure if that will work, though).
>
> I think what some providers do is wrap the URL and at some point later
> (after perhaps the domain or URL itself is listed by an RBL) disable
> the link for when the recipient clicks the link or views the email. I
> don't know how (or whether) they always break DKIM in the process.
> That would be interesting to find out.
>
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Gun Control is marketed to the public using the appealing delusion
that violent criminals will obey the law.
-----------------------------------------------------------------------
Tomorrow: Albert Einstein's 139th Birthday
Re: Dealing with links to malicious documents
Posted by Alex <my...@gmail.com>.
Hi,
On Tue, Mar 13, 2018 at 2:21 PM, John Hardin <jh...@impsec.org> wrote:
> On Tue, 13 Mar 2018, Olivier Coutu wrote:
>
>> In the last few months, we have seen an increase of generic emails (e.g.
>> regarding unpaid invoices) being sent with links to infected legitimate
>> websites hosting malware. This malware often comes in the form of docs with
>> macros e.g. https://pastebin.com/VHz41RUL
>>
>> In a lot of cases, neither the sender nor the URL are listed in any
>> blacklists at send time, and we are looking into ways to deal with these
>> links. We have developed some heuristics based on the text but this is more
>> reactive than proactive and the spams often are very similar to legitimate
>> emails. Ideally we would be able to see what is /really/ behind these links.
>>
>> The technologies we know exist are:
>>
>> a) Link following
>> Whether it is only for url shorteners or for all links, simulating a click
>> could give us info on what will happen, but has implications when the
>> website interprets that like a click from the user and updates their
>> database in some way such as unsubscribing a user.
>>
>> b) Link rewriting
>> Rewrite the link so that it is analysed by the anti-spam provider at
>> click-time. Costly to implement and breaks message integrity/DKIM. Even
>> after 24h, a lot of these infected websites are not listed on blacklists.
>> This method also has privacy implications.
>>
>> c) DNS-based approaches
>> Similar to link rewriting, use a dns-firewall such as Cisco Umbrella to
>> block queries to malicious websites. Our tests indicate that this does not
>> work very well for the aforementioned infected websites. It might work well
>> for C&C servers but we feel like that is a bit late to avoid an infection.
>>
>> Are there other solutions that we have not thought of? Are any of you
>> having trouble with these types of links?
>
>
> d) Don't accept emails from outside your organization that link to hosted
> documents. The document needs to be attached, so that it can be scanned.
> Unfortunately this is not feasible if you're not a (at least
> semi-)monolithic organization where you can apply such policies.
I don't think denying access to direct downloads in email is really a
policy that would work in any moderately sized organization. I tried
looking at this in the recent past, and it would block a ton of
legitimate email.
Also, in this particular case, you don't know that the link downloads
something until you actually click on it or follow it.
> e) if you are a (semi-)monolithic organization that uses a boundary proxy to
> control web access, then add such URLs to that proxy's blocklist until the
> contents can be scanned, or so that the proxy does the redirect-through-AV
> automatically (not sure if that will work, though).
I think what some providers do is wrap the URL and at some point later
(after perhaps the domain or URL itself is listed by an RBL) disable
the link for when the recipient clicks the link or views the email. I
don't know how (or whether) they always break DKIM in the process.
That would be interesting to find out.
Re: Dealing with links to malicious documents
Posted by John Hardin <jh...@impsec.org>.
On Tue, 13 Mar 2018, Olivier Coutu wrote:
> In the last few months, we have seen an increase of generic emails (e.g.
> regarding unpaid invoices) being sent with links to infected legitimate
> websites hosting malware. This malware often comes in the form of docs with
> macros e.g. https://pastebin.com/VHz41RUL
>
> In a lot of cases, neither the sender nor the URL are listed in any
> blacklists at send time, and we are looking into ways to deal with these
> links. We have developed some heuristics based on the text but this is more
> reactive than proactive and the spams often are very similar to legitimate
> emails. Ideally we would be able to see what is /really/ behind these links.
>
> The technologies we know exist are:
>
> a) Link following
> Whether it is only for url shorteners or for all links, simulating a click
> could give us info on what will happen, but has implications when the website
> interprets that like a click from the user and updates their database in some
> way such as unsubscribing a user.
>
> b) Link rewriting
> Rewrite the link so that it is analysed by the anti-spam provider at
> click-time. Costly to implement and breaks message integrity/DKIM. Even after
> 24h, a lot of these infected websites are not listed on blacklists. This
> method also has privacy implications.
>
> c) DNS-based approaches
> Similar to link rewriting, use a dns-firewall such as Cisco Umbrella to block
> queries to malicious websites. Our tests indicate that this does not work
> very well for the aforementioned infected websites. It might work well for
> C&C servers but we feel like that is a bit late to avoid an infection.
>
> Are there other solutions that we have not thought of? Are any of you having
> trouble with these types of links?
d) Don't accept emails from outside your organization that link to hosted
documents. The document needs to be attached, so that it can be scanned.
Unfortunately this is not feasible if you're not a (at least
semi-)monolithic organization where you can apply such policies.
e) if you are a (semi-)monolithic organization that uses a boundary proxy
to control web access, then add such URLs to that proxy's blocklist until
the contents can be scanned, or so that the proxy does the
redirect-through-AV automatically (not sure if that will work, though).
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Gun Control is marketed to the public using the appealing delusion
that violent criminals will obey the law.
-----------------------------------------------------------------------
Tomorrow: Albert Einstein's 139th Birthday
Re: Dealing with links to malicious documents
Posted by Leandro <le...@spfbl.net>.
2018-03-13 15:13 GMT-03:00 Olivier Coutu <ol...@zerospam.ca>:
> In the last few months, we have seen an increase of generic emails (e.g.
> regarding unpaid invoices) being sent with links to infected legitimate
> websites hosting malware. This malware often comes in the form of docs with
> macros e.g. https://pastebin.com/VHz41RUL
>
> In a lot of cases, neither the sender nor the URL are listed in any
> blacklists at send time, and we are looking into ways to deal with these
> links. We have developed some heuristics based on the text but this is more
> reactive than proactive and the spams often are very similar to legitimate
> emails. Ideally we would be able to see what is *really* behind these
> links.
>
> The technologies we know exist are:
>
> a) Link following
> Whether it is only for url shorteners or for all links, simulating a click
> could give us info on what will happen, but has implications when the
> website interprets that like a click from the user and updates their
> database in some way such as unsubscribing a user.
>
>
Our customers decides if the filter must follow all redirections or follow
just shorteners redirections. If they choose all redirections, the filter
will download any file to AV scan, if exists.
We make it very clear what is the filter implications if it follow all
redirects. Most have chosen to let the filter follow all redirects, even
though they may activate unwanted functions on the links. For them, is
preferable to take the risk of activating unwanted functions than to run
the risk of receiving a ransomware.
I believe that web designers need to get used to the idea of programming
double confirmation functions because it is increasingly frequent the
filters that follow all the redirects, because of all the abuse we are
witnessing. Just put a confirm button at web function is enough to avoid
unwanted function.
Re: Dealing with links to malicious documents
Posted by Pedro David Marco <pe...@yahoo.com>.
>a) Link following
>Whether it is only for url shorteners or for all links, simulating a click could give us info on what will happen, but has implications when the website interprets that like a click from the user and updates their database in >some way such as unsubscribing a user.
Be carefull with one time links !!!! Download only headers may be a good idea
Another idea may be to donwload headers with different user-agents . If we get different responses.....
----PedroD