You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Keith Whyte <ke...@media-solutions.ie> on 2004/05/06 20:51:06 UTC

hotmail first HOP?

Would appreciaate some clarification of the firsthop look up rules for BlackLists..
This one is a valid mail from a hotmail account, 
and it nearly got hammered: (i've xxxxxxxx'd out all private info..)
Shouldn't 200.78.33.237 be treated as the first hop (originating host)
and therefore not trigger in all those blacklists?

thnx,
keith.


PROBLEM MESSAGE FOLLOWS:

Return-Path: <xx...@hotmail.com>
Received: from hotmail.com (bay2-f170.bay2.hotmail.com [65.54.247.170])
	by xxxxxxxxxxxxxxxxxxx (8.12.11/8.12.11) with ESMTP id i46IWnSi008992
	for <xxxxxxxxxxxxxxxxxxxxx>; Thu, 6 May 2004 19:32:49 +0100
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Thu, 6 May 2004 11:32:43 -0700
Received: from 200.78.33.237 by by2fd.bay2.hotmail.msn.com with HTTP;
	Thu, 06 May 2004 18:32:42 GMT
X-Originating-IP: [200.78.33.237]
X-Originating-Email: [xxxxxxxxxxxx@hotmail.com]
X-Sender: xxxxxxxxxxxxx@hotmail.com
From: "xxxxxxxxxxx" <xx...@hotmail.com>
To: xxxxxxxxxxxxxxxxxxxxxx
Subject: ** SPAM? (6.522) xxxxxxxxxxxxxxxxx **
Date: Thu, 06 May 2004 18:32:42 +0000
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary="----------=_1083868372-7696-6"
Message-ID: <BA...@hotmail.com>
X-OriginalArrivalTime: 06 May 2004 18:32:43.0210 (UTC) FILETIME=[84F7E2A0:01C43398]
X-Spam-Score: 6.522 (******) HTML_MESSAGE,MIME_HTML_NO_CHARSET,MIME_HTML_ONLY,RCVD_IN_DSBL,RCVD_IN_NJABL,RCVD_IN_NJABL_PROXY,RCVD_IN_SORBS,RCVD_IN_SORBS_HTTP,RCVD_IN_SORBS_MISC,RCVD_IN_SORBS_SOCKS
X-Spam-Level: ******
X-Scanned-By: MIMEDefang 2.42

This is a multi-part message in MIME format...

------------=_1083868372-7696-6
Content-Type: text/html
Content-Disposition: inline

<html>
[ ...
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
...]
</html>

------------=_1083868372-7696-6
Content-Type: text/plain; name="SpamAssassinReport.txt"
Content-Disposition: inline; filename="SpamAssassinReport.txt"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-Mailer: MIME-tools 5.411 (Entity 5.404)

Spam detection software, running on the system "xxxxxxxxxxxxxxx", has
identified this incoming email as possible spam.  The original message
has been attached to this so you can view it (if it isn't spam) or block
similar future email.  If you have any questions, see
the administrator of that system for details.

Content analysis details:   (6.5 points, 9.0 required)

 pts rule name              description
---- ---------------------- --------------------------------------------------
 0.1 MIME_HTML_ONLY         BODY: Message only has text/html MIME parts
 0.0 HTML_MESSAGE           BODY: HTML included in message
 0.7 MIME_HTML_NO_CHARSET   RAW: Message text in HTML without charset
 1.1 RCVD_IN_SORBS_HTTP     RBL: SORBS: sender is open HTTP proxy server
                            [200.78.33.237 listed in dnsbl.sorbs.net]
 1.1 RCVD_IN_SORBS_MISC     RBL: SORBS: sender is open proxy server
                            [200.78.33.237 listed in dnsbl.sorbs.net]
 0.1 RCVD_IN_SORBS          RBL: SORBS: sender is listed in SORBS
                            [200.78.33.237 listed in dnsbl.sorbs.net]
 0.1 RCVD_IN_NJABL          RBL: Received via a relay in dnsbl.njabl.org
                            [200.78.33.237 listed in dnsbl.njabl.org]
 1.1 RCVD_IN_DSBL           RBL: Received via a relay in list.dsbl.org
                            [<http://dsbl.org/listing?ip=200.78.33.237>]
 1.1 RCVD_IN_NJABL_PROXY    RBL: NJABL: sender is an open proxy
                            [200.78.33.237 listed in dnsbl.njabl.org]
 1.1 RCVD_IN_SORBS_SOCKS    RBL: SORBS: sender is open SOCKS proxy server
                            [200.78.33.237 listed in dnsbl.sorbs.net]



------------=_1083868372-7696-6--


Re: hotmail first HOP?

Posted by Keith Whyte <ke...@media-solutions.ie>.
Matt Kettler wrote:

>
> Did you mis-speak and really mean rejecting (at SMTP delivery time)? 
> If not, why are you bouncing at all? Shame on you. Post delivery 
> bounce systems are nearly as bad as spammers.
>
I think i mis-spoke, let me clarify:
My system runs sendmail/mimedefang/spamassassin/clamav
If the incoming mail acheives sufficient spam score, my MTA rejects with 
554 5.7.1 and a text message which depends on rcpt domain.
I also disallow connections based on certain blacklists.
Currently in cetain situations, I accept and simply drop viruses, in 
some cases they are rejected. (yes i understand the issues here.)
I don't believe i'm being a bad net citizen with any of this.

Hey let's not fight - i totally agree, isp's should be pressured to 
assign static ip to highspeed dialup, it would make life much easier,
I'm simply looking to find a solution so that I can keep my users 
mailboxes as spam free as possible, while not confusing people sending 
mail to my users, when their mail is legit.

Keith.


Re: hotmail first HOP?

Posted by Matt Kettler <mk...@evi-inc.com>.
At 04:13 PM 5/6/2004, Keith Whyte wrote:
>but I *have* to do something, currently i risk bouncing perfectly valid 
>mail originating from these IP blocks.

Did you mis-speak and really mean rejecting (at SMTP delivery time)? If 
not, why are you bouncing at all? Shame on you. Post delivery bounce 
systems are nearly as bad as spammers.

>what would you say to counting the hops, then running notfirsthop tests on 
>the SOCKS/PROXY rbl lists and then reducing the score with a meta rule, if 
>there are more than 2 hops.

Don't bother counting hops and other garbage, just use notfirsthop for your 
systems and be done with it.

The "notfirsthop" setting isn't literally "not first hop" it's really "only 
check IPs dropping off at trusted servers", which should achieve what you 
desire.

I on the other hand, prefer the default, and IMO correct, behavior.






Re: hotmail first HOP?

Posted by Keith Whyte <ke...@media-solutions.ie>.
Matt Kettler wrote:

> Not all dynamic blocks are "different every day" like your net cafe 
> example. You can't categorically excuse dynamic IPs from proxy list 
> checks.
>
but I *have* to do something, currently i risk bouncing perfectly valid 
mail originating from these IP blocks.


what would you say to counting the hops, then running notfirsthop tests 
on the SOCKS/PROXY rbl lists and then reducing the score with a meta 
rule, if there are more than 2 hops.

as in:
+ the firsthop is an open relay..
+ then the message was relayed by other relay (how many open relays are 
configured with a smarthost?)
+ the "other relay" is not an open relay, so.....
= ignore the fact that the firsthop is an open relay.


Re: hotmail first HOP?

Posted by Keith Whyte <ke...@media-solutions.ie>.
> Firsthop *ONLY* applies to dialup/dynamic IP lists. None of the lists 
> that fired are dialup lists.
>
is there a *bug* then in the fact that the non-dialup lists are listing 
dial-up IPs?
obviously a dial-up running open-relay/socks/etc/ is temporary?

maybe i should run my own listcheck and use a meta rule to drop the 
score a little if  a SOCKS type rule also matches a dialup?


Re: hotmail first HOP?

Posted by jdow <jd...@earthlink.net>.
From: "Keith Whyte" <ke...@media-solutions.ie>
> Matt Kettler wrote:
>
> > Firsthop *ONLY* applies to dialup/dynamic IP lists. None of the lists
> > that fired are dialup lists.
>
> I just ran a grep on the rules for notfirsthop and realise this now,
> sorry for the lazy post.....
>
> >
> > In this case, it looks like the RBLs are claiming 200.78.33.237 was
> > running an open proxy. SORBS last scanned it august of 2003, but
> > nobody has ever requested rescan or challenged the listing.
> >
> > The SORBS ones can be cleared up pretty easily, if the user goes to
> > www.dnsbl.sorbs.net and looks up their IP, they can request a re-scan.
>
> Yes, but as it was a from a user in an internet cafe on a mexican DSL
> provider's link, chances are there are more listed ip's, and the when
> the café boots up tommorrow equally they'll get a different address.
> (it's prodigy-infinitum - they operate on mostly Dynamic IP DSL) I've
> seen a good few listed recently.
> Should this be considered a bug?  - a score of nearly 6 just for using a
> net café? how could i lower the score here?

Don't use such a broad brush as you get with black lists. Scan via other
means. I do not use BLs here. Of about 200 spams a day I get about 1 missed
spam per day and maybe one false hit every few weeks. With that well
setup filtering why should I bother with the BLs and their overly broad
brushes?

An alternative is to let the person know that he should defer sending
his email until he is home. Internet cafes are prime sites for sending
spam and scams. The recent Nigerian 419 scammer that was caught was
nailed in a wireless hotspot at a Starbucks, I believe.

{^_^}


Re: hotmail first HOP?

Posted by Matt Kettler <mk...@evi-inc.com>.
At 03:24 PM 5/6/2004, Keith Whyte wrote:
>Yes, but as it was a from a user in an internet cafe on a mexican DSL 
>provider's link, chances are there are more listed ip's, and the when the 
>café boots up tommorrow equally they'll get a different address. (it's 
>prodigy-infinitum - they operate on mostly Dynamic IP DSL) I've seen a 
>good few listed recently.

That's not surprising. If there's a vulnerable proxy in that network, 
spammers will keep finding it, and keep abusing it, even as it moves around 
to different IPs.

>Should this be considered a bug?

I'd argue it's not a bug, but it is definitely collateral damage, which 
isn't a good thing.

>- a score of nearly 6 just for using a net café?

Correction.. a score of nearly 6 for using a net cafe which is part of an 
IP network block known to contain open proxies which have been targeted and 
abused to relay spam, and happens to currently be located at an IP address 
which had one of these verified proxies.

>how could i lower the score here?

Dynamic DSL blocks containing open proxies are a problem. Eventually the 
entire subnet gets blocked until the ISP tracks down which of their 
customers has an open proxy and gets them to fix it, or blocks inbound 
proxy requests altogether.

However, from a spam fighting perspective, you KNOW for a fact that a relay 
existed at that location, and keeps jumping IPs in the same block. You also 
know that eventually the offending IP will move back to the same address, 
since it's allocation is going to be semi-random, but from a fixed-size 
pool of IPs. Do you merely ignore these IPs, or do you accept the 
collateral damage of listing this IP, even though that the relay may or may 
not be there today?

Unfortunately, only the ISP itself is the only is capable of dealing with 
this problem. It needs to track down which of it's customers is the relay, 
and then inform the RBLs that they've fixed the problem. End user's can't 
really do much.

The other alternative is to get a static IP service. Technically speaking, 
this is the kind of service a net cafe should have anyway, but it's not 
always available at reasonable cost. As an amusing twist, the ISP's job 
would be a lot easier if the net cafe was static, that way they'd be able 
to quickly identify complaints as being related to the net cafe or not.






Re: hotmail first HOP?

Posted by Keith Whyte <ke...@media-solutions.ie>.
Matt Kettler wrote:

> Firsthop *ONLY* applies to dialup/dynamic IP lists. None of the lists 
> that fired are dialup lists.

I just ran a grep on the rules for notfirsthop and realise this now, 
sorry for the lazy post.....  

>
> In this case, it looks like the RBLs are claiming 200.78.33.237 was 
> running an open proxy. SORBS last scanned it august of 2003, but 
> nobody has ever requested rescan or challenged the listing.
>
> The SORBS ones can be cleared up pretty easily, if the user goes to 
> www.dnsbl.sorbs.net and looks up their IP, they can request a re-scan.

Yes, but as it was a from a user in an internet cafe on a mexican DSL 
provider's link, chances are there are more listed ip's, and the when 
the café boots up tommorrow equally they'll get a different address. 
(it's prodigy-infinitum - they operate on mostly Dynamic IP DSL) I've 
seen a good few listed recently.
Should this be considered a bug?  - a score of nearly 6 just for using a 
net café? how could i lower the score here?


Re: hotmail first HOP?

Posted by Matt Kettler <mk...@evi-inc.com>.
At 02:51 PM 5/6/2004, Keith Whyte wrote:
>Would appreciaate some clarification of the firsthop look up rules for 
>BlackLists..
>This one is a valid mail from a hotmail account, and it nearly got 
>hammered: (i've xxxxxxxx'd out all private info..)
>Shouldn't 200.78.33.237 be treated as the first hop (originating host)
>and therefore not trigger in all those blacklists?


Firsthop *ONLY* applies to dialup/dynamic IP lists. None of the lists that 
fired are dialup lists.

The IP in question is listed in the sorbs based DYNABLOCK dynamic list, but 
that SA rule never matched (and correctly so) because it was firsthop skipped.



>HTML_MESSAGE,MIME_HTML_NO_CHARSET,MIME_HTML_ONLY,RCVD_IN_DSBL,RCVD_IN_NJABL,RCVD_IN_NJABL_PROXY,RCVD_IN_SORBS,RCVD_IN_SORBS_HTTP,RCVD_IN_SORBS_MISC,RCVD_IN_SORBS_SOCKS

In this case, it looks like the RBLs are claiming 200.78.33.237 was running 
an open proxy. SORBS last scanned it august of 2003, but nobody has ever 
requested rescan or challenged the listing.

The SORBS ones can be cleared up pretty easily, if the user goes to 
www.dnsbl.sorbs.net and looks up their IP, they can request a re-scan.


Re: hotmail first HOP?

Posted by Matt Kettler <mk...@evi-inc.com>.
At 03:10 PM 5/6/2004, Dan Wilder wrote:
>No doubt there's a post in the archive that describes it all perfectly,
>but I was unable to find it.  The question crops up often.  I was having
>trouble with it myself just the other day.
>
>The resolution AFAIK revolves around:
>
>*  setting trusted_networks correctly, and

Dan, this isn't Keith's problem.. that will fix "notfirsthop" lists, but 
that part of his config is working correctly, otherwise RCVD_IN_DYNABLOCK 
would have also matched.

The problem he's having is not a misconfiguration. His friends IP is 
blacklisted as an open-proxy spam relay. SA noticed, as it should.

SA is very much intentionally designed to NOT skip open-relay list checks 
on the first hop.

Why? Because some spam is relayed from exploited hosts into legitimate ISP 
relays. The only way to detect these is to detect if the originating IP is 
in any relay/exploit lists. The IP in question is part of a DSL network, 
which makes it a prime target for spammer abuse and all the more important 
to check the relay databases. (Lots of foolish DSL users accidentally 
create open relay mailservers and/or proxies on their home systems as they 
try to evade their work's firewalls and/or content filters)

This is not the same thing as checking the first hop against dialup lists, 
which is inherently bad.


Re: hotmail first HOP?

Posted by Dan Wilder <da...@ssc.com>.
On Thu, May 06, 2004 at 01:51:06PM -0500, Keith Whyte wrote:
> Would appreciaate some clarification of the firsthop look up rules for 
> BlackLists..
> This one is a valid mail from a hotmail account, 
> and it nearly got hammered: (i've xxxxxxxx'd out all private info..)
> Shouldn't 200.78.33.237 be treated as the first hop (originating host)
> and therefore not trigger in all those blacklists?
> 
> thnx,
> keith.

No doubt there's a post in the archive that describes it all perfectly,
but I was unable to find it.  The question crops up often.  I was having
trouble with it myself just the other day.

The resolution AFAIK revolves around:

*  setting trusted_networks correctly, and

*  Having some sort of DNS that resolves your mailhub or
   incoming relay host with an "A" record.  The address the
   "A" record points to needs to be in trusted_networks.

In our case we have in local.cf:

    trusted_networks 192.168. 127.

and incoming mail has:

    Received: ... by mail.ssc.com (Postfix) with ESMTP id 70B2AFF80

added by our mail hub, a bastion host hiding from the Internet
behind a NAT router.  Outside there's an "A" record for mail.ssc.com 
but it points to the outside address of the router, which port-forwards 
to the mail hub.  Inside, the hub is at 192.168.1.3.

Inside the NAT we run internal split DNS which considers itself
definitive for our domain but resolves all the stuff on our LAN
you can't see at all from the outside.

For quite a long time I was having problems with RCVD_IN_DSBL cropping
up for anything that originated from a dynamic address, regardless of
whether it was correctly relayed via a good ISP address or not.

As I've seen it explained, the "Received" header that's tested 
is the last one added prior to arrival at a trusted network.  Thus
not penalizing users who send via smtp from a dynamic address so
long as they relay via their ISP.

The final piece that made it all work for us was adding an "A" record 
for mail.ssc.com to the internal split of the DNS.  Previously we 
had had a CNAME record pointing to another name for the host, by which
it is known for most purposes.

Once the A record for mail.ssc.com became available internally, we
stopped getting those annoying FPs 

> 
> 
> PROBLEM MESSAGE FOLLOWS:
> 
> Return-Path: <xx...@hotmail.com>
> Received: from hotmail.com (bay2-f170.bay2.hotmail.com [65.54.247.170])
> 	by xxxxxxxxxxxxxxxxxxx (8.12.11/8.12.11) with ESMTP id i46IWnSi008992
> 	for <xxxxxxxxxxxxxxxxxxxxx>; Thu, 6 May 2004 19:32:49 +0100
> Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
> 	 Thu, 6 May 2004 11:32:43 -0700
> Received: from 200.78.33.237 by by2fd.bay2.hotmail.msn.com with HTTP;
> 	Thu, 06 May 2004 18:32:42 GMT
> X-Originating-IP: [200.78.33.237]
> X-Originating-Email: [xxxxxxxxxxxx@hotmail.com]
> X-Sender: xxxxxxxxxxxxx@hotmail.com
> From: "xxxxxxxxxxx" <xx...@hotmail.com>
> To: xxxxxxxxxxxxxxxxxxxxxx
> Subject: ** SPAM? (6.522) xxxxxxxxxxxxxxxxx **
> Date: Thu, 06 May 2004 18:32:42 +0000
> Mime-Version: 1.0
> Content-Type: multipart/mixed; boundary="----------=_1083868372-7696-6"
> Message-ID: <BA...@hotmail.com>
> X-OriginalArrivalTime: 06 May 2004 18:32:43.0210 (UTC) 
> FILETIME=[84F7E2A0:01C43398]
> X-Spam-Score: 6.522 (******) 
> HTML_MESSAGE,MIME_HTML_NO_CHARSET,MIME_HTML_ONLY,RCVD_IN_DSBL,RCVD_IN_NJABL,RCVD_IN_NJABL_PROXY,RCVD_IN_SORBS,RCVD_IN_SORBS_HTTP,RCVD_IN_SORBS_MISC,RCVD_IN_SORBS_SOCKS
> X-Spam-Level: ******
> X-Scanned-By: MIMEDefang 2.42
> 
> This is a multi-part message in MIME format...
> 
> ------------=_1083868372-7696-6
> Content-Type: text/html
> Content-Disposition: inline
> 
> <html>
> [ ...
> xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 
> ...]
> </html>
> 
> ------------=_1083868372-7696-6
> Content-Type: text/plain; name="SpamAssassinReport.txt"
> Content-Disposition: inline; filename="SpamAssassinReport.txt"
> Content-Transfer-Encoding: 7bit
> MIME-Version: 1.0
> X-Mailer: MIME-tools 5.411 (Entity 5.404)
> 
> Spam detection software, running on the system "xxxxxxxxxxxxxxx", has
> identified this incoming email as possible spam.  The original message
> has been attached to this so you can view it (if it isn't spam) or block
> similar future email.  If you have any questions, see
> the administrator of that system for details.
> 
> Content analysis details:   (6.5 points, 9.0 required)
> 
> pts rule name              description
> ---- ---------------------- 
> --------------------------------------------------
> 0.1 MIME_HTML_ONLY         BODY: Message only has text/html MIME parts
> 0.0 HTML_MESSAGE           BODY: HTML included in message
> 0.7 MIME_HTML_NO_CHARSET   RAW: Message text in HTML without charset
> 1.1 RCVD_IN_SORBS_HTTP     RBL: SORBS: sender is open HTTP proxy server
>                            [200.78.33.237 listed in dnsbl.sorbs.net]
> 1.1 RCVD_IN_SORBS_MISC     RBL: SORBS: sender is open proxy server
>                            [200.78.33.237 listed in dnsbl.sorbs.net]
> 0.1 RCVD_IN_SORBS          RBL: SORBS: sender is listed in SORBS
>                            [200.78.33.237 listed in dnsbl.sorbs.net]
> 0.1 RCVD_IN_NJABL          RBL: Received via a relay in dnsbl.njabl.org
>                            [200.78.33.237 listed in dnsbl.njabl.org]
> 1.1 RCVD_IN_DSBL           RBL: Received via a relay in list.dsbl.org
>                            [<http://dsbl.org/listing?ip=200.78.33.237>]
> 1.1 RCVD_IN_NJABL_PROXY    RBL: NJABL: sender is an open proxy
>                            [200.78.33.237 listed in dnsbl.njabl.org]
> 1.1 RCVD_IN_SORBS_SOCKS    RBL: SORBS: sender is open SOCKS proxy server
>                            [200.78.33.237 listed in dnsbl.sorbs.net]
> 
> 
> 
> ------------=_1083868372-7696-6--
> 

-- 
Dan Wilder