You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Marc Perkel <su...@junkemailfilter.com> on 2011/11/17 16:27:51 UTC

Detecting serious domains

I'm exploring a variety of ideas to determine the difference between 
"serious" domains down to throw away domains used by spammers. The ideas 
I'm presenting here are not complete but are just a conversation starter.

for example, if the sending domain has no MX records of its own it is 
more likely spam that if there are 3 or more MX records that resolve to 
multiple IPs over more than one network. Generally spam only domains are 
minimally configured, and highly configured domains are not spam only. I 
also think that NS records might indicate that a domain is serious or not.

I think the serious scale could be a useful factor in SA. It doesn't 
determine if it's spam or ham in itself. Yahoo is a serious domain and 
there's lost of spam. Serious domains should not be blacklisted for 
example. We could also look for consistency. Bad RDNS from a serious 
domain might be a spam indicator.

There might be other methods of detecting serious domains. If they are 
using expensive services. Spammers would not have their dns hosted with 
Ultra DNS, or use the expensive registrars, or other services that are 
expensive.

Also - thinking we should slowly mine the whois database and provide 
some sort of DNS based lookup of whois information to be able to 
determine the registrar of a domain, the domain age, or other info that 
would be useful in determining that the domain is serious or not.

Who thinks I'm onto something?


-- 
Marc Perkel - Sales/Support
support@junkemailfilter.com
http://www.junkemailfilter.com
Junk Email Filter dot com
415-992-3400


Re: Detecting serious domains

Posted by Robert Schetterer <ro...@schetterer.org>.
Am 17.11.2011 16:27, schrieb Marc Perkel:
> I'm exploring a variety of ideas to determine the difference between
> "serious" domains down to throw away domains used by spammers. The ideas
> I'm presenting here are not complete but are just a conversation starter.
> 
> for example, if the sending domain has no MX records of its own it is
> more likely spam that if there are 3 or more MX records that resolve to
> multiple IPs over more than one network. Generally spam only domains are
> minimally configured, and highly configured domains are not spam only. I
> also think that NS records might indicate that a domain is serious or not.
> 
> I think the serious scale could be a useful factor in SA. It doesn't
> determine if it's spam or ham in itself. Yahoo is a serious domain and
> there's lost of spam. Serious domains should not be blacklisted for
> example. We could also look for consistency. Bad RDNS from a serious
> domain might be a spam indicator.
> 
> There might be other methods of detecting serious domains. If they are
> using expensive services. Spammers would not have their dns hosted with
> Ultra DNS, or use the expensive registrars, or other services that are
> expensive.
> 
> Also - thinking we should slowly mine the whois database and provide
> some sort of DNS based lookup of whois information to be able to
> determine the registrar of a domain, the domain age, or other info that
> would be useful in determining that the domain is serious or not.
> 
> Who thinks I'm onto something?
> 
> 

in the end this is a reputation question
as far i know this is worked on a more global standart
with ideas from i.e dkim
perhaps you should invest time there
not sure if this is a right link
http://tools.ietf.org/html/draft-kucherawy-reputation-query-dns-00

but i know others , reading this list know better about whats comming up
there, perhaps they answer better

and as far i remember , there where solutions to identify
domains registration age, as there where a possibility
for "tasting" domains, but as i remember this isnt possible anymore


-- 
Best Regards

MfG Robert Schetterer

Germany/Munich/Bavaria

Re: Detecting serious domains

Posted by Michelle Konzack <li...@tamay-dogan.net>.
Hello Marc,

Am 2011-11-17 07:27:51, hacktest Du folgendes herunter:
> determine if it's spam or ham in itself. Yahoo is a serious domain
> and there's lost of spam. Serious domains should not be blacklisted

Ehm?

I block <@yahoo.com> on SMTP level (on my corporated Server), because if
I remove the BLOCK, I would get every day around  20-80.000  Spams  from
Yahoo.

> for example. We could also look for consistency. Bad RDNS from a
> serious domain might be a spam indicator.

Right

> Also - thinking we should slowly mine the whois database and provide
> some sort of DNS based lookup of whois information to be able to
> determine the registrar of a domain, the domain age, or other info
> that would be useful in determining that the domain is serious or
> not.

1+  At least for the Domain Age!

> Who thinks I'm onto something?

Thanks, Greetings and nice Day/Evening
    Michelle Konzack

-- 
##################### Debian GNU/Linux Consultant ######################
   Development of Intranet and Embedded Systems with Debian GNU/Linux
               Internet Service Provider, Cloud Computing
                <http://www.itsystems.tamay-dogan.net/>

itsystems@tdnet                     Jabber  linux4michelle@jabber.ccc.de
Owner Michelle Konzack

Gewerbe Strasse 3                   Tel office: +49-176-86004575
77694 Kehl                          Tel mobil:  +49-177-9351947
Germany                             Tel mobil:  +33-6-61925193  (France)

USt-ID:  DE 278 049 239

Linux-User #280138 with the Linux Counter, http://counter.li.org/

Re: Detecting serious domains

Posted by Michelle Konzack <li...@tamay-dogan.net>.
Hello darxus@chaosreigns.com,

Am 2011-11-17 12:29:41, hacktest Du folgendes herunter:
> There could be a useful correlation there, but I need to point out that if
> a domain has no MX records, the correct thing to do is to send email to the
> A record for the domain, and I've seen legit domains configured that way
> and unwilling to change.  It's not even a violation of RFC.

Right, but MOST spamers act like this AND there IP does not  respond  to
SMTP requests.  So, why waysting time and resource?

Thanks, Greetings and nice Day/Evening
    Michelle Konzack

-- 
##################### Debian GNU/Linux Consultant ######################
   Development of Intranet and Embedded Systems with Debian GNU/Linux
               Internet Service Provider, Cloud Computing
                <http://www.itsystems.tamay-dogan.net/>

itsystems@tdnet                     Jabber  linux4michelle@jabber.ccc.de
Owner Michelle Konzack

Gewerbe Strasse 3                   Tel office: +49-176-86004575
77694 Kehl                          Tel mobil:  +49-177-9351947
Germany                             Tel mobil:  +33-6-61925193  (France)

USt-ID:  DE 278 049 239

Linux-User #280138 with the Linux Counter, http://counter.li.org/

Re: Detecting serious domains

Posted by da...@chaosreigns.com.
On 11/17, Marc Perkel wrote:
> for example, if the sending domain has no MX records of its own it
> is more likely spam that if there are 3 or more MX records that

There could be a useful correlation there, but I need to point out that if
a domain has no MX records, the correct thing to do is to send email to the
A record for the domain, and I've seen legit domains configured that way
and unwilling to change.  It's not even a violation of RFC.

> There might be other methods of detecting serious domains. If they
> are using expensive services. Spammers would not have their dns
> hosted with Ultra DNS, or use the expensive registrars, or other
> services that are expensive.

There *could* be something there.  I'm not familiar with Ultra DNS, but any
service provider that's known to be good about blocking customers for
spamming, it could be useful to detect the use of that provider, and use it
as a kind of white list.  Maybe even some network providers are better
about kicking off spammers than others?

I'll actually forward that idea along to the DNSWL folks for consideration
as criteria for whitelisting.

> Who thinks I'm onto something?

You could be.  Find out and let us know :)
The ASRG list is also generally interested in this type of discussion:
http://irtf.org/mailman/listinfo/asrg

-- 
"Life is either a daring adventure or it is nothing at all."
- Helen Keller
http://www.ChaosReigns.com

Re: Detecting serious domains

Posted by Joe Sniderman <jo...@thoroquel.org>.
On 11/17/2011 10:27 AM, Marc Perkel wrote:
> I'm exploring a variety of ideas to determine the difference between
> "serious" domains down to throw away domains used by spammers. The ideas
> I'm presenting here are not complete but are just a conversation starter.

I'd think domain age would be biggest indicator of *not* being a
throw-away domain. IIRC you already track that in your own dns lists..

> for example, if the sending domain has no MX records of its own it is
> more likely spam that if there are 3 or more MX records that resolve to
> multiple IPs over more than one network. Generally spam only domains are
> minimally configured, and highly configured domains are not spam only. I
> also think that NS records might indicate that a domain is serious or not.

Maybe. Keep in mind that a throw-away domain might be purchased through
a registrar that provides email hosting, and might appear to be nicely
configured. Perhaps in combination with age, this could be a semi-useful
indicator.. ie, new domain, only one (implicit) MX, decent chance of
being a throw away. old domain, probably not a throwaway, new domain,
nicely configured is hard to tell though...

There also are free email hosting options that involve multiple MXs..
Put yourself in spammy's shoes for a moment... if you had a throwaway
domain would you host the incoming email on your own servers, or on
gmail apps? I'd choose gmail in heartbeat. Then again plenty of legit
domains use gmail too, so in and of itself its not a sign of seriousness
nor of lack thereof.

Regarding NS records, yeah there are some nameservers that seem to only
host bad domains, others that seem to only (or mostly) host good
domains, and then a whole bunch that host all kinds of domains.
Nameserver reputation might be a good track to follow IMHO.

If you're trying to predict how serious a newly registered domain is,
I'd look at the nameservers, and the whois.

Private registration, cheap registrar, cheap dns, brand new domain,
probably bad news until more info is available

Proper whois info, expensive registrar, expensive dns and/or dns with a
good reputation, brand new domain, far less suspicious.

Proper whois info, cheap registrar, expensive dns and/or dns with a good
a reputation, brand new domain, somewhere in the middle.

I'm just thinking "outloud" here, does this sound sensible to you?

> I think the serious scale could be a useful factor in SA. It doesn't
> determine if it's spam or ham in itself. Yahoo is a serious domain and
> there's lost of spam. Serious domains should not be blacklisted for
> example. We could also look for consistency. Bad RDNS from a serious
> domain might be a spam indicator.
> 
> There might be other methods of detecting serious domains. If they are
> using expensive services. Spammers would not have their dns hosted with
> Ultra DNS, or use the expensive registrars, or other services that are
> expensive.

Well, spammers might well use UltraDNS and/or expensive registrars, but
probably would only do so for their domains which they intend on keeping
long term, rather than ones they intend to throw away.

> Also - thinking we should slowly mine the whois database and provide
> some sort of DNS based lookup of whois information to be able to
> determine the registrar of a domain, the domain age, or other info that
> would be useful in determining that the domain is serious or not.

That is nothing new. Spam eating monkey AFAIK has some zones which tell
about privitized whois, age of registration, etc. You also have a zone
if I'm not mistaken that approximates how long ago a domain was first
seen (by your users) in email.  Some of the SORBS zones can also be used
in tandem with each other to extrapolate similar info.  Senderbase also
keeps tabs on history of domains as well as IPs..

Question becomes how to use the data in a way to approximate the
"seriousness" of the domain, and then how to use the approximated domain
"seriousness" to improve filtering. Two separate questions if you ask me.

> Who thinks I'm onto something?

I think you *might* be on to something, or might not.

Here's another thought off the top of my head: looking at nameservers
and calculating average age of domains hosted on the nameservers, as
well as the average length of time that the domains hosted on the
nameservers have been on those specific nameservers...

-- 
Joe Sniderman <jo...@thoroquel.org>

Re: Detecting serious domains

Posted by Michelle Konzack <li...@tamay-dogan.net>.
Hello Kevin A. McGrail,

Am 2011-11-17 10:56:52, hacktest Du folgendes herunter:
> For example, I've seen .info domains used a lot by spammers.  I'm
> sure there is a patter there with a registrar probably.

Here I can say, the DOT INFO spam is nearly 60%.

Thanks, Greetings and nice Day/Evening
    Michelle Konzack

-- 
##################### Debian GNU/Linux Consultant ######################
   Development of Intranet and Embedded Systems with Debian GNU/Linux
               Internet Service Provider, Cloud Computing
                <http://www.itsystems.tamay-dogan.net/>

itsystems@tdnet                     Jabber  linux4michelle@jabber.ccc.de
Owner Michelle Konzack

Gewerbe Strasse 3                   Tel office: +49-176-86004575
77694 Kehl                          Tel mobil:  +49-177-9351947
Germany                             Tel mobil:  +33-6-61925193  (France)

USt-ID:  DE 278 049 239

Linux-User #280138 with the Linux Counter, http://counter.li.org/

Re: Detecting serious domains

Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
> for example, if the sending domain has no MX records of its own it is 
> more likely spam that if there are 3 or more MX records that resolve 
> to multiple IPs over more than one network. Generally spam only 
> domains are minimally configured, and highly configured domains are 
> not spam only. I also think that NS records might indicate that a 
> domain is serious or not.
>
> I think the serious scale could be a useful factor in SA. It doesn't 
> determine if it's spam or ham in itself. Yahoo is a serious domain and 
> there's lost of spam. Serious domains should not be blacklisted for 
> example. We could also look for consistency. Bad RDNS from a serious 
> domain might be a spam indicator.
>
> There might be other methods of detecting serious domains. If they are 
> using expensive services. Spammers would not have their dns hosted 
> with Ultra DNS, or use the expensive registrars, or other services 
> that are expensive.
>
> Also - thinking we should slowly mine the whois database and provide 
> some sort of DNS based lookup of whois information to be able to 
> determine the registrar of a domain, the domain age, or other info 
> that would be useful in determining that the domain is serious or not.
>
> Who thinks I'm onto something? 

You have ideas here that have been done before (whois information and 
domain age for example).  You have some others that I would question 
without research.

I would recommend you look at installing something like MIMEDefang that 
you can use to do research on incoming email and gather some statistics 
and see if you can spot a ham/spam trend.

There is certainly merit to the investigation and then you can come back 
and say I found XYZ.

For example, I've seen .info domains used a lot by spammers.  I'm sure 
there is a patter there with a registrar probably.

Regards,
KAM