You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Da...@ChaosReigns.com on 2010/02/14 20:06:56 UTC
MTX public blacklist implemented Re: MTX plugin functionally
complete?
Very simple, via HTTP and a small perl script:
http://www.chaosreigns.com/mtx/#blacklist
It's currently empty.
On 02/14, Darxus@ChaosReigns.com wrote:
> On 02/14, Jonas Eckerman wrote:
> > * I think there should be a way to tell the world wether you are using
> > the scheme for a domain (not host) or not. This could easily be done in
> > DNS.
> > If anyone connects from a host where reverse lookup or HELO puts it in
> > "frukt.org" domain, you know that your should reject or score high unless
> > it has FCDNS and a matching MTX record.
I remembered why (else) I didn't want to do that. It effectively says
"Everything else should be rejected." Which will discourage some people
from using it. So you would at least need to provide a way to say "Yes,
I'm participating, but anything without an MTX record is valid too."
Still needs further consideration.
--
"When in doubt, gas it. It may not solve the problem,
But it ends the suspense." - Steve Moonitz, DoD #2319, 1994
http://www.ChaosReigns.com
Re: MTX - How does it stop spam?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
> On 02/16, Marc Perkel wrote:
> > I'm still waiting for RDNS to be widely adopted enough to penalize for
> > that. There is a lot of good email that comes from misconfigured
> > servers. If we can't get the world to do good RDNS I don't think we can
> > get the world to do some other more complex scheme.
On 16.02.10 15:31, Darxus@ChaosReigns.com wrote:
> If valid RDNS were a usefully unforgable way to detect spam, I like to
> think there would be more of a push to straighten it out. But spammers
> have quite a lot of IPs to use with valid RDNS already.
>
> So I think requiring it for something that has a better chance of blocking
> spam has a better chance of getting RDNS set up properly.
At least some spammers tend to adapt, SPF might be a good example (no, they
did not epxloit it, they only exploited people who misunderstood SPF).
Why do you think that more regular admins than spammers will adapt onto this
scheme?
>From this point of view, MTX is very similar to Spamhaus PBL - they indent
to mark IPs that are either supposed to send mail (MTX) or _not_ supposed to
send mail (PBL). Since not all of admins trust PBL (and SPF, DKIM) at SMTP
level, I'm not sure wheter they will trust MTX at this level.
There can always be some idiot trying to deliver his "correct" mail from
braindeadly broken mail client through misconfigured mail server from an IP
without rdns, listed in every possible blacklist, and admin who thinks this
is a good reason not to trust any of those spam signs.
> On 02/16, Marc Perkel wrote:
> > I'm looking over your MTX site and like SPF I don't understand how it
> > stops spam. Thanks for at least addressing in part the email forwarding
> > issue.
>
> To take an example off the end of my log file:
I think we even do not need examples. MTX mark should clearly indicate which
host is supposed to send mail and which is not.
MTX depends on both reverse and direct domain, which means you need to have
access to both to provide valid MTX record.
This also means that changing either reverse or forward DNS name will
invalidate existing MTX record, which I believe is correct for positive (may
send mail) records, but imho it's incorrect for negative (may not send
mail) records.
It also means that setting up MTX is a bit harder and less error-prone.
> > In order to be a white list you have to do something spammers can't do.
>
> That's why a blacklist of spammers using MTX is necessary.
Blackists are necessary in all ways imho. While correct SPF/DKIM reduces the
need for blacklists/whitelists to domain _name_, lists like PBL/DNSWL reduce
that to IP addresses. MTX attepts to do both, while not attempting to solve
the forging problem that is addressed by SPF/DKIM.
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Save the whales. Collect the whole set.
Re: MTX - How does it stop spam?
Posted by Henrik K <he...@hege.li>.
On Tue, Feb 16, 2010 at 03:36:53PM -0500, Darxus@ChaosReigns.com wrote:
> On 02/16, Kris Deugau wrote:
> > Marc Perkel wrote:
> >> ... Since your idea also
> >> requires blacklists to counter this effect then I'm still not sure what
> >> this adds.
> >
> > *nod* This is the biggest question I still see remaining; who
> > maintains the blacklist? How many spams can come from an "MTX-approved"
> > IP before it can/should be blacklisted?
>
> Because I believe it will be far easier to maintain a list of IPs and / or
> domains which spam *and* use MTX, due to significant reduction in IPs they
> can spam from. My last post went into more detail.
You keep "believing" without actually ever explaining anything in detail.
How is it easier? You STILL need global traps/feeds to have reasonable data
and reputation. Are you going to be the maintainer? I don't see how MTX
would help for example Spamhaus. They list everything anyway and it works
fine.
Some people do maintain own lists, but I don't understand how MTX helps them
blacklist either.
> The problem of deciding at which point you blacklist somebody is the reason
> why my blacklist implementation allows you to set an SA score for each
> domain. So hosts which send legitimate email *and* spam you can give a
> score which only negates the effect of the MTX Pass.
>
> The public blacklist implementation also allows for this distinction.
I'm sure you know that SA is not the only software/appliance out there.
Probably even the minority. Ijf you want to keep a list, it needs to be DNS
based anyway. Do you really think once a day update is enough for data which
can be old in matter of minutes?
Re: MTX - How does it stop spam?
Posted by Da...@ChaosReigns.com.
On 02/16, Kris Deugau wrote:
> Marc Perkel wrote:
>> ... Since your idea also
>> requires blacklists to counter this effect then I'm still not sure what
>> this adds.
>
> *nod* This is the biggest question I still see remaining; who
> maintains the blacklist? How many spams can come from an "MTX-approved"
> IP before it can/should be blacklisted?
Because I believe it will be far easier to maintain a list of IPs and / or
domains which spam *and* use MTX, due to significant reduction in IPs they
can spam from. My last post went into more detail.
The problem of deciding at which point you blacklist somebody is the reason
why my blacklist implementation allows you to set an SA score for each
domain. So hosts which send legitimate email *and* spam you can give a
score which only negates the effect of the MTX Pass.
The public blacklist implementation also allows for this distinction.
--
"Will I ever learn? I hope not, I'm having too much fun."
- Brent "Minime" Avis, motorcycle.com
http://www.ChaosReigns.com
Re: MTX - How does it stop spam?
Posted by Da...@ChaosReigns.com.
On 02/16, Marc Perkel wrote:
> I'm still waiting for RDNS to be widely adopted enough to penalize for
> that. There is a lot of good email that comes from misconfigured
> servers. If we can't get the world to do good RDNS I don't think we can
> get the world to do some other more complex scheme.
If valid RDNS were a usefully unforgable way to detect spam, I like to
think there would be more of a push to straighten it out. But spammers
have quite a lot of IPs to use with valid RDNS already.
So I think requiring it for something that has a better chance of blocking
spam has a better chance of getting RDNS set up properly.
On 02/16, Marc Perkel wrote:
> I'm looking over your MTX site and like SPF I don't understand how it
> stops spam. Thanks for at least addressing in part the email forwarding
> issue.
To take an example off the end of my log file:
You get an email delivered by 163.20.114.1.
You look up the host name (PTR record) of that IP, it's:
dns.mcjh.tpc.edu.tw.
The IP and hostname make the MTX record:
1.114.20.163.mtx.dns.mcjh.tpc.edu.tw.
You look up the value of that DNS "A" record. It comes back "not found".
That's an MTX Fail. So you check the MTX Policy on the domain.
The MTX Policy record for that domain is named: policy.mtx.tpc.edu.tw.
If that DNS "A" record had the value 127.0.0.2 (HardFail, no delegation),
that would give this email an MTX HardFail, and you'd reject it.
Does that answer your question?
> In order to be a white list you have to do something spammers can't do.
That's why a blacklist of spammers using MTX is necessary. I believe
maintaining it will be far easier than maintaining blacklists of all the
spammer domains or IPs, since there are fewer opportunities where a spammer
owns both the IP *and* uses a domain they own.
As I said in the example, MTX will have problems in cases where the spammer
owns the IP and uses a throwaway domain, one which they only use for a
short burst of spam until the blacklists catch up with them.
But I believe that subset of IPs will be easier to maintain a blacklist
of, and if the spammers are pinned down that far, and all we have to
focus our attention on is the problem of throwaway domains, I think
it'll get handled.
SPF does not require the spammer to own the IP. They can use somebody
else's IP, use a domain they own in the MAIL FROM, which points to the SPF
record on their own domain, in which they can include whatever IPs they
want.
> this adds. As you know people register new domain names just to avoid
> being on any list so your idea would seem to be a white list for those
> who exploit that.
*And* own the transmitting IP, yes.
> As to penalizing those who don't participate, I already have enough
I don't recommend that.
> headaches with SPF and others who want to inflict their personal
> standards on the whole of the email community. SPF. which has left me
> with a bad attitude, does nothing for me to catch spam or pass ham. But
> it does result in good email that I forward being blocked.
Yup. That's specifically why I made MTX.
> As to whitelisting - there's actually a far easier solution that I use.
> I do RDNS to get the host name. Then I forward confirm it to verify that
> it is valid. Spammers can't spoof that. Then I look it up in my host
There are plenty of IPs configured to pass that available to spammers, but
your next point covers that problem.
> name white list of hosts who send nothing but good email. This actually
> works extremely well. But like everyone I'm always looking for more
Yup, that sounds good. DNSWL is a public whitelist intended for that kind
of use, but based on IPs instead of host names.
I assume you don't penalize email for not being on your whitelist?
> ideas especially for white rules because in y bustness one good email
> bounced is words that 100 spams not bounced.
I think that's true for most people. It certainly shows in the non-spam /
spam accuracy ratios SpamAssassin aims for when re-scoring the test set.
--
"Anarchy is based on the observation that since few are fit to rule
themselves, even fewer are fit to rule others." -Edward Abbey
http://www.ChaosReigns.com
Re: MTX public blacklist implemented Re: MTX plugin functionally complete?
Posted by Per Jessen <pe...@computer.org>.
Matus UHLAR - fantomas wrote:
> well, the ipv6 addresses are (were?) expected to be allocated by /48
> blocks, so we could need check on this level too, imho.
We got an IPv6 range allocated late last year, it is a /48 block.
/Per Jessen, Zürich
Re: MTX public blacklist implemented Re: MTX plugin functionally
complete?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
> On Sun, 14 Feb 2010, Jonas Eckerman wrote:
>> 1: The participation record is optional, so you only use it if you want
>> "everything else" to be rejected.
On 15.02.10 09:04, Charles Gregory wrote:
> This is why I would support mtamark... It permits the sysadmin to
> determine the default behaviour for his IP range, rather than defining a
> dangerous default in the client.
>
> And I quote:
> This subdomain MAY be inserted at any level in the DNS tree for IPv4
> IN-ADDR.ARPA reverse zones. For IPv6, to limit the number of DNS
> queries, _srv is only queried at the /128 (host), /64 (subnet) and /
> 32 (site) level. That way it can either provide information for a
> specific IP address or for a whole network block. More specific
> information takes precedence over information found closer to the top
> of the tree.
>
> The beauty of this mechanism is that we can 'sell' large ISP's on it by
> saying "you only need to create one 'allow' entry for each legitimate MTA
> and one 'deny' entry for each netblock.
well, the ipv6 addresses are (were?) expected to be allocated by /48 blocks,
so we could need check on this level too, imho.
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Silvester Stallone: Father of the RISC concept.
Re: [sa] Re: MTX - How does it stop spam?
Posted by Charles Gregory <cg...@hwcn.org>.
On Tue, 16 Feb 2010, Kris Deugau wrote:
> *nod* This is the biggest question I still see remaining; who maintains the
> blacklist? How many spams can come from an "MTX-approved" IP before it
> can/should be blacklisted?
Why do we need any new/special blacklist at all? If the spamming from a
given IP is sufficiently large, the regular internet blacklists will
capture this IP and do a far better job of blacklisting, managing removes,
etc, etc. Why reinvent the wheel?
- C
Re: MTX - How does it stop spam?
Posted by Matthias Leisi <ma...@leisi.net>.
Am 16.02.10 21:23, schrieb Kris Deugau:
> *nod* This is the biggest question I still see remaining; who
> maintains the blacklist? How many spams can come from an "MTX-approved"
> IP before it can/should be blacklisted?
It does not necessarily or exclusively need to be a manually maintained
blacklist. Domain-age- and usage-based blacklists would be useful in
that context.
-- Matthias
Re: MTX - How does it stop spam?
Posted by Kris Deugau <kd...@vianet.ca>.
Marc Perkel wrote:
> ... Since your idea also
> requires blacklists to counter this effect then I'm still not sure what
> this adds.
*nod* This is the biggest question I still see remaining; who
maintains the blacklist? How many spams can come from an "MTX-approved"
IP before it can/should be blacklisted?
-kgd
MTX - How does it stop spam?
Posted by Marc Perkel <ma...@perkel.com>.
I'm looking over your MTX site and like SPF I don't understand how it
stops spam. Thanks for at least addressing in part the email forwarding
issue.
In order to be a white list you have to do something spammers can't do.
I don't see what prevents spammers from creating good MTX records like
they do with SPF and whitelisting themselves. Since your idea also
requires blacklists to counter this effect then I'm still not sure what
this adds. As you know people register new domain names just to avoid
being on any list so your idea would seem to be a white list for those
who exploit that.
As to penalizing those who don't participate, I already have enough
headaches with SPF and others who want to inflict their personal
standards on the whole of the email community. SPF. which has left me
with a bad attitude, does nothing for me to catch spam or pass ham. But
it does result in good email that I forward being blocked.
I'm always looking for innovative ideas that actually work. I don't want
to discourage you. But the "actually works" part is very important. I
have had a lot of ideas that I got really excited about and after I
implemented them I found out that the idea wasn't quite as good as I
thought it would be.
As to whitelisting - there's actually a far easier solution that I use.
I do RDNS to get the host name. Then I forward confirm it to verify that
it is valid. Spammers can't spoof that. Then I look it up in my host
name white list of hosts who send nothing but good email. This actually
works extremely well. But like everyone I'm always looking for more
ideas especially for white rules because in y bustness one good email
bounced is words that 100 spams not bounced.
Re: MTX public blacklist implemented Re: MTX plugin functionally
complete?
Posted by Marc Perkel <ma...@perkel.com>.
Darxus@ChaosReigns.com wrote:
>
> In my initial posts I did focus too much on my hope that MTX will
> eventually be sufficiently widely adopted that such mail *can* safely be
> penalized. I also apologized for that communication failure on my part.
>
>
I'm still waiting for RDNS to be widely adopted enough to penalize for
that. There is a lot of good email that comes from misconfigured
servers. If we can't get the world to do good RDNS I don't think we can
get the world to do some other more complex scheme.
Re: MTX public blacklist implemented Re: MTX plugin functionally
complete?
Posted by Da...@ChaosReigns.com.
On 02/16, Charles Gregory wrote:
> You got it. Exactly. And that's why I gave up on MTX. Because the author
> was insisting that exactly that should happen.
I have never recommended that the majority of people penalize email just
because it doesn't get an MTX Pass, before wide spread adoption.
In my initial posts I did focus too much on my hope that MTX will
eventually be sufficiently widely adopted that such mail *can* safely be
penalized. I also apologized for that communication failure on my part.
Perhaps if you read the current version of the web page, you will be more
comfortable with my intentions. It has changed quite a lot since my first
post:
http://www.chaosreigns.com/mtx/
--
"Blessed are the cracked, for they shall let in the light."
http://www.ChaosReigns.com
Re: MTX public blacklist implemented Re: MTX plugin functionally
complete?
Posted by Matus UHLAR - fantomas <uh...@fantomas.sk>.
> On 02/14, Jonas Eckerman wrote:
> > 1: The participation record is optional, so you only use it if you want
> > "everything else" to be rejected.
On 14.02.10 14:48, Darxus@ChaosReigns.com wrote:
> Yeah. I'm thinking of using the 4th octet to indicate participation, and
> the third octet to indicate delegation.
If you want to check participation, you should do it on different level,
e.g. check chaosreigns.com before mail.chaosreigns.com. It of course
requires more DNS lookups, but note that people who do not participate, will
not set ANY record so checking 127.* won't help you.
> Check for the MTX record first, and if it is 127.0.0.1 or 127.0.0.0 you can
> skip this.
>
> 4th octet:
> 0 Not participating.
> 1 (or record not defined) Participating, everything not defined is valid (like SPF neutral).
> 2 Participating, other stuff might be valid (like SPF softfail).
> 3 Participating, everything else is invalid (SPF fail).
>
> 3rd octet:
> 1 All MTX records are at this level.
> 2 All MTX records are at a subdomain.
> 3 Check MTX records at this level and then the subdomain.
>
>
> If the value of the 4th octet changes when going to a subdomain, you
> could say to only check the 4th octet for participating or not if the
> 3rd octet is 2 (all delegated to subdomain). Or you could use the most
> restrictive of the two records.
--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
"Where do you want to go to die?" [Microsoft]
Re: MTX public blacklist implemented Re: MTX plugin functionally
complete?
Posted by Da...@ChaosReigns.com.
On 02/14, Jonas Eckerman wrote:
> 1: The participation record is optional, so you only use it if you want
> "everything else" to be rejected.
Yeah. I'm thinking of using the 4th octet to indicate participation, and
the third octet to indicate delegation.
Check for the MTX record first, and if it is 127.0.0.1 or 127.0.0.0 you can
skip this.
4th octet:
0 Not participating.
1 (or record not defined) Participating, everything not defined is valid (like SPF neutral).
2 Participating, other stuff might be valid (like SPF softfail).
3 Participating, everything else is invalid (SPF fail).
3rd octet:
1 All MTX records are at this level.
2 All MTX records are at a subdomain.
3 Check MTX records at this level and then the subdomain.
If the value of the 4th octet changes when going to a subdomain, you
could say to only check the 4th octet for participating or not if the
3rd octet is 2 (all delegated to subdomain). Or you could use the most
restrictive of the two records.
Still not feeling like I swallowed a cat.
I think it could cause slower adoption to ask people to "Create this one
record for all of your servers, and decide if you want to create this
complicated hierarchical thing defining your participation."
Perhaps spec out a version 2 including this to be implemented at a later
time?
> 2: Make it a policy record rather than a participation record, so you
> can specify more stuff. Either a TXT record or a bitmaped A record for
> example. Call it "_policy._mtx.*".
That sounds likely to get complicated. Details?
--
"Whatever you do will be insignificant, but it is very important that
you do it." - Mahatma Gandhi
http://www.ChaosReigns.com
Re: MTX public blacklist implemented Re: MTX plugin functionally
complete?
Posted by Charles Gregory <cg...@hwcn.org>.
On Tue, 16 Feb 2010, Jonas Eckerman wrote:
>> > 1: The participation record is optional, so you only use it if you
>> > want "everything else" to be rejected.
>> This is why I would support mtamark... It permits the sysadmin to
>> determine the default behaviour for his IP range, rather than defining a
>> dangerous default in the client.
> In what way does the above define a dangerous default?
It doesn't. My comment refers to early messages where the author of
'mtx' said that the 'standard' behaviour in the absence of any mtx
record as being equivalent to a 'deny' condition. That is, the domain
would be scored as 'spammish' if it did not participate.
> The default in the statement above is to consider a domain as *not*
> participating unless otherwise stated by whoever manages the DNS for the
> domain.
Correct. And my comment was that this was a much better alternative to
the 'dangerous default' of having 'not participating' mean 'spammy'.
> If the domain does not participate it should not be punished when a MTX
> record isn't found.
You got it. Exactly. And that's why I gave up on MTX. Because the author
was insisting that exactly that should happen.
- C
Re: MTX public blacklist implemented Re: MTX plugin functionally
complete?
Posted by Jonas Eckerman <jo...@frukt.org>.
On 2010-02-15 15:04, Charles Gregory wrote:
> On Sun, 14 Feb 2010, Jonas Eckerman wrote:
>> 1: The participation record is optional, so you only use it if you
>> want "everything else" to be rejected.
> This is why I would support mtamark... It permits the sysadmin to
> determine the default behaviour for his IP range, rather than defining a
> dangerous default in the client.
In what way does the above define a dangerous default?
The default in the statement above is to consider a domain as *not*
participating unless otherwise stated by whoever manages the DNS for the
domain.
If the domain does not participate it should not be punished when a MTX
record isn't found.
Regards
/Jonas
--
Jonas Eckerman
Fruktträdet & Förbundet Sveriges Dövblinda
http://www.fsdb.org/
http://www.frukt.org/
http://whatever.frukt.org/
Re: MTX public blacklist implemented Re: MTX plugin functionally
complete?
Posted by Charles Gregory <cg...@hwcn.org>.
On Sun, 14 Feb 2010, Jonas Eckerman wrote:
> 1: The participation record is optional, so you only use it if you want
> "everything else" to be rejected.
This is why I would support mtamark... It permits the sysadmin to
determine the default behaviour for his IP range, rather than defining a
dangerous default in the client.
And I quote:
This subdomain MAY be inserted at any level in the DNS tree for IPv4
IN-ADDR.ARPA reverse zones. For IPv6, to limit the number of DNS
queries, _srv is only queried at the /128 (host), /64 (subnet) and /
32 (site) level. That way it can either provide information for a
specific IP address or for a whole network block. More specific
information takes precedence over information found closer to the top
of the tree.
The beauty of this mechanism is that we can 'sell' large ISP's on it by
saying "you only need to create one 'allow' entry for each legitimate MTA
and one 'deny' entry for each netblock.
And for SA there is no need to give it 'starting' scores, like SPF, the
mechanism is effective as soon as it is used, and ignorable if not...
- C
Re: MTX public blacklist implemented Re: MTX plugin functionally
complete?
Posted by Jonas Eckerman <jo...@frukt.org>.
On 2010-02-14 20:06, Darxus@ChaosReigns.com wrote:
> I remembered why (else) I didn't want to do that. It effectively says
> "Everything else should be rejected." Which will discourage some people
> from using it. So you would at least need to provide a way to say "Yes,
> I'm participating, but anything without an MTX record is valid too."
The first two solutions for this that pops into my head:
1: The participation record is optional, so you only use it if you want
"everything else" to be rejected.
2: Make it a policy record rather than a participation record, so you
can specify more stuff. Either a TXT record or a bitmaped A record for
example. Call it "_policy._mtx.*".
More on the other very valid concerns later about this...
Regards
/Jonas
--
Jonas Eckerman
Fruktträdet & Förbundet Sveriges Dövblinda
http://www.fsdb.org/
http://www.frukt.org/
http://whatever.frukt.org/