You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Adam Katz <an...@khopis.com> on 2009/05/09 01:09:06 UTC

Re: [SA] Stacked word sentences.

John Hardin wrote:
> rawbody __TWO_WORD_LINES /^\S\+\s\+\S\+$/
> tflags  __TWO_WORD_LINES multiple
> meta    STACKED_TEXT (__TWO_WORD_LINES > 10)
> 
> Likely somewhat FP-prone...

I think quite FP-prone; think about emailed system logs, lists,
invoices, etc.  Your example used lots of real words, so I'd trust
Bayes to find it.  It also had a URL, so the URIBLs can detect it too.

Finally, IIRC, some of the fuzzy checksum mechanisms go by patterns
that take a keen interest in paragraph structure like that (or at
least one was mentioned as well-loved at the last MIT Spam
Conference), so make sure you're using Razor2, Pyzor, iXhash, and if
permissible, DCC (though I'm not sure which of those use this method
... iXhash certainly does not).

Re: [SA] Stacked word sentences.

Posted by John Hardin <jh...@impsec.org>.
On Fri, 8 May 2009, Adam Katz wrote:

> John Hardin wrote:
>> rawbody __TWO_WORD_LINES /^\S\+\s\+\S\+$/
>> tflags  __TWO_WORD_LINES multiple
>> meta    STACKED_TEXT (__TWO_WORD_LINES > 10)
>>
>> Likely somewhat FP-prone...
>
> I think quite FP-prone; think about emailed system logs, lists, 
> invoices, etc. Your example used lots of real words, so I'd trust Bayes 
> to find it.  It also had a URL, so the URIBLs can detect it too.

Yeah. I certainly wouldn't score such a rule very high, unless in concert 
with something like BAYES > 80 or MIME_HTML_ONLY.

-- 
  John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
  jhardin@impsec.org    FALaholic #11174     pgpk -a jhardin@impsec.org
  key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
   Liberals love sex ed because it teaches kids to be safe around their
   sex organs. Conservatives love gun education because it teaches kids
   to be safe around guns. However, both believe that the other's
   education goals lead to dangers too terrible to contemplate.
-----------------------------------------------------------------------
  Today: the 64th anniversary of VE day

Re: [SA] Stacked word sentences.

Posted by Karsten Bräckelmann <gu...@rudersport.de>.
On Fri, 2009-05-08 at 19:09 -0400, Adam Katz wrote:
> Finally, IIRC, some of the fuzzy checksum mechanisms go by patterns
> that take a keen interest in paragraph structure like that (or at
> least one was mentioned as well-loved at the last MIT Spam
> Conference), so make sure you're using Razor2, Pyzor, iXhash, and if
> permissible, DCC (though I'm not sure which of those use this method
> ... iXhash certainly does not).

Sure does.

Actually, IIRC, that's it's main algo. Strip everything but whitespace,
condense repeating whitespace into a single occurrence, and hash that.


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}