full rules

[Tony Finch <do...@dotat.at>]

sandbox/felicity/70_other.cf says

# Ok, I've said in the past that full rules are evil and that we should never
# use them.  While I do think that they're nasty and are almost always not the
# most efficient way to go about things, in this case, unfortunately,
# I think it actually is the most efficient way to do this search.  :(
# BTW: a raw (non-encoded) NUL char (ASCII 0) is forbidden by RFC2822, though
# it seems to show up in spams a lot these days.  MTAs -- deny these mails!

I'm thinking of trying matches against (parts of) binary attachments. The
only way I can think of doing this is using full rules to match fragments
of base64.

Also, I believe that a lot of legitimate messages (from Microsoft MUAs)
contain NULs, typically at the end of attachments. Therefore my servers
strip NULs rather than rejecting them. Insert usual rant about the quality
of standards conformance in legitimage email :-(

Tony.
-- 
f.a.n.finch  <do...@dotat.at>  http://dotat.at/
FORTIES CROMARTY FORTH TYNE DOGGER: SOUTHERLY 6 TO GALE 8. ROUGH OR VERY
ROUGH. MAINLY FAIR. GOOD.

Re: full rules

[Theo Van Dinter <fe...@apache.org>]

On Mon, Nov 27, 2006 at 04:12:20PM +0000, Justin Mason wrote:
> well, that could argue that we should come up with a MIMEFull rule
> type plugin, similar to the MIMEHeader plugin...

If people would find it useful, sure.  I think the main issue is that a lot of
people want to do more complex things.  Searching for a content-type and then
running a RE on the raw data is trivial.

-- 
Randomly Selected Tagline:
"When in doubt, parenthesize.  At the very least it will let some
 poor schmuck bounce on the % key in vi."   - Larry Wall

Re: full rules

[Tony Finch <do...@dotat.at>]

On Mon, 27 Nov 2006, Theo Van Dinter wrote:
> On Mon, Nov 27, 2006 at 03:04:54PM +0000, Tony Finch wrote:
> >
> > Also, I believe that a lot of legitimate messages (from Microsoft MUAs)
> > contain NULs, typically at the end of attachments. Therefore my servers
> > strip NULs rather than rejecting them. Insert usual rant about the quality
> > of standards conformance in legitimage email :-(
>
> At the end of attachments in terms of the raw or encoded version?  So far:
>
>   1.138   1.3665   0.0016    0.999   0.87    1.00  NULL_IN_BODY
>
> which works for me.  I don't think they're common in any meaningful way.

OK, maybe "a lot" was exaggerating - we see about 1 a day out of about
300,000 messages, but it's enough to cause problems :-(

Tony.
-- 
f.a.n.finch  <do...@dotat.at>  http://dotat.at/
LUNDY FASTNET: EASTERLY 5 TO 7. SLIGHT OR MODERATE BECOMING ROUGH OR VERY
ROUGH. RAIN AT TIMES, SLEET LATER. MODERATE OR GOOD.

Re: full rules

[Theo Van Dinter <fe...@apache.org>]

On Mon, Nov 27, 2006 at 03:04:54PM +0000, Tony Finch wrote:
> I'm thinking of trying matches against (parts of) binary attachments. The
> only way I can think of doing this is using full rules to match fragments
> of base64.

A plugin will do it much better.  For instance, you'll be able to limit your
search against binary attachments. :)

> Also, I believe that a lot of legitimate messages (from Microsoft MUAs)
> contain NULs, typically at the end of attachments. Therefore my servers
> strip NULs rather than rejecting them. Insert usual rant about the quality
> of standards conformance in legitimage email :-(

At the end of attachments in terms of the raw or encoded version?  So far:

  1.138   1.3665   0.0016    0.999   0.87    1.00  NULL_IN_BODY

which works for me.  I don't think they're common in any meaningful way.

-- 
Randomly Selected Tagline:
Q: Don't you know who our President is?
 A: A working class man who started out with nothing in life but two strong
    hands and a brain, and now has to make due with just the hands.
         - http://slashdot.org/comments.pl?sid=189485&cid=15602732