Re: URI DeObfuscation / URIRewrite plugin

[Karsten Bräckelmann <gu...@rudersport.de>]

On Fri, 2009-07-31 at 03:22 +0200, Karsten Bräckelmann wrote:
> Devs and listeners,
> 
> GUDO, the Generic URI (no word-play here) DeObfuscation plugin and the
> URIRewrite plugin are finally available for public dev consumption!
>   http://guenther.dyndns.org/tmp/obfu/

Good, I see I now got backups of the code in quite a few countries on
various continents. :)

> Please have a look.  All and any feedback welcome.

Seriously, feedback?  Thoughts, comments?

What I'm most interested in at this point is feedback on code (and
comments), and of course the POD, settings in general, features or
understandability. Opinions, thoughts, human review. Not necessarily
statistics just yet.

Anything to share?

  guenther

-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: URI DeObfuscation / URIRewrite plugin

[Karsten Bräckelmann <gu...@rudersport.de>]

On Mon, 2009-08-03 at 11:25 +0100, Steve Freegard wrote:
> Karsten Bräckelmann wrote:

> I've had it running on a production box for the last couple of days and
> have been capturing any messages that hit the HAS_ANY_OBFU_URI rule.
> 
> This box *was* getting loads of obfu'd URI messages (I capture messages
> from certain ISP dynamic ranges that have massive bot infestations in my
> MTA and feed them to Bayes automatically); but when I checked the traps
> over the weekend - I didn't see a single message containing obfu's URIs.
>  So it appears that that this particular campaign has gone quiet for the
> time being.

I noticed this myself -- on a few, entirely unrelated accounts. Have
seen the last one of these obfuscated URIs on Sat. Since Sun, I've been
getting the very same URIs in the "bad, good" doodles style with per-
image manipulation.  Coincidence?


Anyway, this plugin is here to stay! I was aware this will cease off
sooner or later again, and said so on the list. My main intention was
and is, to extract the de-obfuscated URIs, and have URI DNSBLs properly
hit them.

Some words on the history:  We've all seen obfuscated URIs before,
multiple times. Especially the (dot) was not new and to be expected
early during this last run.

Ever since (quite a long time ago) some German spammer sent out
obfuscated URIs advertising porn sites with catchy, under-age hints in
the name, I wanted to hack such a plugin.

Even if the end-result of the availability of GUDO eventually means the
end of URI obfuscation, I'm not unhappy and the plugin successfully
accomplished its mission. Even better, if the URI is out there in the
clear, less processing.

The main intention is to *close* that loophole that obfuscation is.


This set of plugins can handle other styles of de-obfuscation and
counter-measures, too.  Hardly documented, but that's intentional. For
now. ;)

  guenther


> However I did have a few obfu messages saved and running them through
> GUDO with the example settings from the perldoc captured them nicely.

-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: URI DeObfuscation / URIRewrite plugin

[Karsten Bräckelmann <gu...@rudersport.de>]

On Mon, 2009-08-03 at 11:25 +0100, Steve Freegard wrote:
> Karsten Bräckelmann wrote:

> > What I'm most interested in at this point is feedback on code (and
> > comments), and of course the POD, settings in general, features or
> > understandability. Opinions, thoughts, human review. Not necessarily
> > statistics just yet.

> As no-one else has commented yet - I thought I'd start the ball rolling.

Thanks for your comments, Steve, much appreciated. :)

(Covered the "coincidental" end of this particular spam run in my
previous post, other sub-thread.)


> However I did have a few obfu messages saved and running them through
> GUDO with the example settings from the perldoc captured them nicely.
> 
> The documentation is straightforward and relatively easy to understand
> given the subject matter being quite complex and the example settings
> appear to work well.

Good to hear, thanks. :)  I still got a feeling I should re-write some
parts of the POD. Though when I looked into it, nothing apparent stuck
out, and it was all clear -- to me. ;)


FWIW, the example settings are quite broad. More sophisticated rules can
easily be written. The defines should help a great deal in structuring.
However, given the tests to weed out non-obfuscated URIs or non-URIs,
even the broad rules should be safe.

Moreover, this plugin is not primarily intended to score on its own. But
to have URI DNSBLs do the scoring. Thus, erroneously picking up innocent
bystanders merely will result in an additional DNS lookup, but not
increase the score of the message.


> All in all - I'd say it works great.

:-)


-- 
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i<l;i++){ i%8? c<<=1:
(c=*++x); c&128 && (s+=h); if (!(h>>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}

Re: URI DeObfuscation / URIRewrite plugin

[Steve Freegard <st...@stevefreegard.com>]

Karsten Bräckelmann wrote:
> 
> Seriously, feedback?  Thoughts, comments?
> 
> What I'm most interested in at this point is feedback on code (and
> comments), and of course the POD, settings in general, features or
> understandability. Opinions, thoughts, human review. Not necessarily
> statistics just yet.
> 
> Anything to share?

As no-one else has commented yet - I thought I'd start the ball rolling.

I've had it running on a production box for the last couple of days and
have been capturing any messages that hit the HAS_ANY_OBFU_URI rule.

This box *was* getting loads of obfu'd URI messages (I capture messages
from certain ISP dynamic ranges that have massive bot infestations in my
MTA and feed them to Bayes automatically); but when I checked the traps
over the weekend - I didn't see a single message containing obfu's URIs.
 So it appears that that this particular campaign has gone quiet for the
time being.

However I did have a few obfu messages saved and running them through
GUDO with the example settings from the perldoc captured them nicely.

The documentation is straightforward and relatively easy to understand
given the subject matter being quite complex and the example settings
appear to work well.

All in all - I'd say it works great.

Kind regards,
Steve.