You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by "McDonald, Dan" <Da...@austinenergy.com> on 2008/01/26 14:09:46 UTC
p0f not catching Windows XP
I'm using amavisd-new and p0f with BOTNET.pl, and some Windows XP
machines are not being caught.
Here are my rules:
header L_P0F_WXP X-Amavis-OS-Fingerprint =~ /^Windows XP(?![^(]*\b2000 SP)/
score L_P0F_WXP 2.3
header L_P0F_W X-Amavis-OS-Fingerprint =~ /^Windows(?! XP)/
score L_P0F_W 1.0
header L_P0F_UNKN X-Amavis-OS-Fingerprint =~ /^UNKNOWN/
score L_P0F_UNKN 0.8
header L_P0F_Unix X-Amavis-OS-Fingerprint =~ /^((Free|Open|Net)BSD|Solaris|HP-UX|Tru64)/
score L_P0F_Unix -1.0
header L_P0F_Linux X-Amavis-OS-Fingerprint =~ /^Linux/
score L_P0F_Linux -0.1
I had a message with the following header:
X-Amavis-OS-Fingerprint: Windows XP SP1+, 2000 SP3 (NAT!), (distance 20,
link: unknown-1490), [83.11.64.39]
It doesn't appear to match either the L_P0F_WXP or the L_P0F_W rule:
[mcdonalddj@mcdonalddj-dc ~]$ grep -P '/^Windows(?! XP)/' Download/foo.eml
[mcdonalddj@mcdonalddj-dc ~]$ grep -P '/^Windows XP(?![^(]*\b2000 SP)/' Download/foo.eml
Does anyone have rules that catch this?
--
Daniel J McDonald, CCIE #2495, CISSP #78281, CNX
Austin Energy
http://www.austinenergy.com
Re: p0f not catching Windows XP
Posted by Mark Martinec <Ma...@ijs.si>.
Matt,
> Yeah, that much made sense to me. I just wasn't sure why there was no
> match at all for the combo one. The FP explanation makes sense. It was
> obvious it was intentional, but the why was not.
>
> However, I might suggest adding a comment to the .cf file, possibly even
> a commented-out rule, explaining it's been disabled due to FPs on
> Microsoft servers. That might avoid some FAQs.
>
> Something like this :
You are right, adding a 'describe' to rules is in order.
Here is the set of rules I'm currently using:
(leaving out my TTL-distance rules)
describe __L_P0F_EXISTS A header field X-Amavis-OS-Fingerprint does exist
header __L_P0F_EXISTS exists:X-Amavis-OS-Fingerprint
describe L_P0F_WXP Remote system is truly a Windows XP, not a Windows 2000
header L_P0F_WXP X-Amavis-OS-Fingerprint =~ /\AWindows XP(?![^(]*\b2000 SP)/m
score L_P0F_WXP 2.3
describe L_P0F_W Remote system is some Windows variant, except Windows XP
header L_P0F_W X-Amavis-OS-Fingerprint =~ /\AWindows(?! XP)/m
score L_P0F_W 1.3
describe L_P0F_UNKN P0f was unable to determine remote OS type
header L_P0F_UNKN X-Amavis-OS-Fingerprint =~ /\AUNKNOWN/m
score L_P0F_UNKN 0.8
describe L_P0F_Unix Remote system is running Unix, not Linux
header L_P0F_Unix X-Amavis-OS-Fingerprint =~ /\A((Free|Open|Net)BSD|Solaris|HP-UX|Tru64|AIX)/m
score L_P0F_Unix -1.0
describe L_P0F_Unix Remote system is running Linux
header L_P0F_Linux X-Amavis-OS-Fingerprint =~ /\ALinux/m
score L_P0F_Linux -0.1
# tame a Botnet plugin, reducing its false positives
score BOTNET 0.1
meta BOTNET_W !DKIM_VERIFIED && (L_P0F_WXP || L_P0F_W) && BOTNET
score BOTNET_W 2.8
meta BOTNET_WU !DKIM_VERIFIED && (L_P0F_UNKN || !__L_P0F_EXISTS) && BOTNET
score BOTNET_WU 2.0
meta BOTNET_OTHER !BOTNET_W && !BOTNET_WU && BOTNET
score BOTNET_OTHER 0.5
# some statistics:
# ham(3): spam(6)
#-----------------------------
#Windows-XP 0.7 % : 99.3 %
#Windows-2000 5.8 % : 94.2 %
#UNKNOWN 16.5 % : 83.5
#Linux 58.8 % : 41.2 %
#Unix 80.3 % : 19.7 %
#Unix+Linux 66.5 % : 33.5 %
Mark
Re: p0f not catching Windows XP
Posted by Matt Kettler <mk...@verizon.net>.
Mark Martinec wrote:
>>> header L_P0F_WXP X-Amavis-OS-Fingerprint =~ /^Windows XP(?![^(]*\b2000
>>> SP)/ score L_P0F_WXP 2.3
>>> header L_P0F_W X-Amavis-OS-Fingerprint =~ /^Windows(?! XP)/
>>> score L_P0F_W 1.0
>>>
> [...]
>
> Matt Kettler wrote:
>
>> Well, that much should be obvious.
>> Both rules are explicitly designed to NOT match that..
>>
>
>
>> The big question is, why?
>>
>
> That is intentional. Some corporate sites use a Microsoft's mailer,
> and it sometimes appears to p0f as 'Windows XP, 2000 SP' or similar.
> There would be too many false positives to treat it the same as
> versions of Windows XP that majority of users run at home, so
> the rule tries to be selective on Windows XP only, erring on the
> safe side.
>
>
>> The ?! is a forward look-ahead that prevents matching.
>> So, the first rule: /^Windows(?! XP)/
>> Will match anything that's windows that's not XP.
>>
>
> That second rule is excluding XP because plain Windows XP was
> already covered by the first rule, and it would not be
> appropriate to score it twice.
>
Yeah, that much made sense to me. I just wasn't sure why there was no
match at all for the combo one. The FP explanation makes sense. It was
obvious it was intentional, but the why was not.
However, I might suggest adding a comment to the .cf file, possibly even
a commented-out rule, explaining it's been disabled due to FPs on
Microsoft servers. That might avoid some FAQs.
Something like this :
# This rule sometimes matches windows server products, causing
# False positives against exchange and other windows-based mail servers.
# header L_P0F_WXP2KUNSURE X-Amavis-OS-Fingerprint =~ /^Windows
XP[^(]*\b2000 SP/
# score L_P0F_WXP2KUNSURE 0.01
# describe L_P0F_WXP2KUNSURE Could be Win XP or 2k possibly a server.
might be nice. That way folks who want it can use it, but it should be
clear to them it can cause FP problems.
Re: p0f not catching Windows XP
Posted by Mark Martinec <Ma...@ijs.si>.
> > header L_P0F_WXP X-Amavis-OS-Fingerprint =~ /^Windows XP(?![^(]*\b2000
> > SP)/ score L_P0F_WXP 2.3
> > header L_P0F_W X-Amavis-OS-Fingerprint =~ /^Windows(?! XP)/
> > score L_P0F_W 1.0
[...]
Matt Kettler wrote:
> Well, that much should be obvious.
> Both rules are explicitly designed to NOT match that..
> The big question is, why?
That is intentional. Some corporate sites use a Microsoft's mailer,
and it sometimes appears to p0f as 'Windows XP, 2000 SP' or similar.
There would be too many false positives to treat it the same as
versions of Windows XP that majority of users run at home, so
the rule tries to be selective on Windows XP only, erring on the
safe side.
> The ?! is a forward look-ahead that prevents matching.
> So, the first rule: /^Windows(?! XP)/
> Will match anything that's windows that's not XP.
That second rule is excluding XP because plain Windows XP was
already covered by the first rule, and it would not be
appropriate to score it twice.
Mark
Re: p0f not catching Windows XP
Posted by Matt Kettler <mk...@verizon.net>.
McDonald, Dan wrote:
> I'm using amavisd-new and p0f with BOTNET.pl, and some Windows XP
> machines are not being caught.
> Here are my rules:
> header L_P0F_WXP X-Amavis-OS-Fingerprint =~ /^Windows XP(?![^(]*\b2000 SP)/
> score L_P0F_WXP 2.3
> header L_P0F_W X-Amavis-OS-Fingerprint =~ /^Windows(?! XP)/
> score L_P0F_W 1.0
> header L_P0F_UNKN X-Amavis-OS-Fingerprint =~ /^UNKNOWN/
> score L_P0F_UNKN 0.8
> header L_P0F_Unix X-Amavis-OS-Fingerprint =~ /^((Free|Open|Net)BSD|Solaris|HP-UX|Tru64)/
> score L_P0F_Unix -1.0
> header L_P0F_Linux X-Amavis-OS-Fingerprint =~ /^Linux/
> score L_P0F_Linux -0.1
>
>
> I had a message with the following header:
> X-Amavis-OS-Fingerprint: Windows XP SP1+, 2000 SP3 (NAT!), (distance 20,
> link: unknown-1490), [83.11.64.39]
>
> It doesn't appear to match either the L_P0F_WXP or the L_P0F_W rule:
> [mcdonalddj@mcdonalddj-dc ~]$ grep -P '/^Windows(?! XP)/' Download/foo.eml
> [mcdonalddj@mcdonalddj-dc ~]$ grep -P '/^Windows XP(?![^(]*\b2000 SP)/' Download/foo.eml
>
Well, that much should be obvious. Both rules are explicitly designed to
NOT match that.. The big question is, why?
The ?! is a forward look-ahead that prevents matching. So, the first rule:
/^Windows(?! XP)/
Will match anything that's windows that's not XP.
The second rule:
/^Windows XP(?![^(]*\b2000 SP)/
Will match anything starting with "Windows XP", but it can't have "2000
SP" before a (.
You could easily write a rule for it:
header L_P0F_WXP2KUNSURE X-Amavis-OS-Fingerprint =~ /^Windows XP[^(]*\b2000 SP/
Which will match any of the headers with both XP and 2000 in it.
>
> Does anyone have rules that catch this?
>
>
>