You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@spamassassin.apache.org by Kris Deugau <kd...@vianet.ca> on 2012/11/29 22:26:41 UTC
FROM_MISSP_* causing FPs
I've just had another couple of reports of false positives due to hits
on one or more of the FROM_MISSP_* rules.
Curious coincidence: Almost all of the reports to date have involved
webform email for real estate companies. Most of the rest have involved
scan-to-email multifunction devices - mostly Xerox.... used by real
estate companies. O_o
It's become enough of a problem that I've sighed, rolled my eyes, and
disabled the scored rules in this cluster, since they're also not making
much of a difference in spam catch rate locally. (They're hitting as
much as ~3% of mail scanned daily, but only making the difference
between ham and spam on ~10-15 of ~80-90K messages daily - including an
unknown number of FPs that don't get reported.)
While I agree that this is triggering on a real RFC-"SHOULD" structural
problem with the emails, it's clear that people writing webform handlers
and people writing email code for multifunction printer/scanner devices
don't see this as a problem. I've tried contacting several websites
offering services to real estate agents, for instance, and I have had no
response.
-kgd
Re: FROM_MISSP_* causing FPs
Posted by Alexandre Boyer <bi...@gmail.com>.
Hi Kevin,
You are right, and by a lot I know what you mean, I see them too :-)
But rare are the one that fake the X-Mailer header. I can't remind
seeing one in fact.
Note: I corrected my __AJB_HAS_XEROX this very morning to:
header __AJB_MAILER_XEROX X-Mailer =~ /^WorkCentre .{3,6}/
I noticed false positives because of the declared Mailer version
(instead of "WorkCentre 1234", it's now "WorkCentre /4.03"). I realy
like version numbers that are consistent in time. This proves how
developers thought about things in the very first place.
Also, some Xerox machine do add some interesting headers:
X-Xerox-Source-IP: 192.168.2.130
X-Xerox-Source-Name: redacted@example.com
X-Xerox-DeviceType: Phaser 3635MFP
X-Xerox-DeviceName: XRX0000AADEF46B
X-Xerox-Mail-Id: 1100856957-758036596-000571194682402-758036596-535680529
I'm building rules with those, as I never saw such faked headers in
spams spoofing the Subject: Scan from a Xerox, but in the case of
forwarded scans, I keep my meta with Thread related rules.
Regards,
Alex, from prypiat.
Yes, I recycle.
On 12-11-30 09:54 AM, Kevin A. McGrail wrote:
> On 11/30/2012 8:15 AM, Alexandre Boyer wrote:
>> As a Mailer agent, I also spotted the Xerox Workcenter to have a
>> dirty bahavior.
>>
>> As I had the very same problem as Kris, I personnaly did not disabled
>> those rules but builded some metas based on X-Mailer and Subject tests:
>>
>> header __AJB_HAS_XEROX X-Mailer =~ /WorkCentre \d{3,5}/
>> header __AJB_XEROX_SUBJ Subject =~ /Scan from a Xerox/
>>
>> I meta those sub-tests with FROM_MISSP_* and I compensate for the
>> scores. As I use some KHOP rules, I also meta this with KHOP_THREADED
>> as well as with some Thread related rules to avoid blocking forwarded
>> scans.
>>
>> I did not made a deep research, I could probably customize
>> __AJB_HAS_XEROX to match specific versions of this "broken" agent,
>> but this work good like that. As they say: "first make it work, then
>> make it better." But when it works, I ususally have something else to
>> do than make it better.
>>
>> Works pretty well indeed.
> Adding to the mix, I see a LOT of phishing attempts with Scan from XYZ...
>
> Regards,
> KAM
Scan-to-email headers? (was: Re: FROM_MISSP_* causing FPs)
Posted by John Hardin <jh...@impsec.org>.
On Fri, 30 Nov 2012, Kevin A. McGrail wrote:
> Adding to the mix, I see a LOT of phishing attempts with Scan from XYZ...
There's also malware distributed that way.
Can anybody provide me (offlist!) the headers from a _legitimate_
scan-to-email from an HP and/or Xerox scanner (both are wanted)? I'd like
to see what real ones look like, to help reduce FPs.
Thanks!
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Justice is justice, whereas "social justice" is code for one set
of rules for the rich, another for the poor; one set for whites,
another set for minorities; one set for straight men, another for
women and gays. In short, it's the opposite of actual justice.
-- Burt Prelutsky
-----------------------------------------------------------------------
15 days until Bill of Rights day
Re: FROM_MISSP_* causing FPs
Posted by "Kevin A. McGrail" <KM...@PCCC.com>.
On 11/30/2012 8:15 AM, Alexandre Boyer wrote:
> As a Mailer agent, I also spotted the Xerox Workcenter to have a dirty
> bahavior.
>
> As I had the very same problem as Kris, I personnaly did not disabled
> those rules but builded some metas based on X-Mailer and Subject tests:
>
> header __AJB_HAS_XEROX X-Mailer =~ /WorkCentre \d{3,5}/
> header __AJB_XEROX_SUBJ Subject =~ /Scan from a Xerox/
>
> I meta those sub-tests with FROM_MISSP_* and I compensate for the
> scores. As I use some KHOP rules, I also meta this with KHOP_THREADED
> as well as with some Thread related rules to avoid blocking forwarded
> scans.
>
> I did not made a deep research, I could probably customize
> __AJB_HAS_XEROX to match specific versions of this "broken" agent, but
> this work good like that. As they say: "first make it work, then make
> it better." But when it works, I ususally have something else to do
> than make it better.
>
> Works pretty well indeed.
Adding to the mix, I see a LOT of phishing attempts with Scan from XYZ...
Regards,
KAM
Re: FROM_MISSP_* causing FPs
Posted by Alexandre Boyer <bi...@gmail.com>.
Alex, from prypiat.
Yes, I recycle.
On 12-12-03 02:04 AM, John Wilcock wrote:
> Le 30/11/2012 18:18, John Hardin a écrit :
>>> header __AJB_HAS_XEROX X-Mailer =~ /WorkCentre \d{3,5}/
>>> header __AJB_XEROX_SUBJ Subject =~ /Scan from a Xerox/
>>
>> Thanks! I will add those to my sandbox.
>>
>> Question: how often do you see that subject _without_ that X-Mailer?
>
> Whenever someone legitimately forwards a scanned document (which is
> quite a common occurrence in offices that have such scanner/copiers).
> Also worth noting that the default subject depends on the copier's
> locale, and can be changed anyway.
Right. But thing is: spammers won't try this, they tend to mimic the
default title to lure unaware/imprudent end-users. Therefore the
relative utility of a meta including the default title ;-)
To answer John Hardin's question: I will have to query my logs, I don't
have much time, but I will answer your question someday :-D
>
> PS: do you want genuine scans from other types of networked copier? I
> can forward a Rex Rotary example offlist if that would be useful.
>
> John.
>
Re: FROM_MISSP_* causing FPs
Posted by John Hardin <jh...@impsec.org>.
On Mon, 3 Dec 2012, John Wilcock wrote:
> Le 30/11/2012 18:18, John Hardin a écrit :
>> > header __AJB_HAS_XEROX X-Mailer =~ /WorkCentre \d{3,5}/
>> > header __AJB_XEROX_SUBJ Subject =~ /Scan from a Xerox/
>>
>> Thanks! I will add those to my sandbox.
>>
>> Question: how often do you see that subject _without_ that X-Mailer?
>
> Whenever someone legitimately forwards a scanned document (which is quite a
> common occurrence in offices that have such scanner/copiers). Also worth
> noting that the default subject depends on the copier's locale, and can be
> changed anyway.
>
> PS: do you want genuine scans from other types of networked copier? I can
> forward a Rex Rotary example offlist if that would be useful.
Sure.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
It is not the place of government to make right every tragedy and
woe that befalls every resident of the nation.
-----------------------------------------------------------------------
12 days until Bill of Rights day
Re: FROM_MISSP_* causing FPs
Posted by John Wilcock <jo...@tradoc.fr>.
Le 30/11/2012 18:18, John Hardin a écrit :
>> header __AJB_HAS_XEROX X-Mailer =~ /WorkCentre \d{3,5}/
>> header __AJB_XEROX_SUBJ Subject =~ /Scan from a Xerox/
>
> Thanks! I will add those to my sandbox.
>
> Question: how often do you see that subject _without_ that X-Mailer?
Whenever someone legitimately forwards a scanned document (which is
quite a common occurrence in offices that have such scanner/copiers).
Also worth noting that the default subject depends on the copier's
locale, and can be changed anyway.
PS: do you want genuine scans from other types of networked copier? I
can forward a Rex Rotary example offlist if that would be useful.
John.
--
-- Over 5000 webcams from ski resorts around the world - www.snoweye.com
-- Translate your technical documents and web pages - www.tradoc.fr
Re: FROM_MISSP_* causing FPs
Posted by John Hardin <jh...@impsec.org>.
On Fri, 30 Nov 2012, Alexandre Boyer wrote:
> As a Mailer agent, I also spotted the Xerox Workcenter to have a dirty
> bahavior.
>
> As I had the very same problem as Kris, I personnaly did not disabled
> those rules but builded some metas based on X-Mailer and Subject tests:
>
> header __AJB_HAS_XEROX X-Mailer =~ /WorkCentre \d{3,5}/
> header __AJB_XEROX_SUBJ Subject =~ /Scan from a Xerox/
Thanks! I will add those to my sandbox.
Question: how often do you see that subject _without_ that X-Mailer?
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
Justice is justice, whereas "social justice" is code for one set
of rules for the rich, another for the poor; one set for whites,
another set for minorities; one set for straight men, another for
women and gays. In short, it's the opposite of actual justice.
-- Burt Prelutsky
-----------------------------------------------------------------------
15 days until Bill of Rights day
Re: FROM_MISSP_* causing FPs
Posted by Alexandre Boyer <bi...@gmail.com>.
As a Mailer agent, I also spotted the Xerox Workcenter to have a dirty
bahavior.
As I had the very same problem as Kris, I personnaly did not disabled
those rules but builded some metas based on X-Mailer and Subject tests:
header __AJB_HAS_XEROX X-Mailer =~ /WorkCentre \d{3,5}/
header __AJB_XEROX_SUBJ Subject =~ /Scan from a Xerox/
I meta those sub-tests with FROM_MISSP_* and I compensate for the
scores. As I use some KHOP rules, I also meta this with KHOP_THREADED as
well as with some Thread related rules to avoid blocking forwarded scans.
I did not made a deep research, I could probably customize
__AJB_HAS_XEROX to match specific versions of this "broken" agent, but
this work good like that. As they say: "first make it work, then make it
better." But when it works, I ususally have something else to do than
make it better.
Works pretty well indeed.
Alex, from prypiat.
Yes, I recycle.
On 12-11-29 08:35 PM, Michael Orlitzky wrote:
> On 11/29/2012 05:43 PM, John Hardin wrote:
>> On Thu, 29 Nov 2012, Kris Deugau wrote:
>>
>>> I've just had another couple of reports of false positives due to hits
>>> on one or more of the FROM_MISSP_* rules.
>>>
>>> Curious coincidence: Almost all of the reports to date have involved
>>> webform email for real estate companies. Most of the rest have involved
>>> scan-to-email multifunction devices - mostly Xerox.... used by real
>>> estate companies. O_o
>> Is there any possibility of getting user agent headers for these FPs? If a
>> particular piece of legit software always does this then obviously those
>> rules should ignore such messages.
>>
> I had one guy actually read the rejection message and contact
> postmaster@ about this.
>
> His sig shows:
>
> Sent from my MOTOROLA ATRIX™ 2 on AT&T
>
> And the headers:
>
> X-Spam-Flag: NO
> X-Spam-Score: 4.224
> X-Spam-Level: ****
> X-Spam-Status: No, score=4.224 required=5 tests=[FREEMAIL_FROM=0.001,
> FROM_MISSP_EH_MATCH=2.499, FROM_MISSP_FREEMAIL=1.723,
> HTML_MESSAGE=0.001] autolearn=disabled
> From: "user@example.com"<us...@example.com>
> X-Mailer: Motorola android mail 1.0
>
> It was relayed through AOL, who you think would clean that up. This
> particular model also base64 encodes the entire message...
Re: FROM_MISSP_* causing FPs
Posted by John Hardin <jh...@impsec.org>.
On Thu, 29 Nov 2012, Michael Orlitzky wrote:
> On 11/29/2012 05:43 PM, John Hardin wrote:
>> On Thu, 29 Nov 2012, Kris Deugau wrote:
>>
>>> I've just had another couple of reports of false positives due to hits
>>> on one or more of the FROM_MISSP_* rules.
>>>
>>> Curious coincidence: Almost all of the reports to date have involved
>>> webform email for real estate companies. Most of the rest have involved
>>> scan-to-email multifunction devices - mostly Xerox.... used by real
>>> estate companies. O_o
>>
>> Is there any possibility of getting user agent headers for these FPs? If a
>> particular piece of legit software always does this then obviously those
>> rules should ignore such messages.
>
> I had one guy actually read the rejection message and contact
> postmaster@ about this.
>
> His sig shows:
>
> Sent from my MOTOROLA ATRIX™ 2 on AT&T
>
> And the headers:
>
> X-Spam-Flag: NO
> X-Spam-Score: 4.224
> X-Spam-Level: ****
> X-Spam-Status: No, score=4.224 required=5 tests=[FREEMAIL_FROM=0.001,
> FROM_MISSP_EH_MATCH=2.499, FROM_MISSP_FREEMAIL=1.723,
> HTML_MESSAGE=0.001] autolearn=disabled
> From: "user@example.com"<us...@example.com>
> X-Mailer: Motorola android mail 1.0
>
> It was relayed through AOL, who you think would clean that up. This
> particular model also base64 encodes the entire message...
Thanks, I will add some MUA rules for this and see what the corpus has to
say, if anything.
Kris, any from you?
Anybody who sees FPs with the FROM_MISSP rules is more than welcome to
send me X-Mailer and/or User-Agent headers directly.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
"Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never
does quite what I want. I wish Christopher Robin was here."
-- Peter da Silva in a.s.r
-----------------------------------------------------------------------
26 days until Christmas
Re: FROM_MISSP_* causing FPs
Posted by Michael Orlitzky <mi...@orlitzky.com>.
On 11/29/2012 05:43 PM, John Hardin wrote:
> On Thu, 29 Nov 2012, Kris Deugau wrote:
>
>> I've just had another couple of reports of false positives due to hits
>> on one or more of the FROM_MISSP_* rules.
>>
>> Curious coincidence: Almost all of the reports to date have involved
>> webform email for real estate companies. Most of the rest have involved
>> scan-to-email multifunction devices - mostly Xerox.... used by real
>> estate companies. O_o
>
> Is there any possibility of getting user agent headers for these FPs? If a
> particular piece of legit software always does this then obviously those
> rules should ignore such messages.
>
I had one guy actually read the rejection message and contact
postmaster@ about this.
His sig shows:
Sent from my MOTOROLA ATRIX™ 2 on AT&T
And the headers:
X-Spam-Flag: NO
X-Spam-Score: 4.224
X-Spam-Level: ****
X-Spam-Status: No, score=4.224 required=5 tests=[FREEMAIL_FROM=0.001,
FROM_MISSP_EH_MATCH=2.499, FROM_MISSP_FREEMAIL=1.723,
HTML_MESSAGE=0.001] autolearn=disabled
From: "user@example.com"<us...@example.com>
X-Mailer: Motorola android mail 1.0
It was relayed through AOL, who you think would clean that up. This
particular model also base64 encodes the entire message...
Re: FROM_MISSP_* causing FPs
Posted by Kris Deugau <kd...@vianet.ca>.
Alexandre Boyer wrote:
> Take care with Xerox versions, it just changed.
Yeah, your meta is probably better for general use.
I disabled most of the cluster outright because while they're hitting a
decent percentage of mail, they're not making the difference between ham
and spam very often... and the FP reports indicate they're making that
difference mostly on the wrong messages. :/
> I do not trust PHP Mailers, as PHP is wrong by design.
Heh. Yeah, I'm not sure if those are examples of bad coding in the
internals of PHP, or bad coding on the part of whoever wrote the webform
handler.
-kgd
Re: FROM_MISSP_* causing FPs
Posted by Alexandre Boyer <bi...@gmail.com>.
Take care with Xerox versions, it just changed.
I mentioned this in my reply to Kris.
I do not trust PHP Mailers, as PHP is wrong by design.
Alex, from prypiat.
Yes, I recycle.
On 12-11-30 10:17 AM, Kris Deugau wrote:
> John Hardin wrote:
>> On Thu, 29 Nov 2012, Kris Deugau wrote:
>>
>>> I've just had another couple of reports of false positives due to hits
>>> on one or more of the FROM_MISSP_* rules.
>>>
>>> Curious coincidence: Almost all of the reports to date have involved
>>> webform email for real estate companies. Most of the rest have involved
>>> scan-to-email multifunction devices - mostly Xerox.... used by real
>>> estate companies. O_o
>> Is there any possibility of getting user agent headers for these FPs? If
>> a particular piece of legit software always does this then obviously
>> those rules should ignore such messages.
> The most recent scan-to-email had:
>
> X-Mailer: WorkCentre 7428
>
> and another couple of older ones showed:
>
> X-Mailer: WorkCentre 7435
>
> Walking Xerox's list of scan-to-email-capable devices will probably turn
> up another couple of possible model numbers.
>
> Digging back in the FP archive, there are a handful of webform messages
> with "X-Mailer: PHP4", and none of the rest have even that much.
>
> None of the ones I've had reported are from desktop or mobile MUAs.
>
> Two are from airlines - one an ezine from Air Canada, the other a
> receipt/itinerary from Air Creebec.
>
> -kgd
Re: FROM_MISSP_* causing FPs
Posted by Kris Deugau <kd...@vianet.ca>.
John Hardin wrote:
> On Thu, 29 Nov 2012, Kris Deugau wrote:
>
>> I've just had another couple of reports of false positives due to hits
>> on one or more of the FROM_MISSP_* rules.
>>
>> Curious coincidence: Almost all of the reports to date have involved
>> webform email for real estate companies. Most of the rest have involved
>> scan-to-email multifunction devices - mostly Xerox.... used by real
>> estate companies. O_o
>
> Is there any possibility of getting user agent headers for these FPs? If
> a particular piece of legit software always does this then obviously
> those rules should ignore such messages.
The most recent scan-to-email had:
X-Mailer: WorkCentre 7428
and another couple of older ones showed:
X-Mailer: WorkCentre 7435
Walking Xerox's list of scan-to-email-capable devices will probably turn
up another couple of possible model numbers.
Digging back in the FP archive, there are a handful of webform messages
with "X-Mailer: PHP4", and none of the rest have even that much.
None of the ones I've had reported are from desktop or mobile MUAs.
Two are from airlines - one an ezine from Air Canada, the other a
receipt/itinerary from Air Creebec.
-kgd
Re: FROM_MISSP_* causing FPs
Posted by John Hardin <jh...@impsec.org>.
On Thu, 29 Nov 2012, Kris Deugau wrote:
> I've just had another couple of reports of false positives due to hits
> on one or more of the FROM_MISSP_* rules.
>
> Curious coincidence: Almost all of the reports to date have involved
> webform email for real estate companies. Most of the rest have involved
> scan-to-email multifunction devices - mostly Xerox.... used by real
> estate companies. O_o
Is there any possibility of getting user agent headers for these FPs? If a
particular piece of legit software always does this then obviously those
rules should ignore such messages.
--
John Hardin KA7OHZ http://www.impsec.org/~jhardin/
jhardin@impsec.org FALaholic #11174 pgpk -a jhardin@impsec.org
key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
"Bother," said Pooh as he struggled with /etc/sendmail.cf, "it never
does quite what I want. I wish Christopher Robin was here."
-- Peter da Silva in a.s.r
-----------------------------------------------------------------------
26 days until Christmas