You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@spamassassin.apache.org by bu...@bugzilla.spamassassin.org on 2016/09/27 01:41:33 UTC
[Bug 7351] New: Unclear to to get removed from razor2 DB
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7351
Bug ID: 7351
Summary: Unclear to to get removed from razor2 DB
Product: Spamassassin
Version: unspecified
Hardware: PC
OS: Mac OS X
Status: NEW
Severity: normal
Priority: P2
Component: Documentation
Assignee: dev@spamassassin.apache.org
Reporter: tfreeman@identityfraud.com
Hi,
Emails that contain links to our domain are triggering the Razor2 DB. I can't
seem to find any information anywhere on how to get us removed from the Razor2
DB. The best bet I have been able to find is paying hundreds of dollars to
ReturnPath for domain certification. This seems like extortion, which I find
unseemly in an open source apache project.
We don't send bulk emails, we only send transactional emails. When our password
reset emails are being blocked, there is obviously something wrong with the way
that the Razor2 DB is working.
See:
http://www.mail-tester.com/web-BpdtMj
vs the nearly identical, but with our old domain:
http://www.mail-tester.com/web-VFbHyY
I understand that spam filtering is hard, but using a opaque, undocumented
blacklisting DB like Razor2 that is associated with a commercial entity seems
extremely unfair to small business with limited IT budgets that do their best
to follow email best practices.
Thanks,
Tyler
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7351] Unclear to to get removed from razor2 DB
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7351
--- Comment #6 from Bill Cole <sa...@billmail.scconsult.com> ---
(In reply to Tyler Freeman from comment #3)
> Thanks RW, anything that reduces the default weight given to Razor2's
> classification of our messages would help us.
>
> I still cannot find anywhere that discusses the removal process from Razor2.
That is because it is not a simple database of obvious tokens directly managed
by a single source. Cloudmark runs the service and ultimately owns the data,
but it is submitted as fuzzy hashes by Razor users around the world who judge
messages to be spam or not spam. In Razor jargon, they "report" messages that
are spam and "revoke" messages that are not spam, and the Razor DB keeps
tallies of what hashes have been reported or revoked and how trustworthy each
reporting user's judgment is. In principle Cloudmark could whitelist whatever
hashes are being generated by your mail and being reported as spam but in
practice they don't, even for paying customers (at least they didn't when I was
a paying customer some years back...) This is reasonable because anyone using
Razor (or Authority, its commercial sibling) can whitelist locally and revoke
non-spam actively. If enough users who have a pattern of accuracy revoke a lot
of similar mail, the Razor DB will start seeing that mail as non-spam.
> If anyone has a better understanding of how entries are added to and removed
> from Razor2 I'd appreciate any help you can offer or any information you can
> point me towards.
Well, http://razor.sourceforge.net/docs/faq.php has a fair bit of information,
as does the rest of the documentation on that site. Users of Razor ultimately
determine what Razor says about any particular message. If you want to be a
Razor user and revoke your own mail to influence its score, there is nothing
fundamentally wrong with that.
One other thing to consider: it is clear that the messages you put through the
mail-tester.com widget are not real, as they use an example.com address. In
addition, even the one which got tagged by Razor2 was scored below the standard
SpamAssassin threshold, so it would not be deemed spam by most systems using
Razor2 via SpamAssassin. This raises a question which may not be obvious to
you, but is important to answer:
Is any of your REAL mail being rejected or relegated to spam folders by any
system, as far as you can tell?
If the answer is "no", then I don't think you have a real problem, only a
hypothetical one. The fact that mail-tester.com can see Razor (and by
implication Authority) as a possible derogatory source regarding your mail does
not mean that Razor is causing you any actual delivery problems now or that it
will in the future. Intrinsically, mail-tester.com cannot test the sources of
all deliverability troubles, they can only test what's freely available.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7351] Unclear to to get removed from razor2 DB
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7351
Joe Quinn <jq...@pccc.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |jquinn+SAbug@pccc.com
Status|REOPENED |RESOLVED
Resolution|--- |INVALID
--- Comment #9 from Joe Quinn <jq...@pccc.com> ---
The original issue of this ticket is not something SA is responsible for, and
appears to have been resolved anyway. A new ticket is created per RW's comment
about razor scores at https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7390.
I am closing this ticket as invalid, unless there are any objections.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7351] Unclear to to get removed from razor2 DB
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7351
--- Comment #8 from Tyler Freeman <tf...@identityfraud.com> ---
And it appears to have worked, we are no longer being flagged by Razor2 for
links to bizlock.net : http://www.mail-tester.com/web-P5kvgH
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7351] Unclear to to get removed from razor2 DB
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7351
RW <rw...@googlemail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |rwmaillists@googlemail.com
--- Comment #2 from RW <rw...@googlemail.com> ---
I think there is a problem here. For a while this month the razor score was
above 4.5 points, and at that level a razor FP combines with BAYES_50 to
produce a misclassification at the default threshold. The current rules are
over-complicated and make it hard to put a sensible limit on the score.
Engine 4 fell-off a long time ago leaving only engine 8.
RAZOR2_CF_RANGE_E8_51_100 and RAZOR2_CF_RANGE_51_100 are now duplicates, so
RAZOR2_CF_RANGE_E4_51_100 and RAZOR2_CF_RANGE_E8_51_100 are now both
superfluous.
Engine 8 produces mostly high confidence results, in the mass checks
RAZOR2_CHECK only hits 0.3% more spam (for 44% more ham). Its precise score
wont be all that important - provided it's low.
What I would suggest is something like:
- Remove RAZOR2_CF_RANGE_E8_51_100 and RAZOR2_CF_RANGE_E4_51_100
- Fix the score for RAZOR2_CHECK at 0 1.7 0 0.8
- Limit the score for RAZOR2_CF_RANGE_51_100 to 2.7
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7351] Unclear to to get removed from razor2 DB
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7351
--- Comment #7 from Tyler Freeman <tf...@identityfraud.com> ---
Bill Cole,
Thanks for taking the time to write an informative response. I appreciate it
greatly
> it is submitted as fuzzy hashes by Razor users around the world who judge messages to be spam or not spam. In Razor jargon, they "report" messages that are spam and "revoke" messages that are not spam, and the Razor DB keeps tallies of what hashes have been reported or revoked and how trustworthy each reporting user's judgment is.
This was my guess of how this might be working since it was described as a
"distributed db". I
> In principle Cloudmark could whitelist whatever hashes are being generated by your mail and being reported as spam but in practice they don't, even for paying customers (at least they didn't when I was a paying customer some years back...)
This is good to know, I contacted Cloudmark through their online contact for
several days ago but have not heard back. I won't waste further time following
up there.
> One other thing to consider: it is clear that the messages you put through the mail-tester.com widget are not real, as they use an example.com address.
I crafted those messages specifically for describing the issues as I didn't
want to post any real user emails.
> Is any of your REAL mail being rejected or relegated to spam folders by any system, as far as you can tell?
Yes, we have had a bunch of emails rejected by a wide variety of ESPs with
response along the lines of the following:
"554 5.7.1 Spam detected by content scanner. Message rejected." or
"554 5.7.1 [P4] Message blocked due to spam content in the message."
I've been contacting each ESP were we've had an issue to request that they stop
blocking our emails (and I'll keep doing so as new ones come in.) It has been
extremely time consuming and many ESPs require confirmation from the recipient
of a blocked message that they wanted to receive it (luckily we have phone
numbers for our customers and provide a high value service, so we can reach out
directly.) I had been hoping that this would translate into eventual removal of
our hashes from Razor2 based on my guess of how it worked. From what you have
said, this may indeed be the case (as long as some of these ESPs are using
Razor2.)
It sounds like what I did is the best way to get a hash removed from the Razor2
DB, since you can take advantage of revocations from a wide range of different
TeS score holders.
The only other solution for a non ESP seems to be installing Cloudmark's
SpamNet Outlook plugin and using that to build up a good TeS score before
revoking the spam classification of your own messages.
The Razor2 SourceForge site appears to be inactive, any ideas on getting the
above information on removal posted so that others will have an easier time
figuring this out than I did? Additionally, it would be neat to have a public
list of ESPs and their various methods/emails/phonenumbers for requesting
unblocking.
Thanks for everyone's help,
Tyler
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7351] Unclear to to get removed from razor2 DB
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7351
Benny Pedersen <me...@junc.eu> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |me@junc.eu
--- Comment #4 from Benny Pedersen <me...@junc.eu> ---
there is nothing to remove, please whitelist local
check razor-whitelist
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7351] Unclear to to get removed from razor2 DB
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7351
Kevin A. McGrail <km...@pccc.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |kmcgrail@pccc.com
--- Comment #10 from Kevin A. McGrail <km...@pccc.com> ---
+1
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7351] Unclear to to get removed from razor2 DB
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7351
Tyler Freeman <tf...@identityfraud.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|INVALID |---
Status|RESOLVED |REOPENED
CC| |tfreeman@identityfraud.com
--- Comment #3 from Tyler Freeman <tf...@identityfraud.com> ---
Thanks RW, anything that reduces the default weight given to Razor2's
classification of our messages would help us.
I still cannot find anywhere that discusses the removal process from Razor2.
I've opened a ticket on the Razor2 bug tracker, but I suspect that bug tracker
is dead since the last closed ticket is from 2006.
https://sourceforge.net/p/razor/support-requests/43/
If anyone has a better understanding of how entries are added to and removed
from Razor2 I'd appreciate any help you can offer or any information you can
point me towards.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7351] Unclear to to get removed from razor2 DB
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7351
Bill Cole <sa...@billmail.scconsult.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |INVALID
CC| |sa-bugz-20080315@billmail.s
| |cconsult.com
Status|NEW |RESOLVED
--- Comment #1 from Bill Cole <sa...@billmail.scconsult.com> ---
The Apache SpamAssassin project does not have any control over the Razor2
service, which is operated by Cloudmark. SpamAssassin can use Razor2 by way of
a Plugin, however that Plugin is NOT loaded by default and requires the
installation of the actual Razor2 software which is not included with
SpamAssassin. The SpamAssassin Razor2 Plugin is documented both in the
SpamAssassin software distribution and at
http://spamassassin.apache.org/full/3.4.x/doc/Mail_SpamAssassin_Plugin_Razor2.html.
As that page states, the documentation of Razor itself is provided by the Razor
project at http://razor.sourceforge.net. The Razor project is not part of
SpamAssassin and is not an ASF project.
--
You are receiving this mail because:
You are the assignee for the bug.
[Bug 7351] Unclear to to get removed from razor2 DB
Posted by bu...@bugzilla.spamassassin.org.
https://bz.apache.org/SpamAssassin/show_bug.cgi?id=7351
John Hardin <jh...@impsec.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |jhardin@impsec.org
--- Comment #5 from John Hardin <jh...@impsec.org> ---
(In reply to Benny Pedersen from comment #4)
> there is nothing to remove, please whitelist local
>
> check razor-whitelist
Unfortunately that doesn't help the sender.
Tyler, you may have to get in touch with Cloudmark to resolve this.
--
You are receiving this mail because:
You are the assignee for the bug.