Need help in playing in rules sandbox

[Sidney Markowitz <si...@sidney.com>]

I haven't done much with rules testing and I seem to be confused.

I once created a sandbox and experimented with some rules. When I was done I deleted them. 
Recently I created some new rules, putting them into 
rules/trunk/sandbox/sidney/72_other.cf and committing it.

When I look at the Rule QA web page the only rules I find named T_SIDNEY* are the old ones 
which should no longer exist. I don't see them anywhere in my svn working directory tree.

The wiki page http://wiki.apache.org/spamassassin/RuleSandboxes says that the sandbox is 
under trunk/rulesrc/sandbox, while http://wiki.apache.org/spamassassin/RulesProjPromotion 
says it is rules/trunk/sandbox. However they seem to contain the same thing, I guess due 
to something that corresponds to a symlink in an svn repository?

In any case the rules I committed do not show up in Rule QA and the ones I no longer see 
anywhere in my svn tree do show up.

Help?

  -- sidney

Re: Need help in playing in rules sandbox

[Justin Mason <jm...@jmason.org>]

On Jan 7, 2008 8:43 AM, Sidney Markowitz <si...@sidney.com> wrote:
> I haven't done much with rules testing and I seem to be confused.
>
> I once created a sandbox and experimented with some rules. When I was done I deleted them.
> Recently I created some new rules, putting them into
> rules/trunk/sandbox/sidney/72_other.cf and committing it.
>
> When I look at the Rule QA web page the only rules I find named T_SIDNEY* are the old ones
> which should no longer exist. I don't see them anywhere in my svn working directory tree.

found it: this is not serious.

basically, when the rule-QA backend runs "hit-frequencies", it does so in a
checkout containing rule files which were last compiled back in July.
hit-frequencies has code to handle listing zero-hit rules, by getting the
entire list of all known rules in ../rules; if any of those rules is not hit in
the mass-check logs, it'll still be output in the "freqs" listing, but with 0
hits.  in this case, that includes your old rules.

not sure how to fix this, other than either:

- not listing zero-hitting rules.  but that's confusing if someone checks in a
  *new* zero-hitter and wants to see the freqs!

- running "svn up" in that hit-frequencies checkout.  will investigate this.

> The wiki page http://wiki.apache.org/spamassassin/RuleSandboxes says that the sandbox is
> under trunk/rulesrc/sandbox, while http://wiki.apache.org/spamassassin/RulesProjPromotion
> says it is rules/trunk/sandbox. However they seem to contain the same thing, I guess due
> to something that corresponds to a symlink in an svn repository?
>
> In any case the rules I committed do not show up in Rule QA and the ones I no longer see
> anywhere in my svn tree do show up.

ok, I think it was just a time delay thing.  when I search for /SIDNEY they
show up now:

http://ruleqa.spamassassin.org/?daterev=20080107-r609531-n&rule=%2FSIDNEY

To explain: here's what can happen.

At 0850 UTC the nightly tag happens, then at 0900 UTC the various mass-checks
all start.  As the day progresses, mass-check results arrive and are put into
ruleqa's output.  However, it's not until a certain point well into the day,
that we can reasonably guess that that nightly's results are all in.

Now, in your case, your new rules were added in r609296, 2008-01-06
11:53:49 +0000.
That's just *after* the nightly tag for the day -- the worst possible
timing!  ;)
so they didn't show up in a nightly until the next day, today:
20080107-r609531-n .  When you took a look earlier on today, that mass-check
wasn't considered "last-night" since it was too fresh, and there were not
going to be enough results in.  (It is now, though.)

This is kind of unavoidable, given that we only tag for mass-checks once
per day, and it takes several hours to run them :(

here's a tip: on the front page, click on 'today's nightly run' for the most
recent nightly -- typically this is only partially completed (depending on when
you view it) but is more likely to have your up-to-date rule changes.
alternatively, every checkin gets an "mc-fast" preflight mass-check, straight
away:

http://ruleqa.spamassassin.org/?daterev=20080106-r609296-b&rule=&srcpath=sidney&g=Change

but those will typically have slightly different (and less reliable) results.
I'd like to fix that by improving the mc-fast corpus sometime....

--j.

>
> Help?
>
>   -- sidney
>
>

Re: Need help in playing in rules sandbox

[Sidney Markowitz <si...@sidney.com>]

Sidney Markowitz wrote, On 7/1/08 9:43 PM:
> rules/trunk/sandbox/sidney/72_other.cf

correction, it was 70_other.cf