Problem with 72_scores.cf generation

[Dave Jones <da...@apache.org>]

Kevin,
I have spent about 5 hours this morning trying to track down the 
72_scores.cf generation problem.  I haven't pinpointed the problem yet 
but here's what I have found so far:

NOTE: su - automc for proper paths below.

1. ~/svn/masses/rule-update-score-gen/generate-new-scores.sh is the 
script in question

https://svn.apache.org/viewvc/spamassassin/trunk/masses/rule-update-score-gen/generate-new-scores.sh?revision=1798589&view=markup

Line 271 runs "runGA"

2. runGA creates 
~/tmp/generate-new-scores/trunk-new-rules-set0/masses/gen-set0-5-5.0-6000-ga/scores

This 'scores' file has 345 scores in it.  I wish we had a copy of this 
file from mid March to see if it also had around the same number of 
scores to confirm the runGA/garescorer is not the problem.  However, 
this file is a temp file that used to be in /tmp so it's probably not 
backed up anywhere and definitely not in SVN.

3. Back in the generate-new-scores.sh at line 289, the 
"extract-new-scores" script creates scores-new from the scores file but 
excludes/culls out anything manually scored in 50_scores.cf.

The culled scores-new file has the same 42 lines and ends at MILLION_USD 
just like our 72_scores.cf so this is the smoking gun but I haven't 
found the what pulled the trigger yet.  There is something different 
about this step than back on March 15th when we had our last good 
72_scores.cf.


THINGS I HAVE CHECKED:

At first I thought that 50_scores.cf changed a lot which caused more 
exclusion/culling in the 72_scores.cf but that's not it.  The revision 
only shows a few minor changes in 50_scores.cf:

https://svn.apache.org/viewvc/spamassassin/trunk/rules/50_scores.cf?view=log

Next I looked a the garescorer since it get's compiled from garescorer.c 
every run.  The garescorer.c is identical in the backups and in SVN so 
that's not it.  Again, I wish I had a 'scores' file to compare to from 
mid March.

-- 
Dave Jones