You are viewing a plain text version of this content. The canonical link for it is here.
Posted to sysadmins@spamassassin.apache.org by John Hardin <jh...@impsec.org> on 2021/02/20 18:30:30 UTC

freqsd on sa-vm masscheck box?

All:

It seems that freqsd was never migrated to the new masscheck VM. I'm 
working on doing that now, but I was wondering whether anybody remembers 
deferring that for a specific reason...?

I don't think this will affect masschecks and rule publication, it looks 
like it's just background processing to support the ruleqa website.


-- 
  John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
  jhardin@impsec.org                         pgpk -a jhardin@impsec.org
  key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
   USMC Rules of Gunfighting #20: The faster you finish the fight,
   the less shot you will get.
-----------------------------------------------------------------------
  2 days until George Washington's 289th Birthday

Re: freqsd on sa-vm masscheck box?

Posted by Henrik K <he...@hege.li>.
On Sat, Feb 20, 2021 at 09:07:48PM -0800, John Hardin wrote:
> 
> It maintains the data supporting these parts of the ruleqa detail web page:
> 
> 1) which file each rule lives in (partially working, seems to have problems
> sometimes with __subrules and T_rules)
> 
> 2) when the source file was last updated (seems to be using checkout date
> rather than last commit date, may look deeper into that)
> 
> 3) collects data for the hit rate over time graphs
> 
> 4) collects data for the corpus quality analysis (I think; that's still not
> working)
> 
> 5) potentially other less-visible stuff, like the "source details" link on
> the right side of the section headers
> 
> 
> What I was most interested in fixing was the source file reference.

There's already lots of stuff in cron, freqsd seems to be running
gen_info_xml and ruleqa.cgi -refresh redundandly etc..  but feel free to
sort these out. :-)


Re: freqsd on sa-vm masscheck box?

Posted by Bill Cole <sa...@billmail.scconsult.com>.
On 21 Feb 2021, at 0:07, John Hardin wrote:

> On Sun, 21 Feb 2021, Henrik K wrote:
>
>> On Sat, Feb 20, 2021 at 10:30:30AM -0800, John Hardin wrote:
>>> All:
>>>
>>> It seems that freqsd was never migrated to the new masscheck
>>> VM. I'm working on doing that now, but I was wondering
>>> whether anybody remembers deferring that for a specific
>>> reason...?
>>>
>>> I don't think this will affect masschecks and rule
>>> publication, it looks like it's just background processing to
>>> support the ruleqa website.
>>
>> What is it supposed to do exactly, looks deprecated?
>
> It maintains the data supporting these parts of the ruleqa detail web 
> page:
>
> 1) which file each rule lives in (partially working, seems to have 
> problems sometimes with __subrules and T_rules)
>
> 2) when the source file was last updated (seems to be using checkout 
> date rather than last commit date, may look deeper into that)
>
> 3) collects data for the hit rate over time graphs
>
> 4) collects data for the corpus quality analysis (I think; that's 
> still not working)
>
> 5) potentially other less-visible stuff, like the "source details" 
> link on the right side of the section headers

I'm pretty sure that these have been broken since before the migration.



-- 
Bill Cole
bill@scconsult.com or billcole@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)
Not Currently Available For Hire

Re: freqsd on sa-vm masscheck box?

Posted by John Hardin <jh...@impsec.org>.
On Sun, 21 Feb 2021, Henrik K wrote:

> On Sat, Feb 20, 2021 at 10:30:30AM -0800, John Hardin wrote:
>> All:
>>
>> It seems that freqsd was never migrated to the new masscheck
>> VM. I'm working on doing that now, but I was wondering
>> whether anybody remembers deferring that for a specific
>> reason...?
>>
>> I don't think this will affect masschecks and rule
>> publication, it looks like it's just background processing to
>> support the ruleqa website.
>
> What is it supposed to do exactly, looks deprecated?

It maintains the data supporting these parts of the ruleqa detail web 
page:

1) which file each rule lives in (partially working, seems to have 
problems sometimes with __subrules and T_rules)

2) when the source file was last updated (seems to be using checkout date 
rather than last commit date, may look deeper into that)

3) collects data for the hit rate over time graphs

4) collects data for the corpus quality analysis (I think; that's still 
not working)

5) potentially other less-visible stuff, like the "source details" link on 
the right side of the section headers


What I was most interested in fixing was the source file reference.



-- 
  John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
  jhardin@impsec.org                         pgpk -a jhardin@impsec.org
  key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
   A superior gunman is one who uses his superior judgment to keep
   himself out of situations that would require the use of his
   superior skills.
-----------------------------------------------------------------------
  2 days until George Washington's 289th Birthday

Re: freqsd on sa-vm masscheck box?

Posted by Henrik K <he...@hege.li>.
On Sat, Feb 20, 2021 at 10:30:30AM -0800, John Hardin wrote:
> All:
> 
> It seems that freqsd was never migrated to the new masscheck
> VM. I'm working on doing that now, but I was wondering
> whether anybody remembers deferring that for a specific
> reason...?
> 
> I don't think this will affect masschecks and rule
> publication, it looks like it's just background processing to
> support the ruleqa website.

What is it supposed to do exactly, looks deprecated?


Re: freqsd on sa-vm masscheck box?

Posted by John Hardin <jh...@impsec.org>.
On Sat, 20 Feb 2021, Kevin A. McGrail wrote:

> I am sure it was just missed.  Henrik did a great job on the box migration
> but that was likely missed.
>
> Any chance you can take a stab at it?

Already got it mostly working. It's getting the last updated date wrong 
but I'm going to let it run overnight and see what happens before 
troubleshooting further.

Is there any interest in running it once manually with a longer cutoff in 
order to catch up more than the last 30 days so that the 
performance-over-time graphs go back further?

> On Sat, Feb 20, 2021, 13:30 John Hardin <jh...@impsec.org> wrote:
>
>> All:
>>
>> It seems that freqsd was never migrated to the new masscheck VM. I'm
>> working on doing that now, but I was wondering whether anybody remembers
>> deferring that for a specific reason...?
>>
>> I don't think this will affect masschecks and rule publication, it looks
>> like it's just background processing to support the ruleqa website.


-- 
  John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
  jhardin@impsec.org                         pgpk -a jhardin@impsec.org
  key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
-----------------------------------------------------------------------
   A superior gunman is one who uses his superior judgment to keep
   himself out of situations that would require the use of his
   superior skills.
-----------------------------------------------------------------------
  2 days until George Washington's 289th Birthday

Re: freqsd on sa-vm masscheck box?

Posted by "Kevin A. McGrail" <km...@apache.org>.
I am sure it was just missed.  Henrik did a great job on the box migration
but that was likely missed.

Any chance you can take a stab at it?

On Sat, Feb 20, 2021, 13:30 John Hardin <jh...@impsec.org> wrote:

> All:
>
> It seems that freqsd was never migrated to the new masscheck VM. I'm
> working on doing that now, but I was wondering whether anybody remembers
> deferring that for a specific reason...?
>
> I don't think this will affect masschecks and rule publication, it looks
> like it's just background processing to support the ruleqa website.
>
>
> --
>   John Hardin KA7OHZ                    http://www.impsec.org/~jhardin/
>   jhardin@impsec.org                         pgpk -a jhardin@impsec.org
>   key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
> -----------------------------------------------------------------------
>    USMC Rules of Gunfighting #20: The faster you finish the fight,
>    the less shot you will get.
> -----------------------------------------------------------------------
>   2 days until George Washington's 289th Birthday
>