Proposal: Temporary Manual Inspection of Auto-Promoted Rules

["Warren Togami Jr." <wt...@gmail.com>]

May I propose that we insert a temporary "short-circuit" into the 
auto-update mechanism to allow for manual inspection prior to live push?

Let all but the DNS update happen then post the URL somewhere public so 
many eyes can download and examine diffs.

Keep it this way for a short while until we've regained confidence that 
auto-promotion wont cause unexpected surprises.  Thereafter make it 
fully automatic again, but make it easy to flip a boolean to switch back 
to manual inspection should we need it again later.

Thoughts?

Warren Togami
warren@togami.com

Re: Proposal: Temporary Manual Inspection of Auto-Promoted Rules

["Kevin A. McGrail" <KM...@PCCC.com>]

On 3/25/2011 2:44 AM, Warren Togami Jr. wrote:
> May I propose that we insert a temporary "short-circuit" into the 
> auto-update mechanism to allow for manual inspection prior to live push?
>
> Let all but the DNS update happen then post the URL somewhere public 
> so many eyes can download and examine diffs.
>
> Keep it this way for a short while until we've regained confidence 
> that auto-promotion wont cause unexpected surprises.  Thereafter make 
> it fully automatic again, but make it easy to flip a boolean to switch 
> back to manual inspection should we need it again later.
Good Proposal Warren,

I was discussing this as well with others to add a pre-flight check.

However, this was apparently vetted ages ago during sa-updates 
development.  There is even code remnants in sa-update that refer to 
"staging".

The trade-off was use sa-updates to stop quick spam trends or risk 
publishing rules with problems.

The choice was to stop quick spam trends.

I think the current problems with rules generation is artificially 
exacerbating the problems and that staging will eventually be more 
problematic than helpful.

Therefore I think that stopping quick spam trends needs to continue as 
the overall goal.  However, some administrators can't take that risk.

So here is my thought on the pre-flightcheck / short circuit after more 
thought:

- Add a 24-hour delay by DEFAULT to sa-update so that sa-update checks 
for an update via DNS and if 24 hours passes then it installs the updates.

- We change the rules versions in DNS to be <YYYYMMDDHH><currentnumber> 
based on UTC so that the math to check the hour is actually included in 
the DNS query.

- Otherwise, if people want to override the wait (i.e. get the latest 
rules), then they just add a flag to their sa-update command such as 
--delay=0 where --delay=24 is the default and voila, we have our staging 
without doing anything major to rules generation.

- We also should think about adding some randomization of +/-  hours to 
sa-update to stop the large jumps in sa-update traffic that are on the hour.

Finally, the "good" news about this is that it brings to light my 
worries with RBLs that don't have sufficient resources and stupidly use 
FPs as an answer when query thresholds are reached.  These type of RBLs 
can never be candidates for default configuration with SA and it gives 
me more impetus to drafting the RBL inclusion rules I've been working on 
for far to long.

Regards,
KAM