easier rule committing?

[Justin Mason <jm...@jmason.org>]

hey,

It strikes me that the process to become an ASF committer is turning out
to be a little too onerous for some of our purposes.

Nowadays we have these rule sandboxes (
http://wiki.apache.org/spamassassin/RuleSandboxes ), and we want to allow
all sorts of people -- including occasional contributors who might add a
rule once every 3 months, for example -- to add rules to them for testing.

However, we still have this process where people who are not committers,
have to go through a committer to get their rules in there, or become a
full committer.   Ideally, it should be possible that they can just write
the rules to the repository themselves so it can be picked up for testing,
without our intervention -- that's what the sandbox system was intended to
help with!

We discussed this when setting up the sandboxes, and decided to go with
committer access only -- because we couldn't see a way to get a less
onerous, more open model that fits with the ASF policies.  But I don't
think this is working.  So:

- (a) Is there a way to add a new type of committer for sandboxes only,
  with a lower "barrier to entry"?    I notice that one of the options
  for the Summer of Code students is to give them freer reign on a side
  branch, which strikes me as a similar setup.

- (b) more importantly: is this worth doing?  Are we likely to get more
  interest in rule development if we do this?


Any thoughts?

--j.

Re: easier rule committing?

[Theo Van Dinter <fe...@apache.org>]

On Sat, Apr 29, 2006 at 05:39:15PM +0100, Justin Mason wrote:
> It strikes me that the process to become an ASF committer is turning out
> to be a little too onerous for some of our purposes.

Yes.  There's supposed to be (iirc) the CA in the works so that people
can get commit access without getting a full ASF account, which is the
main time suck.  Once that's ready, we can, in theory, give out access
much more easily for at least the rules area (though at the moment we
decided to go with granting full commit access).

> - (a) Is there a way to add a new type of committer for sandboxes only,
>   with a lower "barrier to entry"?    I notice that one of the options
>   for the Summer of Code students is to give them freer reign on a side
>   branch, which strikes me as a similar setup.

The issue is commit access requiring a full ASF shell account.  Until that's
fixed, there's not really a way to give people a "limited" commit priv.

> - (b) more importantly: is this worth doing?  Are we likely to get more
>   interest in rule development if we do this?

I think we have general development issues, namely that there are very
few people left who are actively doing development.  Rule development is
important too, but I think "easier" just means "more rules" which doesn't
necessarily help.

I'm tired of constantly discussing how to get more rule developers though.
How do we get more developers in general?

-- 
Randomly Generated Tagline:
"Truly unencumbered by the engineering process."
         - Unknown about the Renault Dauphine

Re: easier rule committing?

[Theo Van Dinter <fe...@apache.org>]

On Sat, Apr 29, 2006 at 09:25:45PM -0400, Kevin A. McGrail wrote:
> standards because I feel that rules built-in to SA have to really be of the 
> highest quality.  The recent addition of sa-update may change my position 
> on that.

I'm not sure why sa-update would change your position.  Just because we can
get rules published and unpublished quickly doesn't mean the rules can be
lower quality.

> Finally, allowing to sa-update to quickly use these new rules would be 
> ideal.  I would setup thresholds (some automatic) like:

Something like this is sort of in place.  We have automatic "graduation"
nightly based on mass-check results, though there are still issues to work out
with it.  It's only working for 3.2 at the moment.

> This would allow something akin to sa-update --threshold=experimental which 
> in addition to the new SA rules would also get the rules that are 
> experimental.

FYI, in sa-update parlance, those would be different channels.  ie:

sa-update --channel expirimental.spamassassin.org

or some such.  Right now just the main channel (updates) is in use because we
don't have any other ones ready yet.

-- 
Randomly Generated Tagline:
Cat \kat'\ n. 1: A dog with an attitude problem

Re: easier rule committing?

[Kenneth Porter <sh...@sewingwitch.com>]

On Saturday, April 29, 2006 9:25 PM -0400 "Kevin A. McGrail" 
<ke...@thoughtworthy.com> wrote:

> However, adding more people writing rules and trying to explain the
> difference between .* and .{0,30} and False Positives could get very
> tiresome.  Some people make it over the hill and start writing good rules
> but I still see a lot of crazy or simply inefficient rules.

Is there a wiki page enumerating common faults in rules?

Re: easier rule committing?

["Kevin A. McGrail" <ke...@thoughtworthy.com>]

I've had a few weeks now to delve into how the rule process works and it is 
convoluted.  However, I'm not sure there is a good, easy fix.  I'm just 
coming up to speed with how best to get rules to "release quality" standards 
because I feel that rules built-in to SA have to really be of the highest 
quality.  The recent addition of sa-update may change my position on that.

However, adding more people writing rules and trying to explain the 
difference between .* and .{0,30} and False Positives could get very 
tiresome.  Some people make it over the hill and start writing good rules 
but I still see a lot of crazy or simply inefficient rules.

That said, there have been a lot of rule writers who have been frustrated 
over the years (myself included) with the difficulty of rules submission and 
rules update.  I think the creation of SARE/rules emporium would be an 
example of this but again sa-update seems like an excellent step towards 
this.

Therefore, I think a web-based rule submission system with attached spamples 
could be very useful for adding rules directly for testing.  It should be 
simple and quick.

After that, I think growing the corpus for testing new rules and getting 
more systems running honeypots and nightly mass check would be the next 
step.

Finally, allowing to sa-update to quickly use these new rules would be 
ideal.  I would setup thresholds (some automatic) like:

brand new (lints but hasn't been run through nightlymasscheck)
experimental (has XYZ result with nightlymasscheck)
...
released (will be in the next release / current sa-update threshold)

This would allow something akin to sa-update --threshold=experimental which 
in addition to the new SA rules would also get the rules that are 
experimental.

Regards,
KAM

> It strikes me that the process to become an ASF committer is turning out
> to be a little too onerous for some of our purposes.

Re: easier rule committing?

[Michael Monnerie <mi...@it-management.at>]

On Samstag, 29. April 2006 18:39 Justin Mason wrote:
> - (b) more importantly: is this worth doing?  Are we likely to get
> more interest in rule development if we do this?

I believe there are more people like me - making rules for their 
language (german in my case). In April alone, I've had 39026 downloads 
of my ruleset so far till yesterday, so people seem to use it.

With an easier system, many more languages could probably be supported, 
which would draw more attention to mass-checks also - and you are 
searching for contributors there also.

Currently, I have to send my rules to Justin to have them in a sandbox, 
and as they change often several times a week, it's a lot of 
unnecessary work for both of us.

mfg zmi
-- 
// Michael Monnerie, Ing.BSc    -----      http://it-management.at
// Tel: 0660/4156531                          .network.your.ideas.
// PGP Key:   "lynx -source http://zmi.at/zmi3.asc | gpg --import"
// Fingerprint: 44A3 C1EC B71E C71A B4C2  9AA6 C818 847C 55CB A4EE
// Keyserver: www.keyserver.net                 Key-ID: 0x55CBA4EE

Re: easier rule committing?

[Craig McLean <cr...@fukka.co.uk>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Justin Mason wrote:
> hey,
> 
> It strikes me that the process to become an ASF committer is turning out
> to be a little too onerous for some of our purposes.
[snippage]
> - (b) more importantly: is this worth doing?  Are we likely to get more
>   interest in rule development if we do this?
> 
> 
> Any thoughts?

My £.02, I have a raft of rules which follow the "works for me"
principle, but I'm a single-issue host. Just me and the wife using this
system so I have limited corpora for masschecking the rules I write. I
would dearly love to release all my rules freely for consideration by
others, but don't want the embarrasment of said rules FPing all over the
place.

I'd love to see some way for me to masscheck and, if applicable, submit
these rules online. Something web-front-endy which allows one to upload
.cf files would be perfect.

With that in mind, I'd be happy to donate processor cycles on my little
PIII-850 dev machine for a distributed masscheck effort if they would be
of any use.

Thanks,
Craig.

- --
Craig McLean		http://fukka.co.uk
craig@fukka.co.uk	Where the fun never starts
	Powered by FreeBSD, and GIN!
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (GNU/Linux)

iD8DBQFEU6zzMDDagS2VwJ4RApDbAJ9iX+Si9wlifE0QMgtMwyOod4qk1ACaAh96
40R5O3ZKirN5kOqIoiwCuE0=
=ahFc
-----END PGP SIGNATURE-----