Client side Hooks?

[Josef Wolf <jw...@raven.inka.de>]

Hello!

In the past 10 years I have built some wired script around cvs to
control my tests/ci/export/build/etc/pp. Now that svn came up and gave
me much hope that the world could be much better than in cvs, I sat
down and looked at those wired scripts and tried to find a better way
to do such things.

Here are some (probably wired) ideas which I would like to discuss on
this list.

What if we would introduce client-side-hooks based on properties? Look
at this example:

 cd $WC

 # refuse commit unless regression tests pass and changelog exists
 #
 echo make check >pre-commit
 echo emacs changelog >>pre-commit
 svn add pre-commit
 svn propset svn:commit:pre /bin/sh pre-commit

 # announce the commit
 #
 echo mail -s '"rev $LastChagedRevision:$ committed"' $recipients >post-commit
 svn propset svn:keywords LastChangedRevision post-commit
 svn propset svn:commit:post /bin/sh post-commit

 # export automatically builds and uploads to server
 #
 echo make dist >post-export
 echo scp 'foo-$LastChangedRevision:$.tar.gz ftp.foo.org:/pub/foo >>post-export
 svn propset svn:keywords LastChangedRevision post-export
 svn propset svn:export:post /bin/sh post-export

This could be extended to include the OS-name or hostname, so you
could make:

 svn propset svn:export:post:Win c:\\command.com post-export.bat
 svn propset svn:export:post:Linux /bin/sh post-export-linux.sh
 
Every subcommand could have its own set of hooks. OK, this is just a
little example. You could think up a lot of funny stuff with such an
extension.

What do you think about that?

PS: Oups? I just noticed that svn lacks an "export" subcommand?

-- 
-- Josef Wolf -- jw@raven.inka.de --

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Client side Hooks?

[Greg Hudson <gh...@MIT.EDU>]

Yeah, -1 on client-side hooks based on server-side information, for
security reasons.  (Client-side hooks based on client-side information,
such as homedir configuration, is fine.)


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Client side Hooks? (rethaught)

["Chris D. Sloan" <cd...@cs.hmc.edu>]

On Sun, May 26, 2002 at 10:41:46PM +0200, Josef Wolf wrote:
> On Sat, May 25, 2002 at 12:20:27PM -0700, Chris D. Sloan wrote:
> 
> > I think that making subversion give the repository direct exectuion
> > priveleges on my system by default is a mistake.
> 
> Right. Such an extension would require a command line switch to enable
> the hook-execution. The default should be not to execute the hooks.

In addition to command line args, I'd want a per-repository and/or
per-working directory way to enable the hook execution.  For example,
if you use Subversion at your company, then it might make sense to
enable hook execution for the whole repository (or a subset thereof).
Then you could just forget about it since you trust the other
developers at your company.

The per-working directory control of hooks might be needless
complication, but here was my thinking: If you had some type of
sandbox, you could run the hooks in there, but you wouldn't want to
run them in working directories not in the sandbox.  A similar
situation is if your config is in you NFS homedirectory, but you have
working directories on multiple machines.  You could enable hooks on
some machines but not on all.  (Say the hooks only run on certain
machines.)


> > One idea for reducing server load is to take a page from ageis and
> > modify it for svn.  [ ... ]
> 
> Are you talking about Aegis (note the spelling) or about yet another
> SCM? Yes, Aegis has many fine Ideas. It would be nice to see some
> (most?) of them in svn, too. But I don't think this will happen
> before svn-3.0 or some such.

I took the observed behavior of aegis and adapted it so that I think
it would work on Subversion.  In particular, the key idea was that the
checkin operation included the "proof" that you did the needed
operations (in this case ran the tests).  Since the proof was passed
to the server, you don't need the client hooks to run the commands
automatically for you.  This gives the user the flexibility to run the
tests (or whatever) on their schedule but still allows the admin to
require certain standards for commited code.

> > >  # announce the commit
> > 
> > The subversion project already does this with a server side hook.
> > Also, the server is known to be setup to send mail.  My client box
> > isn't configured to do so, so it makes sense that it be done at the
> > server.
> 
> This is correct. But with the server-hook the only information you can
> include is information you can get out of the repo. And that is
> _exactly_ what I am missing. Please read ahead.
> 
> 
> After a close look at my scripts I start to notice what I am _really_
> missing. You are right: I don't really need the client side
> hooks. Most of the quirks those scripts do are to get around cvs
> design issues and are not needed anymore with svn. The only thing that
> remains is to mantain additional, independent change-logs. Let me explain
> this a little bit closer:
> 
> Additionally to the log-messages (which are primarily for developers)
> we maintain additional change-logs for testers and for customers. Those
> logs don't go in that detail as the logs for developers, because the
> testers/customers start asking silly questions if you give them too
> much information (not that askig questions were a bad thing. But
> answering them takes a lot of time ;-)
> 
> The structure of those logs is somewhat different. Usually there is
> one entry for every _released_ revision which contains all the changes
> made in previos revisions.
> 
> To help people to maintain those logs, a perl script is used for
> checkin. This script calls an editor to get the changelog-entries,
> does some kludges to integrate them into the main changelog file and
> calls "cvs ci"/"cvs tag"/"cvs tag -b"/etc/pp.
> 
> The thing I am really missing is a way to get (via an editor,
> templates would be nice) additional informations (which will not be
> checked in) from the committer and pass them through to the
> server-hooks.

We used commit templates (with server-side enforcement) with CVS.
This included bug numbers and user-visible chagnes and so on.  The
enforcement was through the CVS server commit hook.  Subversion should
definitely include this capability if it doesn't have it yet.

	Chris

-- 
Chris Sloan
cds@cs.hmc.edu

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Client side Hooks? (rethaught)

[Josef Wolf <jw...@raven.inka.de>]

On Sat, May 25, 2002 at 12:20:27PM -0700, Chris D. Sloan wrote:

> I think that making subversion give the repository direct exectuion
> priveleges on my system by default is a mistake.

Right. Such an extension would require a command line switch to enable
the hook-execution. The default should be not to execute the hooks.

> Regardless, the server is a better place to do many of the examples
> you gave, anyway.

> On Sat, May 25, 2002 at 02:27:36PM +0200, Josef Wolf wrote:
> >  # refuse commit unless regression tests pass and changelog exists
> > 
> This might better belong on the server.  You can't enforce a client
> hook, since anyone could hack the client not to perform it anyway.  As
> a result, most types of commit fitness tests belong on the server.

This one is not to enforce bad guys to run the tests. You can't
enforce this regardless of where the hooks are placed. This is to help
the good guys not to _forget_ to run the tests.

> One idea for reducing server load is to take a page from ageis and
> modify it for svn.  [ ... ]

Are you talking about Aegis (note the spelling) or about yet another
SCM? Yes, Aegis has many fine Ideas. It would be nice to see some
(most?) of them in svn, too. But I don't think this will happen
before svn-3.0 or some such.

> >  # announce the commit
> 
> The subversion project already does this with a server side hook.
> Also, the server is known to be setup to send mail.  My client box
> isn't configured to do so, so it makes sense that it be done at the
> server.

This is correct. But with the server-hook the only information you can
include is information you can get out of the repo. And that is
_exactly_ what I am missing. Please read ahead.


After a close look at my scripts I start to notice what I am _really_
missing. You are right: I don't really need the client side
hooks. Most of the quirks those scripts do are to get around cvs
design issues and are not needed anymore with svn. The only thing that
remains is to mantain additional, independent change-logs. Let me explain
this a little bit closer:

Additionally to the log-messages (which are primarily for developers)
we maintain additional change-logs for testers and for customers. Those
logs don't go in that detail as the logs for developers, because the
testers/customers start asking silly questions if you give them too
much information (not that askig questions were a bad thing. But
answering them takes a lot of time ;-)

The structure of those logs is somewhat different. Usually there is
one entry for every _released_ revision which contains all the changes
made in previos revisions.

To help people to maintain those logs, a perl script is used for
checkin. This script calls an editor to get the changelog-entries,
does some kludges to integrate them into the main changelog file and
calls "cvs ci"/"cvs tag"/"cvs tag -b"/etc/pp.

The thing I am really missing is a way to get (via an editor,
templates would be nice) additional informations (which will not be
checked in) from the committer and pass them through to the
server-hooks.

-- 
-- Josef Wolf -- jw@raven.inka.de --

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Client side Hooks?

["Chris D. Sloan" <cd...@cs.hmc.edu>]

I have a few concerns about client side hooks:

- Security for the user.

- There are real compatibility problems with client hooks, and many of
  the tasks can be achieved equally well on the server.


** Security **

I think my chief concern is this: I read about some interesting open
source project on the net and decide I want to check out the code and
have a look.  I svn checkout the code, and the client hooks covertly
subvert my system.

I think that making subversion give the repository direct exectuion
priveleges on my system by default is a mistake.  Almost no one can
remember to always disable client hooks everytime they access an
untrusted repository.  (Just look how poorly people listen to advice
not to open executable attachments.)  If svn gives me a way to enable
client hooks for a particular repository or a particular working
directory, then, as long as the default is disabled, this might be
ok.

Regardless, the server is a better place to do many of the examples
you gave, anyway.


** Compatibility **

As you allude to near the end of your message, client-side hooks would
depend at the very least on the OS of the host.  In fact, having dealt
with this type of thing before, I've found that it tends to depend on
a lot more than that.

In my project, we had development on windows and solaris mostly.  We
needed to have code run to setup a working directory after checkout
and after updates to certain files.  If we had had the possiblity of
client side hooks, we might have used them since the setup code was a
noop if it didn't need to do anything.  We had some compatibility
problems with Solaris, but Windows was the real pain.  Some computers
didn't have the MKS korn shell installed.  Some had Cygwin installed
instead.  Some didn't have Visual C++ installed.  And so on and so
on...

Of course, if you are a company with a good IT staff, you may be able
to standardize the machines enough to make this work, but we found the
problem more difficult than most people originally expected.

If the project is an open source project where the participants'
machines are not maintained by a common IT staff, then this gets
tricky indeed.


** My Conclusion **

Client hooks may be a desired or needed feature, and these problems
may be solvable, but I think they need to be thought about carefully.

	Chris

On Sat, May 25, 2002 at 02:27:36PM +0200, Josef Wolf wrote:
>  # refuse commit unless regression tests pass and changelog exists

This might better belong on the server.  You can't enforce a client
hook, since anyone could hack the client not to perform it anyway.  As
a result, most types of commit fitness tests belong on the server.

One idea for reducing server load is to take a page from ageis and
modify it for svn.  Under Ageis, when you run tests or you integrate
the build or anything else that the Ageis system wants to verify that
you have done before it proceeds, the results of the operation (output
of the test suite or whatever) are checked in with the changes.  This
allows the server verify that your test results match the expected
results.  (You include the checksums of the source or whatever if you
are worried about simple cheating.)  This system is no easier to fake
than the client hooks, and doesn't load the server.

Oh.  It just occured to me that this system also ensures that the
tests run in your environment (ie. on the local OS).  Running on the
central server would not guarentee that.

> 
>  # announce the commit

The subversion project already does this with a server side hook.
Also, the server is known to be setup to send mail.  My client box
isn't configured to do so, so it makes sense that it be done at the
server.

>  # export automatically builds and uploads to server

I don't really understand the motivation for this.  I sometimes export
from a repository because I want to, for example, import into a local
repository or do something else which is not really release based.
I'm don't understand the value of saving the state at that export
(especially since it should be possible to replicate it later if I
know the revision number when I exported).

> This could be extended to include the OS-name or hostname, so you
> could make:
> 
>  svn propset svn:export:post:Win c:\\command.com post-export.bat
>  svn propset svn:export:post:Linux /bin/sh post-export-linux.sh

My experience (as I mentioned above) is that OS type (for example) is
too course-grained to do really make this type of stuff work in a
complex situation.  Using hostnames would be really difficult to
maintain.

-- 
Chris Sloan
cds@cs.hmc.edu

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org