You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Alfred von Campe <al...@von-campe.com> on 2013/12/02 21:44:10 UTC
Help with post-commit script
I need to implement a post-commit hook that does the following in a "standard" Subversion repository (with trunk, branches, and tags at the top level):
1. Checks for existence of a certain property in the top-level directory of the trunk or branch
2. If property exists, check if any files listed in said property's value has been modified
3. Send an email with all diffs from step 2 to the recipients listed in another property
What makes it tricky is that in all steps above I need to be able to handle trunk as well as any branches. In addition, since it is possible to submit to both trunk and/or one or more branches in the same commit, and the property may exist in one but not the other, I need to walk through all files affected in the commit. I think this means I cannot use just svnlook and will have to parse the output of svn log. My questions are:
1. Is parsing the output of svn log the best approach?
2. If not, what is a better approach?
3. I would prefer to write this in bash or perl, but would appreciate any examples in other languages before I go re-inventing the wheel.
Thanks,
Alfred
P.S. Please CC me as I am not on the list.
Re: Help with post-commit script
Posted by Alfred von Campe <al...@von-campe.com>.
Thanks to all for the suggestions. I actually oversimplified the situation a little bit (this repo has multiple projects with trunk, branches, and tags directories underneath them), which makes the script a little more complicated. But the concepts presented so far have given me a path forward. I'll post the results once I have it working (hopefully by the end of this week).
Thanks again,
Alfred
Re: Help with post-commit script
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Andrew Reedick wrote on Tue, Dec 03, 2013 at 15:04:13 -0500:
>
>
> > -----Original Message-----
> > From: Daniel Shahaf [mailto:d.s@daniel.shahaf.name]
> > Sent: Tuesday, December 03, 2013 1:14 PM
> > To: Andrew Reedick
> > Cc: Alfred von Campe; users@subversion.apache.org
> > Subject: Re: Help with post-commit script
> >
> >
> > > svnlook changed ... > $CHANGED_LIST || exit 1 cat $CHANGED_LIST | sed
> > > 's/^....//g' | perl -ne 'print "$1$2\n" if
> > > /^(trunk)\/|^(branches\/[^\/]*)\//' | sort -u | xargs -n 1 -i svnlook
> > > propget $REPOS_PATH my:filelist_prop "{}" > $FILES_TO_REPORT_ON ||
> > > exit 1
> > >
> > > cat $CHANGED_LIST | while read i
> >
> > 'read' splits on whitespace, so filenames that contain spaces won't
> > DTRT.
>
> Close. Read will drop leading/trailing whitespace. It does respect
> "internal" whitespace though.
>
Thanks for the correction.
> How does 'svnlook changed' output filenames with embedded newlines
> anyway?
It doesn't, but in FSFS repositories they can't be added in the first
place.
> The entries in the filelist svn property would need to be in a regex format. So everything would be escaped already.
>
The pattern is $i, which comes from the dirs-changed output, not from
the property. (I agree with the 'grep -qF' solution, though.)
> SVNLOOK_CMD=/path/to/svnlook
You still need to make sure grep, perl, etc are in the path (or use
abspaths to them). perl won't be in the pth, for example.
RE: Help with post-commit script
Posted by Andrew Reedick <An...@cbeyond.net>.
> -----Original Message-----
> From: Daniel Shahaf [mailto:d.s@daniel.shahaf.name]
> Sent: Tuesday, December 03, 2013 1:14 PM
> To: Andrew Reedick
> Cc: Alfred von Campe; users@subversion.apache.org
> Subject: Re: Help with post-commit script
>
>
> > svnlook changed ... > $CHANGED_LIST || exit 1 cat $CHANGED_LIST | sed
> > 's/^....//g' | perl -ne 'print "$1$2\n" if
> > /^(trunk)\/|^(branches\/[^\/]*)\//' | sort -u | xargs -n 1 -i svnlook
> > propget $REPOS_PATH my:filelist_prop "{}" > $FILES_TO_REPORT_ON ||
> > exit 1
> >
> > cat $CHANGED_LIST | while read i
>
> 'read' splits on whitespace, so filenames that contain spaces won't
> DTRT.
Close. Read will drop leading/trailing whitespace. It does respect "internal" whitespace though.
The fix is to set IFS to null:
$ OLD_IFS=$IFS
$ IFS=
$ echo " a b c d.txt " | while read i
> do
> echo ".$i."
> done
. a b c d.txt .
Now if someone has embedded newlines in their filenames then that would be a problem. At that point you're talking about 'xargs -0 (--null)', sort -z, (--zero-terminated), using perl to chomp the newline and output a null character, etc. How does 'svnlook changed' output filenames with embedded newlines anyway?
>
> > do
> > grep -q "$i" "$FILES_TO_REPORT_ON"
>
> Same for filenames that contain regex metacharacters.
The entries in the filelist svn property would need to be in a regex format. So everything would be escaped already.
If not, then perl's quotemeta and greps --fixed-strings flags would be of use: perl -ne 'chomp; print quotemeta($_) . "\n"' file.txt.
Anyway, IME, it's almost always a better idea to use the --xml option when parsing svn commands, which implies writing a proper perl script. The work can (probably) be done in bash, but with all the whitespace handling and potentially multiple layers of interpolation going on, the code can get unwieldy quickly.
Updated script:
#!/bin/bash
set -o pipefail
REPOS_PATH=$1
REV=$2
SVNLOOK_CMD=/path/to/svnlook
RECIPIENT_LIST=$($SVNLOOK_CMD propget ... my:email_list_prop)
if [[ -z "$RECIPIENT_LIST" ]] then
exit 0
fi
CHANGED_LIST=$(mktemp ...)
FILES_TO_REPORT_ON=$(mktemp ...)
$SVNLOOK_CMD changed ... | perl -ne 'chomp; print quotemeta($_) . "\n"' > $CHANGED_LIST || exit 1
perl -i -pe 's/^....//' $CHANGED_LIST
perl -ne 'chomp; print quotemeta("$1$2") . "\n" if /^(trunk)\/|^(branches\/[^\/]*)\//' $CHANGED_LIST | sort -u | xargs -n 1 -i svnlook propget "$REPOS_PATH" my:filelist_prop "{}" > $FILES_TO_REPORT_ON || exit 1
cat $CHANGED_LIST | while read i
do
grep -q -F "$i" "$FILES_TO_REPORT_ON"
if [[ $? -eq 0 ]]
then
svnlook diff -r ... | sendmail -s " $i was touched in an impure manner" $RECIPIENT_LIST
fi
done
Re: Help with post-commit script
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
Andrew Reedick wrote on Mon, Dec 02, 2013 at 16:07:52 -0600:
> #!/bin/bash
>
> set -o pipefail
>
> REPOS_PATH=$1
> REV=$2 # or is it the other way around?
It's this way around.
> RECIPIENT_LIST=$(svnlook propget ... my:email_list_prop)
Need to set PATH first for this to work.
> svnlook changed ... > $CHANGED_LIST || exit 1
> cat $CHANGED_LIST | sed 's/^....//g' | perl -ne 'print "$1$2\n" if /^(trunk)\/|^(branches\/[^\/]*)\//' | sort -u | xargs -n 1 -i svnlook propget $REPOS_PATH my:filelist_prop "{}" > $FILES_TO_REPORT_ON || exit 1
>
> cat $CHANGED_LIST | while read i
'read' splits on whitespace, so filenames that contain spaces won't DTRT.
> do
> grep -q "$i" "$FILES_TO_REPORT_ON"
Same for filenames that contain regex metacharacters.
> if [[ $? -eq 0 ]]
> then
> sendmail -s " $i was touched in an impure manner" $RECIPIENT_LIST < svnlook diff -r ...
There's an obvious semantic error here, you meant to use a pipe.
But I would suggest using mailer.py here. If nothing else, you can use
it in its "print the mail to stdout" mode and then pipe that to
sendmail. That'll take care of formatting the diff, log message, etc.
> Also, do NOT have any blank links in the "my:filelist_prop". Or does
> that only apply when using 'grep -v'?
Your perl invocation will DTRT on blank lines (that is: skip them
without printing them).
RE: Help with post-commit script
Posted by Andrew Reedick <An...@cbeyond.net>.
> -----Original Message-----
> From: Alfred von Campe [mailto:alfred@von-campe.com]
> Sent: Monday, December 02, 2013 3:44 PM
> To: users@subversion.apache.org
> Subject: Help with post-commit script
>
> I need to implement a post-commit hook that does the following in a "standard" Subversion repository (with trunk, branches, and tags at the top level):
>
> 1. Checks for existence of a certain property in the top-level directory of the trunk or branch
> 2. If property exists, check if any files listed in said property's value has been modified
> 3. Send an email with all diffs from step 2 to the recipients listed in another property
>
> What makes it tricky is that in all steps above I need to be able to handle trunk as well as any branches. In addition, since it is possible to submit to both trunk and/or one or more branches in the same commit, and the property > may exist in one but not the other, I need to walk through all files affected in the commit. I think this means I cannot use just svnlook and will have to parse the output of svn log. My questions are:
>
> 1. Is parsing the output of svn log the best approach?
> 2. If not, what is a better approach?
> 3. I would prefer to write this in bash or perl, but would appreciate any examples in other languages before I go re-inventing the wheel.
>
> Thanks,
> Alfred
>
> P.S. Please CC me as I am not on the list.
1. Probably not.
2. Ssee below
3. See below
Pseudocode. You'll need to fill in the gaps, i.e. the ellipses. And you'll probably want to delete the temp files with a trap (google on: bash delete temp dir)
#!/bin/bash
set -o pipefail
REPOS_PATH=$1
REV=$2 # or is it the other way around?
RECIPIENT_LIST=$(svnlook propget ... my:email_list_prop)
if [[ -z "$RECIPIENT_LIST" ]]
then
exit 0
fi
CHANGED_LIST=$(mktemp ...)
FILES_TO_REPORT_ON=$(mktemp ...)
svnlook changed ... > $CHANGED_LIST || exit 1
cat $CHANGED_LIST | sed 's/^....//g' | perl -ne 'print "$1$2\n" if /^(trunk)\/|^(branches\/[^\/]*)\//' | sort -u | xargs -n 1 -i svnlook propget $REPOS_PATH my:filelist_prop "{}" > $FILES_TO_REPORT_ON || exit 1
cat $CHANGED_LIST | while read i
do
grep -q "$i" "$FILES_TO_REPORT_ON"
if [[ $? -eq 0 ]]
then
sendmail -s " $i was touched in an impure manner" $RECIPIENT_LIST < svnlook diff -r ...
fi
done
Also, do NOT have any blank links in the "my:filelist_prop". Or does that only apply when using 'grep -v'?