You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Jan Warnking <ja...@ujf-grenoble.fr> on 2006/03/10 10:25:29 UTC
svn and shell scripts: managing properties
Hello all,
I'm just discovering subversion and am in the process of setting up a
couple of Linux shell scripts for some common operations we perform on
our code. That involves setting/modifying properties on files
automatically. It was not long before I ran into a couple of
difficulties doing that:
- a minor annoyance of the proplist command is that it cannot be asked
to omit the first line, which reads "Properties on 'mypath':" and which
does not actually contain property names. I expected either the --quiet
or the --non-interactive switch to do that for me. The former does not
exist for propedit, the latter doesn't change the output. This makes
scripts unnecessarily cumbersome. I'm not really good at shellscripts
yet, so any code snipped stripping that line from the output would be
appreciated.
- a bigger annoyance is the algorithm used for checking whether a
property has been modified with a propedit command. If I do:
svn propedit myproperty --editor-cmd "sed -i -e 's/foo/bar/g'" mypath
then svn will respond: "No changes to property 'myproperty' on
'mypath'". This occurs because SVN checks the timestamp (modification
time) and file size of the temporary file containing the property value,
in order to determine whether a change occurred. Sed is fast enough so
that the modification time appears unchanged, and the replacement done
does not change file length. Of course I can change this command to:
svn propedit myproperty --editor-cmd "sleep 2; sed -i -e 's/foo/bar/g'"
mypath
That works, but is slow. The whole point of scripting is that I have
potentially many files to process.
I guess both issues are in part feature requests, (being able to
suppress anything not containing property names/values from the output
of proplist and having an option for MD5SUM comparison of the temporary
file before and after a propedit command to detect changes to the
property value). But for now I'd be glad for any suggestions on how to
work around these quirks.
I'm using version 1.2.1, but observed the same behaviour on 1.3.0.
Jan
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn and shell scripts: managing properties
Posted by Jan Warnking <ja...@ujf-grenoble.fr>.
Actually, I would have thought that the algorithm detecting modified
files would be different between inspecting working-copy files in view
of a commit and detecting an abandoned propedit. The tradeoff between
accuracy and speed mentioned in the discussions cited by Ryan will not
be the same for the propedit for the following reasons:
- there is exactly one file to be checked per transaction
- that file will be tiny in the vast majority of cases
- the original content of the file is known to propedit and need not be
read from a file
- the probability of the file being changed on disk is very close to one
(how often do you cancel a propedit?)
- the chances of the file being younger than one second are big as it is
a temporary file created just for this transaction
As a result, the current algorithm has a much higher chance of failing
in this context compared to checks on working-copy files. And a correct
algorithm would entail a negligible performance hit. Most notably it
would be faster than the currently necessary workarounds (combined
propdel and propset or one second of sleep). I would think that just
reading the file and running a byte-wise comparison for changes would
probably the most efficient algorithm both in terms of execution speed
and coding.
Just my €0.02.
Ryan Schmidt wrote :
>
> On Mar 12, 2006, at 05:37, Kalin KOZHUHAROV wrote:
>
>>> The fact that Subversion doesn't notice differences if the modification
>>> occurs within the same second has come up on the list before. For
>>> example:
>>>
>>> http://svn.haxx.se/users/archive-2005-03/0783.shtml
>>>
>>> I searched http://svn.haxx.se/users for "one second"; there were other
>>> results.
>>
>>
>> OK, I kind of missed most of this thread, but this behavior is a bug,
>> right?
>>
>>>>> svn propedit myproperty --editor-cmd "sed -i -e 's/foo/bar/g'" mypath
>>>>> then svn will respond: "No changes to property 'myproperty' on
>>>>> 'mypath'".
>>>>
>>
>> I couldn't find any related issues in the issue tracker, are there any?
>> Or why is it a WONTFIX bug?
>
>
> There was some other discussion about this on the users list in 2005
> which I cannot find at the moment; the proper search terms elude me. I
> believe it boiled down to the only way to avoid this problem would be
> for Subversion to actually check the files' entire contents every
> time, which would be way too slow to be usable, and so this compromise
> was reached that the mtime and/or file size are checked first.
> According to this developer discussion from last month...
>
> http://svn.haxx.se/dev/archive-2006-02/1034.shtml
>
> ...the algorithm currently is:
>
> if (timestamp is same)
> return not_modified;
> else if (size differs)
> return modified;
> else
> go_into_expensive_further_investigation();
>
> See also other parts of that discussion, like:
>
> http://svn.haxx.se/dev/archive-2006-02/1092.shtml
>
>
> I did also find this developer discussion from late 2002 suggesting
> that the current behavior was fine but just needed to be documented
> better:
>
> http://svn.haxx.se/dev/archive-2002-11/1065.shtml
>
>
>
>> So far the ugly workaround seems:
>>
>>>> using a combination of propdel and propset now. Quite ugly but it
>>>> works.
>>>
>
> Not sure why that works... the workaround would seem to be for your
> script to sleep for 1 second between operations to ensure that the
> timestamp is different.
Neither propdel nor proset need to check a file for changes and
therefore they succeed. Sleeping for a second works, too, but is sloooow
if the script operates on many files.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn and shell scripts: managing properties
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Mar 12, 2006, at 05:37, Kalin KOZHUHAROV wrote:
>> The fact that Subversion doesn't notice differences if the
>> modification
>> occurs within the same second has come up on the list before. For
>> example:
>>
>> http://svn.haxx.se/users/archive-2005-03/0783.shtml
>>
>> I searched http://svn.haxx.se/users for "one second"; there were
>> other
>> results.
>
> OK, I kind of missed most of this thread, but this behavior is a
> bug, right?
>
>>>> svn propedit myproperty --editor-cmd "sed -i -e 's/foo/bar/g'"
>>>> mypath
>>>> then svn will respond: "No changes to property 'myproperty' on
>>>> 'mypath'".
>
> I couldn't find any related issues in the issue tracker, are there
> any?
> Or why is it a WONTFIX bug?
There was some other discussion about this on the users list in 2005
which I cannot find at the moment; the proper search terms elude me.
I believe it boiled down to the only way to avoid this problem would
be for Subversion to actually check the files' entire contents every
time, which would be way too slow to be usable, and so this
compromise was reached that the mtime and/or file size are checked
first. According to this developer discussion from last month...
http://svn.haxx.se/dev/archive-2006-02/1034.shtml
...the algorithm currently is:
if (timestamp is same)
return not_modified;
else if (size differs)
return modified;
else
go_into_expensive_further_investigation();
See also other parts of that discussion, like:
http://svn.haxx.se/dev/archive-2006-02/1092.shtml
I did also find this developer discussion from late 2002 suggesting
that the current behavior was fine but just needed to be documented
better:
http://svn.haxx.se/dev/archive-2002-11/1065.shtml
> So far the ugly workaround seems:
>>> using a combination of propdel and propset now. Quite ugly but it
>>> works.
Not sure why that works... the workaround would seem to be for your
script to sleep for 1 second between operations to ensure that the
timestamp is different.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn and shell scripts: managing properties
Posted by Ryan Schmidt <su...@ryandesign.com>.
On Mar 11, 2006, at 16:35, Jan Warnking wrote:
>> - a bigger annoyance is the algorithm used for checking whether a
>> property has been modified with a propedit command. If I do:
>> svn propedit myproperty --editor-cmd "sed -i -e 's/foo/bar/g'" mypath
>> then svn will respond: "No changes to property 'myproperty' on
>> 'mypath'". This occurs because SVN checks the timestamp
>> (modification time) and file size of the temporary file containing
>> the property value, in order to determine whether a change
>> occurred. Sed is fast enough so that the modification time appears
>> unchanged, and the replacement done does not change file length.
>> Of course I can change this command to:
>> svn propedit myproperty --editor-cmd "sleep 2; sed -i -e 's/foo/
>> bar/g'" mypath
>> That works, but is slow. The whole point of scripting is that I
>> have potentially many files to process.
> Thanks, Marc, for the hint on tail +... . On the issue with svn
> propedit being unusable in scripts I gave up and am using a
> combination of propdel and propset now. Quite ugly but it works.
> I'd still like to hear if this is someting that has come up before
> and if there are plans to fix it.
The fact that Subversion doesn't notice differences if the
modification occurs within the same second has come up on the list
before. For example:
http://svn.haxx.se/users/archive-2005-03/0783.shtml
I searched http://svn.haxx.se/users for "one second"; there were
other results.
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn and shell scripts: managing properties
Posted by Jan Warnking <ja...@ujf-grenoble.fr>.
Thanks, Marc, for the hint on tail +... . On the issue with svn propedit
being unusable in scripts I gave up and am using a combination of
propdel and propset now. Quite ugly but it works. I'd still like to hear
if this is someting that has come up before and if there are plans to
fix it.
I wrote:
> - a bigger annoyance is the algorithm used for checking whether a
> property has been modified with a propedit command. If I do:
> svn propedit myproperty --editor-cmd "sed -i -e 's/foo/bar/g'" mypath
> then svn will respond: "No changes to property 'myproperty' on
> 'mypath'". This occurs because SVN checks the timestamp (modification
> time) and file size of the temporary file containing the property
> value, in order to determine whether a change occurred. Sed is fast
> enough so that the modification time appears unchanged, and the
> replacement done does not change file length. Of course I can change
> this command to:
> svn propedit myproperty --editor-cmd "sleep 2; sed -i -e
> 's/foo/bar/g'" mypath
> That works, but is slow. The whole point of scripting is that I have
> potentially many files to process.
Marc Haisenko said:
>On Friday 10 March 2006 11:25, Jan Warnking wrote:
>
>
>>- a minor annoyance of the proplist command is that it cannot be asked
>>to omit the first line, which reads "Properties on 'mypath':" and which
>>does not actually contain property names. I expected either the --quiet
>>or the --non-interactive switch to do that for me. The former does not
>>exist for propedit, the latter doesn't change the output. This makes
>>scripts unnecessarily cumbersome. I'm not really good at shellscripts
>>yet, so any code snipped stripping that line from the output would be
>>appreciated.
>>
>>
>
>As work around you can use "tail +2": it gives you everything after (and
>including) line 2, so you can do:
># command | tail +2
>or
># tail +2 <file
>
>C'ya,
> Marc
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org
Re: svn and shell scripts: managing properties
Posted by Marc Haisenko <ha...@comdasys.com>.
On Friday 10 March 2006 11:25, Jan Warnking wrote:
> - a minor annoyance of the proplist command is that it cannot be asked
> to omit the first line, which reads "Properties on 'mypath':" and which
> does not actually contain property names. I expected either the --quiet
> or the --non-interactive switch to do that for me. The former does not
> exist for propedit, the latter doesn't change the output. This makes
> scripts unnecessarily cumbersome. I'm not really good at shellscripts
> yet, so any code snipped stripping that line from the output would be
> appreciated.
As work around you can use "tail +2": it gives you everything after (and
including) line 2, so you can do:
# command | tail +2
or
# tail +2 <file
C'ya,
Marc
--
Marc Haisenko
Comdasys AG
Rüdesheimer Straße 7
D-80686 München
Tel: +49 (0)89 - 548 43 33 0
Fax: +49 (0)89 - 548 43 33 29
e-mail: haisenko@comdasys.com
http://www.comdasys.com
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org