You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@subversion.apache.org by Peter Klotz <Pe...@ith-icoserve.com> on 2016/03/04 12:54:42 UTC
Timestamp issue with "svn status" in 1.9.3
Hello Subversion Developers
We encountered a strange issue with Subversion 1.9.3 (running under RHEL 7). The attached script demonstrates the behavior.
When changing the timestamp of a file to an earlier point in time and leaving the file size unchanged yet altering the content, Subversion is not always able to detect this change.
The attached test case is the simplest we could come up with that reliably reproduces the problem.
Regards, Peter.
Re: Timestamp issue with "svn status" in 1.9.3
Posted by Philip Martin <ph...@wandisco.com>.
Peter Klotz <Pe...@ith-icoserve.com> writes:
> When changing the timestamp of a file to an earlier point in time and
> leaving the file size unchanged yet altering the content, Subversion
> is not always able to detect this change.
Not any earlier time, but only the exact earlier time at which
Subversion wrote an unchanged file. Without this optimisation
large working copies would be extremely slow.
$ svnadmin create repo
$ svnmucc -mm -U file://`pwd`/repo put <(echo xx) f
$ svn co file://`pwd`/repo wc
$ stat wc/f | grep Modify
Modify: 2016-03-04 15:15:25.927354821 +0000
$ echo zz > wc/f
$ svn st wc # timestamp younger so modification detected
M wc/f
$ touch -d '2016-03-04 15:15' wc/f
$ svn st wc # timestamp older so modification detected
M wc/f
$ touch -d '2016-03-04 15:15:25.927354821' wc/f
$ svn st wc # timestamp equal so modification not detected
$
--
Philip Martin
WANdisco
RE: AW: Timestamp issue with "svn status" in 1.9.3
Posted by Peter Klotz <Pe...@ith-icoserve.com>.
Hello Philip
> > Instead of "why is the second change not reported" it should be "why
> > is the first change reported".
> >
> > Relevant part of the script:
>
> At this point Subversion has recorded a timestamp for the file.
>
> > echo "2" > x
> > touch -m -t "201212310000.00" x
> > svn -q status <-- this change is detected by Subversion (why?)
>
> The recorded timestamp is not 201212310000.00 so it does not match the
> file's timestamp and Subversion detects the change.
>
> > svn commit -m "" x
>
> Subversion records the current timestamp 201212310000.00.
>
> > echo "3" > x
>
> The file's timestamp is not 201212310000.00 so it does not match the
> recorded timestamp and Subversion detects the change.
>
> > touch -m -t "201212310000.00" x
> > svn -q status <-- this change is not detected by Subversion (as
> expected)
>
> The recorded timestamp and the file's timestamp are both
> 201212310000.00
> and Subversion does not detect the change.
Thank you for explaining what happens here!
I falsely assumed that "svn status" looks for "newer than recorded" timestamps instead of "different than recorded" timestamps. So it of course makes sense that the first change is reported and the second is not.
Regards, Peter.
Re: AW: Timestamp issue with "svn status" in 1.9.3
Posted by Philip Martin <ph...@codematters.co.uk>.
Peter Klotz <Pe...@ith-icoserve.com> writes:
> Instead of "why is the second change not reported" it should be "why
> is the first change reported".
>
> Relevant part of the script:
At this point Subversion has recorded a timestamp for the file.
> echo "2" > x
> touch -m -t "201212310000.00" x
> svn -q status <-- this change is detected by Subversion (why?)
The recorded timestamp is not 201212310000.00 so it does not match the
file's timestamp and Subversion detects the change.
> svn commit -m "" x
Subversion records the current timestamp 201212310000.00.
> echo "3" > x
The file's timestamp is not 201212310000.00 so it does not match the
recorded timestamp and Subversion detects the change.
> touch -m -t "201212310000.00" x
> svn -q status <-- this change is not detected by Subversion (as expected)
The recorded timestamp and the file's timestamp are both 201212310000.00
and Subversion does not detect the change.
--
Philip
AW: Timestamp issue with "svn status" in 1.9.3
Posted by Peter Klotz <Pe...@ith-icoserve.com>.
Hello Stefan
Thank you for the quick and detailed response.
> > When changing the timestamp of a file to an earlier point in time and
> leaving the file size unchanged yet altering the content, Subversion is
> not always able to detect this change.
> > The attached test case is the simplest we could come up with that
> reliably reproduces the problem.
>
> Subversion uses timestamp and size as a short-cut for detecting unmodified
> files. This saves Subversion from having to run a full content comparison
> for every file in the working copy when 'svn status' is run under what
> we'd consider "normal" conditions. You are artificially creating conditions
> which fool Subversion into believing the file was not modified. Because the
> timestamps and size checks which precede a full content comparison indicate
> the file was not modified, the full content comparison is skipped and the
> modification remains undetected.
>
> So, that's why you're seeing this behaviour. What's unclear is why you are
> investigating this behaviour in the first place. What is your use case for
> changing the file's content but leaving it's timestamp unchanged?
The reason for investigating this situation was because a tool did exactly what touch does in the script. A timestamp was changed to a value in the past and a content change went undetected.
Unfortunately the content change included a year (2015 -> 2016) and a version number (3.0 -> 4.0), so the file size did not change.
I see the point in Subversion using timestamps to do a quick check for changed files.
What puzzles me (see the script) is that the first change of the file (including a change of timestamp) is detected and reported by "svn status".
However if after a commit the same procedure is repeated, Subversion does no longer detect the change.
So the question should be changed:
Instead of "why is the second change not reported" it should be "why is the first change reported".
Relevant part of the script:
echo "2" > x
touch -m -t "201212310000.00" x
svn -q status <-- this change is detected by Subversion (why?)
svn commit -m "" x
echo "3" > x
touch -m -t "201212310000.00" x
svn -q status <-- this change is not detected by Subversion (as expected)
Regards, Peter.
Re: Timestamp issue with "svn status" in 1.9.3
Posted by Stefan Sperling <st...@elego.de>.
On Thu, Sep 08, 2016 at 07:40:24AM -0700, xvdessel@gmail.com wrote:
> I believe the timestamp to have a granularity of minutes on Linux, but I'm
> not sure of this. Even if it has second precision, such merge scripts can
> beat that.
This depends on the file system being used. Microseconds are not uncommon.
> In bash code, we can easily add the use-commit-times option. But when using
> the SVN plugin in Jenkins, you cannot pass extra options like this one
> unless you change de default settings of the account.
There is a --config-option command line option which can pass any config
file option on the command line without requiring changes to files on disk.
You should be able to force the use-commit-times option like this:
svn --config-option config:miscellany:use-commit-times=yes
This option exists in Apache Subversion. The Jenkins plugin uses SVNKit which
is a reimplementation of Subversion written in Java, so it might not work.
The options are usually compatible, though, so it's worth a try.
Re: Timestamp issue with "svn status" in 1.9.3
Posted by xv...@gmail.com.
> So, that's why you're seeing this behaviour. What's unclear is why you
are
> investigating this behaviour in the first place. What is your use case
for
> changing the file's content but leaving it's timestamp unchanged?
We have a very clear case for this.
An automated Jenkins process must extract a set of files from SVN and
"merge" them in some way, and check in the result somehwere else.
This process checks outs the source data and the target location, then
performs the merge. The a post-build action tries to commit the resulting
merged file.
When only 1 byte is changed in a source file, the resulting merged file
does not change in length. And as this is an automated process, the merge
script is so fast that the commit occurs within the same timestamp as the
checkout.
I believe the timestamp to have a granularity of minutes on Linux, but I'm
not sure of this. Even if it has second precision, such merge scripts can
beat that.
In bash code, we can easily add the use-commit-times option. But when using
the SVN plugin in Jenkins, you cannot pass extra options like this one
unless you change de default settings of the account.
It would be very nice if a simple flag upon commit or status would allow to
force a checksum check rather than a timestamp/filesize check. Rsync is an
example that does this with the --checksum option.
Re: Timestamp issue with "svn status" in 1.9.3
Posted by Stefan Sperling <st...@elego.de>.
On Fri, Mar 04, 2016 at 12:54:42PM +0100, Peter Klotz wrote:
> Hello Subversion Developers
>
> We encountered a strange issue with Subversion 1.9.3 (running under RHEL 7). The attached script demonstrates the behavior.
>
> When changing the timestamp of a file to an earlier point in time and leaving the file size unchanged yet altering the content, Subversion is not always able to detect this change.
> The attached test case is the simplest we could come up with that reliably reproduces the problem.
>
> Regards, Peter.
Subversion uses timestamp and size as a short-cut for detecting unmodified
files. This saves Subversion from having to run a full content comparison
for every file in the working copy when 'svn status' is run under what
we'd consider "normal" conditions. You are artificially creating conditions
which fool Subversion into believing the file was not modified. Because the
timestamps and size checks which precede a full content comparison indicate
the file was not modified, the full content comparison is skipped and the
modification remains undetected.
So, that's why you're seeing this behaviour. What's unclear is why you are
investigating this behaviour in the first place. What is your use case for
changing the file's content but leaving it's timestamp unchanged?
Does the use-commit-times option help your use case, perhaps?
See http://svnbook.red-bean.com/nightly/en/svn.advanced.confarea.html#svn.advanced.confarea.opts
and search for 'use-commit-times' on this page.