You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Ed <ed...@kdtc.net> on 2010/02/05 02:55:15 UTC
[RFC] MTime functional specifications (v2.0)
Hi,
First, some commentary. I realized that the previous specification was
absolutely terrible and I do appreciate everyone's patience with me
(esp Philip Martin, Philipp Marek and Julian Foad). Their commentary
made me realize that all the while, I had a small monkey on my back.
Over the period, it had grown to become a little too heavy. Anyway,
I think I've gotten it off my back. This time around, it should be
a better piece of work (ok, at least it should be better than the
previous piece.) Still learning the tricks of the trade.
"Learning without thought is labor lost; thought without
learning is perilous. " - Confucious
Btw, thanks to Stefan for indirectly helping me to understand that
most of the stuff on that old functional specification wasn't called
for. Merge. What has that got to do with setting the 'mtime'? Why
did I think it had anything to do with 'mtime'. Or does it?
[Rhetorical question, really]
[Yes. There should be an attachment to this e-mail.]
Now, the log:
[[[
Added the mtime functional specification submission to the notes
directory for RFC.
* notes/mtime_specs.txt
Added
Patch by: Edmund Wong ed{_at_}kdtc.net
Suggested by: julianf
Reviewed by: Philipp Marek
Philip Martin
]]]
Re: [RFC] MTime functional specifications (v2.0)
Posted by Julian Foad <ju...@btopenworld.com>.
Ed, thanks for this. Here are lots of comments and questions. If you
could update the doc wherever it makes sense, think about the hard parts
and come up with proposals for them, and ask any questions wherever it's
not clear, that would be great.
On Fri, 2010-02-05, Ed wrote:
> +OUTLINE OF A
> +
> +FUNCTIONAL (BEHAVIOURAL) SPECIFICATION FOR
> +
> +MTIME PRESERVATION (ISSUE #1256)
> +
> +
> +* Introduction
> +
> +Issue #1256 was entered into the Subversion issue tracker on 24th April
> +2003 and has been opened since then. A few patches have been submitted;
> +but, have not been applied to the source code due to the insufficiency of
> +said patches in one way or another. In the meantime, users have made
> +comments on the issue and several discussions have been put forth within
> +the mailing lists; all to no conclusion.
> +
> +This functional specification which will be submitted for RFC on the dev
> +list:
> +
> + 1) To generate a practical and user-friendly resolution to the issue.
> + 2) To formulate an implementation plan to allow the developers
> + a clear-cut specification to work with.
> +
> +
> +* Current Subversion behaviour
> +
> +The mtime of any file is the time when the file is checked out, updated
> +or merged. Subversion does not save the mtime upon commit, so a file which
> +is commited loses its mtime and when checked out, the mtime is set to the
> +checkout time. This behaviour is nonconducive to users who want to keep
> +track of the mtimes of each file and/or directory.
> +
> +There is a work-around to this issue (taken from IcePic's user response
> +on IRC). Suppose we have a file called foo.txt and we wish to maintain its
> +mtime attribute. We create a pre-commit script which takes the mtime (and
> +other information) of foo.txt and store it in another file, say mtime.txt
> +and then commit foo.txt plus mtime.txt. Then when updating or merging,
> +another script grabs foo.txt and mtime.txt and then gets and saves the
> +original mtime for foo.txt as retrieved from mtime.txt. This is not
> +efficient and quite a tedious way of working around this issue.
> +
> +* High-level semantics we are trying to achieve:
> +
> + - Whenever Subversion puts or modifies a file (or directory) in
> + the WC, it shall set the node's mtime in the WC to the mtime
> + recorded for that node as given by the server.
No, I don't agree. For example, when "svn merge" or "svn patch" modifies
a file, I think Subversion should not set the node's mtime to the mtime
recorded for that node. I would it expect it to use the current time,
probably. Do you agree? Are there other similar situations?
Let's have a "Definitions within this document" section.
Can we define "modifies a file" and "modifies a directory"? How about:
- Subversion "modifies" a file when Subversion changes the file's text
(i.e. content), even if it is just updating the file's keywords or EOL
style.
- Subversion "modifies" a directory ... never.
I don't know if those definitions are good, they're just some
definitions to serve as a starting point to decide if they're good.
These assume that we don't want to count property modifications as part
of the data that mtime refers to, but another option would be to count
property mods too. For directories, another option would be to include
changes to the list child entries.
> + - Whenever Subversion modifies a file (or directory) in the WC, and
> + there are local changes to that node, such as when updating a file
> + that the user has been editing, it shall update the mtime property.
The modification could cause local changes to exist or to stop existing.
To clarify, say "[...], and *after this modification* there are local
changes [...]".
"It shall update the mtime property" ... to what? The current time, I
presume. And I presume it should set the node's mtime to the current
time as well.
For a file with local mods, this rule conflicts with the first rule
above: if Subversion modifies such a file, does it set the node's mtime
to the recorded mtime or set the recorded mtime to the node's mtime?
Please clarify.
> + - Backward compatibility issues:
> +
> + o If the mtime/origmtime property hasn't been set for a node
What's "origmtime" or "origtime"? They are not mentioned until this
point.
> + (most probably because it was stored in the repository
> + prior to this feature being implemented), the mtime/origtime
> + can be set to the current date of modification.
We'll need to specify what cases this rule will apply to, but at first
it's more important to define how it works among clients that have
implemented this support.
> +* Specification of the behaviour in all the cases:
> +
> +Data Storage:
> +
> + Mtime shall be stored in a versioned property named 'svn:mtime'.
> + Any file or directory may have this property. The format of the
> + property value is 'YYYY-MM-DDTHH:MM:SS.UUUUUU' of which the time
> + is UTC.
> +
> +Behaviour of Each Action:
> +
> + The behaviour of each svn action that may affect a node in the
> + WC, is for x where x is a member of {file, dir}:
> +
> + CT = current time
> + M(x) = mtime of x
> + R(x) = recorded mtime 'svn:mtime'
> +
> + All initial values prior to the actions are set to NULL and it is
> + assumed that all the following functions are done with all the
> + necessary checking, such that as an example, after doing a
> + 'svn add', svn will complain if the user repeats the command.
Talking of "svn add", can you add it to this list? And "copy", "update",
and "merge". According to the "High Level Semantics" section, all of
these commands (and more, probably) should have some effect.
> + - import
> + Let f_import(x) be the following process:
> + 1a) if x is already versioned, exit.
> + 1b) Otherwise, get M(x)
> + 2) Set R(x) = M(x)
> +
> + o if x = file, then f_import(x).
> + o if x = dir, then recursively f_import(x)
OK.
> + - checkout
> + Let f_checkout(x) be the following process:
> + 1) Get R(x)
> + 2) Set M(x) = R(x)
> +
> + o if x = file, then f_checkout(x).
> + o if x = dir, then recursively f_checkout(x)
OK.
> + - commit
> + Let f_commit(x) be the following process:
> + 1) Get M(x)
> + 2) Set R(x) = M(x)
> +
> + o if x = file, then f_commit(x).
> + o if x = dir, then recursively f_commit(x)
Potentially OK, but it doesn't seem to match up very well with how I
understood the "High Level Semantics" section.
> + - rename
> + [Feature not yet available. Equivalent to
> + a) svn cp x y
> + b) svn del x
> + but there should be a historical link between
> + y and x. See issue #898.]
> + i.e. svn rename x y
> + Let f_rename(x,y) be the following process:
> +
> + 1) Get R(x).
> + 2) Set R(y) = R(x)
OK.
> +
> + If the 'svn:mtime' property does not exist for a file or
> + directory, it means either it (file or directory) is not
> + versioned, or it existed in the repository and WC before
> + this feature was created. Since the original information
> + pertaining to the file/directory is lost, the options are
> + either to store the current mtime as the original mtime
> + or completely ignore the 'svn:mtime' property for this
> + file or directory. This functional specification takes
> + the tact of setting the 'svn:mtime' property to the current
("takes the tack")
> + mtime as it will give the user at least a starting point
> + to which to make their statistical/informational mtime references.
I'm not sure whether we'll want it to do that by default. We can come
back to this after we've got the basic operation solid.
> +* Controlling the behaviour:
> +
> + Since 'svn:mtime' is a property, its behaviour can be controlled
> + with any command that deals with properties, i.e. svn pset.
What we want to know about controlling the behaviour is how we can
command Subversion to start using svn:mtime properties or to stop using
them, and at what granularity (per user, per repository, per directory,
per file?). For example, a software developer would often want to be
able to specify that the properties shall not be used (at least for
updates), whereas a documentation person using the same repository would
probably want them enabled.
Look at the way that optional features such as "svn:ignore", auto-props
and use-commit-times are controlled. There is a mixture of client-config
options, per-directory/per-file properties, and command-line switches.
> + For instance, the user wanting to set the 'svn:mtime' for an
> + already versioned file which has none of the 'svn:mtime' properties
> + set.
> +
> + svn pset svn:mtime '11-11-2009T12:22:00.000000' foo.txt
> +
> + "How does it interact with the "use-commit-times" option?"
> +
> + With this option, use-commit-times will override all Subversion
> + conditions as mentioned above.
Sorry, I don't understand that sentence. As far as I can see,
use-commit-times was not mentioned above. Please give more details
(maybe in the detailed description section).
> +* Concerns
> +
> + From the desc4 of Issue #1256,
> +
> + " 1) Preserving file timestamps on import and add, so when
> + the file is checked out again, the working file gets the
> + same timestamp as when it was imported/added. "
> +
> + When it means 'added', the author takes the position of when
> + the file/directory is 'committed'. The rationale is as
> + follows. The question is whether the user wishes to keep the
> + mtime of the add or is the commit 'mtime' sufficient enough?
> + This stems from the fact that the scheduled 'add' datetime
> + is not the same as the actual commit datetime. A developer
> + can schedule an add and one day later, commit the actual changes.
> + Should there be another option for allowing the user to use the
> + add 'mtime' and not the commit 'mtime'? If the add 'mtime'
> + is requested, where is this information stored? It does
> + require a temporary storage. This convolutes the whole
> + issue and since only one 'mtime' was requested, it would
> + simply matters by just using the commit 'mtime'.
It sounds like you are saying that the combination of "svn add FILE"
followed by "svn commit FILE" should lead to the file's "mtime" property
recording the time of the commit.
When an unversioned file is added, and thus becomes versioned, its mtime
already exists (in the operating system) and I believe that is the time
we want to record as the file's "mtime" property. I don't understand why
we would want to record the commit time instead.
Note: The commit time is already recorded for every commit, in the
"svn:date" revision property.
- Julian
RE: [RFC] MTime functional specifications (v2.0)
Posted by "Ph. Marek" <ph...@emerion.com>.
Hello Julian!
>> > Due to the potentially ubiquitous nature of this property, some
>> > consideration needs to be given to when and where the mtime property
>> > modification is reported for the affected files and folders.
>> >
>> > Sometime? Always? Never?
>>
>> With the current ra layer the answer would probably be: Always.
>
> I don't think the question wanted a simple answer. The point is that the
> design of this feature must consider whether it would often cause
> "noise" by causing mtime changes to be reported when they are not
> interesting.
>
> There are two sides to this. First on the repository side: if the design
> would often cause mtime property changes to be committed on files that
> have not otherwise changed, then, because of the fact that the current
> RA layer reporting mechanisms would just report "this file has changed",
> that would be bad. That needs to be considered especially after updates
> and reverts, I should think.
In my branch that would be a non-issue.
As the property would be changed on a commit, only files that get
committed (ie. that are really changed, or at least have other
property changes) would have a changed svn:text-time.
> The other half of the issue is when svn is reporting local WC changes in
> a simple "svn status" or "svn diff" command: how does the design ensure
> that only useful changes are reported here?
The same happens locally - the changed svn:text-time is simply not
seen, because it's just a property that gets sent along to the server.
Of course, that means that a mtime-only change (think "touch") cannot
easily be committed.
Regards,
Phil
RE: [RFC] MTime functional specifications (v2.0)
Posted by "Ph. Marek" <ph...@emerion.com>.
Hello Julian!
>> > Due to the potentially ubiquitous nature of this property, some
>> > consideration needs to be given to when and where the mtime property
>> > modification is reported for the affected files and folders.
>> >
>> > Sometime? Always? Never?
>>
>> With the current ra layer the answer would probably be: Always.
>
> I don't think the question wanted a simple answer. The point is that the
> design of this feature must consider whether it would often cause
> "noise" by causing mtime changes to be reported when they are not
> interesting.
>
> There are two sides to this. First on the repository side: if the design
> would often cause mtime property changes to be committed on files that
> have not otherwise changed, then, because of the fact that the current
> RA layer reporting mechanisms would just report "this file has changed",
> that would be bad. That needs to be considered especially after updates
> and reverts, I should think.
In my branch that would be a non-issue.
As the property would be changed on a commit, only files that get
committed (ie. that are really changed, or at least have other
property changes) would have a changed svn:text-time.
> The other half of the issue is when svn is reporting local WC changes in
> a simple "svn status" or "svn diff" command: how does the design ensure
> that only useful changes are reported here?
The same happens locally - the changed svn:text-time is simply not
seen, because it's just a property that gets sent along to the server.
Of course, that means that a mtime-only change (think "touch") cannot
easily be committed.
Regards,
Phil
RE: [RFC] MTime functional specifications (v2.0)
Posted by Julian Foad <ju...@wandisco.com>.
> > Ed wrote:
> > > Btw, regarding the RFC I submitted. Was I supposed to send it as
> > > a diff, or a text file? (I realize it is a moot point right now,
> > > but for future reference, I think it would be nice to get this
> > > clarified. :)).
As we don't have a base version in the repository yet, for any updates I
would prefer you to send both the diff and the new full text.
Bert Huijben wrote:
> Geoff Rowell wrote:
> > Due to the potentially ubiquitous nature of this property, some
> > consideration needs to be given to when and where the mtime property
> > modification is reported for the affected files and folders.
> >
> > Sometime? Always? Never?
>
> With the current ra layer the answer would probably be: Always.
I don't think the question wanted a simple answer. The point is that the
design of this feature must consider whether it would often cause
"noise" by causing mtime changes to be reported when they are not
interesting.
There are two sides to this. First on the repository side: if the design
would often cause mtime property changes to be committed on files that
have not otherwise changed, then, because of the fact that the current
RA layer reporting mechanisms would just report "this file has changed",
that would be bad. That needs to be considered especially after updates
and reverts, I should think.
The other half of the issue is when svn is reporting local WC changes in
a simple "svn status" or "svn diff" command: how does the design ensure
that only useful changes are reported here?
- Julian
[...]
RE: [RFC] MTime functional specifications (v2.0)
Posted by Julian Foad <ju...@wandisco.com>.
> > Ed wrote:
> > > Btw, regarding the RFC I submitted. Was I supposed to send it as
> > > a diff, or a text file? (I realize it is a moot point right now,
> > > but for future reference, I think it would be nice to get this
> > > clarified. :)).
As we don't have a base version in the repository yet, for any updates I
would prefer you to send both the diff and the new full text.
Bert Huijben wrote:
> Geoff Rowell wrote:
> > Due to the potentially ubiquitous nature of this property, some
> > consideration needs to be given to when and where the mtime property
> > modification is reported for the affected files and folders.
> >
> > Sometime? Always? Never?
>
> With the current ra layer the answer would probably be: Always.
I don't think the question wanted a simple answer. The point is that the
design of this feature must consider whether it would often cause
"noise" by causing mtime changes to be reported when they are not
interesting.
There are two sides to this. First on the repository side: if the design
would often cause mtime property changes to be committed on files that
have not otherwise changed, then, because of the fact that the current
RA layer reporting mechanisms would just report "this file has changed",
that would be bad. That needs to be considered especially after updates
and reverts, I should think.
The other half of the issue is when svn is reporting local WC changes in
a simple "svn status" or "svn diff" command: how does the design ensure
that only useful changes are reported here?
- Julian
[...]
RE: [RFC] MTime functional specifications (v2.0)
Posted by Bert Huijben <be...@qqmail.nl>.
> -----Original Message-----
> From: Geoff Rowell [mailto:geoff.rowell@gmail.com]
> Sent: zaterdag 6 februari 2010 14:26
> To: Ed
> Cc: Philipp Marek; dev@subversion.apache.org
> Subject: Re: [RFC] MTime functional specifications (v2.0)
> >
> > I still feel that if this specification goes through RFC and is ok'd,
> > then wouldn't it be easier to take your implementation and modify it
> > to whatever the revised specification states; wouldn't this be a lot
> > easier? While the specification itself is bite sized, I don't think
> > the actual implementation is. (or is it?) Was your solution bite-
> sized?
> >
> > Btw, regarding the RFC I submitted. Was I supposed to send it as
> > a diff, or a text file? (I realize it is a moot point right now,
> > but for future reference, I think it would be nice to get this
> > clarified. :)).
> >
> Due to the potentially ubiquitous nature of this property, some
> consideration needs to be given to when and where the mtime property
> modification is reported for the affected files and folders.
>
> Sometime? Always? Never?
With the current ra layer the answer would probably be: Always.
In functions like 'svn status' and 'svn log' we just get the boolean
indicating that one or more properties are changed. Providing more
information would require extensive RA layer changes.
We have some existing issues that indicate that this is a known problem for
'svn:mergeinfo' and you will most likely see only more changes for the
timestamp property. svn log would only indicate that the file was changed..
not if it is a textual change, a property change or a timestamp change.
(If repository and client are 1.7+ you could see if the change is on a
property or textual; but we are not there yet)
I would suggest that if/when we are going to implement a change from this
change-reporting, that we should also look to svn:mergeinfo.
Bert
Re: [RFC] MTime functional specifications (v2.0)
Posted by Geoff Rowell <ge...@gmail.com>.
On Fri, Feb 5, 2010 at 9:42 PM, Ed <ed...@kdtc.net> wrote:
> Philipp Marek wrote:
>>
>> Hello Ed!
>>
>> On Freitag, 5. Februar 2010, Ed wrote:
>>>
>>> As usual, thanks for the prompt comments.
>>
>> I think I have to apologize - I seem to cause confusion.
>
> No. No need for apologies. Just my tendency to be
> confused easily. :)
>
>>
>>> All of those patches I was referring to were your patches as included in
>>> the issue 1256 attachment list. Just a cursory query. If there exists
>>> a full implementation of 'mtime', (I really should look at the contents
>>> of the branches more often) this means it's already been implemented.
>>
>> *Something* is already implemented.
>> But that's not accepted into mainline, because there was just my
>> implementation, and no consensus about the specification.
>> (Mostly because nobody had time at that time, that was around 0.35, so
>> before the 1.0 release)
>
> I hope there's some sort of consensus this time around.
>
>>
>>> Is the point of a functional spec to flush out the details of the
>>> implementation? Or should I just follow the implementation and
>>> apply the functional specs according to what is implemented. (Or does
>>> this make any sense? )
>>
>> No, a specification should be done first.
>> I had one, for the behaviour I needed - which I implemented then.
>> So my "description" of the branch is the "specification" of it, too.
>>
>> But it's entirely possible that other behaviour is wanted - that's why you
>> re-
>> open the discussion, after all!
>
> Is my spec similar to yours? With Bert's comments in mind, the
> current spec should be changed to reflect that if the mtime property
> doesn't exist then svn shouldn't even bother with adding it. It
> only adds mtime properties to newly imported/added files.
>
>
>>
>>
>>>>>
>>>>> + - Whenever Subversion modifies a file (or directory) in the WC,
>>>>> and
>>>>> + there are local changes to that node, such as when updating a
>>>>> file + that the user has been editing, it shall update the mtime
>>>>> property.
>>>>
>>>> This isn't implemented in the branch, AFAIR.
>>>
>>> I wasn't really sure about this point. I'm assuming that the user
>>> had saved some changes (motto: save often :)), so then the actual
>>> on-disk mtime would change. Then if the user updates his/her WC,
>>> here's where I'm iffy.
>>
>> Sorry, I meant *exactly* that it's (AFAIR) not implemented. It makes a lot
>> of sense, but I fear that the branch will just "apply" the newly fetched
>> R'(x).
>
> It brings me to thinking would the operating system permit svn to update
> the mtime M(x) = R'(x) if the user is still editing the file? Would
> there be an access denied error?
>
>>> If user A edits x with mtime M(x) and repository mtime of R(x), then
>>> user B commits a change to x and commits it then R(x) becomes R'(x).
>>> Then if user A edits his/her WC's x, thus M(x) becomes M'(x) and
>>> then updates his/her WC. Then the system will compare R'(x) and
>>> M'(x). Maybe I'm getting confused, but isn't that in itself
>>> a conflict of properties?
>>
>> In my branch the property was overwritten on the actual commit.
>> So if the user didn't manually change the property, the stored value will
>> locally still be M(x), so the update doesn't see any conflict.
>>
>> M'(x) would be generated on the commit.
>
> Good. Thanks for the clarification.
>
>>> If it's already defined in svn_props.h, then it's a lot easier to
>>> use the existing one (would users expect it to be called 'mtime',
>>> since using 'text-time' might mean that it can only be used for
>>> text files as opposed to binary files?).
>>
>> No, "text" means "data" here, as opposed to "ctime" for "inode time".
>> In the WC library there was the data often named "text", that's where it
>> comes from.
>>
>> (I'd hope that this name doesn't hurt ... and nowadays it's used in at
>> least 3 programs [svntar, fsvs, and svn+branch], so it probably shouldn't be
>> changed now).
>
> Does svntar, fsvs and svn+branch use the text-time property as if it is
> the mtime property? I'm not familiar with svntar, fsvs and svn+branch.
>
>>
>>>>>
>>>>> + ... This functional specification takes
>>>>> + the tact of setting the 'svn:mtime' property to the current
>>>>> + mtime as it will give the user at least a starting point
>>>>> + to which to make their statistical/informational mtime references.
>>>>
>>>> So there's no way to have some part of the WC with mtime-storage, and
>>>> another without? I'm thinking about generated binaries that you want to
>>>> keep.
>>>
>>> Pardon me for being a bit dense. Do you mean that generated binaries
>>> (or binaries in general?) should not have the 'mtime' stored?
>>
>> No ...
>> If you've got some source files, you *don't* want mtime stored for them,
>> so that "make" knows what to re-compile.
>
> Oh. Good point. Didn't think about the 'make' issue.
>
>> But if you want to keep the generated binaries (and especially if there's
>> some 3rd party dependence, eg. on some DLLs), you *want* the mtime - because
>> that's the easiest thing to compare over a phone line.
>
> Now I understand. Thanks.
>
>>
>> So it's entirely possible that some files shall have the mtime stored,
>> while others shouldn't.
>>
>> In my branch that was "solved" by the auto-props - I extended them to
>> apply to directories, too (someway, don't remember now), so that for certain
>> file- (and directory) specifications "svn:text-time" could be set to "yes"
>> on "svn add"; this would automatically be changed to the correct value on
>> commit.
>>
>
> I still feel that if this specification goes through RFC and is ok'd,
> then wouldn't it be easier to take your implementation and modify it
> to whatever the revised specification states; wouldn't this be a lot
> easier? While the specification itself is bite sized, I don't think
> the actual implementation is. (or is it?) Was your solution bite-sized?
>
> Btw, regarding the RFC I submitted. Was I supposed to send it as
> a diff, or a text file? (I realize it is a moot point right now,
> but for future reference, I think it would be nice to get this
> clarified. :)).
>
Due to the potentially ubiquitous nature of this property, some
consideration needs to be given to when and where the mtime property
modification is reported for the affected files and folders.
Sometime? Always? Never?
--
Geoff Rowell
geoff.rowell@gmail.com
Re: [RFC] MTime functional specifications (v2.0)
Posted by "Ph. Marek" <ph...@emerion.com>.
Hello Ed!
Ed <ed...@kdtc.net> wrote:
> Is my spec similar to yours? With Bert's comments in mind, the
> current spec should be changed to reflect that if the mtime property
> doesn't exist then svn shouldn't even bother with adding it.
Well, a few rounds of discussion more and we'll end up with my
implementation, I think ;-)
> It
> only adds mtime properties to newly imported/added files.
My implementation does that via the auto-props.
That way you can eg. say that only "*.dll", "*.exe" and "*.doc" should
get the mtime property.
>> Sorry, I meant *exactly* that it's (AFAIR) not implemented. It
>> makes a lot of sense, but I fear that the branch will just "apply"
>> the newly fetched R'(x).
>
> It brings me to thinking would the operating system permit svn to update
> the mtime M(x) = R'(x) if the user is still editing the file? Would
> there be an access denied error?
Depends on the OS and editor, I think ...
But if the user is actively working on the file, subversion does an
update, but the editor doesn't notice that and just overwrites the
file - then worse things happen than a lost mtime.
>>> If it's already defined in svn_props.h, then it's a lot easier to
>>> use the existing one (would users expect it to be called 'mtime',
>>> since using 'text-time' might mean that it can only be used for
>>> text files as opposed to binary files?).
>> No, "text" means "data" here, as opposed to "ctime" for "inode time".
>> In the WC library there was the data often named "text", that's
>> where it comes from.
>>
>> (I'd hope that this name doesn't hurt ... and nowadays it's used in
>> at least 3 programs [svntar, fsvs, and svn+branch], so it probably
>> shouldn't be changed now).
>
> Does svntar, fsvs and svn+branch use the text-time property as if it is
> the mtime property? I'm not familiar with svntar, fsvs and svn+branch.
Well, "svn+branch" means my implementation on the branch(es).
And FSVS (http://fsvs.tigris.org) and svntar
(http://svn.borg.ch/svntar/) both use that value as mtime, yes.
(After all, both were meant to be compatible with the meta-data
storage in the subversion branches).
> I still feel that if this specification goes through RFC and is ok'd,
> then wouldn't it be easier to take your implementation and modify it
> to whatever the revised specification states; wouldn't this be a lot
> easier? While the specification itself is bite sized, I don't think
> the actual implementation is. (or is it?) Was your solution bite-sized?
Maybe. The actual diff of my implementation wasn't that big, the
update patch in the issue is 273 lines.
But the changes were spread all over the code - partly because
libsvn_wc was a bit of a mess.
With 1.7 that might be much better.
In fact, I believe for 1.7 that will have to be rewritten.
> Btw, regarding the RFC I submitted. Was I supposed to send it as
> a diff, or a text file? (I realize it is a moot point right now,
> but for future reference, I think it would be nice to get this
> clarified. :)).
I don't know.
A normal text file saves the character at the start, and can easily be
applied, too.
Regards,
Phil
Re: [RFC] MTime functional specifications (v2.0)
Posted by Ed <ed...@kdtc.net>.
Philipp Marek wrote:
> Hello Ed!
>
> On Freitag, 5. Februar 2010, Ed wrote:
>> As usual, thanks for the prompt comments.
> I think I have to apologize - I seem to cause confusion.
No. No need for apologies. Just my tendency to be
confused easily. :)
>
>> All of those patches I was referring to were your patches as included in
>> the issue 1256 attachment list. Just a cursory query. If there exists
>> a full implementation of 'mtime', (I really should look at the contents
>> of the branches more often) this means it's already been implemented.
> *Something* is already implemented.
> But that's not accepted into mainline, because there was just my
> implementation, and no consensus about the specification.
> (Mostly because nobody had time at that time, that was around 0.35, so before
> the 1.0 release)
I hope there's some sort of consensus this time around.
>
>> Is the point of a functional spec to flush out the details of the
>> implementation? Or should I just follow the implementation and
>> apply the functional specs according to what is implemented. (Or does
>> this make any sense? )
> No, a specification should be done first.
> I had one, for the behaviour I needed - which I implemented then.
> So my "description" of the branch is the "specification" of it, too.
>
> But it's entirely possible that other behaviour is wanted - that's why you re-
> open the discussion, after all!
Is my spec similar to yours? With Bert's comments in mind, the
current spec should be changed to reflect that if the mtime property
doesn't exist then svn shouldn't even bother with adding it. It
only adds mtime properties to newly imported/added files.
>
>
>>>> + - Whenever Subversion modifies a file (or directory) in the WC, and
>>>> + there are local changes to that node, such as when updating a
>>>> file + that the user has been editing, it shall update the mtime
>>>> property.
>>> This isn't implemented in the branch, AFAIR.
>> I wasn't really sure about this point. I'm assuming that the user
>> had saved some changes (motto: save often :)), so then the actual
>> on-disk mtime would change. Then if the user updates his/her WC,
>> here's where I'm iffy.
> Sorry, I meant *exactly* that it's (AFAIR) not implemented. It makes a lot of
> sense, but I fear that the branch will just "apply" the newly fetched R'(x).
It brings me to thinking would the operating system permit svn to update
the mtime M(x) = R'(x) if the user is still editing the file? Would
there be an access denied error?
>> If user A edits x with mtime M(x) and repository mtime of R(x), then
>> user B commits a change to x and commits it then R(x) becomes R'(x).
>> Then if user A edits his/her WC's x, thus M(x) becomes M'(x) and
>> then updates his/her WC. Then the system will compare R'(x) and
>> M'(x). Maybe I'm getting confused, but isn't that in itself
>> a conflict of properties?
> In my branch the property was overwritten on the actual commit.
> So if the user didn't manually change the property, the stored value will
> locally still be M(x), so the update doesn't see any conflict.
>
> M'(x) would be generated on the commit.
Good. Thanks for the clarification.
>> If it's already defined in svn_props.h, then it's a lot easier to
>> use the existing one (would users expect it to be called 'mtime',
>> since using 'text-time' might mean that it can only be used for
>> text files as opposed to binary files?).
> No, "text" means "data" here, as opposed to "ctime" for "inode time".
> In the WC library there was the data often named "text", that's where it comes
> from.
>
> (I'd hope that this name doesn't hurt ... and nowadays it's used in at least 3
> programs [svntar, fsvs, and svn+branch], so it probably shouldn't be changed
> now).
Does svntar, fsvs and svn+branch use the text-time property as if it is
the mtime property? I'm not familiar with svntar, fsvs and svn+branch.
>
>>>> + ... This functional specification takes
>>>> + the tact of setting the 'svn:mtime' property to the current
>>>> + mtime as it will give the user at least a starting point
>>>> + to which to make their statistical/informational mtime references.
>>> So there's no way to have some part of the WC with mtime-storage, and
>>> another without? I'm thinking about generated binaries that you want to
>>> keep.
>> Pardon me for being a bit dense. Do you mean that generated binaries
>> (or binaries in general?) should not have the 'mtime' stored?
> No ...
> If you've got some source files, you *don't* want mtime stored for them, so
> that "make" knows what to re-compile.
Oh. Good point. Didn't think about the 'make' issue.
> But if you want to keep the generated binaries (and especially if there's some
> 3rd party dependence, eg. on some DLLs), you *want* the mtime - because that's
> the easiest thing to compare over a phone line.
Now I understand. Thanks.
>
> So it's entirely possible that some files shall have the mtime stored, while
> others shouldn't.
>
> In my branch that was "solved" by the auto-props - I extended them to apply to
> directories, too (someway, don't remember now), so that for certain file- (and
> directory) specifications "svn:text-time" could be set to "yes" on "svn add";
> this would automatically be changed to the correct value on commit.
>
I still feel that if this specification goes through RFC and is ok'd,
then wouldn't it be easier to take your implementation and modify it
to whatever the revised specification states; wouldn't this be a lot
easier? While the specification itself is bite sized, I don't think
the actual implementation is. (or is it?) Was your solution bite-sized?
Btw, regarding the RFC I submitted. Was I supposed to send it as
a diff, or a text file? (I realize it is a moot point right now,
but for future reference, I think it would be nice to get this
clarified. :)).
Edmund
Re: [RFC] MTime functional specifications (v2.0)
Posted by Peter Samuelson <pe...@p12n.org>.
[Philipp Marek]
> No, "text" means "data" here, as opposed to "ctime" for "inode time".
> In the WC library there was the data often named "text", that's where
> it comes from.
>
> (I'd hope that this name doesn't hurt ... and nowadays it's used in
> at least 3 programs [svntar, fsvs, and svn+branch], so it probably
> shouldn't be changed now).
That shouldn't really enter into consideration. Anyone who uses a
property in the 'svn:' namespace before that property is accepted by
the Subversion community knows they can expect problems.
--
Peter Samuelson | org-tld!p12n!peter | http://p12n.org/
Re: [RFC] MTime functional specifications (v2.0)
Posted by Ed <ed...@kdtc.net>.
Bert Huijben wrote:
>>> Pardon me for being a bit dense. Do you mean that generated binaries
>>> (or binaries in general?) should not have the 'mtime' stored?
>> No ...
>> If you've got some source files, you *don't* want mtime stored for them,
> so
>> that "make" knows what to re-compile.
>> But if you want to keep the generated binaries (and especially if there's
>> some
>> 3rd party dependence, eg. on some DLLs), you *want* the mtime - because
>> that's
>> the easiest thing to compare over a phone line.
>
> I would suggest making it default *not* to store these timestamps.
>
> If we are going to store them by default, a new client will introduce this
> property everywhere (even on existing repositories). Users of older clients
> can then no longer merge new revisions because this property will always
> conflict with versions on another branch. (We can't learn 1.0-1.6 clients
> that they should always just replace the old property value, when we release
> 1.7/1.8)
That makes sense. So if the mtime (or whatever) property doesn't
already exist then ignore it. Then the property should only be
applied to newly added files/directories, right?
So commit w/ mtime property iff the mtime property exists in the
repository for that file/directory.
> We really need better property conflict resolving before I would enable such
> a change.
Meaning a new issue should be filed/or followed up (if filed)?
>
>> So it's entirely possible that some files shall have the mtime stored,
> while
>> others shouldn't.
>>
>> In my branch that was "solved" by the auto-props - I extended them to
> apply
>> to
>> directories, too (someway, don't remember now), so that for certain file-
>> (and
>> directory) specifications "svn:text-time" could be set to "yes" on "svn
> add";
>> this would automatically be changed to the correct value on commit.
>
> This would be a solution to the problem mentioned above: Only change it if
> the file needs the timestamp stored, just like svn:needs-lock.
> (In that case the user decided that it wants to break certain older client
> features)
As mentioned above, would a 'better property conflict resolution' be a
pre-requisite for this or is it given the auto-props (as extended by
Philipp) be sufficient enough?
Thanks
Edmund
RE: [RFC] MTime functional specifications (v2.0)
Posted by Bert Huijben <be...@qqmail.nl>.
> -----Original Message-----
> From: Philipp Marek [mailto:philipp.marek@emerion.com]
> Sent: vrijdag 5 februari 2010 9:48
> To: dev@subversion.apache.org
> Cc: Ed
> Subject: Re: [RFC] MTime functional specifications (v2.0)
>
> Hello Ed!
>
> On Freitag, 5. Februar 2010, Ed wrote:
> > As usual, thanks for the prompt comments.
> I think I have to apologize - I seem to cause confusion.
>
> > All of those patches I was referring to were your patches as included in
> > the issue 1256 attachment list. Just a cursory query. If there exists
> > a full implementation of 'mtime', (I really should look at the contents
> > of the branches more often) this means it's already been implemented.
> *Something* is already implemented.
> But that's not accepted into mainline, because there was just my
> implementation, and no consensus about the specification.
> (Mostly because nobody had time at that time, that was around 0.35, so
> before
> the 1.0 release)
>
> > Is the point of a functional spec to flush out the details of the
> > implementation? Or should I just follow the implementation and
> > apply the functional specs according to what is implemented. (Or does
> > this make any sense? )
> No, a specification should be done first.
> I had one, for the behaviour I needed - which I implemented then.
> So my "description" of the branch is the "specification" of it, too.
>
> But it's entirely possible that other behaviour is wanted - that's why you
re-
> open the discussion, after all!
>
>
> > >> + - Whenever Subversion modifies a file (or directory) in the WC,
and
> > >> + there are local changes to that node, such as when updating a
> > >> file + that the user has been editing, it shall update the mtime
> > >> property.
> > >
> > > This isn't implemented in the branch, AFAIR.
> >
> > I wasn't really sure about this point. I'm assuming that the user
> > had saved some changes (motto: save often :)), so then the actual
> > on-disk mtime would change. Then if the user updates his/her WC,
> > here's where I'm iffy.
> Sorry, I meant *exactly* that it's (AFAIR) not implemented. It makes a lot
of
> sense, but I fear that the branch will just "apply" the newly fetched
R'(x).
>
> > If user A edits x with mtime M(x) and repository mtime of R(x), then
> > user B commits a change to x and commits it then R(x) becomes R'(x).
> > Then if user A edits his/her WC's x, thus M(x) becomes M'(x) and
> > then updates his/her WC. Then the system will compare R'(x) and
> > M'(x). Maybe I'm getting confused, but isn't that in itself
> > a conflict of properties?
> In my branch the property was overwritten on the actual commit.
> So if the user didn't manually change the property, the stored value will
> locally still be M(x), so the update doesn't see any conflict.
>
> M'(x) would be generated on the commit.
>
>
> > >> + Mtime shall be stored in a versioned property named 'svn:mtime'.
> > >> + Any file or directory may have this property. The format of the
> > >> + property value is 'YYYY-MM-DDTHH:MM:SS.UUUUUU' of which the
> time
> > >> + is UTC.
> > >
> > > Very fine. And if you change that to "svn:text-time", this is already
> > > specified in svn_props.h:
> > >
> > >
> https://svn.apache.org/repos/asf/subversion/trunk/subversion/include/svn
> _
> > >props.h
> >
> > If it's already defined in svn_props.h, then it's a lot easier to
> > use the existing one (would users expect it to be called 'mtime',
> > since using 'text-time' might mean that it can only be used for
> > text files as opposed to binary files?).
> No, "text" means "data" here, as opposed to "ctime" for "inode time".
> In the WC library there was the data often named "text", that's where it
> comes
> from.
>
> (I'd hope that this name doesn't hurt ... and nowadays it's used in at
least 3
> programs [svntar, fsvs, and svn+branch], so it probably shouldn't be
changed
> now).
>
> > >> + ... This functional specification takes
> > >> + the tact of setting the 'svn:mtime' property to the current
> > >> + mtime as it will give the user at least a starting point
> > >> + to which to make their statistical/informational mtime references.
> > >
> > > So there's no way to have some part of the WC with mtime-storage, and
> > > another without? I'm thinking about generated binaries that you want
to
> > > keep.
> >
> > Pardon me for being a bit dense. Do you mean that generated binaries
> > (or binaries in general?) should not have the 'mtime' stored?
> No ...
> If you've got some source files, you *don't* want mtime stored for them,
so
> that "make" knows what to re-compile.
> But if you want to keep the generated binaries (and especially if there's
> some
> 3rd party dependence, eg. on some DLLs), you *want* the mtime - because
> that's
> the easiest thing to compare over a phone line.
I would suggest making it default *not* to store these timestamps.
If we are going to store them by default, a new client will introduce this
property everywhere (even on existing repositories). Users of older clients
can then no longer merge new revisions because this property will always
conflict with versions on another branch. (We can't learn 1.0-1.6 clients
that they should always just replace the old property value, when we release
1.7/1.8)
We really need better property conflict resolving before I would enable such
a change.
When we would introduce this issue 'as-is' on trunk, we would start
introducing this property on the ASF repository now, as at least a few
Subversion developers use trunk for their development work.
I think that would even require changes to the post-commit mailer script
before we actually release 1.7.
(I don't think we would have used a normal svn: property for storing
mergeinfo if we knew back then what we know today.)
> So it's entirely possible that some files shall have the mtime stored,
while
> others shouldn't.
>
> In my branch that was "solved" by the auto-props - I extended them to
apply
> to
> directories, too (someway, don't remember now), so that for certain file-
> (and
> directory) specifications "svn:text-time" could be set to "yes" on "svn
add";
> this would automatically be changed to the correct value on commit.
This would be a solution to the problem mentioned above: Only change it if
the file needs the timestamp stored, just like svn:needs-lock.
(In that case the user decided that it wants to break certain older client
features)
Bert
Re: [RFC] MTime functional specifications (v2.0)
Posted by Philipp Marek <ph...@emerion.com>.
Hello Ed!
On Freitag, 5. Februar 2010, Ed wrote:
> As usual, thanks for the prompt comments.
I think I have to apologize - I seem to cause confusion.
> All of those patches I was referring to were your patches as included in
> the issue 1256 attachment list. Just a cursory query. If there exists
> a full implementation of 'mtime', (I really should look at the contents
> of the branches more often) this means it's already been implemented.
*Something* is already implemented.
But that's not accepted into mainline, because there was just my
implementation, and no consensus about the specification.
(Mostly because nobody had time at that time, that was around 0.35, so before
the 1.0 release)
> Is the point of a functional spec to flush out the details of the
> implementation? Or should I just follow the implementation and
> apply the functional specs according to what is implemented. (Or does
> this make any sense? )
No, a specification should be done first.
I had one, for the behaviour I needed - which I implemented then.
So my "description" of the branch is the "specification" of it, too.
But it's entirely possible that other behaviour is wanted - that's why you re-
open the discussion, after all!
> >> + - Whenever Subversion modifies a file (or directory) in the WC, and
> >> + there are local changes to that node, such as when updating a
> >> file + that the user has been editing, it shall update the mtime
> >> property.
> >
> > This isn't implemented in the branch, AFAIR.
>
> I wasn't really sure about this point. I'm assuming that the user
> had saved some changes (motto: save often :)), so then the actual
> on-disk mtime would change. Then if the user updates his/her WC,
> here's where I'm iffy.
Sorry, I meant *exactly* that it's (AFAIR) not implemented. It makes a lot of
sense, but I fear that the branch will just "apply" the newly fetched R'(x).
> If user A edits x with mtime M(x) and repository mtime of R(x), then
> user B commits a change to x and commits it then R(x) becomes R'(x).
> Then if user A edits his/her WC's x, thus M(x) becomes M'(x) and
> then updates his/her WC. Then the system will compare R'(x) and
> M'(x). Maybe I'm getting confused, but isn't that in itself
> a conflict of properties?
In my branch the property was overwritten on the actual commit.
So if the user didn't manually change the property, the stored value will
locally still be M(x), so the update doesn't see any conflict.
M'(x) would be generated on the commit.
> >> + Mtime shall be stored in a versioned property named 'svn:mtime'.
> >> + Any file or directory may have this property. The format of the
> >> + property value is 'YYYY-MM-DDTHH:MM:SS.UUUUUU' of which the time
> >> + is UTC.
> >
> > Very fine. And if you change that to "svn:text-time", this is already
> > specified in svn_props.h:
> >
> > https://svn.apache.org/repos/asf/subversion/trunk/subversion/include/svn_
> >props.h
>
> If it's already defined in svn_props.h, then it's a lot easier to
> use the existing one (would users expect it to be called 'mtime',
> since using 'text-time' might mean that it can only be used for
> text files as opposed to binary files?).
No, "text" means "data" here, as opposed to "ctime" for "inode time".
In the WC library there was the data often named "text", that's where it comes
from.
(I'd hope that this name doesn't hurt ... and nowadays it's used in at least 3
programs [svntar, fsvs, and svn+branch], so it probably shouldn't be changed
now).
> >> + ... This functional specification takes
> >> + the tact of setting the 'svn:mtime' property to the current
> >> + mtime as it will give the user at least a starting point
> >> + to which to make their statistical/informational mtime references.
> >
> > So there's no way to have some part of the WC with mtime-storage, and
> > another without? I'm thinking about generated binaries that you want to
> > keep.
>
> Pardon me for being a bit dense. Do you mean that generated binaries
> (or binaries in general?) should not have the 'mtime' stored?
No ...
If you've got some source files, you *don't* want mtime stored for them, so
that "make" knows what to re-compile.
But if you want to keep the generated binaries (and especially if there's some
3rd party dependence, eg. on some DLLs), you *want* the mtime - because that's
the easiest thing to compare over a phone line.
So it's entirely possible that some files shall have the mtime stored, while
others shouldn't.
In my branch that was "solved" by the auto-props - I extended them to apply to
directories, too (someway, don't remember now), so that for certain file- (and
directory) specifications "svn:text-time" could be set to "yes" on "svn add";
this would automatically be changed to the correct value on commit.
Regards,
Phil
Re: [RFC] MTime functional specifications (v2.0)
Posted by Ed <ed...@kdtc.net>.
Hi Philipp,
As usual, thanks for the prompt comments.
>> +Issue #1256 was entered into the Subversion issue tracker on 24th April
>> +2003 and has been opened since then. A few patches have been submitted;
> There's even a branch with a full implementation in the subversion
> repository, and it's been ported to relatively current versions - maybe
> that's what you mean by "patches" here.
All of those patches I was referring to were your patches as included in
the issue 1256 attachment list. Just a cursory query. If there exists
a full implementation of 'mtime', (I really should look at the contents
of the branches more often) this means it's already been implemented.
Is the point of a functional spec to flush out the details of the
implementation? Or should I just follow the implementation and
apply the functional specs according to what is implemented. (Or does
this make any sense? )
>
>> + - Whenever Subversion modifies a file (or directory) in the WC, and
>> + there are local changes to that node, such as when updating a file
>> + that the user has been editing, it shall update the mtime property.
> This isn't implemented in the branch, AFAIR.
I wasn't really sure about this point. I'm assuming that the user
had saved some changes (motto: save often :)), so then the actual
on-disk mtime would change. Then if the user updates his/her WC,
here's where I'm iffy.
If user A edits x with mtime M(x) and repository mtime of R(x), then
user B commits a change to x and commits it then R(x) becomes R'(x).
Then if user A edits his/her WC's x, thus M(x) becomes M'(x) and
then updates his/her WC. Then the system will compare R'(x) and
M'(x). Maybe I'm getting confused, but isn't that in itself
a conflict of properties? (Mind you, this doesn't necessarily mean
a conflict of file content since there are updated non-conflict
situations.) I think if there is a conflict in file content then
the user has a bigger problem to resolve.
t
>
>> + Mtime shall be stored in a versioned property named 'svn:mtime'.
>> + Any file or directory may have this property. The format of the
>> + property value is 'YYYY-MM-DDTHH:MM:SS.UUUUUU' of which the time
>> + is UTC.
> Very fine. And if you change that to "svn:text-time", this is already
> specified in svn_props.h:
> https://svn.apache.org/repos/asf/subversion/trunk/subversion/include/svn_props.h
If it's already defined in svn_props.h, then it's a lot easier to
use the existing one (would users expect it to be called 'mtime',
since using 'text-time' might mean that it can only be used for
text files as opposed to binary files?).
>
>> + ... This functional specification takes
>> + the tact of setting the 'svn:mtime' property to the current
>> + mtime as it will give the user at least a starting point
>> + to which to make their statistical/informational mtime references.
> So there's no way to have some part of the WC with mtime-storage, and another
> without? I'm thinking about generated binaries that you want to keep.
Pardon me for being a bit dense. Do you mean that generated binaries
(or binaries in general?) should not have the 'mtime' stored?
Thanks,
Edmund
Re: [RFC] MTime functional specifications (v2.0)
Posted by Philipp Marek <ph...@emerion.com>.
Hello Ed,
some more comments.
On Freitag, 5. Februar 2010, Ed wrote:
>+Issue #1256 was entered into the Subversion issue tracker on 24th April
> +2003 and has been opened since then. A few patches have been submitted;
There's even a branch with a full implementation in the subversion
repository, and it's been ported to relatively current versions - maybe
that's what you mean by "patches" here.
>+ - Whenever Subversion modifies a file (or directory) in the WC, and
>+ there are local changes to that node, such as when updating a file
>+ that the user has been editing, it shall update the mtime property.
This isn't implemented in the branch, AFAIR.
>+ Mtime shall be stored in a versioned property named 'svn:mtime'.
>+ Any file or directory may have this property. The format of the
>+ property value is 'YYYY-MM-DDTHH:MM:SS.UUUUUU' of which the time
>+ is UTC.
Very fine. And if you change that to "svn:text-time", this is already
specified in svn_props.h:
https://svn.apache.org/repos/asf/subversion/trunk/subversion/include/svn_props.h
>+ ... This functional specification takes
>+ the tact of setting the 'svn:mtime' property to the current
>+ mtime as it will give the user at least a starting point
>+ to which to make their statistical/informational mtime references.
So there's no way to have some part of the WC with mtime-storage, and another
without? I'm thinking about generated binaries that you want to keep.
Maybe the property should just be set (on commit) if it already exists.
(That's what the branch does.)
Thank you for working on that!
Regards,
Phil