subversion issue 2516

[Jörg Rebenstorf <jo...@gin.de>]

I have a suggestion for a solution of issue 2516 of the subversion issue
tracker. Please comment on this and forward my suggestion to issue 2516
of the issue tracker database if feasible.

I suggest we shall have a new access method to solve this usability
problem, something that does not need existing svn repositories to be
changed and something that is not incompatible with existing svn
behavior.

The new access option should update the working copy of the primary
target "PATH..." and its externals to a certain inter-repository
consistent state, that is, to update the primary target and all of its
externals content to their appropriate version at a certain point in
time.
The new access method shall be an additional command-line option, for
example "--sync-externals", for the svn client tool's update command
like:

svn update -r212 --sync-externals [PATH...]
or
svn update -r{2015-08-26} --sync-externals [PATH...]

What this new option shall do is, that if an externals definition, that
is involved in the update procedure as-is, does not specify a revision
(or date) information then it shall use the corresponding revision
number (or date) for updating the external so that the external
repository matches the revision of the primary target in time, that is,
the user will update the working copy to that specific state of the
content what the involved repositories had stored at that point in time
(i.e. their former HEAD revision of the past). This should work for
linking via svn:externals inside the same repository as well as for
externals to other repositories.

I believe many users already have linked via svn:externals without a
given revision number for that link definition. So currently these
definitions can only be used to update to the *current* HEAD of these
externals (as defined). But I believe many also want to use this setup
to update via the externals to the HEAD of these linked repositories as
they were at that certain point in time in the past, when updating the
working copy of the repository, that includes the externals definitions,
to a specific revision.

So when an svn:externals definition states a specific revision number to
use, than this specified revision number shall still be used no matter
if --sync-externals is used or not, but else the new option should make
a difference.

I think my solution is better than the "-rPARENT" enhancement (suggested
by Karl Fogel) and similar solutions because needing to specify
something like "-rPARENT" in the externals definition implies to change
existing repositories backward in time to use that feature for existing
ones as these definitions are part of the repository in our situation.

Best regards,
Jörg Rebenstorf

Re: subversion issue 2516

[Julian Foad <ju...@btopenworld.com>]

Re: http://subversion.tigris.org/issues/show_bug.cgi?id=2516

"--revision option is not forwarded to svn:externals references"

Mark Phippard wrote:
> Julian Foad wrote:
>> No, the issue is about when head is r5678 and I request "svn update
>> -r100" then I want the externals to be updated to r100 not to r5678.
>
> OK, I was extrapolating a bit.  Is svn up (implicit HEAD) any different?  If
> HEAD is r5678 when it updates the main WC and r5680 when it gets to the
> external ... to me that is the same problem.

Hi Mark.

This is an interesting tangent.

You're talking about the problem of what I'll call 'skew' that results
if we look up the revision number corresponding to 'head' more than
once in any given repository: the look-ups can give different results,
whereas we would benefit from more consistent results if we use the
same value throughout the whole execution of the recursive update.

If 'svn up' (implicit HEAD) finds head is 5678 and then it updates an
external in the *same* repository to 'head', and looks up head again
and finds r5680, then that is a 'skew' problem.

We can and should eliminate in-repository skew by performing exactly
one 'head' look-up per repository, within any one command. (Long-lived
clients may determine the appropriate scope in some other way than
"one command".)

A similar issue that I'll lump under the general term 'skew' exists
between different repositories.

If 'svn up' (implicit HEAD) finds head is 5678 and then it updates an
external in a *different* repository to 'head', then it has to perform
a look-up of head in the different repository. It is not possible to
perform the two look-ups synchronously (unless the repositories are
both managed by some external system that makes this possible). In
general there is no correlation between the commits in each
repository, so the result of looking up 'head' in the second
repository may be 123 or 1000009 or anything, and also, independently,
it may have been committed a long time earlier or later than our r5678
was.

We could take steps to reduce cross-repository skew. For example, we
could start by determining the set of repositories to be contacted,
and trying to do 'head' look-ups in these repositories all at the same
time as closely as possible. We cannot do this completely, because
until we have performed the first level of checkouts/updates we will
not know what further repositories will be contacted by nested
external definitions.

Skew is also potentially a problem for date look-ups. I can think of
at least three cases:

    * I commited a revision rA to REPO_A and a revision rB to REPO_B
"at the same time". Updating an external defined in REPO_A at rA, it
points to REPO_B, so I look up the date of rA => DATE_A and then I
look up DATE_A in REPO_B. The inevitable clock skew between different
repository servers means that the result of looking up DATE_A in
REPO_B will unpredictably give a result that is somewhere around rB
but just as likely to be (rB - 1) if there is any clock skew at all,
and may be a few revisions either side of rB.

    * We look up a date that is approximately 'now' but is in fact
slightly later than that repository's clock, and the that repo gets
another commit, and then we look up that same date and it returns the
new commit.

    * We look up the same date twice in the same repository and the
result changes because someone modified an svn:date revision property.

The 'skew' problem may exist for any command that makes multiple
accesses to the same or different repositories -- externals are not
needed. It is related to, but different from, almost orthogonal to the
issue at hand: either issue could exist or not without materially
affecting the other.


> As I said, I think if someone really thought the current behavior was what
> they wanted then couldn't they add an explicit -r HEAD to the external
> definition?
>
> This still seems like something we could treat as a bug.

C-Mike already answered that in
subversion.tigris.org/issues/show_bug.cgi?id=2516#desc2 : "That's not
a workaround, because folks can't go back and change old svn:externals
definitions to have that -rHEAD [...]"

- Julian

Re: subversion issue 2516

[Jörg Rebenstorf <jo...@gin.de>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Mark,

If you treat the current behavior as a bug then your could suggest to
define it the other way around:

To introduce a new option --no-sync-externals to get the previous
behavior and just not to use the new option --no-sync-externals to get
the new (you would say corrected) behavior. In this case no option
- --sync-externals would be introduced.

The maintainers have to decide this.

Best regards,
Jörg Rebenstorf

On 08/26/2015 07:32 PM, Mark Phippard wrote:
> On Wed, Aug 26, 2015 at 11:06 AM, Julian Foad
> <ju...@btopenworld.com> wrote:
> 
>> Hi Mark.
>> 
>> No, the issue is about when head is r5678 and I request "svn
>> update -r100" then I want the externals to be updated to r100 not
>> to r5678.
>> 
>> 
> OK, I was extrapolating a bit.  Is svn up (implicit HEAD) any
> different? If HEAD is r5678 when it updates the main WC and r5680
> when it gets to the external ... to me that is the same problem.
> 
> As I said, I think if someone really thought the current behavior
> was what they wanted then couldn't they add an explicit -r HEAD to
> the external definition?
> 
> This still seems like something we could treat as a bug.
> 


- -- 
G.i.N. Gesellschaft für industrielle Netzwerke GmbH
Jörg Rebenstorf
Raiffeisenstr. 15
D-64347 Griesheim

Telefon: +49 6155 - 8259 - 0
Telefax: +49 6155 - 8259 - 11

E-Mail: joerg.rebenstorf@gin.de<ma...@gin.de>
Internet: http://www.gin.de

Sitz: Griesheim
Registergericht: Amtsgericht Darmstadt, HRB 5068
Gerichtsstand: Darmstadt
USt.-ID-Nr. DE 111633284
WEEE-Reg.-Nr. DE 20824942

Geschäftsführer:
Dipl.-Inform. Andreas Schoenberg
Dipl.-Ing. (FH) Kay Wuttke

- -----------------------------------------------------------------------
IMPORTANT NOTICE:
This message is intended only for the use or the
individual or entity to which it is addressed, and
may contain information that is privileged, confidential
and exempt from disclosure under applicable law.
If the reader of this message is not the intended
recipient, or the employee or agent responsible
for delivering the message to the intended recipient,
you are hereby notified that any dissemination,
distribution or copying of this communication is
strictly prohibited.
If you have received this communication in error,
please notify the sender immediately by email.
Thank you.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlXez08ACgkQcakT0GqLJ9nl+ACgjzlhPpV+Q2QG4ksH4yy4WpUZ
Ck8AoLeGtV5ARXeHou+Sxtx7f6WpN/ym
=aKfm
-----END PGP SIGNATURE-----

Re: subversion issue 2516

[Mark Phippard <ma...@gmail.com>]

On Wed, Aug 26, 2015 at 11:06 AM, Julian Foad <ju...@btopenworld.com>
wrote:

> Hi Mark.
>
> No, the issue is about when head is r5678 and I request "svn update
> -r100" then I want the externals to be updated to r100 not to r5678.
>
>
OK, I was extrapolating a bit.  Is svn up (implicit HEAD) any different?
If HEAD is r5678 when it updates the main WC and r5680 when it gets to the
external ... to me that is the same problem.

As I said, I think if someone really thought the current behavior was what
they wanted then couldn't they add an explicit -r HEAD to the external
definition?

This still seems like something we could treat as a bug.

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: subversion issue 2516

[Julian Foad <ju...@btopenworld.com>]

Hi Mark.

No, the issue is about when head is r5678 and I request "svn update
-r100" then I want the externals to be updated to r100 not to r5678.

- Julian


On 26 August 2015 at 15:48, Mark Phippard <ma...@gmail.com> wrote:
> On Wed, Aug 26, 2015 at 10:42 AM, Julian Foad <ju...@btopenworld.com>
> wrote:
>>
>> Mark Phippard wrote:
>> > Julian Foad wrote:
>> >> Jörg Rebenstorf wrote:
>> >> [...]
>> >> > The new access option should update the working copy of the primary
>> >> > target "PATH..." and its externals to a certain inter-repository
>> >> > consistent state, that is, to update the primary target and all of
>> >> > its
>> >> > externals content to their appropriate version at a certain point in
>> >> > time.
>> >> > The new access method shall be an additional command-line option, for
>> >> > example "--sync-externals", for the svn client tool's update command
>> >> > like:
>> >
>> > Why do we even need to introduce a new option here?  What would be
>> > controversial about just doing this automatically (for externals from
>> > the
>> > same repository that do not specify a revision in the external)?
>> >
>> > Isn't this just the expected and desired behavior?
>>
>> Hi, Mark.
>>
>> I don't think we can claim that the existing behaviour has been
>> obviously and totally wrong all along, and just change it. It gave
>> logical and predictable results, just not the results we think most
>> people would expect or want. People will have got used to it, and
>> while it's easy to imagine many cases where that's not desired, I can
>> also imagine there are many cases where that's now being relied on.
>
>
> Maybe I misunderstand the real issue because the issue tracker and this
> thread are focusing more on syntax solutions and assuming the problem is
> understood.
>
> Isn't this ultimately about race condition type scenarios?  I run svn up on
> my WC.  It updates it to r100 and then moves on to the externals.  In the
> meantime, more commits happened in my repository and so the externals from
> the same repository actually update to r102.
>
> I find it hard to believe people have come to rely on this and want it.  The
> results are "understandable", but not "logical and predictable".
>
> Likewise, if I specified svn up -r100 it again seems more "logical and
> predictable" if it did the same to my externals.  They could, after all,
> have -r HEAD added to them if this was not desired.
>
>>
>>
>> When something we designed clearly could never have been useful, then
>> we can call it a bug and change it, but that's not the case here.
>
>
> I think it is a bug because someone could always explicitly add -r HEAD or
> @HEAD to the external to get that behavior.
>
>
> That said, this is total drive-by feedback.  I might not have taken enough
> time to really understand the issue here.

Re: subversion issue 2516

[Mark Phippard <ma...@gmail.com>]

On Wed, Aug 26, 2015 at 10:42 AM, Julian Foad <ju...@btopenworld.com>
wrote:

> Mark Phippard wrote:
> > Julian Foad wrote:
> >> Jörg Rebenstorf wrote:
> >> [...]
> >> > The new access option should update the working copy of the primary
> >> > target "PATH..." and its externals to a certain inter-repository
> >> > consistent state, that is, to update the primary target and all of its
> >> > externals content to their appropriate version at a certain point in
> >> > time.
> >> > The new access method shall be an additional command-line option, for
> >> > example "--sync-externals", for the svn client tool's update command
> >> > like:
> >
> > Why do we even need to introduce a new option here?  What would be
> > controversial about just doing this automatically (for externals from the
> > same repository that do not specify a revision in the external)?
> >
> > Isn't this just the expected and desired behavior?
>
> Hi, Mark.
>
> I don't think we can claim that the existing behaviour has been
> obviously and totally wrong all along, and just change it. It gave
> logical and predictable results, just not the results we think most
> people would expect or want. People will have got used to it, and
> while it's easy to imagine many cases where that's not desired, I can
> also imagine there are many cases where that's now being relied on.
>

Maybe I misunderstand the real issue because the issue tracker and this
thread are focusing more on syntax solutions and assuming the problem is
understood.

Isn't this ultimately about race condition type scenarios?  I run svn up on
my WC.  It updates it to r100 and then moves on to the externals.  In the
meantime, more commits happened in my repository and so the externals from
the same repository actually update to r102.

I find it hard to believe people have come to rely on this and want it.
The results are "understandable", but not "logical and predictable".

Likewise, if I specified svn up -r100 it again seems more "logical and
predictable" if it did the same to my externals.  They could, after all,
have -r HEAD added to them if this was not desired.


>
> When something we designed clearly could never have been useful, then
> we can call it a bug and change it, but that's not the case here.
>

I think it is a bug because someone could always explicitly add -r HEAD or
@HEAD to the external to get that behavior.


That said, this is total drive-by feedback.  I might not have taken enough
time to really understand the issue here.


-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: subversion issue 2516

[Julian Foad <ju...@btopenworld.com>]

Mark Phippard wrote:
> Julian Foad wrote:
>> Jörg Rebenstorf wrote:
>> [...]
>> > The new access option should update the working copy of the primary
>> > target "PATH..." and its externals to a certain inter-repository
>> > consistent state, that is, to update the primary target and all of its
>> > externals content to their appropriate version at a certain point in
>> > time.
>> > The new access method shall be an additional command-line option, for
>> > example "--sync-externals", for the svn client tool's update command
>> > like:
>
> Why do we even need to introduce a new option here?  What would be
> controversial about just doing this automatically (for externals from the
> same repository that do not specify a revision in the external)?
>
> Isn't this just the expected and desired behavior?

Hi, Mark.

I don't think we can claim that the existing behaviour has been
obviously and totally wrong all along, and just change it. It gave
logical and predictable results, just not the results we think most
people would expect or want. People will have got used to it, and
while it's easy to imagine many cases where that's not desired, I can
also imagine there are many cases where that's now being relied on.

When something we designed clearly could never have been useful, then
we can call it a bug and change it, but that's not the case here.

Also, for people who *do* want this new behaviour all the time, they
will have been working around the issue in the past. But this
proposal, not least because of the uncertainty of date specifications,
isn't a totally uncontroversially correct fix in all cases, and so
isn't necessarily going to give the identical results those people
have been achieving in all cases. I suggest that that too is an
argument against just changing it.

Personally, I'd love to see a cleaned-up UI, but the older Subversion
gets the more important it is to preserve quirks so as not to hurt
existing users (including those using scripts).

Is that sufficiently persuasive? It's just my opinion.

- Julian

Re: subversion issue 2516

[Mark Phippard <ma...@gmail.com>]

On Wed, Aug 26, 2015 at 9:36 AM, Julian Foad <ju...@btopenworld.com>
wrote:

> Jörg Rebenstorf wrote:
> > I have a suggestion for a solution of issue 2516 [...]
>
> First, let me quote the summary line and URL of issue 2516, for easier
> reference:
>
> http://subversion.tigris.org/issues/show_bug.cgi?id=2516
>
> "--revision option is not forwarded to svn:externals references"
>
> [...]
> > The new access option should update the working copy of the primary
> > target "PATH..." and its externals to a certain inter-repository
> > consistent state, that is, to update the primary target and all of its
> > externals content to their appropriate version at a certain point in
> > time.
> > The new access method shall be an additional command-line option, for
> > example "--sync-externals", for the svn client tool's update command
> > like:
>
>

Why do we even need to introduce a new option here?  What would be
controversial about just doing this automatically (for externals from the
same repository that do not specify a revision in the external)?

Isn't this just the expected and desired behavior?

-- 
Thanks

Mark Phippard
http://markphip.blogspot.com/

Re: subversion issue 2516

[Julian Foad <ju...@btopenworld.com>]

Jörg Rebenstorf wrote:
> On 08/26/2015 03:36 PM, Julian Foad wrote:
>> http://subversion.tigris.org/issues/show_bug.cgi?id=2516

>> How shall the implementation decide whether to use a revision
>> number or a date for each relevant external?
>
> I think that you question is an implementation issue. I have already
> read in other mail threads that there are some obstacles with this.
>
>> If the user specifies a date for the parent WC, then I think the
>> only sensible option is to update the external to this date.
>
> Right, this seems to be straight forward.
>
>> But if the user specifies a revision number, then in what cases
>> could the code update an external using this revision number? When
>> a client processes an external definition, it does not necessarily
>> know whether the target is in the same repository. (The URL is not
>> sufficiently unique. Maybe it can connect to the repo and then
>> check whether the UUID is the same.) If the client does not know,
>> it must send a date rather than a revnum.
>>
>> Whenever we specify a date, that can lead to ambiguous operation
>> the commit date stamps are not strictly ordered in the repository.
>> That is OK normally, because users usually only specify a date in
>> cases where it works (where date stamps are strictly ordered).
>> However, if this feature would always convert a revnum to a date
>> for updating an external, it may run into problems.
>
> I did not want to suggest to convert a revnum to a revdate.
> Do you say that converting a revnum to a matching revnum in the
> external repository is not possible by any implementation (with
> reasonable effort)?

No, I am not saying it is not possible. I am saying that the only way
to convert a revnum in the parent repository to a matching revnum in
an external repository is first to convert it to a date and then ask
the external repository to convert that date to a revnum.

Thus the procedure looks like this (pseudo-code):

update_recursively(PARENT_URL, REV):
  svn up $PARENT_URL -r $REV
  DATE = `svn propget svn:date $PARENT_URL@$REV`
  for EXTERNAL_URL in find_non_pegged_externals():
      svn up -r{$DATE} $EXTERNAL_URL

Whenever we pass a date in the svn UI, or in an API, Subversion will
always convert it back to a revnum before using it. It does this by a
binary search of the svn:date properties.

So whether we talk about converting to a date or converting to a
foreign-repository revnum, it means the same thing, because Subversion
always converts a date to a revnum before using it.

My concern is that converting a date to a revnum isn't well defined if
the svn:date properties are not completely ordered. However, in most
repositories they probably are completely ordered. This concern is not
specific to this issue of updating externals. So that is just an edge
case we will have to accept. As far as I know, there is no other way
to do it.

> If you are right, then the new option may simply
> support revnum on the command line only in case of externals to the
> same repository and otherwise return an error. If the command involves
> externals to different repositories the user would have to use a
> revdate on the command line if he wants to use this new option so that
> no possibly dangerous implicit conversion from revnum to revdate does
> occur during processing.

No, we can just do the conversion. The conversion is not that
dangerous -- see above.

> I think checking the UUID to know if the svn:externals definition
> leads to the same repository is sufficient.
>
>> So, do we need to check whether the external target repository has
>> the same UUID, and if so then don't change from a revnum to a
>> date?
>
> See above.

So I think (if we have been using a revnum for the parent WC) we
should check the UUID and convert to a date if the UUID is different.


>> Should this new mode apply only to external definitions that do
>> not specify any revision, or also to those which explicitly
>> specify 'HEAD'? An argument in favour of *not* affecting explicit
>> 'head' definitions is that then users can start making use of that
>> distinction in future.
>
> If someone will really need such a feature, that you speak about, just
> add a new keyword (instead of HEAD) to be used in external
> definitions. You can then define that this new keyword means that the
> current head is referenced no matter if --sync-externals is given or not.
>
> But I think that noone needs such a feature urgently as this
> requirement would only derive as an additional feature to be used with
> the suggested option in the future. Noone would have forcasted that a
> distinction between -rHEAD and no revnum in externals would be usable
> with a --sync-externals option that was not defined, so noone can use
> such a distinction using the new --sync-externals option in existing
> repositories.

I agree.

>> An argument in favour of also affecting explicit 'head' definitions
>> is that there has been no distinction between implicit and explicit
>> head in the past, and so there will be repositories where some
>> externals have it explicit and others don't, arbitrarily, and it is
>> better not to start making a distinction now between forms that
>> were considered identical in the past. I think the latter argument
>> wins: it should affect both explicit and implicit head
>> definitions.
>
> Yes, I think so and agree that the latter argument wins. As I said,
> the new feature shall not be incompatible, that is, shall not redefine
> current behavior.

OK, we agree.

>> The solution should be applied uniformly across all commands that
>> can read an old revision and can recurse into externals: checkout,
>> update, switch, export, diff (if and when it supports recursing
>> into externals), etc.
>
> That would be fine.

OK, good.

Thank you.

Do you have any interest in implementing this feature, or do you know
of anyone who could do so? We would help, of course. Writing a test
case, marking it as XFail ("expected fail"), would be one good way to
start. (Even if not, it is still useful to discuss and define how the
feature should or could work, so that it will be easier for someone to
start working on it later.)

- Julian

Re: subversion issue 2516

[Jörg Rebenstorf <jo...@gin.de>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 08/26/2015 03:36 PM, Julian Foad wrote:
> Jörg Rebenstorf wrote:
>> I have a suggestion for a solution of issue 2516 [...]
> 
> First, let me quote the summary line and URL of issue 2516, for
> easier reference:
> 
> http://subversion.tigris.org/issues/show_bug.cgi?id=2516
> 
> "--revision option is not forwarded to svn:externals references"
> 
> [...]
>> The new access option should update the working copy of the
>> primary target "PATH..." and its externals to a certain
>> inter-repository consistent state, that is, to update the primary
>> target and all of its externals content to their appropriate
>> version at a certain point in time. The new access method shall
>> be an additional command-line option, for example
>> "--sync-externals", for the svn client tool's update command 
>> like:
>> 
>> svn update -r212 --sync-externals [PATH...] or svn update
>> -r{2015-08-26} --sync-externals [PATH...]
>> 
>> What this new option shall do is, that if an externals
>> definition, that is involved in the update procedure as-is, does
>> not specify a revision (or date) information then it shall use
>> the corresponding revision number (or date) for updating the
>> external so that the external repository matches the revision of
>> the primary target in time, that is, the user will update the
>> working copy to that specific state of the content what the
>> involved repositories had stored at that point in time (i.e.
>> their former HEAD revision of the past). This should work for 
>> linking via svn:externals inside the same repository as well as
>> for externals to other repositories.
>> 
>> I believe many users already have linked via svn:externals
>> without a given revision number for that link definition. So
>> currently these definitions can only be used to update to the
>> *current* HEAD of these externals (as defined). But I believe
>> many also want to use this setup to update via the externals to
>> the HEAD of these linked repositories as they were at that
>> certain point in time in the past, when updating the working copy
>> of the repository, that includes the externals definitions, to a
>> specific revision.
>> 
>> So when an svn:externals definition states a specific revision
>> number to use, than this specified revision number shall still be
>> used no matter if --sync-externals is used or not, but else the
>> new option should make a difference.
>> 
>> I think my solution is better than the "-rPARENT" enhancement
>> (suggested by Karl Fogel) and similar solutions because needing
>> to specify something like "-rPARENT" in the externals definition
>> implies to change existing repositories backward in time to use
>> that feature for existing ones as these definitions are part of
>> the repository in our situation.
>> 
>> Best regards, Jörg Rebenstorf
> 
> Hello, Jörg. Thank you for this proposal. Let me summarize how I 
> understand your suggestion, and then ask you more about the
> details.
> 
> The proposal is:
> 
> 'svn update' should have an optional new operating mode, enabled by
> some switch (you suggest '--sync-externals'), and in this mode any 
> external definition that does not specify a revision shall be
> updated to the same *revision* or *date* as its parent WC.
> 
> I agree, I think this is basically a good idea.
> 
> More thoughts:
> 
> How shall the implementation decide whether to use a revision
> number or a date for each relevant external?

I think that you question is an implementation issue. I have already
read in other mail threads that there are some obstacles with this.

> If the user specifies a date for the parent WC, then I think the
> only sensible option is to update the external to this date.

Right, this seems to be straight forward.

> But if the user specifies a revision number, then in what cases
> could the code update an external using this revision number? When
> a client processes an external definition, it does not necessarily
> know whether the target is in the same repository. (The URL is not
> sufficiently unique. Maybe it can connect to the repo and then
> check whether the UUID is the same.) If the client does not know,
> it must send a date rather than a revnum.
> 
> Whenever we specify a date, that can lead to ambiguous operation
> the commit date stamps are not strictly ordered in the repository.
> That is OK normally, because users usually only specify a date in
> cases where it works (where date stamps are strictly ordered).
> However, if this feature would always convert a revnum to a date
> for updating an external, it may run into problems.

I did not want to suggest to convert a revnum to a revdate.
Do you say that converting a revnum to a matching revnum in the
external repository is not possible by any implementation (with
reasonable effort)? If you are right, then the new option may simply
support revnum on the command line only in case of externals to the
same repository and otherwise return an error. If the command involves
externals to different repositories the user would have to use a
revdate on the command line if he wants to use this new option so that
no possibly dangerous implicit conversion from revnum to revdate does
occur during processing.
I think checking the UUID to know if the svn:externals definition
leads to the same repository is sufficient.

> So, do we need to check whether the external target repository has
> the same UUID, and if so then don't change from a revnum to a
> date?

See above.

> Should this new mode apply only to external definitions that do
> not specify any revision, or also to those which explicitly
> specify 'HEAD'? An argument in favour of *not* affecting explicit
> 'head' definitions is that then users can start making use of that 
> distinction in future.

If someone will really need such a feature, that you speak about, just
add a new keyword (instead of HEAD) to be used in external
definitions. You can then define that this new keyword means that the
current head is referenced no matter if --sync-externals is given or not.

But I think that noone needs such a feature urgently as this
requirement would only derive as an additional feature to be used with
the suggested option in the future. Noone would have forcasted that a
distinction between -rHEAD and no revnum in externals would be usable
with a --sync-externals option that was not defined, so noone can use
such a distinction using the new --sync-externals option in existing
repositories.

> An argument in favour of also affecting explicit 'head' definitions
> is that there has been no distinction between implicit and explicit
> head in the past, and so there will be repositories where some
> externals have it explicit and others don't, arbitrarily, and it is
> better not to start making a distinction now between forms that
> were considered identical in the past. I think the latter argument
> wins: it should affect both explicit and implicit head 
> definitions.

Yes, I think so and agree that the latter argument wins. As I said,
the new feature shall not be incompatible, that is, shall not redefine
current behavior.

> The solution should be applied uniformly across all commands that
> can read an old revision and can recurse into externals: checkout,
> update, switch, export, diff (if and when it supports recursing
> into externals), etc.

That would be fine.

Best regards,
Jörg Rebenstorf

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlXd92gACgkQcakT0GqLJ9nGFQCg5lJfQuUNSczfaqRKdQDXxwXm
030AoNdLVE44qyWRTI7VLIFsMwnONZd3
=o7KV
-----END PGP SIGNATURE-----

Re: subversion issue 2516

[Julian Foad <ju...@btopenworld.com>]

Jörg Rebenstorf wrote:
> I have a suggestion for a solution of issue 2516 [...]

First, let me quote the summary line and URL of issue 2516, for easier
reference:

http://subversion.tigris.org/issues/show_bug.cgi?id=2516

"--revision option is not forwarded to svn:externals references"

[...]
> The new access option should update the working copy of the primary
> target "PATH..." and its externals to a certain inter-repository
> consistent state, that is, to update the primary target and all of its
> externals content to their appropriate version at a certain point in
> time.
> The new access method shall be an additional command-line option, for
> example "--sync-externals", for the svn client tool's update command
> like:
>
> svn update -r212 --sync-externals [PATH...]
> or
> svn update -r{2015-08-26} --sync-externals [PATH...]
>
> What this new option shall do is, that if an externals definition, that
> is involved in the update procedure as-is, does not specify a revision
> (or date) information then it shall use the corresponding revision
> number (or date) for updating the external so that the external
> repository matches the revision of the primary target in time, that is,
> the user will update the working copy to that specific state of the
> content what the involved repositories had stored at that point in time
> (i.e. their former HEAD revision of the past). This should work for
> linking via svn:externals inside the same repository as well as for
> externals to other repositories.
>
> I believe many users already have linked via svn:externals without a
> given revision number for that link definition. So currently these
> definitions can only be used to update to the *current* HEAD of these
> externals (as defined). But I believe many also want to use this setup
> to update via the externals to the HEAD of these linked repositories as
> they were at that certain point in time in the past, when updating the
> working copy of the repository, that includes the externals definitions,
> to a specific revision.
>
> So when an svn:externals definition states a specific revision number to
> use, than this specified revision number shall still be used no matter
> if --sync-externals is used or not, but else the new option should make
> a difference.
>
> I think my solution is better than the "-rPARENT" enhancement (suggested
> by Karl Fogel) and similar solutions because needing to specify
> something like "-rPARENT" in the externals definition implies to change
> existing repositories backward in time to use that feature for existing
> ones as these definitions are part of the repository in our situation.
>
> Best regards,
> Jörg Rebenstorf

Hello, Jörg. Thank you for this proposal. Let me summarize how I
understand your suggestion, and then ask you more about the details.

The proposal is:

    'svn update' should have an optional new operating mode, enabled
by some switch (you suggest '--sync-externals'), and in this mode any
external definition that does not specify a revision shall be updated
to the same *revision* or *date* as its parent WC.

I agree, I think this is basically a good idea.

More thoughts:

How shall the implementation decide whether to use a revision number
or a date for each relevant external? If the user specifies a date for
the parent WC, then I think the only sensible option is to update the
external to this date. But if the user specifies a revision number,
then in what cases could the code update an external using this
revision number? When a client processes an external definition, it
does not necessarily know whether the target is in the same
repository. (The URL is not sufficiently unique. Maybe it can connect
to the repo and then check whether the UUID is the same.) If the
client does not know, it must send a date rather than a revnum.

Whenever we specify a date, that can lead to ambiguous operation the
commit date stamps are not strictly ordered in the repository. That is
OK normally, because users usually only specify a date in cases where
it works (where date stamps are strictly ordered). However, if this
feature would always convert a revnum to a date for updating an
external, it may run into problems.

So, do we need to check whether the external target repository has the
same UUID, and if so then don't change from a revnum to a date?

Should this new mode apply only to external definitions that do not
specify any revision, or also to those which explicitly specify
'HEAD'? An argument in favour of *not* affecting explicit 'head'
definitions is that then users can start making use of that
distinction in future. An argument in favour of also affecting
explicit 'head' definitions is that there has been no distinction
between implicit and explicit head in the past, and so there will be
repositories where some externals have it explicit and others don't,
arbitrarily, and it is better not to start making a distinction now
between forms that were considered identical in the past. I think the
latter argument wins: it should affect both explicit and implicit head
definitions.

The solution should be applied uniformly across all commands that can
read an old revision and can recurse into externals: checkout, update,
switch, export, diff (if and when it supports recursing into
externals), etc.

- Julian