You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by vijay <vi...@collab.net> on 2012/10/23 16:58:36 UTC
[Issue 4225] Add --include-externals option to 'svn list'
Hi,
I am working on patch to add '--include-externals' option to 'svn list'
[Issue #4225].
Let me try to answer few questions in the issue comment [1].
Please correct me if I am wrong.
1. What happens with externals inside of externals? Are they also
listed? Does
recursion stop at some point?
No. Externals inside of externals are not listed which is similar in
behavior
like other commands checkout, export. My code will read all the
externals set
under given path or URL and call svn_client_list() recursively for all
external URLs, explicitly turning off the flag 'include_externals'.
2. What happens when a repository cannot be contacted, or if authentication
fails?
In case of any external failure, notify it and move on to the next external
definition.
I have implemented "svn list --include-externals" for a given URL.
Sample command-line output:
<snip>
$ /home/vijayaguru/svn-sandbox/subversion/bin/svn ls --include-externals
--depth infinity
file:///home/vijayaguru/svn-sandbox/svn-trunk/vpath/subversion/tests/cmdline/svn-test-work/repositories/externals_tests-1
A/
A/B/
A/B/E/
A/B/E/alpha
A/B/E/beta
A/B/F/
A/B/lambda
A/C/
A/D/
A/D/G/
A/D/G/pi
A/D/G/rho
A/D/G/tau
A/D/H/
A/D/H/chi
A/D/H/omega
A/D/H/psi
A/D/gamma
A/mu
iota
Listing external items at
'file:///home/vijayaguru/svn-sandbox/svn-trunk/vpath/subversion/tests/cmdline/svn-test-work/repositories/externals_tests-1/A/B
- gamma':
gamma
Listing external items at
'file:///home/vijayaguru/svn-sandbox/svn-trunk/vpath/subversion/tests/cmdline/svn-test-work/repositories/externals_tests-1/A/C
- exdir_G':
pi
rho
tau
Listing external items at
'file:///home/vijayaguru/svn-sandbox/svn-trunk/vpath/subversion/tests/cmdline/svn-test-work/repositories/externals_tests-1/A/C
- exdir_H':
chi
omega
psi
Listing external items at
'file:///home/vijayaguru/svn-sandbox/svn-trunk/vpath/subversion/tests/cmdline/svn-test-work/repositories/externals_tests-1/A/D
- exdir_A':
B/
B/E/
B/E/alpha
B/E/beta
B/F/
B/lambda
C/
D/
D/G/
D/G/pi
D/G/rho
D/G/tau
D/H/
D/H/chi
D/H/omega
D/H/psi
D/gamma
mu
Listing external items at
'file:///home/vijayaguru/svn-sandbox/svn-trunk/vpath/subversion/tests/cmdline/svn-test-work/repositories/externals_tests-1/A/D
- exdir_A/G':
pi
rho
tau
Listing external items at
'file:///home/vijayaguru/svn-sandbox/svn-trunk/vpath/subversion/tests/cmdline/svn-test-work/repositories/externals_tests-1/A/D
- exdir_A/H':
chi
omega
psi
Listing external items at
'file:///home/vijayaguru/svn-sandbox/svn-trunk/vpath/subversion/tests/cmdline/svn-test-work/repositories/externals_tests-1/A/D
- x/y/z/blah':
E/
E/alpha
E/beta
F/
lambda
</snip>
Does the above command line output make sense?
If it is okay, I will start writing tests for it.
I have a question to implement the same for a WC path.
'svn list' will always contact the repository for a given WC path.
Can we use libsvn_wc functions here, for e.g
"svn_wc__externals_gather_definitions()"
to read the externals info from a given Working copy path?
Please share your thoughts.
Thanks & Regards,
Vijayaguru
[1] http://subversion.tigris.org/issues/show_bug.cgi?id=4225#desc2
Re: [Issue 4225] Add --include-externals option to 'svn list'
Posted by vijay <vi...@collab.net>.
On Tuesday 23 October 2012 11:42 PM, C. Michael Pilato wrote:
> On 10/23/2012 01:46 PM, Bert Huijben wrote:
>> Before looking at the details some questions about the issue space:
>>
>> How do you handle externals that are stored in svn:externals many directory
>> levels up? (Or even defined from above the current external)
>>
>> How do you handle externals between multiple repositories?
>>
>> How do you avoid infinite recursion (when an svn:external directly or
>> indirectly includes its parent)?
> I would naturally assume that 'svn ls' behaves the same as 'svn export' in
> these situations.
>
> - externals defined outside the subtree which is being listed don't
> get processed.
>
> - externals which point to other repositories get listed from those
> repositories.
>
> - infinite recursion runs infinitely. :-) That's not ideal, I
> know, but it is consistent with what happens today, if I'm not
> mistaken. Any improvements we can make here for export/checkout/
> update/etc. would likely help the list case, too.
>
My implementation behaves the same as 'svn export' in all of the above
situations.
I am getting 'svn:externals' property from svn_ra_get_dir2() and parsing it
with 'svn_wc_parse_externals_description3()' . For each external item,
it will contact
the external item URL and list contents from there.
Some other implementations of externals are using
svn_wc__externals_gather_definitions() to get externals information from
a WC path.
Can we use this function to read externals from a WC path or we should
always get
externals information from repository for a given path or URL?
Thanks & Regards,
Vijayaguru
Re: [Issue 4225] Add --include-externals option to 'svn list'
Posted by "C. Michael Pilato" <cm...@collab.net>.
On 10/23/2012 01:46 PM, Bert Huijben wrote:
> Before looking at the details some questions about the issue space:
>
> How do you handle externals that are stored in svn:externals many directory
> levels up? (Or even defined from above the current external)
>
> How do you handle externals between multiple repositories?
>
> How do you avoid infinite recursion (when an svn:external directly or
> indirectly includes its parent)?
I would naturally assume that 'svn ls' behaves the same as 'svn export' in
these situations.
- externals defined outside the subtree which is being listed don't
get processed.
- externals which point to other repositories get listed from those
repositories.
- infinite recursion runs infinitely. :-) That's not ideal, I
know, but it is consistent with what happens today, if I'm not
mistaken. Any improvements we can make here for export/checkout/
update/etc. would likely help the list case, too.
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Enterprise Cloud Development
RE: [Issue 4225] Add --include-externals option to 'svn list'
Posted by Bert Huijben <be...@qqmail.nl>.
> -----Original Message-----
> From: vijay [mailto:vijay@collab.net]
> Sent: dinsdag 23 oktober 2012 16:59
> To: Subversion Development
> Subject: [Issue 4225] Add --include-externals option to 'svn list'
>
> Hi,
>
> I am working on patch to add '--include-externals' option to 'svn list'
> [Issue #4225].
>
> Let me try to answer few questions in the issue comment [1].
> Please correct me if I am wrong.
Before looking at the details some questions about the issue space:
How do you handle externals that are stored in svn:externals many directory
levels up? (Or even defined from above the current external)
How do you handle externals between multiple repositories?
How do you avoid infinite recursion (when an svn:external directly or
indirectly includes its parent)?
I'm not sure if there is a good and/or complete implementation possible for
this 'problem'/'issue'?
[What limits should we expect from an implementation?]
Thanks for asking on the list before filling in the details.
Bert
Re: [Issue 4225] Add --include-externals option to 'svn list'
Posted by vijay <vi...@collab.net>.
On Wednesday 24 October 2012 07:16 PM, C. Michael Pilato wrote:
> On 10/24/2012 09:25 AM, vijay wrote:
>> On Wednesday 24 October 2012 02:33 AM, Ben Reser wrote:
>>> Does it make more sense to make --include-externals behave something
>>> more like --depth rather than a simple boolean?
>>>
>>> E.G. it'd behave like so:
>>>
>>> No option: current behavior externals are not processed.
>>> --include-externals or --include-externals=infinity : Fully recursive.
>>> --include-externals=immediate : Only externals defined in the repo for
>>> the target.
>
> Externals are today always generated with full recursion, because a) the
> responsibility for deciding at what depth an external working copy is
> interesting lies with the author of that externals definition, and b) today,
> we don't allow authors to make such a declaration (all externals are
> depth-infinity). The shortcomings of our externals definition format
> doesn't, however, change the assignment of responsibility here! So, no, I
> think --include-externals=DEPTH takes us down the wrong path.
>
> In fact, IIRC, our existing commands (svn export) will only process
> externals at all when run with full recursion anyway, so perhaps the
> consistent thing to do is to only honor --include-externals (as a boolean)
> when --depth=infinity for 'svn ls', too.
>
All other commands like svn export, checkout will run with
'--depth=infinity' by default.
But 'svn ls' will run with '--depth=immediates' by default.
If we do like as follows, won't it be easy to understand?
$ svn list --include-externals
The current working copy directory and all externals(including externals
inside of externals) under cwd will be listed with depth 'immediates'.
$ svn list --depth=infinity --include-externals
The current working copy directory and all externals(including externals
inside of externals) under cwd will be listed with depth 'infinity'.
It works the same way as above for 'svn list --depth=files
--include-externals'.
Thanks & Regards,
Vijayaguru
Re: [Issue 4225] Add --include-externals option to 'svn list'
Posted by vijay <vi...@collab.net>.
On Wednesday 24 October 2012 07:16 PM, C. Michael Pilato wrote:
> On 10/24/2012 09:25 AM, vijay wrote:
>> On Wednesday 24 October 2012 02:33 AM, Ben Reser wrote:
>>> Does it make more sense to make --include-externals behave something
>>> more like --depth rather than a simple boolean?
>>>
>>> E.G. it'd behave like so:
>>>
>>> No option: current behavior externals are not processed.
>>> --include-externals or --include-externals=infinity : Fully recursive.
>>> --include-externals=immediate : Only externals defined in the repo for
>>> the target.
>
> Externals are today always generated with full recursion, because a) the
> responsibility for deciding at what depth an external working copy is
> interesting lies with the author of that externals definition, and b) today,
> we don't allow authors to make such a declaration (all externals are
> depth-infinity). The shortcomings of our externals definition format
> doesn't, however, change the assignment of responsibility here! So, no, I
> think --include-externals=DEPTH takes us down the wrong path.
>
> In fact, IIRC, our existing commands (svn export) will only process
> externals at all when run with full recursion anyway, so perhaps the
> consistent thing to do is to only honor --include-externals (as a boolean)
> when --depth=infinity for 'svn ls', too.
>
All other commands like svn export, checkout will run with
'--depth=infinity' by default.
But 'svn ls' will run with '--depth=immediates' by default.
If we do like as follows, won't it be easy to understand?
$ svn list --include-externals
The current working copy directory and all externals(including externals
inside of externals) under cwd will be listed with depth 'immediates'.
$ svn list --depth=infinity --include-externals
The current working copy directory and all externals(including externals
inside of externals) under cwd will be listed with depth 'infinity'.
It works in the same way as above for 'svn list --depth=files
--include-externals'.
Thanks & Regards,
Vijayaguru
Re: [Issue 4225] Add --include-externals option to 'svn list'
Posted by "C. Michael Pilato" <cm...@collab.net>.
On 10/24/2012 09:25 AM, vijay wrote:
> On Wednesday 24 October 2012 02:33 AM, Ben Reser wrote:
>> Does it make more sense to make --include-externals behave something
>> more like --depth rather than a simple boolean?
>>
>> E.G. it'd behave like so:
>>
>> No option: current behavior externals are not processed.
>> --include-externals or --include-externals=infinity : Fully recursive.
>> --include-externals=immediate : Only externals defined in the repo for
>> the target.
Externals are today always generated with full recursion, because a) the
responsibility for deciding at what depth an external working copy is
interesting lies with the author of that externals definition, and b) today,
we don't allow authors to make such a declaration (all externals are
depth-infinity). The shortcomings of our externals definition format
doesn't, however, change the assignment of responsibility here! So, no, I
think --include-externals=DEPTH takes us down the wrong path.
In fact, IIRC, our existing commands (svn export) will only process
externals at all when run with full recursion anyway, so perhaps the
consistent thing to do is to only honor --include-externals (as a boolean)
when --depth=infinity for 'svn ls', too.
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Enterprise Cloud Development
Re: [Issue 4225] Add --include-externals option to 'svn list'
Posted by vijay <vi...@collab.net>.
On Wednesday 24 October 2012 02:33 AM, Ben Reser wrote:
> On Tue, Oct 23, 2012 at 9:58 AM, vijay <vi...@collab.net> wrote:
>> Hi,
>>
>> I am working on patch to add '--include-externals' option to 'svn list'
>> [Issue #4225].
>>
>> Let me try to answer few questions in the issue comment [1].
>> Please correct me if I am wrong.
>>
>> 1. What happens with externals inside of externals? Are they also listed?
>> Does
>> recursion stop at some point?
>>
>> No. Externals inside of externals are not listed which is similar in
>> behavior
>> like other commands checkout, export. My code will read all the externals
>> set
>> under given path or URL and call svn_client_list() recursively for all
>> external URLs, explicitly turning off the flag 'include_externals'.
> Does it make more sense to make --include-externals behave something
> more like --depth rather than a simple boolean?
>
> E.G. it'd behave like so:
>
> No option: current behavior externals are not processed.
> --include-externals or --include-externals=infinity : Fully recursive.
> --include-externals=immediate : Only externals defined in the repo for
> the target.
Yes. It makes sense and it is easy to implement.
But will it be compatible with 'svn commit' which has the same option?
We have to make sure that end users will not get confused with --depth's
depth level and
--include-externals' depth level. Our documentation should help for
users to understand
all of the following cases.
$ svn list --depth=immediates --include-externals=immediates
$ svn list --depth=immediates --include-externals
$ svn list --depth=infinity --include-externals=immediates
$ svn list --depth=infinity --include-externals
$ svn list --depth=files --include-externals=immediates
$ svn list --depth=files --include-externals
Thanks & Regards,
Vijayaguru
P.S: We can even avoid infinite recursion problem (when an svn:external
directly or
indirectly includes its parent) with 'svn list
--include-externals=immediates'.
Re: [Issue 4225] Add --include-externals option to 'svn list'
Posted by Ben Reser <be...@reser.org>.
On Tue, Oct 23, 2012 at 9:58 AM, vijay <vi...@collab.net> wrote:
> Hi,
>
> I am working on patch to add '--include-externals' option to 'svn list'
> [Issue #4225].
>
> Let me try to answer few questions in the issue comment [1].
> Please correct me if I am wrong.
>
> 1. What happens with externals inside of externals? Are they also listed?
> Does
> recursion stop at some point?
>
> No. Externals inside of externals are not listed which is similar in
> behavior
> like other commands checkout, export. My code will read all the externals
> set
> under given path or URL and call svn_client_list() recursively for all
> external URLs, explicitly turning off the flag 'include_externals'.
Does it make more sense to make --include-externals behave something
more like --depth rather than a simple boolean?
E.G. it'd behave like so:
No option: current behavior externals are not processed.
--include-externals or --include-externals=infinity : Fully recursive.
--include-externals=immediate : Only externals defined in the repo for
the target.