WC rewrite questions

["Miller, Eric" <Er...@amd.com>]

A few quick questions on the upcoming WC rewrite.

First - is there a spec for the proposed wc rewrite?  Perhaps my questions are already answered there.

Second - I have a problem with the way the wc adm_crawler is currently implemented in that it finishes the reporter.
In the case of the wc update routines, this effectively causes a new connection to the repository to be created for every path given to the client.
For a large number of input paths this overhead becomes the dominant factor in update times which may be orders of magnitude longer than if done over a single connection.
Will the wc rewrite address this issue and allow the update of multiple wc paths using a single RA connection?

Third - I am concerned about maintaining multi-user access with the new WC model.  Is the current POR to lock the entire WC for write operations?  If so, this could have some very negative consequences in our environment.

Thanks,
Eric

Re: WC rewrite questions

[Listman <li...@burble.net>]

On Nov 3, 2008, at 8:59 PM- Nov 3, 2008, C. Michael Pilato wrote:

> --targets is just a way to get more of what would have been command- 
> line
> arguments into Subversion without tripping OS limitations on command- 
> line
> length or argument counts.
>
> As for the original question, 'update' uses multiple RA sessions  
> because we
> don't allow RA sessions to span repositories, and the various paths  
> passed
> to svn_client_updateN() interfaces may all refer to working copy  
> items that
> came from various repositories.  Additionally, the underlying update
> mechanism is the svn_delta_editor_t interfaces -- a depth-first tree  
> crawl.
> With our current protocol, we'd have to do a significant amount of
> preliminary calculations and transmit significantly more information  
> across
> the wire to "fake" a multi-target update using a single tree crawl.   
> So
> while WC-NG might reduce some of the I/O overhead that would be  
> involved
> ultimately if we were to take this route, there's far more logic in  
> play
> here than that solely of the working copy library.
>

hmm, seems a shame that us single repository types are paying for the  
sins
of those multi repository types.. i wish there was a switch to improve  
on this,
as the OP said, it really slows things down for us..



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: WC rewrite questions

["C. Michael Pilato" <cm...@collab.net>]

--targets is just a way to get more of what would have been command-line
arguments into Subversion without tripping OS limitations on command-line
length or argument counts.

As for the original question, 'update' uses multiple RA sessions because we
don't allow RA sessions to span repositories, and the various paths passed
to svn_client_updateN() interfaces may all refer to working copy items that
came from various repositories.  Additionally, the underlying update
mechanism is the svn_delta_editor_t interfaces -- a depth-first tree crawl.
 With our current protocol, we'd have to do a significant amount of
preliminary calculations and transmit significantly more information across
the wire to "fake" a multi-target update using a single tree crawl.  So
while WC-NG might reduce some of the I/O overhead that would be involved
ultimately if we were to take this route, there's far more logic in play
here than that solely of the working copy library.

Listman wrote:
> 
> if you use the --targets option and pass in the list of objects via a
> file is this an issue?
> 
> On Oct 31, 2008, at 3:52 PM- Oct 31, 2008, Miller, Eric wrote:
> 
>>
>> A few quick questions on the upcoming WC rewrite.
>>
>> First - is there a spec for the proposed wc rewrite?  Perhaps my
>> questions are already answered there.
>>
>> Second - I have a problem with the way the wc adm_crawler is currently
>> implemented in that it finishes the reporter.
>> In the case of the wc update routines, this effectively causes a new
>> connection to the repository to be created for every path given to the
>> client.
>> For a large number of input paths this overhead becomes the dominant
>> factor in update times which may be orders of magnitude longer than if
>> done over a single connection.
>> Will the wc rewrite address this issue and allow the update of
>> multiple wc paths using a single RA connection?
>>
>> Third - I am concerned about maintaining multi-user access with the
>> new WC model.  Is the current POR to lock the entire WC for write
>> operations?  If so, this could have some very negative consequences in
>> our environment.
>>
>> Thanks,
>> Eric
>>
> 


-- 
C. Michael Pilato <cm...@collab.net>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand

Re: WC rewrite questions

[Listman <li...@burble.net>]

if you use the --targets option and pass in the list of objects via a  
file is this an issue?

On Oct 31, 2008, at 3:52 PM- Oct 31, 2008, Miller, Eric wrote:

>
> A few quick questions on the upcoming WC rewrite.
>
> First - is there a spec for the proposed wc rewrite?  Perhaps my  
> questions are already answered there.
>
> Second - I have a problem with the way the wc adm_crawler is  
> currently implemented in that it finishes the reporter.
> In the case of the wc update routines, this effectively causes a new  
> connection to the repository to be created for every path given to  
> the client.
> For a large number of input paths this overhead becomes the dominant  
> factor in update times which may be orders of magnitude longer than  
> if done over a single connection.
> Will the wc rewrite address this issue and allow the update of  
> multiple wc paths using a single RA connection?
>
> Third - I am concerned about maintaining multi-user access with the  
> new WC model.  Is the current POR to lock the entire WC for write  
> operations?  If so, this could have some very negative consequences  
> in our environment.
>
> Thanks,
> Eric
>

Re: WC rewrite questions

[Greg Stein <gs...@gmail.com>]

Hey Eric,

There are some notes at /trunk/notes/wc-ng-design. I wouldn't call it
a "spec" by any stretch. There iare some APIs for a new storage
subsystem located at libsvn_wc/. Basically, I think the WC rewrite is
much more of a research effort. I've been rewriting bits, moving it
towards a place where I can swap out the underlying pieces.

I had no plans regarding the RA connections. That's out of the scope
of what I'm doing right now, so I'd say it will continue to be an
issue.

And for your last point, I believe the new WC will have no locks at
all. That's my intent/goal, at least. So multi-user access should be
just fine.

Cheers,
-g

On Fri, Oct 31, 2008 at 8:52 AM, Miller, Eric <Er...@amd.com> wrote:
>
> A few quick questions on the upcoming WC rewrite.
>
> First - is there a spec for the proposed wc rewrite?  Perhaps my questions
> are already answered there.
>
> Second - I have a problem with the way the wc adm_crawler is currently
> implemented in that it finishes the reporter.
> In the case of the wc update routines, this effectively causes a new
> connection to the repository to be created for every path given to the
> client.
> For a large number of input paths this overhead becomes the dominant factor
> in update times which may be orders of magnitude longer than if done over a
> single connection.
> Will the wc rewrite address this issue and allow the update of multiple wc
> paths using a single RA connection?
>
> Third - I am concerned about maintaining multi-user access with the new WC
> model.  Is the current POR to lock the entire WC for write operations?  If
> so, this could have some very negative consequences in our environment.
>
> Thanks,
> Eric

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org