Re: Copies via update -- net win or net loss? (Was: svn commit: r996914 ...)

[Daniel Shahaf <d....@daniel.shahaf.name>]

C. Michael Pilato wrote on Tue, Sep 14, 2010 at 10:58:20 -0400:
> On 09/14/2010 10:25 AM, rhuijben@apache.org wrote:
> > Author: rhuijben
> > Date: Tue Sep 14 14:25:52 2010
> > New Revision: 996914
> > 
> > URL: http://svn.apache.org/viewvc?rev=996914&view=rev
> > Log:
> > When mixing the two major hacks of the update editor (copy_from location
> > and file externals), we can get into an unexpected state. Update an
> > assertion to handle this state properly and slightly update the expected
> > test result.
> 
> In light of our vision for the future regarding the pristine cache, and the
> seeming flakiness of the special-case code added to handle copies during
> update, should we kill that feature?

Could someone summarize the feature/hack/special-case being discussed
here?  Is it about using the copyfrom info to optimize some over-the-wire
transmissions during an update, as opposed to always asking for the fulltext
of a copied file?  (e.g., I found locate_copyfrom() in update-editor.c)

Thanks,

Daniel.

> I've never been convinced that it was
> truly beneficial as written anyway -- it seems to just sorta throws "what
> ifs" across the wire and then burdens the client with verification and
> handling before falling back to doing what it has always done for non-copied
> files.
> 
> -- 
> C. Michael Pilato <cm...@collab.net>
> CollabNet   <>   www.collab.net   <>   Distributed Development On Demand
> 

Re: Copies via update -- net win or net loss? (Was: svn commit: r996914 ...)

["C. Michael Pilato" <cm...@collab.net>]

> At this point, I'm not even really asking for permission any more.  If
> somebody doesn't beat me to it, I'll gut the feature myself after I wrap up
> some other things that hold my attention right now.

Tracking this work in issue #3711.

-- 
C. Michael Pilato <cm...@collab.net>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand

Re: Copies via update -- net win or net loss? (Was: svn commit: r996914 ...)

["C. Michael Pilato" <cm...@collab.net>]

On 09/15/2010 12:01 PM, Daniel Shahaf wrote:
> C. Michael Pilato wrote on Tue, Sep 14, 2010 at 10:58:20 -0400:
>> On 09/14/2010 10:25 AM, rhuijben@apache.org wrote:
>>> Author: rhuijben
>>> Date: Tue Sep 14 14:25:52 2010
>>> New Revision: 996914
>>>
>>> URL: http://svn.apache.org/viewvc?rev=996914&view=rev
>>> Log:
>>> When mixing the two major hacks of the update editor (copy_from location
>>> and file externals), we can get into an unexpected state. Update an
>>> assertion to handle this state properly and slightly update the expected
>>> test result.
>>
>> In light of our vision for the future regarding the pristine cache, and the
>> seeming flakiness of the special-case code added to handle copies during
>> update, should we kill that feature?
> 
> Could someone summarize the feature/hack/special-case being discussed
> here?  Is it about using the copyfrom info to optimize some over-the-wire
> transmissions during an update, as opposed to always asking for the fulltext
> of a copied file?  (e.g., I found locate_copyfrom() in update-editor.c)

That's the one.

The server does a little extra work to calculate a copyfrom path, a little
extra work to authorize that path.  If it all checks out, the server sends
that copyfrom info plus any additional content/prop deltas against that
copyfrom source to the client instead of just transmitting the new contents
of the copied thing in full.

If the client sees copyfrom info, it then goes to try to find that same
thing in the working copy.  If it manages to find something, it does a local
copy and then applies those extra deltas (if any) from the server.  If it
doesn't find the copyfrom source locally, it then asks the server for the
pristine version of the copyfrom source so it has something to apply those
deltas to.

Yeah.  locate_copyfrom().  Eek.

I don't doubt that in some cases, this code might save someone a bit of
network traffic.  But our vision for this code in the future involves much
more reliable, less-heuristically-driven approaches.  SHA1 checksums into a
database keyed on as much, etc.  I seriously believe we'd be better off
losing this codepath today, if only so we can simplify what is already a
very complex operation (update).

At this point, I'm not even really asking for permission any more.  If
somebody doesn't beat me to it, I'll gut the feature myself after I wrap up
some other things that hold my attention right now.

-- 
C. Michael Pilato <cm...@collab.net>
CollabNet   <>   www.collab.net   <>   Distributed Development On Demand