SVN-4827 svn x-shelve gives E200014: Checksum mismatch

[Daniel Sahlberg <da...@gmail.com>]

Hi,

I've been trying to dig into shelving and bug 4827. I've been debugging the
regression test from Julian. Please excuse me if I'm making up terms for
things called something else, I'm still new at this.

As I understand it, Subversion is:
1. Checking a new WC for the shelf (.svn/experimental/shelves/v3/<encoded
name>-version.wc)
2. Applying the changes from the "normal" WC to the shelf-WC using the
delta editor

The file to be shelved is:
$Rev: 1$\n
New line\n
(where the actual change is the new line - I like Julian's self-explanatory
test code!)

The base file is :
$Rev$\n

In step 1 keyword expansion occurs so the file in the "shelf-WC":
$Rev: 1$\n

The problem seems to be that the patching in step 2 is putting the changes
in the position relative to the base file and not considering the keyword
expansion. So the file in the "shelf-WC" after patching is
$Rev: New line\n
This file matches the "actual" checksum.

The "expected" checksum is based on this file:
$Rev$\n
New line\n
(which doesn't exist except in my mail).

I guess there are two ways forward:
* To prevent keyword expansion in the "shelf-WC"
* To make sure the delta editor applies the patch relative to the expanded
keyword (in this case three bytes later in the file).

Before I try to make a patch out of this I would like to ask which way
would be prefered. It the "shelf-WC" supposed to be just like any other WC
(in that case keyword expanded), or should it receive special treatment to
work better with the delta editor?

Kind regards,
Daniel Sahlberg

Re: SVN-4827 svn x-shelve gives E200014: Checksum mismatch

[Julian Foad <ju...@apache.org>]

Daniel Sahlberg wrote:
> I've been trying to dig into shelving and bug 4827. I've been debugging 
> the regression test from Julian. Please excuse me if I'm making up terms 
> for things called something else, I'm still new at this.

Thank you for digging into this.  Sorry for being slow to respond.

> As I understand it, Subversion is:
> 1. Checking a new WC for the shelf 
> (.svn/experimental/shelves/v3/<encoded name>-version.wc)
> 2. Applying the changes from the "normal" WC to the shelf-WC using the 
> delta editor
> 
> The file to be shelved is:
> $Rev: 1$\n
> New line\n
> (where the actual change is the new line - I like Julian's 
> self-explanatory test code!)
> 
> The base file is :
> $Rev$\n
> 
> In step 1 keyword expansion occurs so the file in the "shelf-WC":
> $Rev: 1$\n
> 
> The problem seems to be that the patching in step 2 is putting the 
> changes in the position relative to the base file and not considering 
> the keyword expansion. So the file in the "shelf-WC" after patching is
> $Rev: New line\n
> This file matches the "actual" checksum.
> 
> The "expected" checksum is based on this file:
> $Rev$\n
> New line\n
> (which doesn't exist except in my mail).
> 
> I guess there are two ways forward:
> * To prevent keyword expansion in the "shelf-WC"

That's the better option.  Details follow.

> * To make sure the delta editor applies the patch relative to the 
> expanded keyword (in this case three bytes later in the file).
> 
> Before I try to make a patch out of this I would like to ask which way 
> would be prefered. It the "shelf-WC" supposed to be just like any other 
> WC (in that case keyword expanded), or should it receive special 
> treatment to work better with the delta editor?

The shelf-WC should not do keyword expansion and EOL-style conversion. 
(EOL-style conversion is the other kind of non-committable local change 
that goes side-by-side with keyword expansion.)  The design of 
shelving-v3 is that only committable changes, which means changes that 
are representable in the "editor API", are to be stored in a shelf [1]. 
The shelf-WC has no purpose other than for the shelving code to store 
changes to it and extract the same changes from it.  When a change is 
shelved from the main WC, keywords should be unexpanded and EOL-style 
normalized to "repository form".  When a change is replayed from a 
shelf-WC back to the main WC, any required keyword expansion and 
EOL-style conversion should be applied to the main WC at that point.

If I remember correctly, the interface for getting changes out of the 
main WC used the commit editor, so that must be returning keywords and 
EOL style to repository form already.  The two areas that need to be 
checked and fixed are (1) the "checkout" code (which you have observed 
is expanding keywords), which, from memory, I think probably has an 
option that can be passed in the client "config" to suppress that 
functionality; and (2) the code being used to apply the patch (coming 
from the main WC) to the shelf WC, which, from memory, is code that I 
had to write and might not have such an option and might have 
inconsistent behaviour.

If you can do anything towards making it work as intended, that would be 
great.

Don't hesitate to follow up with questions here or on the #svn-dev 
channel (which I access these days through Matrix [2], and is still also 
accessible in Freenode IRC).  I forgot I had marked your message for 
attention this time.  I usually would respond much quicker.

Thanks,
- Julian


[1] If there is ever a future enhancement to this design to enable 
shelving WC-only kinds of local changes, I envisage it would be done by 
adding another layer of "editor API" on top of the existing one, that 
has the job of supporting WC-only kinds of local changes, which could 
conceivably include keyword expansion and EOL-style.

[2] #svn-dev via Matrix: https://matrix.to/#/#freenode_#svn-dev:matrix.org