Improved patch format - SoC

[Nicolás Lichtmaier <ni...@reloco.com.ar>]

Hi, when I learned about SoC I started looking at the patch format task. 
But I didn't post anything and that task has been already asked. Anyway, 
near one of the deadlines, I submited a proposal. But I don't want to 
step into Charles Acknin toes, it would be great if we can cooperate in 
both, implementhing this and learning about Subversion coding. Or I can 
do some other Subversion task, if that's undesirable. Or even nothing 
=). Anyway, I'll post here some of the thoughts I had about the issue.

This is what I have written:

== Introduction ==

The diff/patch tools have been great tools. The format they use (called 
"unified diff") has become a standard, and has been integrated into many 
development processes, often with tools that provide means to attach 
patches to issues. In designing a new format, we should learn from their 
success points, those are:

 * A format which serves both for automatic patching and reviewing.
 * A format so simple that you can even edit it with a text editor.
 * A terse format. Metadata gets out of the way, so it has become the 
preferred format to review changes.

An interesting use case of a patch format is bugzilla. Bugzilla was 
designed around CVS, and it helps a community to coordinate the work by 
attaching patches to issues. Those patches are reviewed, approved and 
commited.

Just as Subversion took the best of CVS concepts and created a better 
version control system, we should take this format and enhance it so 
that it can serve Subversion needs.  Those new needs are three:

 * The ability of describe tree modifications (renames, deletions, etc.),
 * The ability to describe modifications to properties.
 * Handle binary files' modifications.

== Requirements ==

 * The format must be easily readable, and the metadata should go out of 
the way. It shouldn't be verbose. E.g: I wouldn't put information in the 
style of RFC-822 headers. The current format's terse metadata is an 
example of this.

 * The patch should convey all the meaning implied in a 'svn merge' 
operation that is considered reasonable. A patch should be like tearing 
the merge action into two steps, i.e.: 'diff + patch = merge'.

 * As the patch format should be designed so it can be used by other 
tools, all Subversion specific references should be tagged as such. This 
doesn't necesarilly mean complicating the format or overengineering it. 
It's just leaving some place for expansion (e.g. ignoring unknown 
merge-tracking info)

* The old format must keep working. This new feature should me 
implemented with a new switch. Or... currently there's no support for 
tree modifications, perhaps if there are no tree modifications the 
format would be compatible...

* Support for binary diffs. Binary diffs would just be diffs created 
with the binary diff algorithm already present in subversion, and then 
encoded in base64 to get a text representation. The algorithm name 
should be stated to allow for change in the future.

* The new format should handle properties. IMO they should be diffed as 
if each property value were a file (easily see which lines had been 
added to svn:ignore).

== Relationship with merge-tracking ==

As patching from an improved patch should work like a normal merge there 
are implications related to the merge-tracking functionality. Of course, 
this would be in case of patching in the same repository.

At first look, it seems that just including the merge info property 
would suffice. It would need to be special-cased though, to resolve the 
"elision", i.e. to include the merge info when that info being inherited 
from a parent directory.

The most common case will be that patches and WCs are all from the same 
repository, as patches are exchanged in a given coding community working 
around a common repository. Cross-repository diff+patch case should be 
studied carefuly.  I would just ignore the merge info in cross-repo 
pathes (perhaps a "force" switch could be used).  This means that the 
GUID of the repository should be included in the patch.

== Design notes ==

I don't think the new format should be 'unified diff' compatible. I see 
much more value in showing more clearly the tree modifications. E.g.: A 
file rename should be marked as a file rename. Not as the whole file 
disappearing (with '-' lines) and reappearing (with '+' lines).

Implementation idea for the above: Each diff part could have a 
'copy-from' field and a 'copy-to' field. This would allow for a clear 
display of files that have been modified after being copied.  The pure 
rename case would only be a copy-from, copy-to part.

To handle this new format, the idea would be to have a standalone 
patching program. This standalone code could be, hopefuly, later added  
by GNU patch or other tools (I wouldn't use APR for this).

== Optional future steps ==

It could be posible to have a standalone tool to convert back and forth 
the standard "unified diff" format and the new one. This tool would fail 
when converting to unified diff a patch which has tree modifications.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Improved patch format - SoC

[Nicolás Lichtmaier <ni...@reloco.com.ar>]

> Just wanted to throw out the idea that allowing comments in the patch
> file might be useful for enabling both justification for individual
> code changes and suggestions for improvement if a patch was being
> passed back and forth. If comments were supported in the format then
> there would be the potential for integration with merge GUIs etc.

It's an interesting idea. In SVK generating a patch is like commiting. 
Including in a patch format a place for adding a patch description (a 
changelog entry, bah) sounds like a nice idea. That patch description 
could be used in a future commits of that patch. I like it.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Improved patch format - SoC

[Chris Morgan <ch...@gmail.com>]

On 3/31/07, Nicolás Lichtmaier <ni...@reloco.com.ar> wrote:
> Hi, when I learned about SoC I started looking at the patch format task.
> But I didn't post anything and that task has been already asked. Anyway,
> near one of the deadlines, I submited a proposal. But I don't want to
> step into Charles Acknin toes, it would be great if we can cooperate in
> both, implementhing this and learning about Subversion coding. Or I can
> do some other Subversion task, if that's undesirable. Or even nothing
> =). Anyway, I'll post here some of the thoughts I had about the issue.
>
> This is what I have written:
>
> == Introduction ==
>
> The diff/patch tools have been great tools. The format they use (called
> "unified diff") has become a standard, and has been integrated into many
> development processes, often with tools that provide means to attach
> patches to issues. In designing a new format, we should learn from their
> success points, those are:
>
>  * A format which serves both for automatic patching and reviewing.
>  * A format so simple that you can even edit it with a text editor.
>  * A terse format. Metadata gets out of the way, so it has become the
> preferred format to review changes.
>
> An interesting use case of a patch format is bugzilla. Bugzilla was
> designed around CVS, and it helps a community to coordinate the work by
> attaching patches to issues. Those patches are reviewed, approved and
> commited.
>
> Just as Subversion took the best of CVS concepts and created a better
> version control system, we should take this format and enhance it so
> that it can serve Subversion needs.  Those new needs are three:
>
>  * The ability of describe tree modifications (renames, deletions, etc.),
>  * The ability to describe modifications to properties.
>  * Handle binary files' modifications.
>
> == Requirements ==
>
>  * The format must be easily readable, and the metadata should go out of
> the way. It shouldn't be verbose. E.g: I wouldn't put information in the
> style of RFC-822 headers. The current format's terse metadata is an
> example of this.
>
>  * The patch should convey all the meaning implied in a 'svn merge'
> operation that is considered reasonable. A patch should be like tearing
> the merge action into two steps, i.e.: 'diff + patch = merge'.
>
>  * As the patch format should be designed so it can be used by other
> tools, all Subversion specific references should be tagged as such. This
> doesn't necesarilly mean complicating the format or overengineering it.
> It's just leaving some place for expansion (e.g. ignoring unknown
> merge-tracking info)
>
> * The old format must keep working. This new feature should me
> implemented with a new switch. Or... currently there's no support for
> tree modifications, perhaps if there are no tree modifications the
> format would be compatible...
>
> * Support for binary diffs. Binary diffs would just be diffs created
> with the binary diff algorithm already present in subversion, and then
> encoded in base64 to get a text representation. The algorithm name
> should be stated to allow for change in the future.
>
> * The new format should handle properties. IMO they should be diffed as
> if each property value were a file (easily see which lines had been
> added to svn:ignore).
>
> == Relationship with merge-tracking ==
>
> As patching from an improved patch should work like a normal merge there
> are implications related to the merge-tracking functionality. Of course,
> this would be in case of patching in the same repository.
>
> At first look, it seems that just including the merge info property
> would suffice. It would need to be special-cased though, to resolve the
> "elision", i.e. to include the merge info when that info being inherited
> from a parent directory.
>
> The most common case will be that patches and WCs are all from the same
> repository, as patches are exchanged in a given coding community working
> around a common repository. Cross-repository diff+patch case should be
> studied carefuly.  I would just ignore the merge info in cross-repo
> pathes (perhaps a "force" switch could be used).  This means that the
> GUID of the repository should be included in the patch.
>
> == Design notes ==
>
> I don't think the new format should be 'unified diff' compatible. I see
> much more value in showing more clearly the tree modifications. E.g.: A
> file rename should be marked as a file rename. Not as the whole file
> disappearing (with '-' lines) and reappearing (with '+' lines).
>
> Implementation idea for the above: Each diff part could have a
> 'copy-from' field and a 'copy-to' field. This would allow for a clear
> display of files that have been modified after being copied.  The pure
> rename case would only be a copy-from, copy-to part.
>
> To handle this new format, the idea would be to have a standalone
> patching program. This standalone code could be, hopefuly, later added
> by GNU patch or other tools (I wouldn't use APR for this).
>
> == Optional future steps ==
>
> It could be posible to have a standalone tool to convert back and forth
> the standard "unified diff" format and the new one. This tool would fail
> when converting to unified diff a patch which has tree modifications.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
> For additional commands, e-mail: dev-help@subversion.tigris.org
>
>

(Replying to all this time ;-)

Just wanted to throw out the idea that allowing comments in the patch
file might be useful for enabling both justification for individual
code changes and suggestions for improvement if a patch was being
passed back and forth. If comments were supported in the format then
there would be the potential for integration with merge GUIs etc.

Chris

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Improved patch format - SoC

[Nicolás Lichtmaier <ni...@reloco.com.ar>]

> All I'm saying is, if you get the distributed rcs people involved again
> to design a One True Portable Changeset Format, I wouldn't be surprised
> to see the process bog down as everybody attempts to add their own
> requirements that have little to do with Subversion.  Everybody, that
> is, who isn't too busy peddling their own existing patchset formats, of
> which I imagine there are some.
>   

Yes, but I think that if someone comes with some logical improvements to 
the "lowest common denominator" that unidiff is, many just start using it.

The two things that need to be added to unidiff so that it can be a base 
to an improved format are: Copies and deletions. And perhaps that's as 
easy as:

--- subversion/include/svn_config.h     (revision 24348)
+++ subversion/include/svn_config2.h     (working copy)
@@ -146,8 +146,8 @@
 /** Read configuration data from @a file (a file or registry path) into
  * @a *cfgp, allocated in @a pool.
[...]

... and ...

--- subversion/include/svn_config.h     (revision 24348)
+++*deleted* (working copy)
[...]

(no space between the '*' and the '+' for differentiating this from a 
file called "*deleted*" =) )

Everything else can be added in ignoreable chunks: properties, binary 
diffs, repository information, etc.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Improved patch format - SoC

[Branko Čibej <br...@xbc.nu>]

Peter Samuelson wrote:
> [Branko ??ibej]
>   
>> No, diff looks at two files. plain 'svn diff" looks at the the file in
>> the working copy, and at what that file looked like when it was last
>> updated from the repository, e.g., it's most recent "ancestor".
>>
>> Merge, on the other hand, looks at three files: The file you're merging
>> from, the file you're merging to, and the common ancestor of these two
>> files. You could say that 'diff3 + patch == merge', but 'diff' and
>> 'diff3' don't produce the same results.
>>     
>
> No, diff + patch _also_ looks at 3 files: diff looks at 2 files, and
> patch looks at a third file.  The third file may be the same as the
> first file, similar to the 'merge -rHEAD:nnn filename' case.
>   

Please see above. merge = diff3 + patch. diff3 != diff.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Improved patch format - SoC

[Peter Samuelson <pe...@p12n.org>]

[Branko ??ibej]
> No, diff looks at two files. plain 'svn diff" looks at the the file in
> the working copy, and at what that file looked like when it was last
> updated from the repository, e.g., it's most recent "ancestor".
> 
> Merge, on the other hand, looks at three files: The file you're merging
> from, the file you're merging to, and the common ancestor of these two
> files. You could say that 'diff3 + patch == merge', but 'diff' and
> 'diff3' don't produce the same results.

No, diff + patch _also_ looks at 3 files: diff looks at 2 files, and
patch looks at a third file.  The third file may be the same as the
first file, similar to the 'merge -rHEAD:nnn filename' case.

> I was of course referring to binary diff/patch/merge. Since you can't
> do inexact binary patch/merge in general, it's not useful for code
> exchange, so you might as well not bother with a generic binary diff
> format.

I think the point of supporting binary diffs in a next-gen patch format
is that a single human-readable, human-editable format could serve some
of the same purposes dumpfiles serve now.

> One more thing ... I think it would be a big mistake if we (the
> Subversion project) invented a new diff/patch format ourselves, in
> isolation. We should make an effort to coordinate with other
> projects, like Mercurial, Arch, Git (brrrr....), OpenCM, etc. etc.,

That's been tried before, several years ago when Subversion and Arch
were young, and as I recall, it quickly broke down into a flamewar when
Tom Lord insisted that unique file IDs absolutely _had_ to be supported
(so that diffs to a file can be tracked through arbitrary renames and
merges across multiple repositories) - and several other people saw no
need for them.

All I'm saying is, if you get the distributed rcs people involved again
to design a One True Portable Changeset Format, I wouldn't be surprised
to see the process bog down as everybody attempts to add their own
requirements that have little to do with Subversion.  Everybody, that
is, who isn't too busy peddling their own existing patchset formats, of
which I imagine there are some.

Re: Improved patch format - SoC

[Branko Čibej <br...@xbc.nu>]

Nicolás Lichtmaier wrote:
>>> Just as Subversion took the best of CVS concepts and created a better
>>> version control system, we should take this format and enhance it so
>>> that it can serve Subversion needs.  Those new needs are three:
>>>
>>> * The ability of describe tree modifications (renames, deletions,
>>> etc.),
>>>     
>>
>> This is more complicated than it looks, if you insist on compatibility
>> with current diff/patch. Diff is file-based, whereas tree modifications
>> are not.
>>
>> For example, patches are separable: you can take a patch which contains
>> diffs of several files, split it apart, and apply each file's hunk
>> separately. When you add tree-modification metadata to such patches,
>> this is (in general) no longer true; and to be safe, a patch program
>> that accepts such an enhanced diff format should be able to warn you
>> that you're missing some tree modifications when you apply a patch.
>>
>> I certainly don't know how to do this by simply extending, e.g.,
>> unidiff.
>>
>> [...]
>>   
>
> Sorry, perhaps I wasn't clear. I'm not proposing a new format
> compatible with unidiff. I'm in fact arguing for the opposite. A new
> format which is "inspired" by unidiff goals, achievements and looks,
> but completely incompatible. I would even change the "---" and "+++"
> markers so as to make it clear that the new format is not compatible,
> although it might look similar. The rationale for not even trying to
> be partially compatible is not providing something that sometimes
> works and sometimes it doesn't.

Oh, all right. That makes all the difference. :)

[...]

>>> * The patch should convey all the meaning implied in a 'svn merge'
>>> operation that is considered reasonable. A patch should be like
>>> tearing the merge action into two steps, i.e.: 'diff + patch = merge'.
>>>     
>>
>> diff + patch != merge
>>
>> I don't know how that misconception came about, but merge typically
>> looks at three sources, not two (and note that there's a prototype 4-way
>> merge in Subversion's source tree).
>>   
>
> Perhaps there's something I'm not understaning. As I see it there the
> 3 ways are:
>
> 1) The common ancestor (which is the "from" revision in the diff)
> 2) The proposed end result
> 3) The current local source. All of them are present in a normal diff
> operation.

No, diff looks at two files. plain 'svn diff" looks at the the file in
the working copy, and at what that file looked like when it was last
updated from the repository, e.g., it's most recent "ancestor".

Merge, on the other hand, looks at three files: The file you're merging
from, the file you're merging to, and the common ancestor of these two
files. You could say that 'diff3 + patch == merge', but 'diff' and
'diff3' don't produce the same results.

And a good thing, too, otherwise you'd get way too many extraneous
conflicts during merges.

[...]

>>
>> If you don't have inexact patch, then the diff/patch thing becomes
>> pretty much useless for code exchange (/and/ for merging).
>>   
>
> The same as regular Subversion merges. Binary merges are aborted and
> left as an exercise for the user. That is not making Subversion
> merging useless, so I don't see why it would render a patch merging
> useles...

I was of course referring to binary diff/patch/merge. Since you can't do
inexact binary patch/merge in general, it's not useful for code
exchange, so you might as well not bother with a generic binary diff format.

By all means make your patch format extensible in such a way that binary
diffs (or deltas, a different beast) can be added, but I don't believe
actually defining and implementing something like that is feasible for a
SoC project.

-- Brane

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Improved patch format - SoC

[Branko Čibej <br...@xbc.nu>]

One more thing ... I think it would be a big mistake if we (the
Subversion project) invented a new diff/patch format ourselves, in
isolation. We should make an effort to coordinate with other projects,
like Mercurial, Arch, Git (brrrr....), OpenCM, etc. etc., and try to
come up with a format that would be at least marginally useful for
echanging patches with tree-modification data between different version
control systems.

-- Brane

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Improved patch format - SoC

[Nicolás Lichtmaier <ni...@reloco.com.ar>]

> Correction: "unified diff" is in fact a later addition to the set of
> diff formats (the original diff didn't have it), and is by no means
> universally accepted as "the standard". IIRC the GCC project, for
> example, insists that patshes be submitted in context-diff, not
> unified-diff format. Reading man diff(3) will give you a list of
> different formats that diff can output, and consequently patch can accept.
>   

I know that. But is "unified diff" the format that is clearly widely 
preferred. IMO it's the one that should be used as inspiration for the 
new format.

> An improved diff format that aims to be(come) compatible with patch
> should be applicable to at least the original plain diff, context-diff
> and unified-diff; possible even ed-script diff (see diff -e), though I
> suspect that one looses too much context to be useful for patches.
>
> [...]
>
>   
>> Just as Subversion took the best of CVS concepts and created a better
>> version control system, we should take this format and enhance it so
>> that it can serve Subversion needs.  Those new needs are three:
>>
>> * The ability of describe tree modifications (renames, deletions, etc.),
>>     
>
> This is more complicated than it looks, if you insist on compatibility
> with current diff/patch. Diff is file-based, whereas tree modifications
> are not.
>
> For example, patches are separable: you can take a patch which contains
> diffs of several files, split it apart, and apply each file's hunk
> separately. When you add tree-modification metadata to such patches,
> this is (in general) no longer true; and to be safe, a patch program
> that accepts such an enhanced diff format should be able to warn you
> that you're missing some tree modifications when you apply a patch.
>
> I certainly don't know how to do this by simply extending, e.g., unidiff.
>
> [...]
>   

Sorry, perhaps I wasn't clear. I'm not proposing a new format compatible 
with unidiff. I'm in fact arguing for the opposite. A new format which 
is "inspired" by unidiff goals, achievements and looks, but completely 
incompatible. I would even change the "---" and "+++" markers so as to 
make it clear that the new format is not compatible, although it might 
look similar. The rationale for not even trying to be partially 
compatible is not providing something that sometimes works and sometimes 
it doesn't.

>> * The patch should convey all the meaning implied in a 'svn merge'
>> operation that is considered reasonable. A patch should be like
>> tearing the merge action into two steps, i.e.: 'diff + patch = merge'.
>>     
>
> diff + patch != merge
>
> I don't know how that misconception came about, but merge typically
> looks at three sources, not two (and note that there's a prototype 4-way
> merge in Subversion's source tree).
>   

Perhaps there's something I'm not understaning. As I see it there the 3 
ways are:

1) The common ancestor (which is the "from" revision in the diff)
2) The proposed end result
3) The current local source. All of them are present in a normal diff 
operation.

>> * Support for binary diffs. Binary diffs would just be diffs created
>> with the binary diff algorithm already present in subversion, and then
>> encoded in base64 to get a text representation. The algorithm name
>> should be stated to allow for change in the future.
>>     
> This one is tricky. The fundamental reason for the success of the
> diff/patch pair is that the most common diff formats contain enough
> context to allow inexact patches. Patch is smart enough to find where to
> apply a patch even if a file has been (slightly) modified. To do so, it
> makes a number of assumptions about how text files are organized; these
> assumptions typically work for source code and consistently formatted
> text, but will usually fall down when, for example, your text format is
> one paragraph per line (which is fairly common in word processors and
> the like).
>
> With a generic binary file, you can define context in a similar way
> (though definitely not in combination with a block-copy delta
> algorithm!), but you can't invent good heuristics that would make
> inexact patching work, except in very, very limited cases -- that is,
> when your patch program knows everything about the format of the binary
> file it's handling.
>
> If you don't have inexact patch, then the diff/patch thing becomes
> pretty much useless for code exchange (/and/ for merging).
>   

The same as regular Subversion merges. Binary merges are aborted and 
left as an exercise for the user. That is not making Subversion merging 
useless, so I don't see why it would render a patch merging useles...

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: Improved patch format - SoC

[Daniel Berlin <db...@dberlin.org>]

On 4/1/07, Branko Čibej <br...@xbc.nu> wrote:
> Nicolás Lichtmaier wrote:
> > == Introduction ==
> >
> > The diff/patch tools have been great tools. The format they use
> > (called "unified diff")
>
> Correction: "unified diff" is in fact a later addition to the set of
> diff formats (the original diff didn't have it), and is by no means
> universally accepted as "the standard". IIRC the GCC project, for
> example, insists that patshes be submitted in context-diff, not
> unified-diff format.

Nope.

"If you are accessing the SVN repository at gcc.gnu.org, please
configure your svn to use GNU diff and generate patches using the
"-up" or "-c3p" options. "

We've found some people like unified, some people like context, and
almost everyone likes one or the other better for different
circumstances.


>
> [...]
>
> > Just as Subversion took the best of CVS concepts and created a better
> > version control system, we should take this format and enhance it so
> > that it can serve Subversion needs.  Those new needs are three:
> >
> > * The ability of describe tree modifications (renames, deletions, etc.),
>
> This is more complicated than it looks, if you insist on compatibility
> with current diff/patch. Diff is file-based, whereas tree modifications
> are not.
>
> For example, patches are separable: you can take a patch which contains
> diffs of several files, split it apart, and apply each file's hunk
> separately. When you add tree-modification metadata to such patches,
> this is (in general) no longer true; and to be safe, a patch program
> that accepts such an enhanced diff format should be able to warn you
> that you're missing some tree modifications when you apply a patch.
>
> I certainly don't know how to do this by simply extending, e.g., unidiff.
1.  include the tree modifications as a whole (IE a block that cannot
be split, such as just a base64 blob.
2. include a checksum of the entire patch at the beginning of the tree
mod section (which would appear like a comment/unrecognized hunk to
regular patch).
3. If the patch checksum doesn't match, something has munged with it
or you are missing data.
> [...]
>
> > * The patch should convey all the meaning implied in a 'svn merge'
> > operation that is considered reasonable. A patch should be like
> > tearing the merge action into two steps, i.e.: 'diff + patch = merge'.
>
> diff + patch != merge
>
> I don't know how that misconception came about, but merge typically
> looks at three sources, not two (and note that there's a prototype 4-way
> merge in Subversion's source tree).
>
> If you want a format that can convey all the meaning of a merge
> operation, then I thing you're better off breaking compatibility with
> the current format.
Why?
All you need to know are the parents.

>
> [...]
> > * Support for binary diffs. Binary diffs would just be diffs created
> > with the binary diff algorithm already present in subversion, and then
> > encoded in base64 to get a text representation. The algorithm name
> > should be stated to allow for change in the future.
>
> This one is tricky. The fundamental reason for the success of the
> diff/patch pair is that the most common diff formats contain enough
> context to allow inexact patches.

I seriously disagree with this statement.  The fundamental reason for
success of diff/patch, is, IMHO, that it is easy to transport, and
still readable in it's basic form.  So you can send them to people and
they can tell what you are doing to the file with only a minimum of
understanding of the format.

> [...]
>
> To sum up: To do what you propose, you'd have to first invent theory
> that I at least haven't seen yet. I don't think this project is suitable
> for a two-month summer hack ...
>
I do believe it is.  Nobody says we have to have a perfect result.  We
simply want something that is useful in 90% of the cases, to 90% of
our users.
I don't believe you have to *invent* anything to do that, it's already
done by others.

Re: Improved patch format - SoC

[Branko Čibej <br...@xbc.nu>]

Nicolás Lichtmaier wrote:
> == Introduction ==
>
> The diff/patch tools have been great tools. The format they use
> (called "unified diff")

Correction: "unified diff" is in fact a later addition to the set of
diff formats (the original diff didn't have it), and is by no means
universally accepted as "the standard". IIRC the GCC project, for
example, insists that patshes be submitted in context-diff, not
unified-diff format. Reading man diff(3) will give you a list of
different formats that diff can output, and consequently patch can accept.

An improved diff format that aims to be(come) compatible with patch
should be applicable to at least the original plain diff, context-diff
and unified-diff; possible even ed-script diff (see diff -e), though I
suspect that one looses too much context to be useful for patches.

[...]

> Just as Subversion took the best of CVS concepts and created a better
> version control system, we should take this format and enhance it so
> that it can serve Subversion needs.  Those new needs are three:
>
> * The ability of describe tree modifications (renames, deletions, etc.),

This is more complicated than it looks, if you insist on compatibility
with current diff/patch. Diff is file-based, whereas tree modifications
are not.

For example, patches are separable: you can take a patch which contains
diffs of several files, split it apart, and apply each file's hunk
separately. When you add tree-modification metadata to such patches,
this is (in general) no longer true; and to be safe, a patch program
that accepts such an enhanced diff format should be able to warn you
that you're missing some tree modifications when you apply a patch.

I certainly don't know how to do this by simply extending, e.g., unidiff.

[...]

> * The patch should convey all the meaning implied in a 'svn merge'
> operation that is considered reasonable. A patch should be like
> tearing the merge action into two steps, i.e.: 'diff + patch = merge'.

diff + patch != merge

I don't know how that misconception came about, but merge typically
looks at three sources, not two (and note that there's a prototype 4-way
merge in Subversion's source tree).

If you want a format that can convey all the meaning of a merge
operation, then I thing you're better off breaking compatibility with
the current format.

[...]
> * Support for binary diffs. Binary diffs would just be diffs created
> with the binary diff algorithm already present in subversion, and then
> encoded in base64 to get a text representation. The algorithm name
> should be stated to allow for change in the future.

This one is tricky. The fundamental reason for the success of the
diff/patch pair is that the most common diff formats contain enough
context to allow inexact patches. Patch is smart enough to find where to
apply a patch even if a file has been (slightly) modified. To do so, it
makes a number of assumptions about how text files are organized; these
assumptions typically work for source code and consistently formatted
text, but will usually fall down when, for example, your text format is
one paragraph per line (which is fairly common in word processors and
the like).

With a generic binary file, you can define context in a similar way
(though definitely not in combination with a block-copy delta
algorithm!), but you can't invent good heuristics that would make
inexact patching work, except in very, very limited cases -- that is,
when your patch program knows everything about the format of the binary
file it's handling.

If you don't have inexact patch, then the diff/patch thing becomes
pretty much useless for code exchange (/and/ for merging).

[...]

To sum up: To do what you propose, you'd have to first invent theory
that I at least haven't seen yet. I don't think this project is suitable
for a two-month summer hack ...

-- Brane

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org