You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by se...@germane-software.com on 2005/06/13 15:36:05 UTC
Block-level commits
Hi all,
I have a couple of questions related to block level commits. I'll lay
them out in advance, so you can avoid reading this post if they don't
interest you.
1) Is anybody interested in what I'm doing?
2) What are the fringe cases that I'm missing?
3) Is there a better way of doing this?
One thing I really like about darcs is block level commits; so much so,
that when I'm going to be doing a lot of work on a project, I often switch
to darcs temporarily to manage the commits, and then export the patches to
Subversion. While this works, it is a bit tedious, so I hacked up a
script to act as an interface to 'svn ci' that lets me cherry-pick the
code that is going into a particular commit. Basically, I'm getting finer
granularity over my commits.
The way the script currently works, it first executes an 'svn diff',
passing to it any arguments it has been given. It parses the output and
deconstructs it into files and blocks, and then presents this information
to the user a block at a time, allowing them to choose or bypass
individual blocks or entire files. I've shamelessly copied the darcs
workflow, including the command set. The script relies on the 'svn' and
'patch' commands.
What I'm wondering about is if there's an obvious logical flaw to the
process that I've missed. Here's what happens:
1) svn diff
2) break apart the patches, choose which will be accepted.
a) Write the entire diff, as a backup
b) Construct a patch including all of the accepted blocks
c) Construct a patch including all of the rejected blocks
3) svn stat
a) Keep track of the Adds/Deletes
4) svn revert
5) Delete all Adds
6) Apply the accepted patches (which will re-created the Adds)
7) Re-add the accepted Adds, re-delete the accepted Deletes
8) Commit the changes
9) Apply the rejected patches
10) Re-add the rejected Adds, re-delete the accepted Deletes
There's heavy error checking throughout this entire process, and attempts
to clean up should anything fail. In particular, the commit is a
dangerous point because of conflicts. If the commit fails, I am left with
a choice of (a) reverting and patching from the entire original patch, or
(b) resolving conflicts and then resuming at #8.
The entire algorithm was rather simple and straightforward until Adds and
Deletes entered the picture; then it started to look hackey. Still, it
works, and my concern is with fringe cases, such as properties; I strongly
suspect that I'm reaching the limit of what I can do without either
hooking into the Subversion bindings or turning this little script into a
full application.
There are a number of directions that I can go with this, but only two
interest me. The first is to rewrite this using language bindings, and
the second is to hack the svn client itself to support block-level
commits. If it won't be accepted formally into the SVN code, or if I have
to maintain it, I won't be doing the second option.
I welcome any feedback on this.
--- SER
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Block-level commits
Posted by kf...@collab.net.
I'm confused as to why we're discussing block-level commits as a
feature of Subversion, when they can easily be implemented as a
feature of the editing enviroment -- which I would think would be
preferable, because then the interface could be as fitting and
intuitive as possible for that environment.
?
-Karl
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Block-level commits
Posted by Chia-liang Kao <cl...@clkao.org>.
<ser <at> germane-software.com> writes:
> > And, as John Peacock said to him, you should also look at the patch
> > management capabilities of SVK:
>
> My main problem with SVK is twofold:
>
> 1) I've never set it up and *not* had trouble installing it.
It's nicely packaged by people for Mac OS X and win32, and is available
in most distribution as binary.
> 2) It adds a whole level of infrastructure that I'm entirely uninterested
> in. I don't want to use a local repository and push changes back to a
> local server. If I wanted to do that, then -- honestly -- I'd just use
> darcs.
You don't need a local branch, you commit to checkout of the mirror and it works
just as normal svn client. And you can use commit -P to generate patch rather
than actually committing.
For the local repository, hust think it as a wc replacement in your case.
At work we use svk, as the repository is 1.3G, svn checkout of trunk takes
20min and 2.2G space; where as svk mirror is, of course, 1.3G, and the checkout
takes 7min and 750M, so you actually save time and space.
Cheers,
CLK
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Block-level commits
Posted by se...@germane-software.com.
> ser@germane-software.com wrote:
>> 10) Re-add the rejected Adds, re-delete the accepted Deletes
>
> ----------------------------------------------^ rejected ^------
Right.
> I suppose this is fine for simple changes, or changes to human-readable
> files where errors don't matter much or can't be checked automatically.
> For software source code, I would always want to break this proceure in
> half and build the version about to be committed, to check that it is
> syntactically corect, and also run regression tests if it was a
> non-trivial change.
Darcs allows pre-commit checks on a patch set. I'd could easily add a
flag forcing the script to break just before the commit, providing the
user with a working copy that just contains the accepted patch set. They
could then resume the commit. This is what currently happens
automatically if there's a commit error (such as a version conflict). If
I wanted to get more fancy, the flag could take an argument that is
executed, and only if the process fails does the script exit. Say,
something like:
svb -t 'make test'
That's a good idea, thanks.
Honestly, though... before we go too much further than this, I'm really
trying to restrict what this thing does... see my comments about SVK,
below.
> You ought to read the thread "(SoC) Feature proposal: Patch-awareness" by
> Matthijs Kooijman <m....@student.utwente.nl>, at
> <http://svn.haxx.se/dev/archive-2005-06/0428.shtml>. You and he both
Thanks.
> And, as John Peacock said to him, you should also look at the patch
> management capabilities of SVK:
My main problem with SVK is twofold:
1) I've never set it up and *not* had trouble installing it.
2) It adds a whole level of infrastructure that I'm entirely uninterested
in. I don't want to use a local repository and push changes back to a
local server. If I wanted to do that, then -- honestly -- I'd just use
darcs.
My little script is exactly that: one file, with three dependencies: Ruby,
svn, and patch, and Ruby is the only one that might be an additional
install.
SVK is an impressive project, but is overkill for what I want, which is
finer control over what gets committed.
--- SER
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Block-level commits
Posted by Julian Foad <ju...@btopenworld.com>.
ser@germane-software.com wrote:
> 1) svn diff
> 2) break apart the patches, choose which will be accepted.
> a) Write the entire diff, as a backup
> b) Construct a patch including all of the accepted blocks
> c) Construct a patch including all of the rejected blocks
> 3) svn stat
> a) Keep track of the Adds/Deletes
> 4) svn revert
> 5) Delete all Adds
> 6) Apply the accepted patches (which will re-created the Adds)
> 7) Re-add the accepted Adds, re-delete the accepted Deletes
> 8) Commit the changes
> 9) Apply the rejected patches
> 10) Re-add the rejected Adds, re-delete the accepted Deletes
----------------------------------------------^ rejected ^------
> There's heavy error checking throughout this entire process, and attempts
> to clean up should anything fail. In particular, the commit is a
> dangerous point because of conflicts. If the commit fails, I am left with
> a choice of (a) reverting and patching from the entire original patch, or
> (b) resolving conflicts and then resuming at #8.
I suppose this is fine for simple changes, or changes to human-readable files
where errors don't matter much or can't be checked automatically. For software
source code, I would always want to break this proceure in half and build the
version about to be committed, to check that it is syntactically corect, and
also run regression tests if it was a non-trivial change.
> The entire algorithm was rather simple and straightforward until Adds and
> Deletes entered the picture; then it started to look hackey. Still, it
> works, and my concern is with fringe cases, such as properties; I strongly
> suspect that I'm reaching the limit of what I can do without either
> hooking into the Subversion bindings or turning this little script into a
> full application.
Yes - you will silently lose Subversion properties on added files if you are
just using the traditional "patch" utility.
> There are a number of directions that I can go with this, but only two
> interest me. The first is to rewrite this using language bindings, and
> the second is to hack the svn client itself to support block-level
> commits. If it won't be accepted formally into the SVN code, or if I have
> to maintain it, I won't be doing the second option.
You ought to read the thread "(SoC) Feature proposal: Patch-awareness" by
Matthijs Kooijman <m....@student.utwente.nl>, at
<http://svn.haxx.se/dev/archive-2005-06/0428.shtml>. You and he both want to
do the same thing, so it would be a good idea if you showed him what you have
already done, and shared ideas and work.
And, as John Peacock said to him, you should also look at the patch management
capabilities of SVK:
<http://svk.elixus.org>
since Chia-liang Kao <cl...@clkao.org> has already got something working. He
uses a traditional "patch" format but with any property changes, etc, stored in
a seperate chunk at the end (encoded) which the conventional 'patch' program
will helpfully ignore. CLK said, "What svk patch stored is just serialized
delta editor calls, along with the target tree/rev that the editor call applies
to. svk patch --apply is able to use such information to perform a 3-way merge
when the target has been changed; it can also update the patch painlessly to
provide a cleanly applicable patch, which makes life easier for people using
plain old /usr/bin/patch."
- Julian
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Block-level commits
Posted by Daniel Rall <dl...@finemaltcoding.com>.
On Mon, 2005-06-13 at 12:02 -0700, ser@germane-software.com wrote:
>> I don't see this offering much over performing a diff and committing the
>> specific portions of your working copy which make up a change set. The
>> primary benefit seems to be the ability to commit directory props
>> without committing modified files from that directory, at the cost of
>> completely losing those directory props -- all props, for that matter --
>> when the patch file is re-applied.
>
>How do you "commit specific portions of your working copy"? Just to
>clarify, I'm talking about a mechanism which allows you to commit not only
>whole files, but just parts of a file.
Thanks for the clarification; this is a key distinction. I was
referring to on a file-by-file basis (specified explicitly on the
command-line). More on this below.
...
>A use case example for the problem that I'm trying to solve is:
>
>I'm working on some new feature. Along the way, I stumble across some
>previously undiscovered bug in my code -- something minor, perhaps a typo
>-- so I fix them. I'd like to commit *just* those changes, but I've also
>been changing the file in other places.
Thanks for providing a use case, always helpful.
>The only ways that I know of to do this in SVN is to break out of what I'm
>doing, check out another copy and make the changes there; or try to
>remember what I need to change and implement it after I'm done with what
>I'm doing. Both of these methods interrupts my workflow.
Right. Or, create a patch containing your local changes, revert the
files which contain changes from separate change sets, cherry-pick the
portions of the patch relevant to your first commit, and possibly edit
them out of the patch for subsequent application back to your previous
state. Which I gather is what your script automates, to an extent.
Personally, I keep around multiple working copies on a couple different
machines. Which approach I use depends on the scope of the changes I
want to commit, and the state of my working copies.
Using traditional patch, how do you deal with changes from multiple
change sets within a given hunk? patch's inability to gracefully deal
with this is one of the factors which drives me to use multiple working
copies (build/run/test state of the code being the other primary
factor).
>Just to head off a cost/value discussion: I'm not particularly
>interested
>in trying to convince anybody that block-level commits AKA partial commits
>AKA cherry picked commits is a useful feature; I've already decided that
>it is, and having once wielded the power of block-level commits in darcs,
>I'm unwilling to do without them. My only interest, then, is gaining this
>functionality in SVN, and possibly providing it to others who might be
>interested in it.
Without a doubt, "working copy changeset management" a very useful
feature! I would definitely prefer to reduce the number of working
copies I need to keep around to a minimum.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Block-level commits
Posted by se...@germane-software.com.
> I don't see this offering much over performing a diff and committing the
> specific portions of your working copy which make up a change set. The
> primary benefit seems to be the ability to commit directory props
> without committing modified files from that directory, at the cost of
> completely losing those directory props -- all props, for that matter --
> when the patch file is re-applied.
How do you "commit specific portions of your working copy"? Just to
clarify, I'm talking about a mechanism which allows you to commit not only
whole files, but just parts of a file. If you're talking about manually
generating, editing, and applying the diffs against a clean copy, then my
method simply automates the process. And my method is entirely incapable
of handling property changes, and is probably broken in a dozen other ways
as well.
A use case example for the problem that I'm trying to solve is:
I'm working on some new feature. Along the way, I stumble across some
previously undiscovered bug in my code -- something minor, perhaps a typo
-- so I fix them. I'd like to commit *just* those changes, but I've also
been changing the file in other places.
The only ways that I know of to do this in SVN is to break out of what I'm
doing, check out another copy and make the changes there; or try to
remember what I need to change and implement it after I'm done with what
I'm doing. Both of these methods interrupts my workflow.
Just to head off a cost/value discussion: I'm not particularly interested
in trying to convince anybody that block-level commits AKA partial commits
AKA cherry picked commits is a useful feature; I've already decided that
it is, and having once wielded the power of block-level commits in darcs,
I'm unwilling to do without them. My only interest, then, is gaining this
functionality in SVN, and possibly providing it to others who might be
interested in it.
--- SER
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Block-level commits
Posted by Daniel Rall <dl...@finemaltcoding.com>.
I don't see this offering much over performing a diff and committing the
specific portions of your working copy which make up a change set. The
primary benefit seems to be the ability to commit directory props
without committing modified files from that directory, at the cost of
completely losing those directory props -- all props, for that matter --
when the patch file is re-applied.
What am I missing?
On Mon, 2005-06-13 at 08:36 -0700, ser@germane-software.com wrote:
>Hi all,
>
>I have a couple of questions related to block level commits. I'll lay
>them out in advance, so you can avoid reading this post if they don't
>interest you.
>
>1) Is anybody interested in what I'm doing?
>2) What are the fringe cases that I'm missing?
>3) Is there a better way of doing this?
>
>One thing I really like about darcs is block level commits; so much so,
>that when I'm going to be doing a lot of work on a project, I often switch
>to darcs temporarily to manage the commits, and then export the patches to
>Subversion. While this works, it is a bit tedious, so I hacked up a
>script to act as an interface to 'svn ci' that lets me cherry-pick the
>code that is going into a particular commit. Basically, I'm getting finer
>granularity over my commits.
>
>The way the script currently works, it first executes an 'svn diff',
>passing to it any arguments it has been given. It parses the output and
>deconstructs it into files and blocks, and then presents this information
>to the user a block at a time, allowing them to choose or bypass
>individual blocks or entire files. I've shamelessly copied the darcs
>workflow, including the command set. The script relies on the 'svn' and
>'patch' commands.
>
>What I'm wondering about is if there's an obvious logical flaw to the
>process that I've missed. Here's what happens:
>
> 1) svn diff
> 2) break apart the patches, choose which will be accepted.
> a) Write the entire diff, as a backup
> b) Construct a patch including all of the accepted blocks
> c) Construct a patch including all of the rejected blocks
> 3) svn stat
> a) Keep track of the Adds/Deletes
> 4) svn revert
> 5) Delete all Adds
> 6) Apply the accepted patches (which will re-created the Adds)
> 7) Re-add the accepted Adds, re-delete the accepted Deletes
> 8) Commit the changes
> 9) Apply the rejected patches
> 10) Re-add the rejected Adds, re-delete the accepted Deletes
>
>There's heavy error checking throughout this entire process, and attempts
>to clean up should anything fail. In particular, the commit is a
>dangerous point because of conflicts. If the commit fails, I am left with
>a choice of (a) reverting and patching from the entire original patch, or
>(b) resolving conflicts and then resuming at #8.
>
>The entire algorithm was rather simple and straightforward until Adds and
>Deletes entered the picture; then it started to look hackey. Still, it
>works, and my concern is with fringe cases, such as properties; I strongly
>suspect that I'm reaching the limit of what I can do without either
>hooking into the Subversion bindings or turning this little script into a
>full application.
>
>There are a number of directions that I can go with this, but only two
>interest me. The first is to rewrite this using language bindings, and
>the second is to hack the svn client itself to support block-level
>commits. If it won't be accepted formally into the SVN code, or if I have
>to maintain it, I won't be doing the second option.
>
>I welcome any feedback on this.
>
>--- SER
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
>For additional commands, e-mail: dev-help@subversion.tigris.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org