You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by cm...@collab.net on 2002/03/19 14:56:26 UTC
Tracing and tracking commits in the new system.
It occurs to me that by switching to a URL-based commit drive (instead
of the current working-copy-path-based system), we throw a wrench into
several other things. For those not familiar with the new system I'm
trying to complete, it replaces a depth-first working copy
crawl/commit with a depth-first crawl/commit of the URLs associated
with the working copy things that have been modified.
Why the switch? Well, `switch` for one thing. ;-)
If I have, say, a checked-out working copy of the Subversion source
tree, and I want to work on a branch of libsvn_fs for a while, I would
do this:
svn cp http://svn.collab.net/repos/svn/trunk/subversion/libsvn_fs \
http://svn.collab.net/repos/svn/branches/cmpilato-branch
svn switch ./subversion/libsvn_fs \
http://svn.collab.net/repos/svn/branches/cmpilato-branch
This means that at libsvn_fs in my working copy, all of a sudden my
URL ancestry is no longer part of the same heirarchy as the rest of my
tree. This poses a problem for the current crawler if I have changes
in both my libsvn_fs (switched) dir and my libsvn_repos (not-switched)
dir. If I use `svn ci ./subversion/libsvn_fs ./subversion/libsvn_repos`
now, the common parent of "./subversion" will be chosen as the root of
my depth-first commit drive, and with it the URL of which it is a
reflection (http://svn.collab.net/repos/svn/trunk). But if that URL
is chosen as the TOP of my depth-first drive, it becomes rather
obvious that there is no way to even reach "http:// ... /svn/branches"!
Thus one of several reasons for the new system, which would, instead
of looking at paths (./subversion/libsvn_fs, ./subversion/libsvn_repos)
look instead the URLs for those things, choose the common parent of
http://svn.collab.net/repos/svn as the root of the commit, and commit
to "branches/cmpilato-branch" and "trunk/subversion/libsvn_repos".
So, enough about the background of the change. On to the ickiness.
------
Wait, one more piece of background. Before the commit ever begins,
and well after it ends, libsvn_client (and libsvn_client alone,
really) has the ability to map a committed thing's URL to a working
copy path and vice-versa.
------
The trace-commit editor is responsible for printing the path which is
being committed, and a verb to go with it:
Adding /home/cmpilato/tests/wc/boo
Sending /home/cmpilato/tests/wc/bah
...
Now, this particular editor could be switch to talk about repository
URLs, instead. So instead of seeing absolute disk paths, you'd see:
Adding http://servername.com/svn/cmpilato/boo
Sending http://servername.com/svn/cmpilato/bah
...
Alternatively, we could lose the trace-commit editor altogether and
simply have a custom callback vtable that would take WC paths, and
while those paths wouldn't necessarily be in depth-first order, they
would still be present.
Other ideas are welcome.
------
The commit-track editor is responsible for keeping track of what
things changed as part of a commit so that revisions can be bumped to
the new revision after the commit has succeeded. It lives inside the
RA layer, and is composed with the commit editor.
Now, I admit to not really understanding why this information has to
be kept inside of the RA layer. That is, why couldn't the working
copy just keep track of what it had committed, have the RA layer
simply answer the question "What is the new revision?", and then the
working copy bump its own revisions? But then again, I can't think of
a good way for the RA layer to answer that question, either.
At any rate, we now have this commit track editor which will know
nothing about the heirarchy of the working copy, and will instead
simply know that a bunch of URLs were changed.
I'm thinking that I'd love to knock that tracking editor out of the
picture since the new commit driver will know, prior to committing,
all of the committable items anyway -- why track them twice? If it
can only have the new-revision question answered, it can call
svn_wc_process_committed on each of the items, and there would be no
more need of the commit-track-hack editor.
------
I feel very strongly that a URL-based commit drive is Goodness, but
there are some definite deviations from the history thus far of
Subversion's commit processes, and that's presenting some challenges
to the idea. Ideas are soooooo welcome.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Tracing and tracking commits in the new system.
Posted by cm...@collab.net.
Greg Stein <gs...@lyra.org> writes:
> Actually, it appears that we still use the returned list. If a returned item
> is in the hash, then it gets bumped. Otherwise, we look in the hash for a
> parent of the returned item, which has a 'recursion' flag set on it.
>
>
> Oh... and wait a second. We also tweak some WC props in there. Thus, ra_dav
> is still going to need access to the list of committed targets, and it will
> still be doing some work. If the commit driver wants to do the bump *after*
> the close_edit() returns, then things should be fine (and we can remove the
> call to the bump_func from ra_dav).
Is this really true? I mean, if you put aside the revision bumping
stuff, is it harmful for ra_dav to tweak those WC props for items that
were returned by the server but not in the committed items list? If
those extra tweaks are non-fatal (or perhaps just inefficient), then
ra_dav doesn't truly need the list of committed items.
> I'd still prefer if the server just returned non-bubble directories and we
> drove the bump/wcprop stuff from that, rather than keeping a big, mother
> list in memory. But hey, if the client is going to generate it in the first
> place to deal with disjoint WCs, then the RA layers definitely shouldn't be
> building a duplicate.
Right. But the server should still not return bubble-up directories,
don't you think?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Tracing and tracking commits in the new system.
Posted by Greg Stein <gs...@lyra.org>.
On Tue, Mar 19, 2002 at 07:08:30PM -0600, Karl Fogel wrote:
> Greg Stein <gs...@lyra.org> writes:
>...
> > Of course, this *does* mean we have to accept the reality of buffering all
> > of that data in memory, in the client.
>
> Buffering a list of names is okay, imho. Just not the contents. :-)
>
> (Yes, someday there will be a directory with ten billion changed
> files. I'm sure we'll get a patch when that happens. :-) )
That's my fear :-)
> > (*) ra_dav had the server tell it what was actually changed in the commit,
> > which meant that the client didn't have to record it. however, the
> > server included bubbled-up directories in its change list. if we omitted
> > the bubble-up changes from the server's response, then we could remove
> > the tracking in ra_dav. this latter algorithm is my favored approach.
>
> We ignore them anyway, right?
Actually, it appears that we still use the returned list. If a returned item
is in the hash, then it gets bumped. Otherwise, we look in the hash for a
parent of the returned item, which has a 'recursion' flag set on it.
Oh... and wait a second. We also tweak some WC props in there. Thus, ra_dav
is still going to need access to the list of committed targets, and it will
still be doing some work. If the commit driver wants to do the bump *after*
the close_edit() returns, then things should be fine (and we can remove the
call to the bump_func from ra_dav).
I'd still prefer if the server just returned non-bubble directories and we
drove the bump/wcprop stuff from that, rather than keeping a big, mother
list in memory. But hey, if the client is going to generate it in the first
place to deal with disjoint WCs, then the RA layers definitely shouldn't be
building a duplicate.
(it just needs to be determined whether they truly are duplicates; if not,
then what differences can be ironed out to bring them into alignment)
Cheers,
-g
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Tracing and tracking commits in the new system.
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
Greg Stein <gs...@lyra.org> writes:
> RA needed a bump function because ra_dav didn't need to track the items in
> memory(*). At some point, however, we added the tracking to ra_dav. Thus,
> all RA implementations are tracking; thus, we can now move the bump handling
> outside of RA.
>
> Of course, this *does* mean we have to accept the reality of buffering all
> of that data in memory, in the client.
Buffering a list of names is okay, imho. Just not the contents. :-)
(Yes, someday there will be a directory with ten billion changed
files. I'm sure we'll get a patch when that happens. :-) )
> (*) ra_dav had the server tell it what was actually changed in the commit,
> which meant that the client didn't have to record it. however, the
> server included bubbled-up directories in its change list. if we omitted
> the bubble-up changes from the server's response, then we could remove
> the tracking in ra_dav. this latter algorithm is my favored approach.
We ignore them anyway, right?
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Tracing and tracking commits in the new system.
Posted by Greg Stein <gs...@lyra.org>.
On Tue, Mar 19, 2002 at 02:27:06PM -0600, Karl Fogel wrote:
> cmpilato@collab.net writes:
> > Ooooh...I did so forget about this. Hm...this would mean the RA layer
> > wouldn't need a bump function at all...nice...
>
> Yah. I guess it's always been the case that it doesn't need a bump
> function, but we'd been doing it from the RA layer because it had to
> set RA-specific wcprops anyway (I think that was the reason...).
RA needed a bump function because ra_dav didn't need to track the items in
memory(*). At some point, however, we added the tracking to ra_dav. Thus,
all RA implementations are tracking; thus, we can now move the bump handling
outside of RA.
Of course, this *does* mean we have to accept the reality of buffering all
of that data in memory, in the client.
Cheers,
-g
(*) ra_dav had the server tell it what was actually changed in the commit,
which meant that the client didn't have to record it. however, the
server included bubbled-up directories in its change list. if we omitted
the bubble-up changes from the server's response, then we could remove
the tracking in ra_dav. this latter algorithm is my favored approach.
--
Greg Stein, http://www.lyra.org/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Tracing and tracking commits in the new system.
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
cmpilato@collab.net writes:
> Ooooh...I did so forget about this. Hm...this would mean the RA layer
> wouldn't need a bump function at all...nice...
Yah. I guess it's always been the case that it doesn't need a bump
function, but we'd been doing it from the RA layer because it had to
set RA-specific wcprops anyway (I think that was the reason...).
-K
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Tracing and tracking commits in the new system.
Posted by cm...@collab.net.
Karl Fogel <kf...@newton.ch.collab.net> writes:
> cmpilato@collab.net writes:
> > The trace-commit editor is responsible for printing the path which is
> > being committed, and a verb to go with it:
> >
> > Adding /home/cmpilato/tests/wc/boo
> > Sending /home/cmpilato/tests/wc/bah
> > ...
> >
> > Now, this particular editor could be switch to talk about repository
> > URLs, instead. So instead of seeing absolute disk paths, you'd see:
> >
> > Adding http://servername.com/svn/cmpilato/boo
> > Sending http://servername.com/svn/cmpilato/bah
> > ...
> >
> > Alternatively, we could lose the trace-commit editor altogether and
> > simply have a custom callback vtable that would take WC paths, and
> > while those paths wouldn't necessarily be in depth-first order, they
> > would still be present.
> >
> > Other ideas are welcome.
>
> Personally prefer the current way. It conserves rightward space
> enough so that one can usually fit the whole path on one line. If we
> switched to URLs, many more lines would wrap past 80 cols.
As I noted on the phone, currently this is not much of a loss since
the commit-trace editor uses absolute paths in its output (through no
fault of its own ... it's opened with an absolute root path). The one
argument I can think of for sticking with a URL output is for
consistency among commit-y things. Actions like 'svn mkdir REPOS_URL',
'svn delete REPOS_URL', 'svn mv REPOS_URL REPOS_URL' and
'svn cp REPOS_URL REPOS_URL' have no working copy to talk about at
all, so it seems weird to have some commit-y things show URLs and
some show working copy paths.
Hm...note the CVS commit output, showing *both* pieces of data...
Checking in .bbdb-data;
/usr/local/cvs/cmpilato/.bbdb-data,v <-- .bbdb-data
new revision: 1.25; previous revision: 1.24
> > Now, I admit to not really understanding why this information has to
> > be kept inside of the RA layer. That is, why couldn't the working
> > copy just keep track of what it had committed, have the RA layer
> > simply answer the question "What is the new revision?", and then the
> > working copy bump its own revisions? But then again, I can't think of
> > a good way for the RA layer to answer that question, either.
>
> Doesn't the RA layer already answer this question, in the *NEW_REV
> argument to ra_lib->get_commit_editor()?
>
> svn_error_t *(*get_commit_editor) (void *session_baton,
> const svn_delta_editor_t **editor,
> void **edit_baton,
> svn_revnum_t *new_rev,
> const char **committed_date,
> const char **committed_author,
> svn_stringbuf_t *log_msg);
>
> So after the commit, libsvn_client knows the new revision, and could
> call pass it to some libsvn_wc post-commit function...
Ooooh...I did so forget about this. Hm...this would mean the RA layer
wouldn't need a bump function at all...nice...
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org
Re: Tracing and tracking commits in the new system.
Posted by Karl Fogel <kf...@newton.ch.collab.net>.
cmpilato@collab.net writes:
> The trace-commit editor is responsible for printing the path which is
> being committed, and a verb to go with it:
>
> Adding /home/cmpilato/tests/wc/boo
> Sending /home/cmpilato/tests/wc/bah
> ...
>
> Now, this particular editor could be switch to talk about repository
> URLs, instead. So instead of seeing absolute disk paths, you'd see:
>
> Adding http://servername.com/svn/cmpilato/boo
> Sending http://servername.com/svn/cmpilato/bah
> ...
>
> Alternatively, we could lose the trace-commit editor altogether and
> simply have a custom callback vtable that would take WC paths, and
> while those paths wouldn't necessarily be in depth-first order, they
> would still be present.
>
> Other ideas are welcome.
Personally prefer the current way. It conserves rightward space
enough so that one can usually fit the whole path on one line. If we
switched to URLs, many more lines would wrap past 80 cols.
> The commit-track editor is responsible for keeping track of what
> things changed as part of a commit so that revisions can be bumped to
> the new revision after the commit has succeeded. It lives inside the
> RA layer, and is composed with the commit editor.
>
> Now, I admit to not really understanding why this information has to
> be kept inside of the RA layer. That is, why couldn't the working
> copy just keep track of what it had committed, have the RA layer
> simply answer the question "What is the new revision?", and then the
> working copy bump its own revisions? But then again, I can't think of
> a good way for the RA layer to answer that question, either.
Doesn't the RA layer already answer this question, in the *NEW_REV
argument to ra_lib->get_commit_editor()?
svn_error_t *(*get_commit_editor) (void *session_baton,
const svn_delta_editor_t **editor,
void **edit_baton,
svn_revnum_t *new_rev,
const char **committed_date,
const char **committed_author,
svn_stringbuf_t *log_msg);
So after the commit, libsvn_client knows the new revision, and could
call pass it to some libsvn_wc post-commit function...
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org