You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@subversion.apache.org by Hyrum K Wright <hy...@wandisco.com> on 2011/11/28 23:34:14 UTC
svn_delta_path_driver(), its purpose and future
In working on the Ev2 shims, I discovered svn_delta_path_driver(), a
nifty little API whose purpose I haven't totally yet discerned. It
looks to be some kind of hybridization of the editor, allowing a
caller to handle some portion of the editor drive manually, though the
documentation has enough ifs, buts, and wherefores to make me a bit
unsure.
If I am correct, however, it would appear that this API is not
compatible with our current editor shims, which queue up all changes
made in an editor, and then play them back through the new shim.
Any illumination by folks knowing more than I would be appreciated.
-Hyrum
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com/
Re: svn_delta_path_driver(), its purpose and future
Posted by Daniel Shahaf <d....@daniel.shahaf.name>.
The docs seem pretty clear to me: svn_delta_path_driver() produces a depth-
first editor traversal, where for each PATH in PATHS the portion in-and-
below PATH is handled by the callback (including the open_*() or add_*()
calls for PATH itself), and the portion of opening/closing the parent
and ancestors of PATH is handled by svn_delta_path_driver() itself.
I have only read its docstring --- not its definition or callers --- so
I might be oversimplifying things. :)
----
What context did you run into this in? Trying to revv that function?
To upgrade its callers to Ev2?
On Monday, November 28, 2011 4:34 PM, "Hyrum K Wright" <hy...@wandisco.com> wrote:
> In working on the Ev2 shims, I discovered svn_delta_path_driver(), a
> nifty little API whose purpose I haven't totally yet discerned. It
> looks to be some kind of hybridization of the editor, allowing a
> caller to handle some portion of the editor drive manually, though the
> documentation has enough ifs, buts, and wherefores to make me a bit
> unsure.
>
> If I am correct, however, it would appear that this API is not
> compatible with our current editor shims, which queue up all changes
> made in an editor, and then play them back through the new shim.
>
> Any illumination by folks knowing more than I would be appreciated.
>
> -Hyrum
>
> --
>
> uberSVN: Apache Subversion Made Easy
> http://www.uberSVN.com/
>
Re: svn_delta_path_driver(), its purpose and future
Posted by "C. Michael Pilato" <cm...@collab.net>.
On 11/30/2011 10:08 AM, Hyrum K Wright wrote:
>> I guess this is the part where I'm confused. You can't "delay" an
>> open_directory() call -- it must execute, it must return, and when it
>> returns it must provide a new directory baton or throw an error.
>
> Actually, such calls can be delayed. Quoth the docs:
>
> * At least one implementation of the editor interface is
> * asynchronous; an error from one operation may be detected some
> * number of operations later. As a result, an editor driver must not
> * assume that an error from an editing function resulted from the
> * particular operation being detected. Moreover, once an editing
> * function returns an error, the edit is dead; the only further
> * operation which may be called on the editor is abort_edit.
>
> Now, that doesn't mean that calls to the SDPD callback shouldn't be
> similarly delayed as well, just that calling an editor handler
> function does not mean that the effects will happen *right now*.
You misunderstand me, or the docs, or both.
What is described here is *not* the delay of an open_directory() call, but
the delay of its effect on the receiver's end, and the delay of the
detection of any possible failures which might occur when said effect is
eventually manifested. We may all rest quite assured that when we called
editor->open_directory(), by golly the code on the other side of that
function call is going to run, and that not a thing more is going to happen
in the sender until that call stack returns. Or, we may all checks
ourselves into the nearest asylum.
What is further not described here is any deviance in the *ordering* of
editor calls or the effects of those calls. It stands against all reason
that the effect of add_file('A/B/NEWDIR/newfile') should "take" before the
effect of the add_directory('A/B/NEWDIR') which immediately preceded it, at
least insofar as Ev1 semantics are concerned.
> Just brain storming here, but since the users of SDPD are already
> operating in a way which is similar to Ev2, I'm wondering if they
> would be primary candidates for converting to Ev2 initially. Of
> course, that'd mean other work would need to be done to give them
> something to interact with, but it may be a possible way forward.
Seems likely. Of course, there's nothing to be done here until there's
actually an Ev2 that need a-drivin'. Or, if you wish to do the exercise
early, you could change those current consumers of SDPD to drive an Ev2->Ev1
adapter.
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Re: svn_delta_path_driver(), its purpose and future
Posted by Hyrum K Wright <hy...@wandisco.com>.
On Wed, Nov 30, 2011 at 8:12 AM, C. Michael Pilato <cm...@collab.net> wrote:
> On 11/30/2011 12:59 AM, Hyrum K Wright wrote:
>> Let me offer a concrete example, in the hopes that I can make some
>> sense. I use the term "sender" to mean "the thing that is invoking
>> the editor callbacks" and "receiver" to mean "the thing who is
>> providing callbacks to be invoked". I believe these are the
>> traditional uses of the terms; if not, I've got some more reeducation
>> to go through.
>
> (I've always used "editor driver" and "editor implementation", but whatever.)
Yep, and after I sent that mail I realized that after writing that
paragraph, I didn't use any of those terms the rest of the mail.
Whatever indeed. :)
>> In libsvn_client/mergeinfo.c there is a function named
>> svn_client__elide_mergeinfo_catalog() (or SCEMC for short). SCEMC
>> uses SDPD to elider mergeinfo and uses a default Ev1 editor with only
>> custom open_root() and open_directory() handlers[1]. It provides this
>> editor to SDPD.
>>
>> Without the shims, everything works well, but when the delayed-run
>> action of the shims is inserted, we get crashes. The reason for this
>> is that with the shims, the SDPD callback is invoked prior to any
>> open_directory() handlers of the editor are invoked, and the SDPD
>> callback depends upon state which the open_directory() handler sets.
>>
>> Basically, the delayed processing of the Ev1 open_directory() handler
>> is combined with the *immediate* handling of the SDPD callback, which
>> results in out-of-order execution of the SDPD callback, and from there
>> badness ensues. My thought is to move the shim insertion inside SDPD
>> and make it clever enough to delay calling the SDPD callback until the
>> Ev1 queue is played back, thus ensuring the ordering is then correct.
>
> I guess this is the part where I'm confused. You can't "delay" an
> open_directory() call -- it must execute, it must return, and when it
> returns it must provide a new directory baton or throw an error.
Actually, such calls can be delayed. Quoth the docs:
* At least one implementation of the editor interface is
* asynchronous; an error from one operation may be detected some
* number of operations later. As a result, an editor driver must not
* assume that an error from an editing function resulted from the
* particular operation being detected. Moreover, once an editing
* function returns an error, the edit is dead; the only further
* operation which may be called on the editor is abort_edit.
Now, that doesn't mean that calls to the SDPD callback shouldn't be
similarly delayed as well, just that calling an editor handler
function does not mean that the effects will happen *right now*.
> I'm starting to wonder if these shims fit into a different junction than I'd
> assumed. In fact, in this case, there should be no shims in use at all --
> it's perfectly okay for SCEMC to use SDPD and an Ev1 editor now and forever.
> We should only be using shims where they are *required* to fix up interface
> mismatches. No such mismatch exists at this call site which, as you say,
> owns both the sender and the receiver.
True, and I've got a very simple patch which removes the use of the
shims from the above example site. I'll need to look elsewhere to see
if other places need a similar adjustment.
Just brain storming here, but since the users of SDPD are already
operating in a way which is similar to Ev2, I'm wondering if they
would be primary candidates for converting to Ev2 initially. Of
course, that'd mean other work would need to be done to give them
something to interact with, but it may be a possible way forward.
> Ah... I see what's going on now. You're trying to perform a double
> transformation (delta->editor->delta) here. Now I understand what you mean
> by delaying the open_directory(). It's the open_directory() of that
> tail-end Ev1 that's delayed. So again, in this instance, the use of shims
> is completely unnecessary. You must know that, so may I assume that you're
> just trying to do this stuff for the sake of working out problems with the
> shims themselves?
Yes! Instead of writing individual tests and guessing at all the
different scenarios the shims would encounter, the easiest solution
was to insert the matching pairs of shims in places where editors are
used or created, and then just fix the bugs. While that may seem like
something of a shotgun approach, it guarantees that the shims will
handle what we need them to in the transition period from Ev1 to Ev2.
As some point, we'll be able to decouple the Ev1->Ev2->Ev1 path, and
replace individual consumers or producers at our leisure. That's the
theory, at least.
-Hyrum
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com/
Re: svn_delta_path_driver(), its purpose and future
Posted by "C. Michael Pilato" <cm...@collab.net>.
On 11/30/2011 12:59 AM, Hyrum K Wright wrote:
> Let me offer a concrete example, in the hopes that I can make some
> sense. I use the term "sender" to mean "the thing that is invoking
> the editor callbacks" and "receiver" to mean "the thing who is
> providing callbacks to be invoked". I believe these are the
> traditional uses of the terms; if not, I've got some more reeducation
> to go through.
(I've always used "editor driver" and "editor implementation", but whatever.)
> In libsvn_client/mergeinfo.c there is a function named
> svn_client__elide_mergeinfo_catalog() (or SCEMC for short). SCEMC
> uses SDPD to elider mergeinfo and uses a default Ev1 editor with only
> custom open_root() and open_directory() handlers[1]. It provides this
> editor to SDPD.
>
> Without the shims, everything works well, but when the delayed-run
> action of the shims is inserted, we get crashes. The reason for this
> is that with the shims, the SDPD callback is invoked prior to any
> open_directory() handlers of the editor are invoked, and the SDPD
> callback depends upon state which the open_directory() handler sets.
>
> Basically, the delayed processing of the Ev1 open_directory() handler
> is combined with the *immediate* handling of the SDPD callback, which
> results in out-of-order execution of the SDPD callback, and from there
> badness ensues. My thought is to move the shim insertion inside SDPD
> and make it clever enough to delay calling the SDPD callback until the
> Ev1 queue is played back, thus ensuring the ordering is then correct.
I guess this is the part where I'm confused. You can't "delay" an
open_directory() call -- it must execute, it must return, and when it
returns it must provide a new directory baton or throw an error.
I'm starting to wonder if these shims fit into a different junction than I'd
assumed. In fact, in this case, there should be no shims in use at all --
it's perfectly okay for SCEMC to use SDPD and an Ev1 editor now and forever.
We should only be using shims where they are *required* to fix up interface
mismatches. No such mismatch exists at this call site which, as you say,
owns both the sender and the receiver.
Ah... I see what's going on now. You're trying to perform a double
transformation (delta->editor->delta) here. Now I understand what you mean
by delaying the open_directory(). It's the open_directory() of that
tail-end Ev1 that's delayed. So again, in this instance, the use of shims
is completely unnecessary. You must know that, so may I assume that you're
just trying to do this stuff for the sake of working out problems with the
shims themselves?
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Re: svn_delta_path_driver(), its purpose and future
Posted by Hyrum K Wright <hy...@wandisco.com>.
On Tue, Nov 29, 2011 at 4:20 PM, C. Michael Pilato <cm...@collab.net> wrote:
> On 11/29/2011 04:56 PM, Hyrum K Wright wrote:
>> I understand and expect that consumers of SDPD as currently
>> implemented will need to use Ev1. The problem is that they don't take
>> very kindly to instances where Ev1 is driven in an asynchronous mode,
>> which doesn't technically violate the Ev1 API. But I think this is a
>> problem that we can work around.
>
> But ... but ... the code sites which are using SDPD *are* the drivers of the
> editor, not the implementors.
>
> Maybe you meant, "they don't take very kindly to instances where Ev1 doesn't
> immediately return an error because all the real work is done by a nested
> Ev2 in a big batch at the end of the Ev1 drive" ?
Let me offer a concrete example, in the hopes that I can make some
sense. I use the term "sender" to mean "the thing that is invoking
the editor callbacks" and "receiver" to mean "the thing who is
providing callbacks to be invoked". I believe these are the
traditional uses of the terms; if not, I've got some more reeducation
to go through.
In libsvn_client/mergeinfo.c there is a function named
svn_client__elide_mergeinfo_catalog() (or SCEMC for short). SCEMC
uses SDPD to elider mergeinfo and uses a default Ev1 editor with only
custom open_root() and open_directory() handlers[1]. It provides this
editor to SDPD.
Without the shims, everything works well, but when the delayed-run
action of the shims is inserted, we get crashes. The reason for this
is that with the shims, the SDPD callback is invoked prior to any
open_directory() handlers of the editor are invoked, and the SDPD
callback depends upon state which the open_directory() handler sets.
Basically, the delayed processing of the Ev1 open_directory() handler
is combined with the *immediate* handling of the SDPD callback, which
results in out-of-order execution of the SDPD callback, and from there
badness ensues. My thought is to move the shim insertion inside SDPD
and make it clever enough to delay calling the SDPD callback until the
Ev1 queue is played back, thus ensuring the ordering is then correct.
-Hyrum
[1] In this case at least, the code site where SDPD is used *is* the
implementor of the editor, not the driver. SCEMC provides an editor
to SDPD which it then drives.
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com/
Re: svn_delta_path_driver(), its purpose and future
Posted by "C. Michael Pilato" <cm...@collab.net>.
On 11/29/2011 04:56 PM, Hyrum K Wright wrote:
> It sounds very much like SDPD is simply a mechanism for doing with Ev1
> what Ev2 is explicitly designed for: operating on arbitrary paths in
> the delta tree. All SDPD does is fill in the gaps so that the sender
> doesn't have to worry about all the intermediate calls between the
> paths of interest, right?
Yup!
> I understand and expect that consumers of SDPD as currently
> implemented will need to use Ev1. The problem is that they don't take
> very kindly to instances where Ev1 is driven in an asynchronous mode,
> which doesn't technically violate the Ev1 API. But I think this is a
> problem that we can work around.
But ... but ... the code sites which are using SDPD *are* the drivers of the
editor, not the implementors.
Maybe you meant, "they don't take very kindly to instances where Ev1 doesn't
immediately return an error because all the real work is done by a nested
Ev2 in a big batch at the end of the Ev1 drive" ?
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Re: svn_delta_path_driver(), its purpose and future
Posted by Hyrum K Wright <hy...@wandisco.com>.
On Tue, Nov 29, 2011 at 3:02 PM, C. Michael Pilato <cm...@collab.net> wrote:
> On 11/29/2011 02:33 PM, Hyrum K Wright wrote:
>> Ev2 doesn't bother with a depth-first tree traversal, the sender just
>> invokes paths in whatever order it sees fit. There are a few obvious
>> restrictions, such as directories need to exist before their children
>> can be processed, for example, but it's much looser than the delta
>> editor. Also, the APIs are a bit more streamlined, in that it's not
>> legal to set the properties on a node multiple times in the same
>> drive, for instance.
>>
>> To handle the conversion from the delta editor to Ev2, we create an
>> editor which receives all the calls from the sender, queues them up,
>> and then drives an Ev2 editor in the receiver. Likewise, a similar
>> shim exists which receives Ev2 calls, queues them up and then drives a
>> delta editor in the proper order. It's all terribly inefficient due
>> to the fact that the entire drive is stored in memory prior to being
>> translated and replayed to the target editor (whether Ev2 or the delta
>> editor), but it's also meant as a temporary measure.
>>
>> However, svn_delta_path_driver() looks like it interacts with the
>> editor while the thing is being driven. It counts on all the prior
>> calls actually reaching the target editor before the callback is
>> invoked, which means it will have an adverse reaction to something
>> which queues up the calls and then replays them, since none of the
>> preceding calls will have succeeded by the time its callback is
>> invoked.
>
> Of course svn_delta_path_driver() interacts with the editor while it's being
> driven -- it's helping to drive it!
>
> Look, what this interface does is pretty straightforward. It examines a
> list of interesting paths provided it. Interesting paths are paths which
> are added, deleted, or modified. It sorts them into depth-first order, and
> drives the provided editor in such a way that all those paths are reached as
> part of the editor drive. As each one is reached, it calls its callback
> function to say, "Hey, we're at the point of the editor drive where you can
> do whatever it is to path FOO that you needed done, so do it and then return
> when you're finished." And then it continues along, opening and closing
> directories until it reaches the next interesting path, continuing until
> every interesting path has been reached as part of the drive and all
> intermediate directories have been closed.
>
> For example, say I have a list of paths and the actions I want to do to them:
>
> 'trunk/mu': delete it
> 'trunk/A/D/G/florp': add it as a directory
> 'trunk/A/B/E/alpha': set the property 'foo' to 'bar' on it
>
> I can tell that the longest common ancestor of these paths is 'trunk', so I
> anchor the editor which is supposed to absorb these changes (maybe its an RA
> commit editor) there, convert my list of paths to be relative to that
> location, then pass the list and the editor to svn_delta_path_driver().
>
> svn_delta_path_driver (SDPD) sorts my relative paths by depth:
>
> 'mu'
> 'A/B/E/alpha'
> 'A/D/G/florp'
>
> and then starts driving the editor to handle intermediate directories
> between and around my interesting paths. In my scenario, it doesn't get far
> before it reaches one of my interesting paths:
>
> SDPD: db1 = open_root('')
>
> Now, it's at the point where it can act on 'mu', but it doesn't know what
> needs to happen to 'mu'. So it calls the path driver callback (PDCB) I
> provided to allow that sucker to do the real work on 'mu', providing 'mu's
> parent directory baton to the callback. My callback function knows that I
> wanted to delete 'mu' (I had that information stuffed into its baton), so it
> does so by calling the delete_entry() function on the same editor that
> svn_path_delta_driver() is using.
>
> PDCB: delete_entry(db1, 'mu')
>
> Then it returns.
>
> SDPD continues, driving the editor to the next interesting path
> ('A/B/E/alpha'), which requires a few more open-dir's:
>
> SDPD: db2 = open_directory(db1, 'A')
> SDPD: db3 = open_directory(db2, 'A/B')
> SDPD: db4 = open_directory(db3, 'A/B/E')
>
> Having arrived at the next interesting path, it calls my callback to do the
> interesting work, and the callback fills in some more of the missing editor
> calls:
>
> PDCB: fb = open_file(db4, 'A/B/E/alpha')
> PDCB: change_file_prop(fb, 'foo', 'bar')
> PDCB: close_file(fb)
>
> Back in SDPD again, we drive to the next destination, which requires moving
> up a directory and back down again into another subtree:
>
> SDPD: close_directory(db4)
> SDPD: db4 = open_directory(db3, 'A/D/G')
>
> Then it calls the callback again for 'A/D/G/florp', which does:
>
> PDCB: db5 = add_directory(db4, 'A/D/G/florp')
>
> This time the callback does something a little different, though -- because
> it opened or added a directory, it returns the dir baton for that directory
> so that SDPD can use it to address immediate children (if any) of this
> directory. In my example, there are none, so when program control returns
> to SDPD, it just closes my new directory as part of its wrap-up of open
> directories back to the root.
>
> SDPD: close_directory(db5)
> SDPD: close_directory(db4)
> SDPD: close_directory(db3)
> SDPD: close_directory(db2)
> SDPD: close_directory(db1)
>
> And the edit drive is complete save for the close_edit() called from outside
> the svn_delta_path_driver() call tree.
Thank you very much for this example, it was quite illuminating.
It sounds very much like SDPD is simply a mechanism for doing with Ev1
what Ev2 is explicitly designed for: operating on arbitrary paths in
the delta tree. All SDPD does is fill in the gaps so that the sender
doesn't have to worry about all the intermediate calls between the
paths of interest, right?
>> Now, after having written all then, I wonder if there would be a way
>> to set up a system whereby the delta_path_driver callback was invoked
>> during the replay of the queued items, rather than the initial calls
>> to them. It would involve a bit more shim code, but this may be the
>> better solution. Hmmm.....
>
> Any consumer of the svn_delta_path_driver() will need to be using Ev1 --
> that's what's supported by the API. If their particular editor is a shim to
> an Ev2 under the hood, that's fine -- the caller need never be aware of that
> any more than they would be if they were manually driving the editor. And
> it sounds as if we don't need to rev svn_delta_path_driver() for an Ev2
> flavor because Ev2 solves the primary problem that SDPD exists to solve:
> the depth-first requirement of editor drives.
I understand and expect that consumers of SDPD as currently
implemented will need to use Ev1. The problem is that they don't take
very kindly to instances where Ev1 is driven in an asynchronous mode,
which doesn't technically violate the Ev1 API. But I think this is a
problem that we can work around.
Thanks for the help!
-Hyrum
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com/
Re: svn_delta_path_driver(), its purpose and future
Posted by "C. Michael Pilato" <cm...@collab.net>.
On 11/29/2011 02:33 PM, Hyrum K Wright wrote:
> Ev2 doesn't bother with a depth-first tree traversal, the sender just
> invokes paths in whatever order it sees fit. There are a few obvious
> restrictions, such as directories need to exist before their children
> can be processed, for example, but it's much looser than the delta
> editor. Also, the APIs are a bit more streamlined, in that it's not
> legal to set the properties on a node multiple times in the same
> drive, for instance.
>
> To handle the conversion from the delta editor to Ev2, we create an
> editor which receives all the calls from the sender, queues them up,
> and then drives an Ev2 editor in the receiver. Likewise, a similar
> shim exists which receives Ev2 calls, queues them up and then drives a
> delta editor in the proper order. It's all terribly inefficient due
> to the fact that the entire drive is stored in memory prior to being
> translated and replayed to the target editor (whether Ev2 or the delta
> editor), but it's also meant as a temporary measure.
>
> However, svn_delta_path_driver() looks like it interacts with the
> editor while the thing is being driven. It counts on all the prior
> calls actually reaching the target editor before the callback is
> invoked, which means it will have an adverse reaction to something
> which queues up the calls and then replays them, since none of the
> preceding calls will have succeeded by the time its callback is
> invoked.
Of course svn_delta_path_driver() interacts with the editor while it's being
driven -- it's helping to drive it!
Look, what this interface does is pretty straightforward. It examines a
list of interesting paths provided it. Interesting paths are paths which
are added, deleted, or modified. It sorts them into depth-first order, and
drives the provided editor in such a way that all those paths are reached as
part of the editor drive. As each one is reached, it calls its callback
function to say, "Hey, we're at the point of the editor drive where you can
do whatever it is to path FOO that you needed done, so do it and then return
when you're finished." And then it continues along, opening and closing
directories until it reaches the next interesting path, continuing until
every interesting path has been reached as part of the drive and all
intermediate directories have been closed.
For example, say I have a list of paths and the actions I want to do to them:
'trunk/mu': delete it
'trunk/A/D/G/florp': add it as a directory
'trunk/A/B/E/alpha': set the property 'foo' to 'bar' on it
I can tell that the longest common ancestor of these paths is 'trunk', so I
anchor the editor which is supposed to absorb these changes (maybe its an RA
commit editor) there, convert my list of paths to be relative to that
location, then pass the list and the editor to svn_delta_path_driver().
svn_delta_path_driver (SDPD) sorts my relative paths by depth:
'mu'
'A/B/E/alpha'
'A/D/G/florp'
and then starts driving the editor to handle intermediate directories
between and around my interesting paths. In my scenario, it doesn't get far
before it reaches one of my interesting paths:
SDPD: db1 = open_root('')
Now, it's at the point where it can act on 'mu', but it doesn't know what
needs to happen to 'mu'. So it calls the path driver callback (PDCB) I
provided to allow that sucker to do the real work on 'mu', providing 'mu's
parent directory baton to the callback. My callback function knows that I
wanted to delete 'mu' (I had that information stuffed into its baton), so it
does so by calling the delete_entry() function on the same editor that
svn_path_delta_driver() is using.
PDCB: delete_entry(db1, 'mu')
Then it returns.
SDPD continues, driving the editor to the next interesting path
('A/B/E/alpha'), which requires a few more open-dir's:
SDPD: db2 = open_directory(db1, 'A')
SDPD: db3 = open_directory(db2, 'A/B')
SDPD: db4 = open_directory(db3, 'A/B/E')
Having arrived at the next interesting path, it calls my callback to do the
interesting work, and the callback fills in some more of the missing editor
calls:
PDCB: fb = open_file(db4, 'A/B/E/alpha')
PDCB: change_file_prop(fb, 'foo', 'bar')
PDCB: close_file(fb)
Back in SDPD again, we drive to the next destination, which requires moving
up a directory and back down again into another subtree:
SDPD: close_directory(db4)
SDPD: db4 = open_directory(db3, 'A/D/G')
Then it calls the callback again for 'A/D/G/florp', which does:
PDCB: db5 = add_directory(db4, 'A/D/G/florp')
This time the callback does something a little different, though -- because
it opened or added a directory, it returns the dir baton for that directory
so that SDPD can use it to address immediate children (if any) of this
directory. In my example, there are none, so when program control returns
to SDPD, it just closes my new directory as part of its wrap-up of open
directories back to the root.
SDPD: close_directory(db5)
SDPD: close_directory(db4)
SDPD: close_directory(db3)
SDPD: close_directory(db2)
SDPD: close_directory(db1)
And the edit drive is complete save for the close_edit() called from outside
the svn_delta_path_driver() call tree.
> Now, after having written all then, I wonder if there would be a way
> to set up a system whereby the delta_path_driver callback was invoked
> during the replay of the queued items, rather than the initial calls
> to them. It would involve a bit more shim code, but this may be the
> better solution. Hmmm.....
Any consumer of the svn_delta_path_driver() will need to be using Ev1 --
that's what's supported by the API. If their particular editor is a shim to
an Ev2 under the hood, that's fine -- the caller need never be aware of that
any more than they would be if they were manually driving the editor. And
it sounds as if we don't need to rev svn_delta_path_driver() for an Ev2
flavor because Ev2 solves the primary problem that SDPD exists to solve:
the depth-first requirement of editor drives.
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand
Re: svn_delta_path_driver(), its purpose and future
Posted by Hyrum K Wright <hy...@wandisco.com>.
On Tue, Nov 29, 2011 at 6:24 AM, C. Michael Pilato <cm...@collab.net> wrote:
> On 11/28/2011 05:34 PM, Hyrum K Wright wrote:
>> In working on the Ev2 shims, I discovered svn_delta_path_driver(), a
>> nifty little API whose purpose I haven't totally yet discerned. It
>> looks to be some kind of hybridization of the editor, allowing a
>> caller to handle some portion of the editor drive manually, though the
>> documentation has enough ifs, buts, and wherefores to make me a bit
>> unsure.
>
> I created svn_delta_path_driver() for those situations where we have a set
> of paths to be modified, the full set of which we know in advance -- we
> aren't discovering them via an editor-compatible-deep-crawl that's easy to
> hook directly up to an editor, for example. It's primary value is that it
> does the work of figuring out which open_dir/close_dir pairs to fire off for
> intermediate directories (which generally *aren't* in that set of
> paths-to-be-changed), allowing it callback to act *only* on the "really
> changed" paths.
>
> The foremost uses which come to mind are client commits (which *do* perform
> a WC crawl, but flatten out the list of modified things long before using
> them to drive the commit editor) and anything that's driving an editor from
> the repository's "fs-paths-changed" output hash.
This sounds sensible, but I guess I'm having trouble meshing it with
my (albeit somewhat constrained) understanding of the delta editor.
If A wants to describe tree changes to B, B provides callbacks to be
invoked by A and A does so (within the confines of the API). Now this
new API shows up which changes those semantics somehow?
I'm just not sure what the interactions are, and hence I don't know
the best way to proceed in adapting them to Ev2. Will it require a
similar API, or will svn_delta_path_driver() just be obsoleted by Ev2?
And in the meantime, is there a way to bridge the two? I just don't
know enough to answer those kinds of questions.
>> If I am correct, however, it would appear that this API is not
>> compatible with our current editor shims, which queue up all changes
>> made in an editor, and then play them back through the new shim.
>>
>> Any illumination by folks knowing more than I would be appreciated.
>
> I'd have to know more about the new editor to make a value judgment here.
> But I can probably help you grok the ..._path_driver() if need it.
Ev2 doesn't bother with a depth-first tree traversal, the sender just
invokes paths in whatever order it sees fit. There are a few obvious
restrictions, such as directories need to exist before their children
can be processed, for example, but it's much looser than the delta
editor. Also, the APIs are a bit more streamlined, in that it's not
legal to set the properties on a node multiple times in the same
drive, for instance.
To handle the conversion from the delta editor to Ev2, we create an
editor which receives all the calls from the sender, queues them up,
and then drives an Ev2 editor in the receiver. Likewise, a similar
shim exists which receives Ev2 calls, queues them up and then drives a
delta editor in the proper order. It's all terribly inefficient due
to the fact that the entire drive is stored in memory prior to being
translated and replayed to the target editor (whether Ev2 or the delta
editor), but it's also meant as a temporary measure.
However, svn_delta_path_driver() looks like it interacts with the
editor while the thing is being driven. It counts on all the prior
calls actually reaching the target editor before the callback is
invoked, which means it will have an adverse reaction to something
which queues up the calls and then replays them, since none of the
preceding calls will have succeeded by the time its callback is
invoked.
Now, after having written all then, I wonder if there would be a way
to set up a system whereby the delta_path_driver callback was invoked
during the replay of the queued items, rather than the initial calls
to them. It would involve a bit more shim code, but this may be the
better solution. Hmmm.....
-Hyrum
--
uberSVN: Apache Subversion Made Easy
http://www.uberSVN.com/
Re: svn_delta_path_driver(), its purpose and future
Posted by "C. Michael Pilato" <cm...@collab.net>.
On 11/28/2011 05:34 PM, Hyrum K Wright wrote:
> In working on the Ev2 shims, I discovered svn_delta_path_driver(), a
> nifty little API whose purpose I haven't totally yet discerned. It
> looks to be some kind of hybridization of the editor, allowing a
> caller to handle some portion of the editor drive manually, though the
> documentation has enough ifs, buts, and wherefores to make me a bit
> unsure.
I created svn_delta_path_driver() for those situations where we have a set
of paths to be modified, the full set of which we know in advance -- we
aren't discovering them via an editor-compatible-deep-crawl that's easy to
hook directly up to an editor, for example. It's primary value is that it
does the work of figuring out which open_dir/close_dir pairs to fire off for
intermediate directories (which generally *aren't* in that set of
paths-to-be-changed), allowing it callback to act *only* on the "really
changed" paths.
The foremost uses which come to mind are client commits (which *do* perform
a WC crawl, but flatten out the list of modified things long before using
them to drive the commit editor) and anything that's driving an editor from
the repository's "fs-paths-changed" output hash.
> If I am correct, however, it would appear that this API is not
> compatible with our current editor shims, which queue up all changes
> made in an editor, and then play them back through the new shim.
>
> Any illumination by folks knowing more than I would be appreciated.
I'd have to know more about the new editor to make a value judgment here.
But I can probably help you grok the ..._path_driver() if need it.
--
C. Michael Pilato <cm...@collab.net>
CollabNet <> www.collab.net <> Distributed Development On Demand