push/pull problem

[Ben Collins-Sussman <su...@collab.net>]

The 'dumpstream' parser isn't quite done.  I've got a push-pull problem.

So we have this parser reading a dumpstream and driving a vtable.
Currently, we have one vtable implementation which commits new
revisions to a repository.

When the parser encounters new fulltext for a file, it calls a vtable
callback (set_fulltext), which is supposed to provide a writable
svn_stream_t that the parser can "push" the fulltext at.  (If the
vtable doesn't want any text, it can return a NULL stream instead.)

So now I'm implementing this set_fulltext callback.  The problem is
that the only way to set/change a file's contents is with
svn_fs_apply_textdelta.  This function also returns a stream that you
can 'push' txdelta windows at.

So how do we generate txdelta windows?  We have a single diffing
algorithm for that: svn_txdelta().  The function takes two
svn_stream_t's, "pulls" data from them, and emits windows that you can
then "push" at the filesystem.

Confused yet?  :-)

The problem in a nutshell, then, is that the vtable callback is having
data "pushed" at it by the parser, but the only way to move this data
into the filesystem is via a mechanism that wants to "pull" data.

I can't just set up some funky pipe whereby someone is pushing data at
one end, and someone else is pulling data at the other end.  There's a
fundamental conflict in flow-control there, unless I fork a child
process. :-)

So what are some possible solutions?

1.  Have the callback implement a stream which accepts all incoming
    data into a tmp file.  When the parser closes the stream, allow
    the fs to pull the data from the tmpfile.  ugh.

2.  Change the callback prototype, so that it pulls data from a parser
    stream.  If a vtable didn't want any text, it would immediately
    close the stream, and the parser would just skip-over any
    remaining textdata (as it does now, when the vtable isn't
    interested.)

Thoughts?



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: push/pull problem

[Ben Collins-Sussman <su...@collab.net>]

Branko Čibej <br...@xbc.nu> writes:

> >Thoughts?
> >
> svn_fs_set_contents will set the contents of any node; apply_textdelta
> calls it internally.
> Oh, that function doesn't exist? Well, you know what to do, then. :-)

Duh, you're right.

I just chatted with cmpilato, and certainly it would only be a Good
Thing to make the svn_fs.h API richer.  Give users a choice:  either
have them push txdelta windows at a file, or let them push fulltext
directly.  

And it turns out that the dag layer *already* has this routine
(svn_fs__dag_get_edit_stream) -- so I'm just going to wrap it in a
public function!

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: push/pull problem

[Branko Čibej <br...@xbc.nu>]

Ben Collins-Sussman wrote:

>So what are some possible solutions?
>
>1.  Have the callback implement a stream which accepts all incoming
>    data into a tmp file.  When the parser closes the stream, allow
>    the fs to pull the data from the tmpfile.  ugh.
>
>2.  Change the callback prototype, so that it pulls data from a parser
>    stream.  If a vtable didn't want any text, it would immediately
>    close the stream, and the parser would just skip-over any
>    remaining textdata (as it does now, when the vtable isn't
>    interested.)
>
>Thoughts?
>  
>
svn_fs_set_contents will set the contents of any node; apply_textdelta 
calls it internally.
Oh, that function doesn't exist? Well, you know what to do, then. :-)

-- 
Brane Čibej   <br...@xbc.nu>   http://www.xbc.nu/brane/



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org