New WC [not on disk]

[Miha Vitorovic <mv...@nil.si>]

Greg,

I hope this is the right way to ask (and if it's already answered 
somewhere feel free to point me to it). I have a question regarding the 
new WC, but first some background:

we primarily develop Lotus Domino applications, and a few years back we 
decided to start using a real development environment (at least for code), 
instead of  what Lotus thinks it is (well, they don't, but have never 
fixed it anyway). So we moved all our code to Java and switched to 
Eclipse. To do that, we dump all our code to the disk, develop there and 
re-import to Domino when finished. Seems like quite a long way, but isn't 
really. Eclipse also gave the ability to use the Subclipse plug-in and 
Subversion for source control. 

Since then two things have happened: 
- the new Notes client is based on Eclipse,
- you started your work on the WC with its centralized meta data 
possibility. 

So this opens up possibility to extend Subclipse to link directly with the 
new Domino client, having all sources on the Domino server, and the meta 
data on the developer's disk. But...
When I thought about it, I also read the old notes about the new WC. There 
it says that all operations work on streams, except the diff (IIRC), which 
needs FILE. It seems to me, that replacing the disk for network access 
would not be hard for streams, but quite impossible for FILE. So my first 
idea was that for that one (few) operation(s) that actually needs FILE to 
dump it to TMP in the background and give the API a handle to that...

So basically, what I'm asking (while you're still in the API design 
phase), do you think the new WC will enable us to do that? Or, if 
currently not, would it be possible to tweak the API so that it would?

If I was unclear in any way, please let me know.

Thanks and best regards,
---
  Miha Vitorovic
  Inženir v tehničnem področju
  Customer Support Engineer

   NIL Data Communications,  Tivolska cesta 48,  1000 Ljubljana,  Slovenia
   Phone +386 1 4746 500      Fax +386 1 4746 501     http://www.NIL.si

Re: New WC [not on disk]

[Erik Huelsmann <eh...@gmail.com>]

>> The choice here really depends on the DIFF library. I haven't looked
>> at it (yet). If it rewinds "often", then we'd want to cache
>> decompressed output to disk, and pass that file to DIFF (or do the
>> "open a stream again for a rewind" option).
>
> If I remember correctly our libsvn_diff library just keeps a token buffer
> instead of using rewinding to create the diff datastructure. It only rewinds
> when creating the actual output.

Right. You remember correctly.

Bye,

Erik.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

RE: New WC [not on disk]

[Bert Huijben <be...@vmoo.com>]

> -----Original Message-----
> From: Greg Stein [mailto:gstein@gmail.com]
> Sent: woensdag 17 september 2008 13:51
> To: Miha Vitorovic
> Cc: dev@subversion.tigris.org
> Subject: Re: New WC [not on disk]
> 
> Hey Miha,
> 
> My plan for the DIFF library was to do something like:
> 
> * ask WC for a FILE if available
>   * if available, then pass it to DIFF
>   * if not, then read the stream in its entirety and write to TMP on
> disk. pass TMP to DIFF
> 
> or:
> 
> * teach diff to use a custom "rewind" callback; the WC will get a
> stream again
> 
> The latter is no problem *if* the stream corresponds directly to a
> file on disk. But if the WC needs to decompress a base text in order
> to return the stream, then it could be expensive (we'd be
> decompressing multiple times). It could be possible to cache the
> decompressed version.
> 
> The choice here really depends on the DIFF library. I haven't looked
> at it (yet). If it rewinds "often", then we'd want to cache
> decompressed output to disk, and pass that file to DIFF (or do the
> "open a stream again for a rewind" option).

If I remember correctly our libsvn_diff library just keeps a token buffer
instead of using rewinding to create the diff datastructure. It only rewinds
when creating the actual output.

But you can (on the libsvn_client and wc level) just override our internal
diff and diff3 with an external application in the config file, and that
would require a (temporary) file anyway.


Grepping the code on its usage even shows the external diff3 tool can be
used by svn_wc_cleanup2() to perform merges. 
(I really have no idea why cleanup would launch a merge... I just hope WC2
will explain all this and maybe removes merges from cleanup)

	Bert


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@subversion.tigris.org
For additional commands, e-mail: dev-help@subversion.tigris.org

Re: New WC [not on disk]

[Greg Stein <gs...@gmail.com>]

Hey Miha,

My plan for the DIFF library was to do something like:

* ask WC for a FILE if available
  * if available, then pass it to DIFF
  * if not, then read the stream in its entirety and write to TMP on
disk. pass TMP to DIFF

or:

* teach diff to use a custom "rewind" callback; the WC will get a stream again

The latter is no problem *if* the stream corresponds directly to a
file on disk. But if the WC needs to decompress a base text in order
to return the stream, then it could be expensive (we'd be
decompressing multiple times). It could be possible to cache the
decompressed version.

The choice here really depends on the DIFF library. I haven't looked
at it (yet). If it rewinds "often", then we'd want to cache
decompressed output to disk, and pass that file to DIFF (or do the
"open a stream again for a rewind" option).

Now... to apply to your use case. If you're keeping all your source on
the network, then we'd say your ACTUAL tree is remote. The BASE and
WORKING would still be local and stored in your metadata store. I'm
not sure of all the touch points between WC and ACTUAL. I doubt that
it will be trivial, but it *does* seem possible. One of the goals of
the rewrite is to make it more obvious *which* tree you're talking
about. In that sense, it may be pretty easy to track down all
references to ACTUAL and flip those over to a network-based
"filesystem".

And blue-sky would provide a pluggable filesystem for ACTUAL to more
easily enable your scenario.

Cheers,
-g

2008/9/17 Miha Vitorovic <mv...@nil.si>:
>
> Greg,
>
> I hope this is the right way to ask (and if it's already answered somewhere
> feel free to point me to it). I have a question regarding the new WC, but
> first some background:
>
> we primarily develop Lotus Domino applications, and a few years back we
> decided to start using a real development environment (at least for code),
> instead of  what Lotus thinks it is (well, they don't, but have never fixed
> it anyway). So we moved all our code to Java and switched to Eclipse. To do
> that, we dump all our code to the disk, develop there and re-import to
> Domino when finished. Seems like quite a long way, but isn't really. Eclipse
> also gave the ability to use the Subclipse plug-in and Subversion for source
> control.
>
> Since then two things have happened:
> - the new Notes client is based on Eclipse,
> - you started your work on the WC with its centralized meta data
> possibility.
>
> So this opens up possibility to extend Subclipse to link directly with the
> new Domino client, having all sources on the Domino server, and the meta
> data on the developer's disk. But...
> When I thought about it, I also read the old notes about the new WC. There
> it says that all operations work on streams, except the diff (IIRC), which
> needs FILE. It seems to me, that replacing the disk for network access would
> not be hard for streams, but quite impossible for FILE. So my first idea was
> that for that one (few) operation(s) that actually needs FILE to dump it to
> TMP in the background and give the API a handle to that...
>
> So basically, what I'm asking (while you're still in the API design phase),
> do you think the new WC will enable us to do that? Or, if currently not,
> would it be possible to tweak the API so that it would?
>
> If I was unclear in any way, please let me know.
>
> Thanks and best regards,
> ---
>  Miha Vitorovic
>  Inženir v tehničnem področju
>  Customer Support Engineer
>
>   NIL Data Communications,  Tivolska cesta 48,  1000 Ljubljana,  Slovenia
>   Phone +386 1 4746 500      Fax +386 1 4746 501     http://www.NIL.si