Subversion efficiency while handling Zope Data.fs files?

[Marcin Kasperski <Ma...@softax.com.pl>]

Has anyone tried using subversion for backup and versioning Zope 
Data.fs files? How efficient is this?

For those who do not know: Zope Data.fs file is usually fairly 
large binary file (database of all the objects and scripts 
making up the whole website) which in most cases is changed only 
by appending small chunks of data at the end of the file. So 
versioning Data.fs can mean scenario like: we have 50MB binary 
file, 500 times a day something appends new 50, 100 or 500 bytes 
at the end of it. 

How efficient can be commit in such a scenario?

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Subversion efficiency while handling Zope Data.fs files?

[Branko Čibej <br...@xbc.nu>]

Ben Collins-Sussman wrote:

>On Wed, 2004-04-14 at 12:15, Marcin Kasperski wrote:
>
>  
>
>>... and the question is how efficient will be the diff 
>>operation...
>>    
>>
>
>"How efficient?"  How do you want to measure it?  Assuming that Data.fs
>isn't already compressed, it will be exactly as efficient as all our
>other binary diff operations.  We have a single binary diff algorithm
>that we use on everything.  It works about as efficiently as xdelta,
>unless it's operating on data that's already compressed.
>  
>
Well, to be pedantic, it works about as efficiently as xdelta on 
compressed files, too -- xdelta will fall on its nose just as horribly 
as vdelta. :-)

-- Brane

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Subversion efficiency while handling Zope Data.fs files?

[Ben Collins-Sussman <su...@collab.net>]

On Wed, 2004-04-14 at 12:15, Marcin Kasperski wrote:

> ... and the question is how efficient will be the diff 
> operation...

"How efficient?"  How do you want to measure it?  Assuming that Data.fs
isn't already compressed, it will be exactly as efficient as all our
other binary diff operations.  We have a single binary diff algorithm
that we use on everything.  It works about as efficiently as xdelta,
unless it's operating on data that's already compressed.



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Subversion efficiency while handling Zope Data.fs files?

[Marcin Kasperski <Ma...@softax.com.pl>]

> > Has anyone tried using subversion for backup and versioning
> > Zope Data.fs files? How efficient is this?
>
> That's funny.  I *just* started teaching myself Zope over the
> weekend, and was wondering about similar things:  how does one
> version control a Zope application?

IMHO the troubles with version control are among main drawbacks 
of Zope. In fact, it seems that most people live with Zope 
internal versions mechanism and store somewhere Data.fs every 
time they decide to pack the database.

I remember some kind of CVS plugin for Zope but (at least when I 
reviewed it some time ago) it was tended to the 'operate on 
single file/object at the time'  concept, without natural 
support for commiting/updating the whole trees.

> I've been wondering if it makes more sense to export an entire
> "Zope folder" as XML, and keep that huge XML file (which
> contains everything: python code, data objects, page
> templates, all directory structure) in a Subversion
> repository.

You could. You could also probably use Zope XML-RPC access to 
write python scripts which would perform the whole task.

There is also some interesting experiment one could perform. Zope 
has layered architecture with fairly well separated database 
handling code and multiple implementations of database interface  
(in Zope slang called 'Storage'). Apart from the standard  
storage (Data.fs) there are storages using BerkeleyDB, Oracle 
and even filesystem-based DirectoryStorage.

I could imagine someone attempting to write Zope storage based 
on ... Subversion repository.

> > versioning Data.fs can mean scenario like: we have 50MB
> > binary file, 500 times a day something appends new 50, 100
> > or 500 bytes at the end of it.
> >
> > How efficient can be commit in such a scenario?
>
> Well, that means that 'svn commit' will only send the newly
> appended data to the repository.  And that same binary-diff
> will be used to store successive revisions of Data.fs in the
> repository.  The fact that Data.fs is binary is irrelevant; 
> Subversion treats all files and diffs as binary.

... and the question is how efficient will be the diff 
operation...

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org

Re: Subversion efficiency while handling Zope Data.fs files?

[Ben Collins-Sussman <su...@collab.net>]

On Wed, 2004-04-14 at 09:30, Marcin Kasperski wrote:
> Has anyone tried using subversion for backup and versioning Zope 
> Data.fs files? How efficient is this?

That's funny.  I *just* started teaching myself Zope over the weekend,
and was wondering about similar things:  how does one version control a
Zope application?

According to a person I met in irc://irc.freenode.net/zope, he claims
that most Zope developers keep their main business-logic python code in
external version control, but that the internal database file is simply
backed up.  (After all, Zope has its own internal version control on
objects.)

But the problem with this strategy, though, is that keeping only your
python code in SVN is incomplete.  If my friend checks out the SVN
project, they have a bunch of code, but no idea how to arrange it within
a Zope tree... no templates, no data objects, nothing.

I've been wondering if it makes more sense to export an entire "Zope
folder" as XML, and keep that huge XML file (which contains everything: 
python code, data objects, page templates, all directory structure) in a
Subversion repository.


> versioning Data.fs can mean scenario like: we have 50MB binary 
> file, 500 times a day something appends new 50, 100 or 500 bytes 
> at the end of it. 
> 
> How efficient can be commit in such a scenario?

Well, that means that 'svn commit' will only send the newly appended
data to the repository.  And that same binary-diff will be used to store
successive revisions of Data.fs in the repository.  The fact that
Data.fs is binary is irrelevant;  Subversion treats all files and diffs
as binary.



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@subversion.tigris.org
For additional commands, e-mail: users-help@subversion.tigris.org