Re: Persistent storage (storable preferred as mechanism)

[David Nicol <da...@gmail.com>]

On 7/9/05, Perrin Harkins <pe...@elem.com> wrote:
> Steven Lembark wrote:
> > Also the dumped data is not (usually)
> > transferrable between platforms.
> 
> It has always worked for me if I use nfreeze() instead of freeze().
> 
> - Perrin

My gripe with Storable is that it is not possible to access into the storable
object without thawing the whole thing.  An idea has just occurred to
me however,
which is, it may be possible to create some kind of tie wrapper that would 
appear to be thawing a Storable object, but all data would continue to reside in
the disk file instead of in memory, with new data getting tacked onto the end.
Locking of course would be among the fairly short list of tricky parts
to implement
this.  So here's the question:  does it already exist?  Has anyone written a
wrapper around the Storable on-disk persistence format that gives a
set of tied objects
that continue to refer to the file?  Sort of like those tools that
open a compressed
archive as a directory?


-- 
David L Nicol
Aesop's fables, with text-sensitive advertising:
http://cronos.advenge.com/pc/aesop/start.html

Re: Persistent storage (storable preferred as mechanism)

["C. Jon Larsen" <jl...@richweb.com>]

On Mon, 11 Jul 2005, David Nicol wrote:

> My gripe with Storable is that it is not possible to access into the storable
> object without thawing the whole thing.  An idea has just occurred to
> me however,

Ideas are great but ....

Have you actually benchmarked Storable ? Its pretty fast indeed. I have 
been using it with file caches (its part of Cache::Cache) and databases 
(postgres, mysql, firebird) for quite some time and thawing an object or 
data structure just does not have that much overhead.

If you have really large data structures and you want to freeze and thaw 
only certain slices of them perhaps Tie::MLDBM might be what you are 
looking for.

-jon

Re: Persistent storage (storable preferred as mechanism)

[Perrin Harkins <pe...@elem.com>]

On Mon, 2005-07-11 at 12:48 -0500, David Nicol wrote:
> My gripe with Storable is that it is not possible to access into the storable
> object without thawing the whole thing.

So use Data::Dumper or XML, if that's what you want.  If you want speed,
use Storable.

- Perrin

Re: [Poop-group] Re: Persistent storage (storable preferred as mechanism)

[Sam Vilain <sa...@vilain.net>]

David Nicol wrote:
> My gripe with Storable is that it is not possible to access into the storable
> object without thawing the whole thing.  An idea has just occurred to
> me however,
> which is, it may be possible to create some kind of tie wrapper that would 
> appear to be thawing a Storable object, but all data would continue to reside in
> the disk file instead of in memory, with new data getting tacked onto the end.
> Locking of course would be among the fairly short list of tricky parts
> to implement
> this.  So here's the question:  does it already exist?  Has anyone written a
> wrapper around the Storable on-disk persistence format that gives a
> set of tied objects
> that continue to refer to the file?  Sort of like those tools that
> open a compressed
> archive as a directory?

You sound like you're after Prevayler-style persistence - ie, "persist
everything, don't make me specify the data schema, and lazy load when you
get to the objects I explicitly asked to save".

This is the storage style that Pixie brought to Perl, and I shamelessly
stole for Tangram::IDBIF and friends.

If you are using Tangram, and you have a "storable" column or "idbif"
property set that maps to a particular column, then you may freely have
other Storage objects in the data structure which is stored.  ie, every
point in the data structure which you said "$storage->insert($foo)",
will be subject to this lazy loading/updating principle.

There are plenty of caveats to the approach, but feel free to ask any
questions you might have here, on the t2-users list[1], or in #tangram
on irc.perl.org.

And yes, this uses tie ;)

Sam.

1. http://tangram.utsl.gen.nz/lists/t2-users/

Re: Persistent storage (storable preferred as mechanism)

[Steven Lembark <le...@wrkhors.com>]

> My gripe with Storable is that it is not possible to access into the

Break the structure into smaller pieces (e.g., a heavily
nested hash into separate values) and store them individually.
This works nicely with a database backend using blobs to 
manage access to the frozen data.


-- 
Steven Lembark                                       85-09 90th Street
Workhorse Computing                                Woodhaven, NY 11421
lembark@wrkhors.com                                     1 888 359 3508