You are viewing a plain text version of this content. The canonical link for it is here.
Posted to modperl@perl.apache.org by Bill Whillers <mo...@whohasit.com> on 2005/07/08 09:01:40 UTC
Persistent storage
Are there any peculiarities or recommendations with using either YAML,
Storable (or another means) to efficiently store session hashes and bring
them back to life safely from rdbms (mysql)?
So far, I've avoided using Data::Dumper (and eval) and just used 'split' with
stored comma-delimited hashes or generated require-able packages with
everything "ready". But now that we're under mod_perl, I'd like to find a
faster, safe way do this without needing to use Apache::Reload or other
avoidable overhead.
BW
Re: Persistent storage
Posted by Collin Winter <co...@gmail.com>.
Depending on how advanced your serialisation needs are, you might have
a look at Data::Dump::Streamer
(http://search.cpan.org/~yves/Data-Dump-Streamer-1.14/). Yves has put
quite a lot of work into making it handle just about every twisted
case you can devise; this includes sane dumping of closures (which
Storable and B::Deparse can't) and blessed regexes with overloaded
stringification. It's not as fast as some other modules, but it can't
be beat for flexibility.
Collin
On 7/8/05, Perrin Harkins <pe...@elem.com> wrote:
> On Fri, 2005-07-08 at 07:15 -0400, Steven Lembark wrote:
> > > So far, I've avoided using Data::Dumper (and eval) and just used 'split'
> > > with stored comma-delimited hashes or generated require-able packages
> > > with everything "ready". But now that we're under mod_perl, I'd like to
> > > find a faster, safe way do this without needing to use Apache::Reload or
> > > other avoidable overhead.
> >
> > <http://search.cpan.org/~ams/Storable-2.15/Storable.pm>
>
> Seconded. Storable has no competition.
>
> - Perrin
>
>
Re: Persistent storage (storable preferred as mechanism)
Posted by Perrin Harkins <pe...@elem.com>.
On Mon, 2005-07-11 at 12:48 -0500, David Nicol wrote:
> My gripe with Storable is that it is not possible to access into the storable
> object without thawing the whole thing.
So use Data::Dumper or XML, if that's what you want. If you want speed,
use Storable.
- Perrin
Re: Persistent storage (storable preferred as mechanism)
Posted by "C. Jon Larsen" <jl...@richweb.com>.
On Mon, 11 Jul 2005, David Nicol wrote:
> My gripe with Storable is that it is not possible to access into the storable
> object without thawing the whole thing. An idea has just occurred to
> me however,
Ideas are great but ....
Have you actually benchmarked Storable ? Its pretty fast indeed. I have
been using it with file caches (its part of Cache::Cache) and databases
(postgres, mysql, firebird) for quite some time and thawing an object or
data structure just does not have that much overhead.
If you have really large data structures and you want to freeze and thaw
only certain slices of them perhaps Tie::MLDBM might be what you are
looking for.
-jon
Re: [Poop-group] Re: Persistent storage (storable preferred as mechanism)
Posted by Sam Vilain <sa...@vilain.net>.
David Nicol wrote:
> My gripe with Storable is that it is not possible to access into the storable
> object without thawing the whole thing. An idea has just occurred to
> me however,
> which is, it may be possible to create some kind of tie wrapper that would
> appear to be thawing a Storable object, but all data would continue to reside in
> the disk file instead of in memory, with new data getting tacked onto the end.
> Locking of course would be among the fairly short list of tricky parts
> to implement
> this. So here's the question: does it already exist? Has anyone written a
> wrapper around the Storable on-disk persistence format that gives a
> set of tied objects
> that continue to refer to the file? Sort of like those tools that
> open a compressed
> archive as a directory?
You sound like you're after Prevayler-style persistence - ie, "persist
everything, don't make me specify the data schema, and lazy load when you
get to the objects I explicitly asked to save".
This is the storage style that Pixie brought to Perl, and I shamelessly
stole for Tangram::IDBIF and friends.
If you are using Tangram, and you have a "storable" column or "idbif"
property set that maps to a particular column, then you may freely have
other Storage objects in the data structure which is stored. ie, every
point in the data structure which you said "$storage->insert($foo)",
will be subject to this lazy loading/updating principle.
There are plenty of caveats to the approach, but feel free to ask any
questions you might have here, on the t2-users list[1], or in #tangram
on irc.perl.org.
And yes, this uses tie ;)
Sam.
1. http://tangram.utsl.gen.nz/lists/t2-users/
Re: Persistent storage (storable preferred as mechanism)
Posted by Steven Lembark <le...@wrkhors.com>.
> My gripe with Storable is that it is not possible to access into the
Break the structure into smaller pieces (e.g., a heavily
nested hash into separate values) and store them individually.
This works nicely with a database backend using blobs to
manage access to the frozen data.
--
Steven Lembark 85-09 90th Street
Workhorse Computing Woodhaven, NY 11421
lembark@wrkhors.com 1 888 359 3508
Re: Persistent storage (storable preferred as mechanism)
Posted by David Nicol <da...@gmail.com>.
On 7/9/05, Perrin Harkins <pe...@elem.com> wrote:
> Steven Lembark wrote:
> > Also the dumped data is not (usually)
> > transferrable between platforms.
>
> It has always worked for me if I use nfreeze() instead of freeze().
>
> - Perrin
My gripe with Storable is that it is not possible to access into the storable
object without thawing the whole thing. An idea has just occurred to
me however,
which is, it may be possible to create some kind of tie wrapper that would
appear to be thawing a Storable object, but all data would continue to reside in
the disk file instead of in memory, with new data getting tacked onto the end.
Locking of course would be among the fairly short list of tricky parts
to implement
this. So here's the question: does it already exist? Has anyone written a
wrapper around the Storable on-disk persistence format that gives a
set of tied objects
that continue to refer to the file? Sort of like those tools that
open a compressed
archive as a directory?
--
David L Nicol
Aesop's fables, with text-sensitive advertising:
http://cronos.advenge.com/pc/aesop/start.html
Re: Persistent storage
Posted by Perrin Harkins <pe...@elem.com>.
Steven Lembark wrote:
> Also the dumped data is not (usually)
> transferrable between platforms.
It has always worked for me if I use nfreeze() instead of freeze().
- Perrin
Re: Persistent storage
Posted by Steven Lembark <le...@wrkhors.com>.
>> > <http://search.cpan.org/~ams/Storable-2.15/Storable.pm>
>>
>> Seconded. Storable has no competition.
Main problem is its advantage: lower overhead than
Data::Dumper but you cannot look at it to tell what's
in the thing. Also the dumped data is not (usually)
transferrable between platforms.
--
Steven Lembark 85-09 90th Street
Workhorse Computing Woodhaven, NY 11421
lembark@wrkhors.com 1 888 359 3508
Re: Persistent storage
Posted by Bryan Lhuillier <br...@coveragesystems.com>.
Great. I'm using Storable now in a test environment.
Thanks for reckoning.
On Friday 08 July 2005 08:19, Perrin Harkins wrote:
> On Fri, 2005-07-08 at 07:15 -0400, Steven Lembark wrote:
> > > So far, I've avoided using Data::Dumper (and eval) and just used
> > > 'split' with stored comma-delimited hashes or generated require-able
> > > packages with everything "ready". But now that we're under mod_perl,
> > > I'd like to find a faster, safe way do this without needing to use
> > > Apache::Reload or other avoidable overhead.
> >
> > <http://search.cpan.org/~ams/Storable-2.15/Storable.pm>
>
> Seconded. Storable has no competition.
>
> - Perrin
Re: Persistent storage
Posted by Perrin Harkins <pe...@elem.com>.
On Fri, 2005-07-08 at 07:15 -0400, Steven Lembark wrote:
> > So far, I've avoided using Data::Dumper (and eval) and just used 'split'
> > with stored comma-delimited hashes or generated require-able packages
> > with everything "ready". But now that we're under mod_perl, I'd like to
> > find a faster, safe way do this without needing to use Apache::Reload or
> > other avoidable overhead.
>
> <http://search.cpan.org/~ams/Storable-2.15/Storable.pm>
Seconded. Storable has no competition.
- Perrin
Re: Persistent storage
Posted by Steven Lembark <le...@wrkhors.com>.
> So far, I've avoided using Data::Dumper (and eval) and just used 'split'
> with stored comma-delimited hashes or generated require-able packages
> with everything "ready". But now that we're under mod_perl, I'd like to
> find a faster, safe way do this without needing to use Apache::Reload or
> other avoidable overhead.
<http://search.cpan.org/~ams/Storable-2.15/Storable.pm>
--
Steven Lembark 85-09 90th Street
Workhorse Computing Woodhaven, NY 11421
lembark@wrkhors.com 1 888 359 3508