Re: Persitant data accross processes

["Aaron E. Ross" <ro...@coreference.com>]

>Hi all,
>
>create a new dbh for every process. I've looked at IPC::Shareable, but =

 why is a new dbh for each process a problem? how else would it work?

>it has to copy data. Meaning that I can only have a certain amount of =
>complexity to my data structures.

 I'm not sure why IPC::Shareable doesn't work, but maybe if you explain the problem 
 we can help.

 I've had the best success with BerkeleyDB for most tasks, I like the balance of simplicity, 
 speed, locking, and concurrency.  Just be sure to use BerkeleyDB and not DB_File, ok?

 You might also take a look at Cache::Cache for some sessions.

 There are also a number of modules that provide persistance frameworks: Alzabo, 
 Persistent::*, Tangram, Storable. See Data Type Marshalling and Persistent Storage
 in the Modules list on CPAN: http://www.cpan.org/modules/00modlist.long.htm

 Let me know how it works out.

 Aaron

Re: Persitant data accross processes

[Rodney Broom <rb...@Desert.NET>]

Hi all,

Thanks for  the input. I've replied to three messages in this one. In order,
they are Joachim, Olivier and Aaron. In my reply to Aaron, I've outlined in
better specificity what I'm actually after. One further note on my needs,
I'm not after session data. I've got a big information caching systems that
sits in memory between my database everything else. What I'm looking for is
some heavy performance improvements.

-------------

From: Joachim Zobel <nc...@netcologne.de>
JZ> We are using Data::Dumper to put the data structures as TEXT into a
MySQL
JZ> table, where they can retrieved via an id.

JZ> This is pretty fast for low to medium traffic and has the advantage that
JZ> the data is persistent (restarting apache won't zap the carts) and human
JZ> readable (debugging is easy).

Wow, ouch! Here's a performance gain for you:
- use Storable qw{freeze thaw store retrieve};
- Take your
    '$dumpered = Dumper($ref)' statement and changed it to
    '$frozen = freeze($ref)'
- Take your
    'eval $dumpered' and changed it to
    '$data = thaw($frozen)'
- For a human to read frozen data: print Dumper(thaw($frozen))

MUCH better gains than what I've just shown you are available. But this is
sure to be cheaper than evaling text.


-------------
From: Olivier Poitrey <rs...@rhapsodyk.net>
OP> I'm working on two modules that can help you to do this job. There names
are
OP> Apache::SharedMem and Apache::Cache. You can find them on :


-------------
From: Aaron E. Ross <ro...@coreference.com>
AER>  why is a new dbh for each process a problem? how else would it work?

Well, every new connection is overhead. So any reuse is good.


RB> >it has to copy data. Meaning that I can only have a certain amount of
RB> >complexity to my data structures.

AER>  I'm not sure why IPC::Shareable doesn't work, but maybe if you explain
the problem
AER>  we can help.

Hmm, you're right. I wasn't specific enough. What I'm after is having Perl
namespace level access to specific slots in memory. Not just the data that
~was~ once in that variable, but the variable itself. So, with two processes
I could say:

  pid 1
   $superclass::thing = "Here is PID $$";
  pid 2
   print $superclass::thing,"\n";   # This would contain PID 1

I can emulate this with any number of devices that look at a third location
for data and return the value. Options include IPC::Shareable, Berkley DB
files, RDBMSs and stone tables manged by carrier pigeons. Depending on the
transient data device, this works for any type of data, including simple
scalars and multi-level hashes and arrays. But the variables returned are
only copies of the data, not the actual variable itself.

- For example:
When you use IPC::Shareable to tie() a var to a chunk of shared memory, you
aren't accessing the memory that Perl has allocated for that variable, you
are accessing a chunk of "Shared Memory" allocated via your system's shmget
function. That means that IPC::Shareable has to do things like checking the
shared memory segment every time the var is accessed. And complex vars don't
get updated in shared memory in some circumstances.
  # Process 1
  tie %big_hash, 'IPC::Shareable",...
  $big_hash{x} = {hr => \%really_deap_hash}

  # Process 2
  tie %big_hash, 'IPC::Shareable",...
  $big_hash{x}->{this}->{that}->{the_other} = 'Not visible to PID 1';

I know, my example is wrong since IPC::Shareable ~does~ handle this
properly. But that package has some serious shared memory management
problems. So I've implimented a version that doesn't. But, I digress. The
IPC::Shareable example also has this problem:

  # Process 1
  tie %hash, 'IPC::Shareable",...
  $hash{x} = {simpe => 'data'}

  # Process 2
  tie %hash, 'IPC::Shareable",...
  $key_ref = $hash{x};

In process 2, $key_ref will maintain it's data even once $hash{x} changes.
That's because IPC::Shareable isn't really presenting me with Perl's memory
location, he's copied a frozen data structure out of shared memory and
letting me have a copy of it.


Again, thanks for all of the replies. It seems as though I never get replies
to my questions. ;-P

---
Rodney Broom
Programmer: Desert.Net