RE: child-parent memory access / mod_perl / shared mem /inter-proccess communication , etc..

["Jim Morrison [Mailing-Lists]" <ju...@mediaisotope.com>]

Perrin,

> I suggest you look at IPC::MM or IPC::Shareable.  IPC::Shareable is
more
> transparent, but IPC::MM has better performance.  IPC::MM simply
creates
> a hash in shared memory and lets you write to it.  Either of these
will
> allow you to share data between processes.

Thanks, I'll have a play around with these!

>> What I really want to do is to write a module that can be accessed
from
>> any of the child apache processes such that some work can be done at
one
>> stage, and if the second request comes through to another child, that
>> child can pick up on the work of the first..

> That sounds pretty sketchy to me.  Why are you trying to do that?
There
> may be a much simpler way to achieve what you're after.

Hmm.. Yes, it sounds pretty sketchy to me too!  Immediately what I am
playing with is the idea of keeping parsed XML (XML::LibXML)in memory
between requests.  Is this a completely barmy idea?

ATB, J.

RE: child-parent memory access / mod_perl / shared mem /inter-proccess communication , etc..

[Perrin Harkins <pe...@elem.com>]

On Mon, 2003-02-24 at 05:40, Jim Morrison [Mailing-Lists] wrote:
> Hmm.. Yes, it sounds pretty sketchy to me too!  Immediately what I am
> playing with is the idea of keeping parsed XML (XML::LibXML)in memory
> between requests.  Is this a completely barmy idea?

Caching is a good idea, but it's not possible to share objects between
Perl processes without serializing them.  Try using Cache::Mmap,
Cache::FileCache, or MLDBM::Sync.

- Perrin

Re: child-parent memory access / mod_perl / shared mem / inter-proccess communication , etc..

[Perrin Harkins <pe...@elem.com>]

Matt already answered most of your questions.

Jim Morrison [Mailing-Lists] wrote:
 > And I'm getting some mileage out of
> experimenting with this anyhow - if it doesn't work for LibXML it looks
> like it could be very useful for other stuff, and if it ever works
> bug-free(-ish) I'd like to give something back to all the CPAN-people
> who've written things I use.. 

No offense, but you are reinventing the wheel.  There are many modules 
available for sharing cached data.  They all use Storable, so none will 
work on LibXML objects, but they are pretty well-known and mature 
modules: Cache::Mmap, Cache::Cache (which includes an option to use 
IPC::Sharelite), IPC::MM, MLDBM::Sync, etc.

- Perrin

RE: child-parent memory access / mod_perl / shared mem / inter-proccess communication , etc..

["Jim Morrison [Mailing-Lists]" <ju...@mediaisotope.com>]

> 
> Jim Morrison [Mailing-Lists] wrote:
> > Sniff..  I've kind of got something working... Enough such that one
> > httpd can request an XYZ, and if a second httpd comes along a little
> > later and requests the same XYZ then it will get it from shared
memory.
> 
> I hope you used one of the modules I suggested rather than writing the
> whole thing from scratch.

Perrin,

No :-) 'course I did. Thanks.. I had a play with the two IPC ones you
sent, Shareable and MM.. I settled on IPC::ShareLite for now.. which
seems the simplest to use.

> > As expected though.. trying to do this with LibXML objects fails.. a
> > lot.
> 
> LibXML is an XS module, so that may interfere with serialization.

Umm. Ok... Does that mean I'm doomed? (not sure what an XS module is) 


> > CAVEAT : I know it's probably looked at as a silly idea, but if I
could
> > keep my parsed stylesheets/xml's shared somewhere I'd save my self
> > having to re-parse for every request wouldn't I? . .
> 
> Why don't you try caching the results instead?

The CMS we use builds static content.. (is that what you mean?) so it
caches the whole site in one go I guess.. never been one for dynamic
content ;-)..  But there are places where I'm going to need dynamic
content - I would like to be able to do this in principle.. I mean, I
would like to cache my parsed stylesheets/xml in mem. rather than
re-parsing all the time..  And I'm getting some mileage out of
experimenting with this anyhow - if it doesn't work for LibXML it looks
like it could be very useful for other stuff, and if it ever works
bug-free(-ish) I'd like to give something back to all the CPAN-people
who've written things I use.. 

Kindest,

Jim.

Re: child-parent memory access / mod_perl / shared mem / inter-proccess communication , etc..

[Perrin Harkins <pe...@elem.com>]

Jim Morrison [Mailing-Lists] wrote:
> Sniff..  I've kind of got something working... Enough such that one
> httpd can request an XYZ, and if a second httpd comes along a little
> later and requests the same XYZ then it will get it from shared memory.

I hope you used one of the modules I suggested rather than writing the 
whole thing from scratch.

> As expected though.. trying to do this with LibXML objects fails.. a
> lot.

LibXML is an XS module, so that may interfere with serialization.

> CAVEAT : I know it's probably looked at as a silly idea, but if I could
> keep my parsed stylesheets/xml's shared somewhere I'd save my self
> having to re-parse for every request wouldn't I? . . 

Why don't you try caching the results instead?

- Perrin

Re: child-parent memory access / mod_perl / shared mem / inter-proccess communication , etc..

[Matt Sergeant <ma...@sergeant.org>]

On Monday, Feb 24, 2003, at 19:47 Europe/London, Jim Morrison 
[Mailing-Lists] wrote:

> What I thought was that if I kept the handle to the parsed XML open
> somewhere else then I would be able to use it.. so a separate process
> does the parsing and keeps hold of the handles of the currently 
> 'shared'
> bits of XML..  What comes out the other end is something along the 
> lines
> of:
>
> 	bless( do{\(my $o = 137110200)}, 'XML::LibXML::Document' )
>
> But trying to toString() it for instance gives me a :
>
> 	XML::LibXML::Document::_toString() -- self contains no data at
> [snip]/LibXML.pm line 659.
>
> To what does the "my $o = 137110200" refer?  Is there no way I can get
> the parsed object out of LibXML2 into another process?

It's a pointer to a region of memory. Since you have protected memory 
in your operating system, only the process that malloced that region of 
memory can access it, hence you can't access it from another process. 
And no, you can't make XML::LibXML allocate in shared memory.

The only conceivable way you could do it is create the DOM before the 
fork and rely on copy-on-write.

> CAVEAT : I know it's probably looked at as a silly idea, but if I could
> keep my parsed stylesheets/xml's shared somewhere I'd save my self
> having to re-parse for every request wouldn't I? . .

Before you try this, *please* investigate parsing every time your 
output needs to change (caching a-la AxKit). A very large site I know 
tried this and discovered that XML::LibXML/LibXSLT parses stylesheets 
*so* fast that it made zero difference to them to cache the stylesheets 
in memory (shared or otherwise).

Try it. You may discover you don't need all this caching.

Matt.

RE: child-parent memory access / mod_perl / shared mem / inter-proccess communication , etc..

["Jim Morrison [Mailing-Lists]" <ju...@mediaisotope.com>]

>> Hmm.. Yes, it sounds pretty sketchy to me too!  Immediately what I am
>> playing with is the idea of keeping parsed XML (XML::LibXML)in memory
>> between requests.  Is this a completely barmy idea?

> Probably, because you'll confuse XML::LibXML's garbage collector.

Sniff..  I've kind of got something working... Enough such that one
httpd can request an XYZ, and if a second httpd comes along a little
later and requests the same XYZ then it will get it from shared memory..
There's a max limit, and when that's reached the oldest XYZ is rotated
out in favor of the new one.

The 'XYZ' can be a string, hash, array, object etc (as far as I've
tried) by using Storable.. to serialize it.. 

As expected though.. trying to do this with LibXML objects fails.. a
lot.

What I thought was that if I kept the handle to the parsed XML open
somewhere else then I would be able to use it.. so a separate process
does the parsing and keeps hold of the handles of the currently 'shared'
bits of XML..  What comes out the other end is something along the lines
of:

	bless( do{\(my $o = 137110200)}, 'XML::LibXML::Document' )

But trying to toString() it for instance gives me a :

	XML::LibXML::Document::_toString() -- self contains no data at
[snip]/LibXML.pm line 659.

(Incidentally, the XML can be happily toString'ed from the process that
originally parsed it, long after.. so I know it's still lurking about
somewhere..)

To what does the "my $o = 137110200" refer?  Is there no way I can get
the parsed object out of LibXML2 into another process?  

CAVEAT : I know it's probably looked at as a silly idea, but if I could
keep my parsed stylesheets/xml's shared somewhere I'd save my self
having to re-parse for every request wouldn't I? . . 

ATB,

Jimbo 

RE: child-parent memory access / mod_perl / shared mem /inter-proccess communication , etc..

[Matt Sergeant <ma...@sergeant.org>]

On Mon, 24 Feb 2003, Jim Morrison [Mailing-Lists] wrote:

> Hmm.. Yes, it sounds pretty sketchy to me too!  Immediately what I am
> playing with is the idea of keeping parsed XML (XML::LibXML)in memory
> between requests.  Is this a completely barmy idea?

Probably, because you'll confuse XML::LibXML's garbage collector.

-- 
<!-- Matt -->
<:->get a SMart net</:->
Spam trap - do not mail: spam-sig@spamtrap.messagelabs.com