Shared Memory Hash Table / SMS

[Ian Holsman <ia...@cnet.com>]

Just wondering if any has got one of these going, 

how does the proposed SMS shared library handle pointers?


..Ian

Re: Shared Memory Hash Table / SMS

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

On Mon, Jul 02, 2001 at 07:10:27PM +0200, Sander Striker wrote:
> > On Mon, Jul 02, 2001 at 09:38:37AM -0700, Ian Holsman wrote:
> > > Just wondering if any has got one of these going, 
> > > 
> > > how does the proposed SMS shared library handle pointers?
> > 
> > Can you please expand a bit further on what you mean?  
> > 
> > Oh, do you mean, "How does the proposed SMS shared memory system 
> > handle pointers in data structures that it has?"  Oh, that, I dunno.  
> > Prolly with some type of offset/fixup routine.  But, that may not 
> > be ideal.  -- justin
> 
> We basically still need to work that out :)
> I'm personally still not sold on the offsets idea. I'm still not
> getting _why_ we would need offsets instead of a simple pointer
> to a piece of mem.

when you request that a shared memory space be mapped into real
memory on different processes, it is extremely unlikely that it
will be mapped to the *same* physical location in memory on
each process (probability: 2^^-32 or 2^^-64...)

so therefore, if you store any pointers from one process _into_
the shared memory block, when they are retrieved by _another_
process, they will of course point to garbage.

there are a number of techniques for dealing with this, but
basically they boil down to structure-packing, effectively
treating the memory exactly as you would a very very fast file.

all best,

luke

RE: Shared Memory Hash Table / SMS

[Sander Striker <st...@apache.org>]

> On Mon, Jul 02, 2001 at 09:38:37AM -0700, Ian Holsman wrote:
> > Just wondering if any has got one of these going, 
> > 
> > how does the proposed SMS shared library handle pointers?
> 
> Can you please expand a bit further on what you mean?  
> 
> Oh, do you mean, "How does the proposed SMS shared memory system 
> handle pointers in data structures that it has?"  Oh, that, I dunno.  
> Prolly with some type of offset/fixup routine.  But, that may not 
> be ideal.  -- justin

We basically still need to work that out :)
I'm personally still not sold on the offsets idea. I'm still not
getting _why_ we would need offsets instead of a simple pointer
to a piece of mem.

Sander

Re: Shared Memory Hash Table / SMS

[Justin Erenkrantz <je...@ebuilt.com>]

On Mon, Jul 02, 2001 at 09:38:37AM -0700, Ian Holsman wrote:
> Just wondering if any has got one of these going, 
> 
> how does the proposed SMS shared library handle pointers?

Can you please expand a bit further on what you mean?  

Oh, do you mean, "How does the proposed SMS shared memory system 
handle pointers in data structures that it has?"  Oh, that, I dunno.  
Prolly with some type of offset/fixup routine.  But, that may not 
be ideal.  -- justin

RE: Shared Memory Hash Table / SMS

[Ian Holsman <ia...@cnet.com>]

On 02 Jul 2001 19:07:52 +0200, Sander Striker wrote:
> > Just wondering if any has got one of these going, 
> > how does the proposed SMS shared library handle pointers?
> 
> Sorry Ian, we're not quite there yet, at least, if you
> mean a shared mem sms.
> 
> I hope to commit the sms that handles multi threading in
> a nicer way tonight. I had planned on doing that yesterday,
> but was to tired to finish it.
> 
> David and I briefly discussed a dual memory pool to
> implement shared memory. One for accounting, one to be
> the actual memory. There will probably be more discussion
> needed on the list first.
> 
> What do you mean exactly with 'shared mem hash table / sms'?
> I mean, what kind of image is in your head? [curious]
> 
I have to store info on about ~100-700,000 entries (~10-64M) which will
sit in a sparse number range. These will get hit on each page
request, so it needs to be fast. Originally the number range was dense
so I could use an array.

what I was thinking of doing was passing the 'max' to the hash_create
function, and using offsets instead of pointers in the structures (which
the function would allocate)

For my app, I don't need to worry about locking, as the table is
write-once (at start) read-many

If sms has the shared memory type I would use that, but for the moment
I'll just use the apr_shm functions 

I'll submit it for addition to apr-util if I can get it working.

> > ...Ian
> 
> Sander

RE: Shared Memory Hash Table / SMS

[Sander Striker <st...@apache.org>]

> Just wondering if any has got one of these going, 
> how does the proposed SMS shared library handle pointers?

Sorry Ian, we're not quite there yet, at least, if you
mean a shared mem sms.

I hope to commit the sms that handles multi threading in
a nicer way tonight. I had planned on doing that yesterday,
but was to tired to finish it.

David and I briefly discussed a dual memory pool to
implement shared memory. One for accounting, one to be
the actual memory. There will probably be more discussion
needed on the list first.

What do you mean exactly with 'shared mem hash table / sms'?
I mean, what kind of image is in your head? [curious]

> ...Ian

Sander