Shared-Memory Abstraction

[Charles Randall <cr...@matchlogic.com>]

I've implemented a shared-memory abstraction that has been integrated
into our Apache-based server (That code is owned by the company, but I
could recreate something similar for Apache),

I believe that a shared-memory abstraction and a mutex/semaphore
abstraction go hand-in-hand. I did not provide a locking mechanism
because our data structures were read-only after initialization.

My implementation provides the following interface:

	MM_MALLOC MEMORY MANAGER

	A new general-purpose memory manager using memory-mapped memory
via the
	mmap() call was written. The following API is provided,

	Shared-Memory Pool Creation/Destruction:

	    struct mmap_pool *mm_create(size_t maxsize);
	    void  mm_destroy(struct mmap_pool *mmp);

	Shared-Memory Allocation:

	    void *mm_malloc(struct mmap_pool *mmp, size_t n);
	    void *mm_calloc(struct mmap_pool *mmp, size_t n, size_t
size);
	    void *mm_realloc(struct mmap_pool *mmp, void *p, size_t n);
	    void  mm_free(struct mmap_pool *mmp, void *p);

	Auxiliary Functions:

	    char *mm_strdup(struct mmap_pool *mmp, char *s);
	    int   mm_lockall(struct mmap_pool *mmp); /* lock entire
region as read-only */
	    int   mm_trim(struct mmap_pool *mmp); /* trim unused portion
of pool */

A typical use is:

o Guess a maximum size for the shared region
o Created a memory pool of that size
o Malloc/free/strdup/etc from that pool
o trim memory pool if necessary
o lock read-only if necessary
o destroy pool when done

This uses mmap() internally, but could be #ifdef'd to support other
systems such as Win32.

Would something like this fit into the model anyone else is thinking of?

Charles F. Randall
crandall@matchlogic.com
MatchLogic, Inc.

Re: Shared-Memory Abstraction

[Rodent of Unusual Size <Ke...@Golux.Com>]

Just WBSing it..  One thing I would like to see is a way for
modules to declare a shared-memory pool before the spawn.
This is several issues at once: a reliable shared-memory
interface supported in the server, allowing the pool routines
to work within a specific address range (the shared area),
and module actions performed by the parent.

I keep thinking how nice it would be for a module to share
information (tables, for instance) between the various children.
Implementing a stat() cache in the core with these basic pieces
would be relatively simple, it seems to me.

2.0 stuff, of course - but just a thought.

#ken	P-)}

Re: Shared-Memory Abstraction

[Dean Gaudet <dg...@arctic.org>]

One problem I see with this, comparing with how the scoreboard works now,
is that it can't be implemented on systems without shared memory
support... yeah I know that sounds illogical.  But the scoreboard right
now works, mostly, on systems where it has to be done via a file.  This is
a specific type of "shared" memory though, where each object in it has one
writer and maybe multiple readers, and has no real need for
synchronization.  (Which incidentally is the same need that the SNMP stuff
has.) 

Yup generally shared mem means a need for mutexes.  So they need to be
abstracted as well. 

To support this stuff portably I'm certain that we have to allow shared
memory and mutex allocation in the parent only, at init() time. ... and in
that case realloc and free aren't very useful. 

Dean

On Tue, 20 Jan 1998, Charles Randall wrote:

> I've implemented a shared-memory abstraction that has been integrated
> into our Apache-based server (That code is owned by the company, but I
> could recreate something similar for Apache),
> 
> I believe that a shared-memory abstraction and a mutex/semaphore
> abstraction go hand-in-hand. I did not provide a locking mechanism
> because our data structures were read-only after initialization.
> 
> My implementation provides the following interface:
> 
> 	MM_MALLOC MEMORY MANAGER
> 
> 	A new general-purpose memory manager using memory-mapped memory
> via the
> 	mmap() call was written. The following API is provided,
> 
> 	Shared-Memory Pool Creation/Destruction:
> 
> 	    struct mmap_pool *mm_create(size_t maxsize);
> 	    void  mm_destroy(struct mmap_pool *mmp);
> 
> 	Shared-Memory Allocation:
> 
> 	    void *mm_malloc(struct mmap_pool *mmp, size_t n);
> 	    void *mm_calloc(struct mmap_pool *mmp, size_t n, size_t
> size);
> 	    void *mm_realloc(struct mmap_pool *mmp, void *p, size_t n);
> 	    void  mm_free(struct mmap_pool *mmp, void *p);
> 
> 	Auxiliary Functions:
> 
> 	    char *mm_strdup(struct mmap_pool *mmp, char *s);
> 	    int   mm_lockall(struct mmap_pool *mmp); /* lock entire
> region as read-only */
> 	    int   mm_trim(struct mmap_pool *mmp); /* trim unused portion
> of pool */
> 
> A typical use is:
> 
> o Guess a maximum size for the shared region
> o Created a memory pool of that size
> o Malloc/free/strdup/etc from that pool
> o trim memory pool if necessary
> o lock read-only if necessary
> o destroy pool when done
> 
> This uses mmap() internally, but could be #ifdef'd to support other
> systems such as Win32.
> 
> Would something like this fit into the model anyone else is thinking of?
> 
> Charles F. Randall
> crandall@matchlogic.com
> MatchLogic, Inc.
> 
> 
>