mem locking (was: Re: cvs commit: apr/memory/unix apr_sms.c)

[Greg Stein <gs...@lyra.org>]

This thread is getting rather long, but I don't see the motivation.

What is the requirement for this range locking? We don't have to do
everything imaginable simply because we can. Every time we load "features"
into APR(UTIL), that just means we need to maintain them. Why do we *need*
this feature? And will it be widely useful?

I'm not seeing it.

Cheers,
-g

On Mon, Jun 11, 2001 at 09:22:59AM +0100, Ben Laurie wrote:
> Luke Kenneth Casson Leighton wrote:
> > On Sat, Jun 09, 2001 at 06:34:21PM +0100, Ben Laurie wrote:
> > > Luke Kenneth Casson Leighton wrote:
> > > > uh... i thought i should point out: the original proposal
> > > > i recommended for the sms locking routine should pass
> > > > in the area of memory to be locked and its length,
> > > > and ditto for the unlocking.
> > >
> > > +1.
> > >
> > > > in this way, let's say you do mmap on a file, you can
> > > > use posix locking as the implementation of the locking
> > > > on the memory.
> > > >
> > > > [or can you?  i dunno, for sure :) ]
> > >
> > > I believe so.
> > >
> > > > and, if the argument is NULL, it means ' do a Global Lock'.
> > > >
> > > > if the unlock() argument is NULL, it means 'do a Global
> > > > Unlock - i.e. free all locks'.
> > >
> > > And if the locking mechanism doesn't support regions? (This is
> > > particularly important if you allow unlocking of subregions). The
> > > obvious answer being that we implement them in APR, of course.
> > 
> > well, that is what made me think that perhaps a separate
> > lock and lock_region be done, such that one or other
> > cold be supported.
> > 
> > or, you mandate providing locking, emulating regions by
> > ... errr.... providing ref-counting on a global lock (yuck!)
> > 
> > hm, that don't work.  nope: have to do the lock and
> > lock_region, then have a means for Users to detect
> > that a lock_region function is available / supported.
> > 
> > you know, this is all tending to suggest having a
> > 'apr_sms_get_features()' fn, returning a bitfield
> > of supported features.
> 
> You can emulate regions with global locks and shared memory, of course.
> And still tell people which you support natively.
> 
> Don't know whether its worth it - does anyone have any views?
> 
> Cheers,
> 
> Ben.
> 
> --
> http://www.apache-ssl.org/ben.html
> 
> "There is no limit to what a man can do or how far he can go if he
> doesn't mind who gets the credit." - Robert Woodruff

-- 
Greg Stein, http://www.lyra.org/

Re: mem locking (was: Re: cvs commit: apr/memory/unix apr_sms.c)

[Ben Laurie <be...@algroup.co.uk>]

David Reid wrote:
> 
> > This thread is getting rather long, but I don't see the motivation.
> >
> > What is the requirement for this range locking? We don't have to do
> > everything imaginable simply because we can. Every time we load "features"
> > into APR(UTIL), that just means we need to maintain them. Why do we *need*
> > this feature? And will it be widely useful?
> >
> > I'm not seeing it.
> 
> Me neither.  It'll be a PITA to implement and not worth our while IMHO.

Sheesh! I don't see it either, which is why I asked. :-)

Cheers,

Ben.

--
http://www.apache-ssl.org/ben.html

"There is no limit to what a man can do or how far he can go if he
doesn't mind who gets the credit." - Robert Woodruff

Re: mem locking (was: Re: cvs commit: apr/memory/unix apr_sms.c)

[David Reid <dr...@jetnet.co.uk>]

I can see why it might be useful in shared memory, so if it can be worked
into any shared sms's then fine (I can see a number of ways of doing it
there, but all are messy) but it's not something we need to add to the
general purpose sms's, IMHO.

It can be added to the individual sms's header file with no trouble.
Optional support isn't worth the hassle right now and I can't really imagine
it'll be needed.

david


> On Mon, Jun 11, 2001 at 10:04:09AM -0400, Bill Stoddard wrote:
> >
> > > > This thread is getting rather long, but I don't see the motivation.
> > > >
> > > > What is the requirement for this range locking? We don't have to do
> > > > everything imaginable simply because we can. Every time we load
"features"
> > > > into APR(UTIL), that just means we need to maintain them. Why do we
*need*
> > > > this feature? And will it be widely useful?
> > > >
> > > > I'm not seeing it.
> > >
> > > Me neither.  It'll be a PITA to implement and not worth our while
IMHO.
> > >
> > > david
> > >
> > >
> >
> > I agree
>
> having read ben's message [regarding emulation of range_lock() if
> it is not supported], i agree that emulating range locking is
> tricky, and may even ... *thinks* ... be unworkeable.  i dunno.
> have to think about it.  which means it's going to be tricky.
> which means complex.  which means complex code to maintain,
> which means nightmare, don't do it with big alarm bells.
>
> given this, david, bill, others, having optional support for
> sms->range_lock() in sms that [sms] Users have to detect and
> cater for if it they want or need it [for speed/optimising
> purposes], or just use sms->global_lock() if they can't
> be bothered:
>
> is _that_ an acceptable thing to do?
>
> not too complex, etc.
>
> lukes
>

Re: mem locking (was: Re: cvs commit: apr/memory/unix apr_sms.c)

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

On Mon, Jun 11, 2001 at 10:04:09AM -0400, Bill Stoddard wrote:
> 
> > > This thread is getting rather long, but I don't see the motivation.
> > >
> > > What is the requirement for this range locking? We don't have to do
> > > everything imaginable simply because we can. Every time we load "features"
> > > into APR(UTIL), that just means we need to maintain them. Why do we *need*
> > > this feature? And will it be widely useful?
> > >
> > > I'm not seeing it.
> > 
> > Me neither.  It'll be a PITA to implement and not worth our while IMHO.
> > 
> > david
> > 
> > 
> 
> I agree

having read ben's message [regarding emulation of range_lock() if
it is not supported], i agree that emulating range locking is
tricky, and may even ... *thinks* ... be unworkeable.  i dunno.
have to think about it.  which means it's going to be tricky.
which means complex.  which means complex code to maintain,
which means nightmare, don't do it with big alarm bells.

given this, david, bill, others, having optional support for
sms->range_lock() in sms that [sms] Users have to detect and
cater for if it they want or need it [for speed/optimising
purposes], or just use sms->global_lock() if they can't
be bothered:

is _that_ an acceptable thing to do?

not too complex, etc.

lukes

Re: mem locking (was: Re: cvs commit: apr/memory/unix apr_sms.c)

[Bill Stoddard <bi...@wstoddard.com>]

> > This thread is getting rather long, but I don't see the motivation.
> >
> > What is the requirement for this range locking? We don't have to do
> > everything imaginable simply because we can. Every time we load "features"
> > into APR(UTIL), that just means we need to maintain them. Why do we *need*
> > this feature? And will it be widely useful?
> >
> > I'm not seeing it.
> 
> Me neither.  It'll be a PITA to implement and not worth our while IMHO.
> 
> david
> 
> 

I agree

Bill

Re: mem locking (was: Re: cvs commit: apr/memory/unix apr_sms.c)

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

On Mon, Jun 11, 2001 at 11:45:07AM +0100, David Reid wrote:
> > This thread is getting rather long, but I don't see the motivation.
> >
> > What is the requirement for this range locking? 

multiple threads accessing same memory range.

see tdb for the motivation.

tdb is gdbm-like simultaneous reader, simultaneous WRITER
[unlike all of the other *dbm databases, which have funny things
like, if you write to this db, all bets are off.  ??? :)]
db-access, with optional mmap support.

the implementation for the data-access is hash-chains.
each hash-chain is *individually* locked, with a byte-range
posix lock.

they have some intriguing double-dealing with enumeration,
and are not recommending the use of the first/next as a result,
but recommend using the tdb_traverse() instead.

anyway.  if an sms-implementor of an mmap or thread-accessed
memory module decided to optimise it by having the admin-memory
byte-range-locked, that should be their choice to be able
to do that.

by not providing an sms->range_lock() fn, or equiv., that
optimisation possibility is pretty much guaranteed ruled out.

in a stackeable manner, anyway.


> We don't have to do
> > everything imaginable simply because we can. Every time we load "features"
> > into APR(UTIL), that just means we need to maintain them. Why do we *need*
> > this feature? And will it be widely useful?

do you think that allowing [sms] Developers to optimise
memory-administrative management is good?

and if so, does that help justify providing range locking
in sms? 

> > I'm not seeing it.

that's because i didn't explain/justify why i was recommending it.

which would help :)

luke

Re: mem locking (was: Re: cvs commit: apr/memory/unix apr_sms.c)

[David Reid <dr...@jetnet.co.uk>]

> This thread is getting rather long, but I don't see the motivation.
>
> What is the requirement for this range locking? We don't have to do
> everything imaginable simply because we can. Every time we load "features"
> into APR(UTIL), that just means we need to maintain them. Why do we *need*
> this feature? And will it be widely useful?
>
> I'm not seeing it.

Me neither.  It'll be a PITA to implement and not worth our while IMHO.

david

Re: mem locking (was: Re: cvs commit: apr/memory/unix apr_sms.c)

[Luke Kenneth Casson Leighton <lk...@samba-tng.org>]

> > > or, you mandate providing locking, emulating regions by
> > > ... errr.... providing ref-counting on a global lock (yuck!)
> > > 
> > > hm, that don't work.  nope: have to do the lock and
> > > lock_region, then have a means for Users to detect
> > > that a lock_region function is available / supported.
> > > 
> > > you know, this is all tending to suggest having a
> > > 'apr_sms_get_features()' fn, returning a bitfield
> > > of supported features.
> > 
> > You can emulate regions with global locks and shared memory, of course.
> > And still tell people which you support natively.
> > 
> > Don't know whether its worth it - does anyone have any views?

hiya ben,

well... yes, i do: i wouldn't recommend making anything
such as emulation of range locking 'mandatory' in sms.

it's quite complex, i guarantee you: see samba's
emulation of NT byte-locking on top of posix range locks!
given that... okay, it's a long story.  trust me: you
don't ever want to try it :) :)

anyway - the point of sms is KISS.  to the extent that we're
having quite complex discussions right now, as experienced
experts and developers, to stave off and help justify _not_
adding in complexity.

the fine line between simplicity/complexity and necessary/nice-to-have
_can_ be walked :)

so, my recommendation is, make range_locking() either
non-existent [favoured by some because i didn't mention
a justification for adding it, *duuh* :)] or optional.

i mean, to be honest, the number of smses that are likely
to _use_ range_locking() is probably going to be quite
small, although i can't speak for the people whose job it
is to worry about speedspeedspeedoptimiseoptimiseoptimise
in apr/httpd, esp. in the area memory handling :)  i only
recommend range_locking() as a possible way to provide
that speedoptimising if it's thought necessary.

otherwise, it doesn't bother me particularly if it's
not added.

all best ben,

luke