You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@apr.apache.org by William A Rowe Jr <wr...@rowe-clan.net> on 2017/03/31 02:21:28 UTC
Default Linux mutex method
So almost two decades later, this is still odd.
apr.h:#define APR_USE_SHMEM_MMAP_TMP 0
apr.h:#define APR_USE_SHMEM_MMAP_SHM 0
apr.h:#define APR_USE_SHMEM_MMAP_ZERO 0
apr.h:#define APR_USE_SHMEM_SHMGET_ANON 0
apr.h:#define APR_USE_SHMEM_SHMGET 1
apr.h:#define APR_USE_SHMEM_MMAP_ANON 1
apr.h:#define APR_USE_SHMEM_BEOS 0
apr.h:#define APR_USE_FLOCK_SERIALIZE 0
apr.h:#define APR_USE_SYSVSEM_SERIALIZE 1
apr.h:#define APR_USE_POSIXSEM_SERIALIZE 0
apr.h:#define APR_USE_FCNTL_SERIALIZE 0
apr.h:#define APR_USE_PROC_PTHREAD_SERIALIZE 0
apr.h:#define APR_USE_PTHREAD_SERIALIZE 1
Now the _USE_ macro indicated the *decision* to elect one
specific default. Not a multiple choice question. What you might
be thinking of are APR_HAS_* macros.
SYSVSEM > PTHREAD in the code logic, so we are still using
sysv semaphores with all their defects over the choice of pthread
mutexes where both are supported on Linux.
Do we want to fix this in the 1.6.0 release?
Re: Default Linux mutex method
Posted by Jim Jagielski <ji...@jaguNET.com>.
> On Mar 30, 2017, at 10:21 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>
>
> SYSVSEM > PTHREAD in the code logic, so we are still using
> sysv semaphores with all their defects over the choice of pthread
> mutexes where both are supported on Linux.
>
Which is faster?
Re: Default Linux mutex method
Posted by Florian Weimer <fw...@deneb.enyo.de>.
* William A. Rowe, Jr.:
> Are modern pthread mutexes sufficiently reliable this decade?
glibc and the Linux kernel have bugs in this area, but the basic
functionality is there.
One problem could be that a switch might not be backwards-compatible
from an application point of view. 32-bit and 64-bit applications
would not be able to use the same mutex.
Upgrading glibc could become more complicated if the shared mutex
isn't recreated on boot, but persisted in the file system. Berkeley
DB has this problem with process-shared condition variables, but the
same problem applies to mutexes as well:
<https://bugzilla.redhat.com/show_bug.cgi?id=1394862>
POSIX requires that the mutex is reinitialized after the last process
detaches from a file mapping (perhaps due to a system reboot), so this
is not a conformance issue, strictly speaking.
Re: Default Linux mutex method
Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Fri, Mar 31, 2017 at 10:22 AM, Jim Jagielski <ji...@jagunet.com> wrote:
> Reading the configure.in file, it looks like we are due
> for some updates for the 21st century:
>
> # See which lock mechanism we'll select by default on this system.
> # The last APR_DECIDE to execute sets the default.
> # At this stage, we match the ordering in Apache 1.3
> # which is (highest to lowest): sysvsem -> fcntl -> flock.
> # POSIX semaphores and cross-process pthread mutexes are not
> # used by default since they have less desirable behaviour when
> # e.g. a process holding the mutex segfaults.
LOL - it was re-reading Roy's 1.2/1.3 timeframe cautions about
avoiding sysvsem that brought me back to this topic, actually.
http://httpd.apache.org/docs/2.0/misc/perf-tuning.html
The pthread mutexes certainly should be cleaned up on a segv.
The sysvsem mutexes will not be cleaned up, just found another
annoyed user on Wednesday which reminded me to look at this.
Are modern pthread mutexes sufficiently reliable this decade?
On Fri, Mar 31, 2017 at 9:58 AM, Nick Kew <ni...@apache.org> wrote:
>
> I’ve just (finally) got the toolchain to build & test this on MacOS,
> and I’ve got identical APR_USE results there. Perhaps our build logic
> really isn’t doing anything very smart at all?
locks/unix/proc_mutex.c is the sole apr consumer of these flags, see
line 1177 within proc_mutex_choose_method(). Posix is dead last,
followed by pthread, fcntl, sysv and file in ascending priority. Without
changing the autoconf logic in the slightest, I would think we aught to
simply flip the prioritization of pthread and sysv.
But we do need BSD/Solaris/OSX feedback here.
Re: Default Linux mutex method
Posted by Jim Jagielski <ji...@jaguNET.com>.
Reading the configure.in file, it looks like we are due
for some updates for the 21st century:
# See which lock mechanism we'll select by default on this system.
# The last APR_DECIDE to execute sets the default.
# At this stage, we match the ordering in Apache 1.3
# which is (highest to lowest): sysvsem -> fcntl -> flock.
# POSIX semaphores and cross-process pthread mutexes are not
# used by default since they have less desirable behaviour when
# e.g. a process holding the mutex segfaults.
Re: Default Linux mutex method
Posted by Rainer Jung <ra...@kippdata.de>.
Am 01.04.2017 um 19:02 schrieb Yann Ylavic:
> On Fri, Mar 31, 2017 at 4:58 PM, Nick Kew <ni...@apache.org> wrote:
>>
>>> On 31 Mar 2017, at 03:21, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>>>
>>> So almost two decades later, this is still odd.
>>
>> Is this a reference to some technical discussion on the subject you recollect?
>>
>> Have you verified that all APR_USE_*_SERIALIZE are mutually exclusive,
>> or could it be that we use both in different contexts, perhaps for a valid reason?
>>
>>> apr.h:#define APR_USE_SHMEM_MMAP_TMP 0
>>> apr.h:#define APR_USE_SHMEM_MMAP_SHM 0
>>> apr.h:#define APR_USE_SHMEM_MMAP_ZERO 0
>>> apr.h:#define APR_USE_SHMEM_SHMGET_ANON 0
>>> apr.h:#define APR_USE_SHMEM_SHMGET 1
>>> apr.h:#define APR_USE_SHMEM_MMAP_ANON 1
>>> apr.h:#define APR_USE_SHMEM_BEOS 0
>>> apr.h:#define APR_USE_FLOCK_SERIALIZE 0
>>> apr.h:#define APR_USE_SYSVSEM_SERIALIZE 1
>>> apr.h:#define APR_USE_POSIXSEM_SERIALIZE 0
>>> apr.h:#define APR_USE_FCNTL_SERIALIZE 0
>>> apr.h:#define APR_USE_PROC_PTHREAD_SERIALIZE 0
>>> apr.h:#define APR_USE_PTHREAD_SERIALIZE 1
>>
>> I\u2019ve just (finally) got the toolchain to build & test this on MacOS,
>> and I\u2019ve got identical APR_USE results there. Perhaps our build logic
>> really isn\u2019t doing anything very smart at all?
>
> Looks good to me (as said in the other message), nothing mutually
> exclusive AFAICT.
>
>>
>>> SYSVSEM > PTHREAD in the code logic, so we are still using
>>> sysv semaphores with all their defects over the choice of pthread
>>> mutexes where both are supported on Linux.
>
> The main defect being the leak of an (limited) IPC entry when httpd
> does not shutdown cleanly, and that it's slower than pthreads, though
> it's hardly measurable probably.
>
>>
>> Probably applies more widely than just Linux. But might a general fix
>> risk breaking some minority Unix platform or cross-compile?
>
> SYSVSEM are robust (can be recovered after a process owning one
> crashes), so the switch to PTHREAD should depend on
> HAVE_PTHREAD_MUTEX_ROBUST only I think (which should be the case for
> Linux and Solaris at least).
>
> Solaris had issue a while ago ([1], which seems to be the "why" for
> prefering SYSVSEM over some shared-pthread implementations), but it's
> probably addressed now (Rainer maybe?).
First current autodetect results for Solaris 8 (ancient) and 10 (still
in use). I don't have 11 results easily at hand but could provide them:
Solaris 8
#define APR_USE_SHMEM_MMAP_TMP 0
#define APR_USE_SHMEM_MMAP_SHM 0
#define APR_USE_SHMEM_MMAP_ZERO 0
#define APR_USE_SHMEM_SHMGET_ANON 0
#define APR_USE_SHMEM_SHMGET 1
#define APR_USE_SHMEM_MMAP_ANON 1
#define APR_USE_SHMEM_BEOS 0
#define APR_USE_FLOCK_SERIALIZE 0
#define APR_USE_SYSVSEM_SERIALIZE 0
#define APR_USE_POSIXSEM_SERIALIZE 0
#define APR_USE_FCNTL_SERIALIZE 1
#define APR_USE_PROC_PTHREAD_SERIALIZE 0
#define APR_USE_PTHREAD_SERIALIZE 1
Solaris 10:
the same except for
#define APR_USE_FCNTL_SERIALIZE 0 (instead of 1)
#define APR_USE_PROC_PTHREAD_SERIALIZE 1 (instead of 0)
Both versions detect/set HAVE_PTHREAD_MUTEX_ROBUST during configure.
I don't have first hand knowledge, but the topic came up a couple of
times. There's this old conversation between Joe and Jeff:
http://marc.info/?t=108720981900003&r=1&w=2
Then we have included 5 years later since APR 1.3.10:
http://svn.apache.org/viewvc?rev=881301&view=rev
From the log:
Solaris 10 or later: Use pthread by default for cross-process mutex
instead of fcntl; the latter is not thread-aware, leading to EDEADLK
failures with some multi-threaded, multi-process applications.
Proper recovery after a crash of the mutex owner was tested on Solaris
10 U5 and OpenSolaris 2009.06. Earlier Solaris versions weren't tested,
and there were reports in httpd-land long ago about failures to recover
the mutex.
The list discussion (triggered by Jim originally not restricted to
Solaris) that lead to this decision can be found here:
http://marc.info/?t=123843657300003&r=1&w=2
It contains some more details.
I do not know anything more recent then contained there.
>>> Do we want to fix this in the 1.6.0 release?
>>
>> How confident are you of not breaking anything with a fix?
>
> The only proc_pthread issue I can think of is [2] (long thread), which
> we addressed in both 1.5.x and 1.6.x, so +1 for me for the switch
> (when HAVE_PTHREAD_MUTEX_ROBUST).
>
>
> Regards,
> Yann.
>
> [1] https://bz.apache.org/bugzilla/show_bug.cgi?id=21322#c18
> [2] https://bz.apache.org/bugzilla/show_bug.cgi?id=49504
Regards,
Rainer
Re: Default Linux mutex method
Posted by Yann Ylavic <yl...@gmail.com>.
On Fri, Mar 31, 2017 at 4:58 PM, Nick Kew <ni...@apache.org> wrote:
>
>> On 31 Mar 2017, at 03:21, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>>
>> So almost two decades later, this is still odd.
>
> Is this a reference to some technical discussion on the subject you recollect?
>
> Have you verified that all APR_USE_*_SERIALIZE are mutually exclusive,
> or could it be that we use both in different contexts, perhaps for a valid reason?
>
>> apr.h:#define APR_USE_SHMEM_MMAP_TMP 0
>> apr.h:#define APR_USE_SHMEM_MMAP_SHM 0
>> apr.h:#define APR_USE_SHMEM_MMAP_ZERO 0
>> apr.h:#define APR_USE_SHMEM_SHMGET_ANON 0
>> apr.h:#define APR_USE_SHMEM_SHMGET 1
>> apr.h:#define APR_USE_SHMEM_MMAP_ANON 1
>> apr.h:#define APR_USE_SHMEM_BEOS 0
>> apr.h:#define APR_USE_FLOCK_SERIALIZE 0
>> apr.h:#define APR_USE_SYSVSEM_SERIALIZE 1
>> apr.h:#define APR_USE_POSIXSEM_SERIALIZE 0
>> apr.h:#define APR_USE_FCNTL_SERIALIZE 0
>> apr.h:#define APR_USE_PROC_PTHREAD_SERIALIZE 0
>> apr.h:#define APR_USE_PTHREAD_SERIALIZE 1
>
> I’ve just (finally) got the toolchain to build & test this on MacOS,
> and I’ve got identical APR_USE results there. Perhaps our build logic
> really isn’t doing anything very smart at all?
Looks good to me (as said in the other message), nothing mutually
exclusive AFAICT.
>
>> SYSVSEM > PTHREAD in the code logic, so we are still using
>> sysv semaphores with all their defects over the choice of pthread
>> mutexes where both are supported on Linux.
The main defect being the leak of an (limited) IPC entry when httpd
does not shutdown cleanly, and that it's slower than pthreads, though
it's hardly measurable probably.
>
> Probably applies more widely than just Linux. But might a general fix
> risk breaking some minority Unix platform or cross-compile?
SYSVSEM are robust (can be recovered after a process owning one
crashes), so the switch to PTHREAD should depend on
HAVE_PTHREAD_MUTEX_ROBUST only I think (which should be the case for
Linux and Solaris at least).
Solaris had issue a while ago ([1], which seems to be the "why" for
prefering SYSVSEM over some shared-pthread implementations), but it's
probably addressed now (Rainer maybe?).
>
>> Do we want to fix this in the 1.6.0 release?
>
> How confident are you of not breaking anything with a fix?
The only proc_pthread issue I can think of is [2] (long thread), which
we addressed in both 1.5.x and 1.6.x, so +1 for me for the switch
(when HAVE_PTHREAD_MUTEX_ROBUST).
Regards,
Yann.
[1] https://bz.apache.org/bugzilla/show_bug.cgi?id=21322#c18
[2] https://bz.apache.org/bugzilla/show_bug.cgi?id=49504
Re: Default Linux mutex method
Posted by Nick Kew <ni...@apache.org>.
> On 31 Mar 2017, at 03:21, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>
> So almost two decades later, this is still odd.
Is this a reference to some technical discussion on the subject you recollect?
Have you verified that all APR_USE_*_SERIALIZE are mutually exclusive,
or could it be that we use both in different contexts, perhaps for a valid reason?
> apr.h:#define APR_USE_SHMEM_MMAP_TMP 0
> apr.h:#define APR_USE_SHMEM_MMAP_SHM 0
> apr.h:#define APR_USE_SHMEM_MMAP_ZERO 0
> apr.h:#define APR_USE_SHMEM_SHMGET_ANON 0
> apr.h:#define APR_USE_SHMEM_SHMGET 1
> apr.h:#define APR_USE_SHMEM_MMAP_ANON 1
> apr.h:#define APR_USE_SHMEM_BEOS 0
> apr.h:#define APR_USE_FLOCK_SERIALIZE 0
> apr.h:#define APR_USE_SYSVSEM_SERIALIZE 1
> apr.h:#define APR_USE_POSIXSEM_SERIALIZE 0
> apr.h:#define APR_USE_FCNTL_SERIALIZE 0
> apr.h:#define APR_USE_PROC_PTHREAD_SERIALIZE 0
> apr.h:#define APR_USE_PTHREAD_SERIALIZE 1
I’ve just (finally) got the toolchain to build & test this on MacOS,
and I’ve got identical APR_USE results there. Perhaps our build logic
really isn’t doing anything very smart at all?
> SYSVSEM > PTHREAD in the code logic, so we are still using
> sysv semaphores with all their defects over the choice of pthread
> mutexes where both are supported on Linux.
Probably applies more widely than just Linux. But might a general fix
risk breaking some minority Unix platform or cross-compile?
> Do we want to fix this in the 1.6.0 release?
How confident are you of not breaking anything with a fix?
—
Nick Kew
Re: Default Linux mutex method
Posted by Yann Ylavic <yl...@gmail.com>.
On Mon, Apr 3, 2017 at 9:06 PM, Jim Jagielski <ji...@jagunet.com> wrote:
> Considering the sad affair w/ pthread on OSX, I would
> recommend we stay w/ using sems.
pthread_cond_timedlock() is not really mandatory IMHO (we did not have
it in 1.5.x), and anyway it's not implemented with any other mechanism
either...
If shared pthreads are robust and have no issues with latests Linux,
BSDs (including MacOS) and Solaris, I think we should use them by
default, and fall back to the current default for older versions
(known to not work as expected) only.
Re: Default Linux mutex method
Posted by Jim Jagielski <ji...@jaguNET.com>.
I think that "issue" is that all such options exist on all
our (modern) platforms and we don't adjust them for specific
OSs... We prefer sysv sems and pthread mutexes.
> On Apr 4, 2017, at 6:27 AM, Nick Kew <ni...@apache.org> wrote:
>
> On Tue, 2017-04-04 at 10:07 +0200, Stefan Sperling wrote:
>> For APR 1.5.2 on OpenBSD 6.0 we have this:
>> (chop)
>
> Thanks. Every little helps.
>
> Looks identical to what we've seen on Linux and MacOSX
> platforms. Have platforms converged, or has our build
> rusted away? It would be interesting to see reports
> from more diverse platforms, including non-x86-based.
>
> Is anyone thinking of tinkering with this on a timescale
> for 1.6.0? If so, please make it a config option and
> default to existing options, on ain't-broke-don't-fix
> principles.
>
> --
> Nick Kew
>
Re: Default Linux mutex method
Posted by Nick Kew <ni...@apache.org>.
On Tue, 2017-04-04 at 10:07 +0200, Stefan Sperling wrote:
> For APR 1.5.2 on OpenBSD 6.0 we have this:
> (chop)
Thanks. Every little helps.
Looks identical to what we've seen on Linux and MacOSX
platforms. Have platforms converged, or has our build
rusted away? It would be interesting to see reports
from more diverse platforms, including non-x86-based.
Is anyone thinking of tinkering with this on a timescale
for 1.6.0? If so, please make it a config option and
default to existing options, on ain't-broke-don't-fix
principles.
--
Nick Kew
Re: Default Linux mutex method
Posted by Stefan Sperling <st...@stsp.name>.
On Mon, Apr 03, 2017 at 05:36:02PM -0500, William A Rowe Jr wrote:
> On Mon, Apr 3, 2017 at 5:00 PM, Yann Ylavic <yl...@gmail.com> wrote:
> > On Mon, Apr 3, 2017 at 9:24 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> >
> >> Given that we didn't elect PROC_PTHREAD mutexes as an alternate
> >> _USE_ schema, I don't have a strong opinion about changing this
> >> for 1.6.0. It can be overridden by the implementer.
> >
> > Yes, ./configure apr_lock_method=USE_PROC_PTHREAD_SERIALIZE works.
> >
> >> It seems we really
> >> should change it on trunk and get that tested in the wild by fellow devs,
> >> at least.
> >
> > I can live with this too :)
>
> For completeness, FC25's default after their patches;
>
> /usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_MMAP_TMP 0
> /usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_MMAP_SHM 0
> /usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_MMAP_ZERO 0
> /usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_SHMGET_ANON 0
> /usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_SHMGET 1
> /usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_MMAP_ANON 1
> /usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_BEOS 0
> /usr/include/apr-1/apr-x86_64.h:#define APR_USE_FLOCK_SERIALIZE 0
> /usr/include/apr-1/apr-x86_64.h:#define APR_USE_SYSVSEM_SERIALIZE 1
> /usr/include/apr-1/apr-x86_64.h:#define APR_USE_POSIXSEM_SERIALIZE 0
> /usr/include/apr-1/apr-x86_64.h:#define APR_USE_FCNTL_SERIALIZE 0
> /usr/include/apr-1/apr-x86_64.h:#define APR_USE_PROC_PTHREAD_SERIALIZE 0
> /usr/include/apr-1/apr-x86_64.h:#define APR_USE_PTHREAD_SERIALIZE 1
>
> So RH hasn't seen fit to override the default. Feel free to compare to your
> $favorite_distro.
For APR 1.5.2 on OpenBSD 6.0 we have this:
#define APR_USE_SHMEM_MMAP_TMP 0
#define APR_USE_SHMEM_MMAP_SHM 0
#define APR_USE_SHMEM_MMAP_ZERO 0
#define APR_USE_SHMEM_SHMGET_ANON 0
#define APR_USE_SHMEM_SHMGET 1
#define APR_USE_SHMEM_MMAP_ANON 1
#define APR_USE_SHMEM_BEOS 0
#define APR_USE_FLOCK_SERIALIZE 0
#define APR_USE_SYSVSEM_SERIALIZE 1
#define APR_USE_POSIXSEM_SERIALIZE 0
#define APR_USE_FCNTL_SERIALIZE 0
#define APR_USE_PROC_PTHREAD_SERIALIZE 0
#define APR_USE_PTHREAD_SERIALIZE 1
The pthread functions pthread_cond_timedwait() and pthread_mutex_timedlock()
mentioned earlier in this discussion both exist.
Once the defaults are changed on APR trunk, I can test APR trunk on
OpenBSD 6.0 (and 6.1, due for release May 1).
If anything is missing I can poke OpenBSD developers who are working
on pthreads and try to get them to add interfaces APR needs.
I can't say anything about other BSDs, I'm afraid. I don't have
the time to keep up with all of them.
Re: Default Linux mutex method
Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Mon, Apr 3, 2017 at 5:00 PM, Yann Ylavic <yl...@gmail.com> wrote:
> On Mon, Apr 3, 2017 at 9:24 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
>
>> Given that we didn't elect PROC_PTHREAD mutexes as an alternate
>> _USE_ schema, I don't have a strong opinion about changing this
>> for 1.6.0. It can be overridden by the implementer.
>
> Yes, ./configure apr_lock_method=USE_PROC_PTHREAD_SERIALIZE works.
>
>> It seems we really
>> should change it on trunk and get that tested in the wild by fellow devs,
>> at least.
>
> I can live with this too :)
For completeness, FC25's default after their patches;
/usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_MMAP_TMP 0
/usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_MMAP_SHM 0
/usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_MMAP_ZERO 0
/usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_SHMGET_ANON 0
/usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_SHMGET 1
/usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_MMAP_ANON 1
/usr/include/apr-1/apr-x86_64.h:#define APR_USE_SHMEM_BEOS 0
/usr/include/apr-1/apr-x86_64.h:#define APR_USE_FLOCK_SERIALIZE 0
/usr/include/apr-1/apr-x86_64.h:#define APR_USE_SYSVSEM_SERIALIZE 1
/usr/include/apr-1/apr-x86_64.h:#define APR_USE_POSIXSEM_SERIALIZE 0
/usr/include/apr-1/apr-x86_64.h:#define APR_USE_FCNTL_SERIALIZE 0
/usr/include/apr-1/apr-x86_64.h:#define APR_USE_PROC_PTHREAD_SERIALIZE 0
/usr/include/apr-1/apr-x86_64.h:#define APR_USE_PTHREAD_SERIALIZE 1
So RH hasn't seen fit to override the default. Feel free to compare to your
$favorite_distro.
Re: Default Linux mutex method
Posted by Yann Ylavic <yl...@gmail.com>.
On Mon, Apr 3, 2017 at 9:24 PM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> Given that we didn't elect PROC_PTHREAD mutexes as an alternate
> _USE_ schema, I don't have a strong opinion about changing this
> for 1.6.0. It can be overridden by the implementer.
Yes, ./configure apr_lock_method=USE_PROC_PTHREAD_SERIALIZE works.
> It seems we really
> should change it on trunk and get that tested in the wild by fellow devs,
> at least.
I can live with this too :)
Re: Default Linux mutex method
Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Mon, Apr 3, 2017 at 2:06 PM, Jim Jagielski <ji...@jagunet.com> wrote:
> Considering the sad affair w/ pthread on OSX, I would
> recommend we stay w/ using sems.
For darwin (and other BSD? or do they still fcntl?), agreed 100%.
Linux seems to be in a completely different state of affairs. Unless I'm
confusing proc mutexes with thread mutexes again, pthread locks are
the one-and-only implementation now with OpenSSL 1.1.0 (the user-
replaceable locking callbacks are now no-ops / never invoked.) They
don't have timed mutexes I'm aware of, so that additional headache
isn't part of their puzzle. But it's a good endorsement for a change on
Linux, at least.
Given that we didn't elect PROC_PTHREAD mutexes as an alternate
_USE_ schema, I don't have a strong opinion about changing this
for 1.6.0. It can be overridden by the implementer. It seems we really
should change it on trunk and get that tested in the wild by fellow devs,
at least.
Re: Default Linux mutex method
Posted by Jim Jagielski <ji...@jaguNET.com>.
Considering the sad affair w/ pthread on OSX, I would
recommend we stay w/ using sems.
Re: Default Linux mutex method
Posted by Jim Jagielski <ji...@jaguNET.com>.
Of course, it depends on the number of threads, natch...
> On Apr 3, 2017, at 2:07 PM, Jim Jagielski <ji...@jaguNET.com> wrote:
>
> Using:
>
> https://github.com/jimjag/benchmarks/blob/master/lock/lock_test.c
>
> under OSX/macOS, pthread_mutex_lock is much quicker. Which makes
> sense since sysv sems and posix sems do much more than simply
> "lock"
Re: Default Linux mutex method
Posted by Jim Jagielski <ji...@jaguNET.com>.
Using:
https://github.com/jimjag/benchmarks/blob/master/lock/lock_test.c
under OSX/macOS, pthread_mutex_lock is much quicker. Which makes
sense since sysv sems and posix sems do much more than simply
"lock"
Re: Default Linux mutex method
Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Mon, Apr 3, 2017 at 5:05 AM, Yann Ylavic <yl...@gmail.com> wrote:
>
> Yes I got this, my point is that there are no duplicates here, one and
> only one per:
> - pthread mutex => PTHREAD_SERIALIZE (kind of implicit since all
> unixes use pthreads, but not to be confused with
> PROC_PTHREAD_SERIALIZE...).
Herp derp... yup that was the error I made.
So the remaining question - whether it would be good to swap SYSVSEM
with PROC_PTHREAD on linux - should be based on more study, Jim's
performance question first and foremost, and also observations about
similar projects and the intrinsic flaw of lingering semaphores after abend.
It may be worthwhile to note that my default FC25 box has zero sysv
semaphores during normal operation, which suggests it is greatly avoided.
Re: Default Linux mutex method
Posted by Yann Ylavic <yl...@gmail.com>.
On Mon, Apr 3, 2017 at 6:16 AM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> On Apr 1, 2017 4:07 AM, "Yann Ylavic" <yl...@gmail.com> wrote:
>
> On Fri, Mar 31, 2017 at 4:21 AM, William A Rowe Jr <wr...@rowe-clan.net>
> wrote:
>> So almost two decades later, this is still odd.
>>
>> apr.h:#define APR_USE_SHMEM_MMAP_TMP 0
>> apr.h:#define APR_USE_SHMEM_MMAP_SHM 0
>> apr.h:#define APR_USE_SHMEM_MMAP_ZERO 0
>> apr.h:#define APR_USE_SHMEM_SHMGET_ANON 0
>> apr.h:#define APR_USE_SHMEM_SHMGET 1
>> apr.h:#define APR_USE_SHMEM_MMAP_ANON 1
>> apr.h:#define APR_USE_SHMEM_BEOS 0
>> apr.h:#define APR_USE_FLOCK_SERIALIZE 0
>> apr.h:#define APR_USE_SYSVSEM_SERIALIZE 1
>> apr.h:#define APR_USE_POSIXSEM_SERIALIZE 0
>> apr.h:#define APR_USE_FCNTL_SERIALIZE 0
>> apr.h:#define APR_USE_PROC_PTHREAD_SERIALIZE 0
>> apr.h:#define APR_USE_PTHREAD_SERIALIZE 1
>
> What's odd?
>
>
>
> APR_HAS_ describes support of a feature/mechanism.
>
> APR_USE_ was an imperitive. The presence, in this case of multiple multiples
> indicates broken autoconf logic.
Yes I got this, my point is that there are no duplicates here, one and
only one per:
- named based SHM => SHMEM_SHMGET,
- anonymous SHM => MMAP_ANON,
- proc mutex => SYSVSEM_SERIALIZE,
- pthread mutex => PTHREAD_SERIALIZE (kind of implicit since all
unixes use pthreads, but not to be confused with
PROC_PTHREAD_SERIALIZE...).
Re: Default Linux mutex method
Posted by William A Rowe Jr <wr...@rowe-clan.net>.
On Apr 1, 2017 4:07 AM, "Yann Ylavic" <yl...@gmail.com> wrote:
On Fri, Mar 31, 2017 at 4:21 AM, William A Rowe Jr <wr...@rowe-clan.net>
wrote:
> So almost two decades later, this is still odd.
>
> apr.h:#define APR_USE_SHMEM_MMAP_TMP 0
> apr.h:#define APR_USE_SHMEM_MMAP_SHM 0
> apr.h:#define APR_USE_SHMEM_MMAP_ZERO 0
> apr.h:#define APR_USE_SHMEM_SHMGET_ANON 0
> apr.h:#define APR_USE_SHMEM_SHMGET 1
> apr.h:#define APR_USE_SHMEM_MMAP_ANON 1
> apr.h:#define APR_USE_SHMEM_BEOS 0
> apr.h:#define APR_USE_FLOCK_SERIALIZE 0
> apr.h:#define APR_USE_SYSVSEM_SERIALIZE 1
> apr.h:#define APR_USE_POSIXSEM_SERIALIZE 0
> apr.h:#define APR_USE_FCNTL_SERIALIZE 0
> apr.h:#define APR_USE_PROC_PTHREAD_SERIALIZE 0
> apr.h:#define APR_USE_PTHREAD_SERIALIZE 1
What's odd?
APR_HAS_ describes support of a feature/mechanism.
APR_USE_ was an imperitive. The presence, in this case of multiple
multiples indicates broken autoconf logic.
Re: Default Linux mutex method
Posted by Yann Ylavic <yl...@gmail.com>.
On Fri, Mar 31, 2017 at 4:21 AM, William A Rowe Jr <wr...@rowe-clan.net> wrote:
> So almost two decades later, this is still odd.
>
> apr.h:#define APR_USE_SHMEM_MMAP_TMP 0
> apr.h:#define APR_USE_SHMEM_MMAP_SHM 0
> apr.h:#define APR_USE_SHMEM_MMAP_ZERO 0
> apr.h:#define APR_USE_SHMEM_SHMGET_ANON 0
> apr.h:#define APR_USE_SHMEM_SHMGET 1
> apr.h:#define APR_USE_SHMEM_MMAP_ANON 1
> apr.h:#define APR_USE_SHMEM_BEOS 0
> apr.h:#define APR_USE_FLOCK_SERIALIZE 0
> apr.h:#define APR_USE_SYSVSEM_SERIALIZE 1
> apr.h:#define APR_USE_POSIXSEM_SERIALIZE 0
> apr.h:#define APR_USE_FCNTL_SERIALIZE 0
> apr.h:#define APR_USE_PROC_PTHREAD_SERIALIZE 0
> apr.h:#define APR_USE_PTHREAD_SERIALIZE 1
What's odd?
My understanding is that SHMEM_SHMGET and MMAP_ANON are not mutually
exclusive because the former is ENOTIMPL in apr_shm_create when
filename is NULL (i.e. anonymous SHM), hence the latter would be used.
Regarding USE_SYSVSEM_SERIALIZE vs PTHREAD_SERIALIZE, the former is
for proc_mutex whereas the latter is for thread_mutex, so I think they
don't collide either.