You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Ian Holsman <li...@holsman.net> on 2007/08/24 02:56:07 UTC

OS/X + Cisco VPN == trouble

Hi.

This one is frustrating me to no end, and was wondering if some BSD/OSX 
guru can help  me out a bit.

I'm using the trunk, and trying to start apache, but I keep getting a 
lock/sem problem

[Fri Aug 24 10:51:53 2007] [emerg] (28)No space left on device: Couldn't 
create accept lock

this only happens if I have run the VPN. when I reboot it works great 
(until I check into the VPN).

from what I can see this error is usually caused by running out of 
sempahores, but ipcs doesn't show any.
and to make matters worse, it only started happening about 2 weeks ago.

regards
Ian

Re: OS/X + Cisco VPN == trouble

Posted by Graham Dumpleton <gr...@gmail.com>.
BTW, I totally forgot to join the dots and point out the following
configuration option for mod_python to drop the number of mutex locks
it uses.

    /* On some systems the locking mechanism chosen uses valuable
       system resources, notably on RH 8 it will use sysv ipc for
       which Linux by default provides only 128 semaphores
       system-wide, and on many other systems flock is used, which
       results in a relatively large number of open files.

       The maximum number of locks can be specified at
       compile time using "./configure --with-max-locks value" or
       at run time with "PythonOption mod_python.mutex_locks value".

       If the PythonOption directive is used, it must be in a
       server config context, otherwise it will be ignored.

       The optimal number of necessary locks is not clear, perhaps a
       small number is more than sufficient - if someone took the
       time to run some research on this, that'd be most welcome!
    */

Thus perhaps just tell mod_python to use less locks.

Graham

On 24/08/07, Graham Dumpleton <gr...@gmail.com> wrote:
> On 24/08/07, Ian Holsman <li...@holsman.net> wrote:
> > Graham Dumpleton wrote:
> > > On 24/08/07, Ian Holsman <li...@holsman.net> wrote:
> > >
> > >> Hi.
> > >>
> > >> This one is frustrating me to no end, and was wondering if some BSD/OSX
> > >> guru can help  me out a bit.
> > >>
> > >> I'm using the trunk, and trying to start apache, but I keep getting a
> > >> lock/sem problem
> > >>
> > >> [Fri Aug 24 10:51:53 2007] [emerg] (28)No space left on device: Couldn't
> > >> create accept lock
> > >>
> > >> this only happens if I have run the VPN. when I reboot it works great
> > >> (until I check into the VPN).
> > >>
> > >> from what I can see this error is usually caused by running out of
> > >> sempahores, but ipcs doesn't show any.
> > >> and to make matters worse, it only started happening about 2 weeks ago.
> > >>
> > >
> > > As a workaround while you resolve why you are running out of mutexes
> > > and/or how to increase the number, use AcceptMutex directive to have
> > > it use a different mutex mechanism.
> > >
> > >   http://httpd.apache.org/docs/2.2/mod/mpm_common.html#acceptmutex
> > >
> > > This question was perhaps more appropriate for user list and not
> > > developers list.
> > >
> > > Graham
> > >
> > >
> > Thanks Graham,
> >
> > I'll remember the users list in the future. as I was using the trunk, I
> > thought it might have been a recent patch to apr/httpd which has caused
> > it, thats all.
> >
> > BTW.. the acceptmutex works for stock apache, but not for mod_python
> > which doesn't have the ability to change the mutex mechanism it uses.
>
> For mod_python problem see:
>
>   https://issues.apache.org/jira/browse/MODPYTHON-202
>
> Suggested quick think right at end.
>
> If only using mod_python to host Python WSGI applications and not
> using mod_python specific features at all, consider using mod_wsgi
> instead. The mod_wsgi module will work as it is the global mutexes
> that mod_python creates for session management that cause the problem
> and mod_wsgi doesn't have such high level features, only supporting
> basic WSGI interface.
>
> Graham
>

Re: OS/X + Cisco VPN == trouble

Posted by Graham Dumpleton <gr...@gmail.com>.
On 24/08/07, Ian Holsman <li...@holsman.net> wrote:
> Graham Dumpleton wrote:
> > On 24/08/07, Ian Holsman <li...@holsman.net> wrote:
> >
> >> Hi.
> >>
> >> This one is frustrating me to no end, and was wondering if some BSD/OSX
> >> guru can help  me out a bit.
> >>
> >> I'm using the trunk, and trying to start apache, but I keep getting a
> >> lock/sem problem
> >>
> >> [Fri Aug 24 10:51:53 2007] [emerg] (28)No space left on device: Couldn't
> >> create accept lock
> >>
> >> this only happens if I have run the VPN. when I reboot it works great
> >> (until I check into the VPN).
> >>
> >> from what I can see this error is usually caused by running out of
> >> sempahores, but ipcs doesn't show any.
> >> and to make matters worse, it only started happening about 2 weeks ago.
> >>
> >
> > As a workaround while you resolve why you are running out of mutexes
> > and/or how to increase the number, use AcceptMutex directive to have
> > it use a different mutex mechanism.
> >
> >   http://httpd.apache.org/docs/2.2/mod/mpm_common.html#acceptmutex
> >
> > This question was perhaps more appropriate for user list and not
> > developers list.
> >
> > Graham
> >
> >
> Thanks Graham,
>
> I'll remember the users list in the future. as I was using the trunk, I
> thought it might have been a recent patch to apr/httpd which has caused
> it, thats all.
>
> BTW.. the acceptmutex works for stock apache, but not for mod_python
> which doesn't have the ability to change the mutex mechanism it uses.

For mod_python problem see:

  https://issues.apache.org/jira/browse/MODPYTHON-202

Suggested quick think right at end.

If only using mod_python to host Python WSGI applications and not
using mod_python specific features at all, consider using mod_wsgi
instead. The mod_wsgi module will work as it is the global mutexes
that mod_python creates for session management that cause the problem
and mod_wsgi doesn't have such high level features, only supporting
basic WSGI interface.

Graham

Re: OS/X + Cisco VPN == trouble

Posted by Ian Holsman <li...@holsman.net>.
Graham Dumpleton wrote:
> On 24/08/07, Ian Holsman <li...@holsman.net> wrote:
>   
>> Hi.
>>
>> This one is frustrating me to no end, and was wondering if some BSD/OSX
>> guru can help  me out a bit.
>>
>> I'm using the trunk, and trying to start apache, but I keep getting a
>> lock/sem problem
>>
>> [Fri Aug 24 10:51:53 2007] [emerg] (28)No space left on device: Couldn't
>> create accept lock
>>
>> this only happens if I have run the VPN. when I reboot it works great
>> (until I check into the VPN).
>>
>> from what I can see this error is usually caused by running out of
>> sempahores, but ipcs doesn't show any.
>> and to make matters worse, it only started happening about 2 weeks ago.
>>     
>
> As a workaround while you resolve why you are running out of mutexes
> and/or how to increase the number, use AcceptMutex directive to have
> it use a different mutex mechanism.
>
>   http://httpd.apache.org/docs/2.2/mod/mpm_common.html#acceptmutex
>
> This question was perhaps more appropriate for user list and not
> developers list.
>
> Graham
>
>   
Thanks Graham,

I'll remember the users list in the future. as I was using the trunk, I 
thought it might have been a recent patch to apr/httpd which has caused 
it, thats all.

BTW.. the acceptmutex works for stock apache, but not for mod_python 
which doesn't have the ability to change the mutex mechanism it uses.

Re: OS/X + Cisco VPN == trouble

Posted by Graham Dumpleton <gr...@gmail.com>.
On 24/08/07, Ian Holsman <li...@holsman.net> wrote:
> Hi.
>
> This one is frustrating me to no end, and was wondering if some BSD/OSX
> guru can help  me out a bit.
>
> I'm using the trunk, and trying to start apache, but I keep getting a
> lock/sem problem
>
> [Fri Aug 24 10:51:53 2007] [emerg] (28)No space left on device: Couldn't
> create accept lock
>
> this only happens if I have run the VPN. when I reboot it works great
> (until I check into the VPN).
>
> from what I can see this error is usually caused by running out of
> sempahores, but ipcs doesn't show any.
> and to make matters worse, it only started happening about 2 weeks ago.

As a workaround while you resolve why you are running out of mutexes
and/or how to increase the number, use AcceptMutex directive to have
it use a different mutex mechanism.

  http://httpd.apache.org/docs/2.2/mod/mpm_common.html#acceptmutex

This question was perhaps more appropriate for user list and not
developers list.

Graham

Re: OS/X + Cisco VPN == trouble

Posted by Graham Dumpleton <gr...@gmail.com>.
On 25/08/07, Graham Dumpleton <gr...@gmail.com> wrote:
> On 24/08/07, Jim Jagielski <ji...@jagunet.com> wrote:
> >
> > On Aug 23, 2007, at 8:56 PM, Ian Holsman wrote:
> >
> > > Hi.
> > >
> > > This one is frustrating me to no end, and was wondering if some BSD/
> > > OSX guru can help  me out a bit.
> > >
> > > I'm using the trunk, and trying to start apache, but I keep getting
> > > a lock/sem problem
> > >
> > > [Fri Aug 24 10:51:53 2007] [emerg] (28)No space left on device:
> > > Couldn't create accept lock
> > >
> > > this only happens if I have run the VPN. when I reboot it works
> > > great (until I check into the VPN).
> > >
> > > from what I can see this error is usually caused by running out of
> > > sempahores, but ipcs doesn't show any.
> > > and to make matters worse, it only started happening about 2 weeks
> > > ago.
> > >
> > > regards
> > > Ian
> > >
> >
> > IIRC, aren't the default sems on OS X Posix sems, not SysV sems??
> > ipcs just shows SysV stats
>
> The source has:
>
>     case APR_LOCK_DEFAULT:
> #if APR_USE_FLOCK_SERIALIZE
>         new_mutex->inter_meth = &mutex_flock_methods;
> #elif APR_USE_SYSVSEM_SERIALIZE
>         new_mutex->inter_meth = &mutex_sysv_methods;
> #elif APR_USE_FCNTL_SERIALIZE
>         new_mutex->inter_meth = &mutex_fcntl_methods;
> #elif APR_USE_PROC_PTHREAD_SERIALIZE
>         new_mutex->inter_meth = &mutex_proc_pthread_methods;
> #elif APR_USE_POSIXSEM_SERIALIZE
>         new_mutex->inter_meth = &mutex_posixsem_methods;
> #else
>         return APR_ENOTIMPL;
> #endif
>
> For my MacOS X PPC system, the apr.h header file has:
>
> #define APR_USE_FLOCK_SERIALIZE           0
> #define APR_USE_SYSVSEM_SERIALIZE         1
> #define APR_USE_POSIXSEM_SERIALIZE        0
> #define APR_USE_FCNTL_SERIALIZE           0
> #define APR_USE_PROC_PTHREAD_SERIALIZE    0
> #define APR_USE_PTHREAD_SERIALIZE         1
>
> So default will always map to SYSVSEM.
>
> You need to check your apr.h header file to see what is enabled and
> would thus be used as default based on order that code checks.

Just to confirm this for my system at least, setting LogLevel to debug yields:

[Sat Jun 09 17:53:08 2007] [debug] worker.c(1715): AcceptMutex:
sysvsem (default: sysvsem)

Yet, when I run ipcs I don't see anything either.

Message Queues:
T     ID     KEY        MODE       OWNER    GROUP

Shared Memory:
T     ID     KEY        MODE       OWNER    GROUP

Semaphores:
T     ID     KEY        MODE       OWNER    GROUP

Same thing if I explicitly set AcceptMutex to sysvsem.

Must be something odd about MacOS X.

Graham

Re: OS/X + Cisco VPN == trouble

Posted by Graham Dumpleton <gr...@gmail.com>.
On 24/08/07, Jim Jagielski <ji...@jagunet.com> wrote:
>
> On Aug 23, 2007, at 8:56 PM, Ian Holsman wrote:
>
> > Hi.
> >
> > This one is frustrating me to no end, and was wondering if some BSD/
> > OSX guru can help  me out a bit.
> >
> > I'm using the trunk, and trying to start apache, but I keep getting
> > a lock/sem problem
> >
> > [Fri Aug 24 10:51:53 2007] [emerg] (28)No space left on device:
> > Couldn't create accept lock
> >
> > this only happens if I have run the VPN. when I reboot it works
> > great (until I check into the VPN).
> >
> > from what I can see this error is usually caused by running out of
> > sempahores, but ipcs doesn't show any.
> > and to make matters worse, it only started happening about 2 weeks
> > ago.
> >
> > regards
> > Ian
> >
>
> IIRC, aren't the default sems on OS X Posix sems, not SysV sems??
> ipcs just shows SysV stats

The source has:

    case APR_LOCK_DEFAULT:
#if APR_USE_FLOCK_SERIALIZE
        new_mutex->inter_meth = &mutex_flock_methods;
#elif APR_USE_SYSVSEM_SERIALIZE
        new_mutex->inter_meth = &mutex_sysv_methods;
#elif APR_USE_FCNTL_SERIALIZE
        new_mutex->inter_meth = &mutex_fcntl_methods;
#elif APR_USE_PROC_PTHREAD_SERIALIZE
        new_mutex->inter_meth = &mutex_proc_pthread_methods;
#elif APR_USE_POSIXSEM_SERIALIZE
        new_mutex->inter_meth = &mutex_posixsem_methods;
#else
        return APR_ENOTIMPL;
#endif

For my MacOS X PPC system, the apr.h header file has:

#define APR_USE_FLOCK_SERIALIZE           0
#define APR_USE_SYSVSEM_SERIALIZE         1
#define APR_USE_POSIXSEM_SERIALIZE        0
#define APR_USE_FCNTL_SERIALIZE           0
#define APR_USE_PROC_PTHREAD_SERIALIZE    0
#define APR_USE_PTHREAD_SERIALIZE         1

So default will always map to SYSVSEM.

You need to check your apr.h header file to see what is enabled and
would thus be used as default based on order that code checks.

Graham

Re: OS/X + Cisco VPN == trouble

Posted by Jim Jagielski <ji...@jaguNET.com>.
On Aug 23, 2007, at 8:56 PM, Ian Holsman wrote:

> Hi.
>
> This one is frustrating me to no end, and was wondering if some BSD/ 
> OSX guru can help  me out a bit.
>
> I'm using the trunk, and trying to start apache, but I keep getting  
> a lock/sem problem
>
> [Fri Aug 24 10:51:53 2007] [emerg] (28)No space left on device:  
> Couldn't create accept lock
>
> this only happens if I have run the VPN. when I reboot it works  
> great (until I check into the VPN).
>
> from what I can see this error is usually caused by running out of  
> sempahores, but ipcs doesn't show any.
> and to make matters worse, it only started happening about 2 weeks  
> ago.
>
> regards
> Ian
>

IIRC, aren't the default sems on OS X Posix sems, not SysV sems??
ipcs just shows SysV stats