You are viewing a plain text version of this content. The canonical link for it is here.
Posted to bugs@httpd.apache.org by bu...@apache.org on 2014/08/02 01:55:52 UTC

[Bug 56803] New: Apache web server hung with this message "[emerg] (22)Invalid argument: couldn't grab the accept mutex"

https://issues.apache.org/bugzilla/show_bug.cgi?id=56803

            Bug ID: 56803
           Summary: Apache web server hung with this message "[emerg]
                    (22)Invalid argument: couldn't grab the accept mutex"
           Product: Apache httpd-2
           Version: 2.2.15
          Hardware: PC
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Core
          Assignee: bugs@httpd.apache.org
          Reporter: rajengineer30@gmail.com

I can see the below snippets in the apache error log.During this time, the
apache hung and killing all the httpd and the restart of apache solved the
issue..

Could anyone please give some tips? 
Which module is throwing this messages? what could have caused this?

<snip>
[Tue Jun 17 15:56:08 2014] [notice] child pid 15415 exit signal Aborted (6)
[Tue Jun 17 15:56:08 2014] [alert] Child 15416 returned a Fatal error... Apache
is exiting!
[Tue Jun 17 15:56:08 2014] [emerg] (22)Invalid argument: couldn't grab the
accept mutex
..
..
..
[Tue Jun 17 15:56:08 2014] [emerg] (22)Invalid argument: couldn't grab the
accept mutex
[Tue Jun 17 15:56:08 2014] [emerg] (22)Invalid argument: couldn't grab the
accept mutex
[Tue Jun 17 15:56:08 2014] [emerg] (22)Invalid argument: couldn't grab the
accept mutex
[Tue Jun 17 15:56:08 2014] [emerg] (22)Invalid argument: couldn't grab the
accept mutex
</snip>

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 56803] Apache web server hung with this message "[emerg] (22)Invalid argument: couldn't grab the accept mutex"

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56803

--- Comment #1 from Mike Rumph <mi...@oracle.com> ---
Hello Rajeshwaran,

This message is issued from the following code segment in
server/mpm/prefork/prefork.c:

static void accept_mutex_on(void)
{
    apr_status_t rv = apr_proc_mutex_lock(accept_mutex);
    if (rv != APR_SUCCESS) {
        const char *msg = "couldn't grab the accept mutex";

        if (ap_my_generation !=
            ap_scoreboard_image->global->running_generation) {
            ap_log_error(APLOG_MARK, APLOG_DEBUG, rv, NULL, "%s", msg);
            clean_child_exit(0);
        }
        else {
            ap_log_error(APLOG_MARK, APLOG_EMERG, rv, NULL, "%s", msg);
            exit(APEXIT_CHILDFATAL);
        }
    }
}

Thanks,

Mike Rumph

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 56803] Apache web server hung with this message "[emerg] (22)Invalid argument: couldn't grab the accept mutex"

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56803

Jeff Trawick <tr...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEEDINFO                    |RESOLVED
         Resolution|---                         |INVALID

--- Comment #4 from Jeff Trawick <tr...@apache.org> ---
The first error (SIGABORT) is definitely mod_pagespeed, based on the extra
error information you posted.

I don't know for sure what causes this:  Child 15416 returned a Fatal error... 
Apache is exiting!

mod_pagespeed might be exiting a process with APEXIT_CHILDFATAL.

I don't think we can help further; the mod_pagespeed issue must be addressed.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 56803] Apache web server hung with this message "[emerg] (22)Invalid argument: couldn't grab the accept mutex"

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56803

--- Comment #3 from rajeshwaran <ra...@gmail.com> ---
Thanks for your response..

(In reply to Jeff Trawick from comment #2)
> >Which module is throwing this messages? what could have caused this?
> >[Tue Jun 17 15:56:08 2014] [notice] child pid 15415 exit signal Aborted (6)
> 
> Get a core dump, use gdb to get a backtrace for all threads from the
> coredump.  We need to see that information to investigate this issue.
> 
> Hopefully this helps you with that:
> http://httpd.apache.org/dev/debugging.html
--------------------------------------------------------------------------------------------------------------------------------------
-------> This is happening only in main production system.The problem is that
we couldn't reproduce this in local.I had searched for the earlier error
message([mod_pagespeed 0.9.17.7-716 @15415]
[0617/155607:FATAL:net/instaweb/apache/), that is before the error 'child pid
15415 exit signal Aborted (6)' and found the below links..But i'm not sure ,
whether the 'mod pagespeed' brought the apache down..The said that this
modpagespeed error would not occur once we upgrade the mod pagespeed to
mod_pagespeed 0.9.18.7(we are currently using mod_pagespeed 0.9.17.716)..Since,
we unable to reproduce this we are doubt on upgrading the modpagespeed will fix
it or not..

for the mod page speed error, i found the below links,

https://code.google.com/p/modpagespeed/issues/detail?id=334

https://code.google.com/p/modpagespeed/issues/detail?id=334#c9
--------------------------------------------------------------------------------------------------------------------------------------
> 
> >[Tue Jun 17 15:56:08 2014] [alert] Child 15416 returned a Fatal error... Apache is exiting!
> 
> There should be some earlier message describing the problem.  
--------------------------------------------------------------------------------------------------------------------------------------
----> Yes, the earlier message is the below,
--------------------------------------------------------------------------------------------------------------------------------------
Tue Jun 17 15:56:07 2014] [alert] [mod_pagespeed 0.9.17.7-716 @15415]
[0617/155607:FATAL:net/instaweb/apache/
serf_url_async_fetcher.cc(466)] Check failed: 0 ==
apr_thread_create(&thread_id_, __null, SerfThreadFn, this, pool_) (0 vs.
11)\nBacktrace:\n\t/opt/collabnet/teamforge/dist/pagespeed/mod_pagespeed.so(+0x36b1a)
[0x7fe0d7489b1a]\n\t/opt/collabnet/teamforge/dist/pagespeed/mod_pagespeed.so(+0x24efa)
[0x7fe0d7477efa]\n\t/opt/collabnet/teamforge/dist/pagespeed/mod_pagespeed.so(+0x38b6d)
[0x7fe0d748bb6d]\n\t/opt/collabnet/teamforge/dist/pagespeed/mod_pagespeed.so(+0x34982)
[0x7fe0d7487982]\n\t/opt/collabnet/teamforge/dist/pagespeed/mod_pagespeed.so(+0x2b0e7)
[0x7fe0d747e0e7]\n\t/opt/collabnet/teamforge/dist/pagespeed/mod_pagespeed.so(+0x9b33b)
[0x7fe0d74ee33b]\n\t/opt/collabnet/teamforge/dist/pagespeed/mod_pagespeed.so(+0x9a3d5)
[0x7fe0d74ed3d5]\n\t/opt/collabnet/teamforge/dist/pagespeed/mod_pagespeed.so(+0x9a4fc)
[0x7fe0d74ed4fc]\n\t/opt/collabnet/teamforge/dist/pagespeed/mod_pagespeed.so(+0x23b24)
[0x7fe0d7476b24]\n\t/usr/sbin/httpd(ap_run_translate_name+0x68)
[0x7fe0e3b748d8]\n\t/usr/sbin/httpd(ap_process_request_internal+0x126)
[0x7fe0e3b76ab6]\n\t/usr/sbin/httpd(ap_process_request+0x1b0)
[0x7fe0e3b88b20]\n\t/usr/sbin/httpd(+0x379a8)
[0x7fe0e3b859a8]\n\t/usr/sbin/httpd(ap_run_process_connection+0x68)
[0x7fe0e3b816b8]\n\t/usr/sbin/httpd(+0x3f977)
[0x7fe0e3b8d977]\n\t/usr/sbin/httpd(+0x3fc8a)
[0x7fe0e3b8dc8a]\n\t/usr/sbin/httpd(ap_mpm_run+0xc1c)
[0x7fe0e3b8e90c]\n\t/usr/sbin/httpd(main+0xb60)
[0x7fe0e3b65900]\n\t/lib64/libc.so.6(__libc_start_main+0xfd)
[0x7fe0e20a7d1d]\n\t/usr/sbin/httpd(+0x16899) [0x7fe0e3b64899]
[Tue Jun 17 15:56:07 2014] [warn] [mod_pagespeed 0.9.17.7-716 @14792]
https://eforge.ericsson.se/sf/tracker/do/listArtifacts/projects.titansim/tracker.support_ticket:858:
Retaining contents of script tag even though script is external.
..
[Tue Jun 17 15:56:01 2014] [notice] child pid 15075 exit signal Aborted (6)
[Tue Jun 17 15:56:01 2014] [notice] child pid 15092 exit signal Aborted (6)
[Tue Jun 17 15:56:01 2014] [notice] child pid 15276 exit signal Aborted (6)
[Tue Jun 17 15:56:01 2014] [notice] child pid 15277 exit signal Aborted (6)
[Tue Jun 17 15:56:01 2014] [notice] child pid 15279 exit signal Aborted (6)
...
[Tue Jun 17 15:56:08 2014] [notice] child pid 15415 exit signal Aborted (6)
--------------------------------------------------------------------------------------------------------------------------------------

> 
> >[Tue Jun 17 15:56:08 2014] [emerg] (22)Invalid argument: couldn't grab the accept mutex
> 
> Have you configured the AcceptMutex directive in any of your .conf files? 
> If so, what is the entire directive?
--------------------------------------------------------------------------------------------------------------------------------------
----------->We have not configure the AcceptMutex directive..We had enabled the
apache log to debug and found that it's using default mutex 'sysvsem'..refer
below log snippet..
---------->[debug] prefork.c(1018): AcceptMutex: sysvsem (default: sysvsem)
--------------------------------------------------------------------------------------------------------------------------------------
> 
> If this is Linux and you configured AcceptMutex to something, I'd comment
> out the AcceptMutex directive.
--------------------------------------------------------------------------------------------------------------------------------------
-----> yes this is a linux system...
--------------------------------------------------------------------------------------------------------------------------------------

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 56803] Apache web server hung with this message "[emerg] (22)Invalid argument: couldn't grab the accept mutex"

Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=56803

Jeff Trawick <tr...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |NEEDINFO

--- Comment #2 from Jeff Trawick <tr...@apache.org> ---
>Which module is throwing this messages? what could have caused this?
>[Tue Jun 17 15:56:08 2014] [notice] child pid 15415 exit signal Aborted (6)

Get a core dump, use gdb to get a backtrace for all threads from the coredump. 
We need to see that information to investigate this issue.

Hopefully this helps you with that: http://httpd.apache.org/dev/debugging.html

>[Tue Jun 17 15:56:08 2014] [alert] Child 15416 returned a Fatal error... Apache is exiting!

There should be some earlier message describing the problem.  

>[Tue Jun 17 15:56:08 2014] [emerg] (22)Invalid argument: couldn't grab the accept mutex

Have you configured the AcceptMutex directive in any of your .conf files?  If
so, what is the entire directive?

If this is Linux and you configured AcceptMutex to something, I'd comment out
the AcceptMutex directive.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org