You are viewing a plain text version of this content. The canonical link for it is here.
Posted to bugs@httpd.apache.org by bu...@apache.org on 2013/12/05 15:28:34 UTC
[Bug 55849] New: child pid 11702 exit signal Segmentation fault
(11),
https://issues.apache.org/bugzilla/show_bug.cgi?id=55849
Bug ID: 55849
Summary: child pid 11702 exit signal Segmentation fault (11),
Product: Apache httpd-2
Version: 2.2.23
Hardware: Sun
OS: Solaris
Status: NEW
Severity: major
Priority: P2
Component: All
Assignee: bugs@httpd.apache.org
Reporter: sivasankar.manu@gmail.com
Created attachment 31094
--> https://issues.apache.org/bugzilla/attachment.cgi?id=31094&action=edit
Core analysis and find some out put
Hi Team,
We have Segmentation Fault error since one year in our PROD and Non_Prod
environments.
Server version: Apache/2.2.23 (Unix)
Server built: Jan 16 2013 17:16:27
Server's Module Magic Number: 20051115:31
Server loaded: APR 1.4.6, APR-Util 1.4.1
Compiled using: APR 1.4.6, APR-Util 1.4.1
Architecture: 32-bit
Server MPM: Worker
threaded: yes (fixed thread count)
forked: yes (variable process count)
Errors:
[Thu Nov 14 02:41:30 2013] [notice] child pid 11702 exit signal Segmentation
fault (11), possible coredump in /ufscmsw1/apache
[Thu Nov 14 02:42:51 2013] [notice] child pid 13294 exit signal Segmentation
fault (11), possible coredump in /ufscmsw1/apache
[Thu Nov 14 02:42:54 2013] [notice] child pid 12880 exit signal Segmentation
fault (11), possible coredump in /ufscmsw1/apache
[Wed Nov 20 02:36:24 2013] [notice] child pid 12494 exit signal Segmentation
fault (11), possible coredump in /ufscmsw1/apache
[Wed Nov 20 02:36:27 2013] [notice] child pid 12496 exit signal Segmentation
fault (11), possible coredump in /ufscmsw1/apache
[Wed Nov 20 02:39:02 2013] [notice] child pid 12493 exit signal Segmentation
fault (11), possible coredump in /ufscmsw1/apache
[Wed Nov 20 02:39:06 2013] [notice] child pid 12495 exit signal Segmentation
fault (11), possible coredump in /ufscmsw1/apache
[Tue Nov 26 07:54:38 2013] [notice] child pid 12701 exit signal Segmentation
fault (11), possible coredump in /ufscmsw1/apache
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 55849] httpd child crashes in ct_auth_check_user
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55849
--- Comment #10 from Mike Rumph <mi...@oracle.com> ---
Hello Siva,
You can find information on the Apache user mailing list at the following link:
- http://httpd.apache.org/userslist.html
Thanks,
Mike Rumph
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 55849] httpd child crashes in ct_auth_check_user
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55849
Jeff Trawick <tr...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|FIXED |INVALID
--- Comment #12 from Jeff Trawick <tr...@apache.org> ---
I'm so glad you resolved it.
As a bookkeeping matter, I'm changing this back to INVALID since an httpd code
fix or documentation change was not required to resolve your issue.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 55849] httpd child crashes in ct_auth_check_user
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55849
--- Comment #8 from Jeff Trawick <tr...@apache.org> ---
>But we are using same setup in SIT/LT/UAT environments. There we don't have any issue.
That's not an uncommon scenario unfortunately; possibly that information will
be of use to the third-party module vendor.
If you have further questions about this issue, please bring it up on the user
support mailing list.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 55849] httpd child crashes in ct_auth_check_user
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55849
--- Comment #6 from Jeff Trawick <tr...@apache.org> ---
The module in question is /dinftksw/apache2/lib/libct_apache22_agent.so -- RSA
Access Manager Agents.
.
Your enterprise has presumably licensed that from RSA. They need to be
contacted about this crash.
.
FWIW, you also are using the WLS plugin from Oracle
(/dinftksw/apache2/modules/mod_wl_22.so), but it doesn't seem to be related to
the problem.
.
Something else interesting:
/dinftksw/apache2/lib/isso_wax_apache2.2.4_solaris10.so; I have no idea what
that is.
.
That's all I can help with. Find somebody at your company that is the liason
with RSA.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 55849] httpd child crashes in ct_auth_check_user
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55849
--- Comment #5 from Siva <si...@gmail.com> ---
(In reply to Siva from comment #4)
> Created attachment 31096 [details]
> pmap output file
=====================
).So look in the pmap output to see what shared library contains fe7fd8b4.
Please find the attached pmap complete output and suggest me if anything need
to do
2).If it is some third-party module, which I think it is, you'd need to contact
the provider of it.
How to find is it third party tool or not. Please help me
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 55849] httpd child crashes in ct_auth_check_user
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55849
Jeff Trawick <tr...@apache.org> changed:
What |Removed |Added
----------------------------------------------------------------------------
Status|NEW |RESOLVED
Resolution|--- |INVALID
Summary|child pid 11702 exit signal |httpd child crashes in
|Segmentation fault (11), |ct_auth_check_user
--- Comment #1 from Jeff Trawick <tr...@apache.org> ---
Is this ct_* stuff RSA ClearTrust? If so, report it to them.
This third-party module passed NULL to fprintf() and crashed. We can't handle
it here.
Thread 1 is stuck in read(); it can't segfault. (I think something that httpd
is doing with signals may trigger a common mis-identification of thread 1 by
pflags.)
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 55849] httpd child crashes in ct_auth_check_user
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55849
--- Comment #3 from Jeff Trawick <tr...@apache.org> ---
>The function fprintf is causing the crash. It is located in memory address fee013f0.
To be more accurate: The caller of fprintf is causing the crash by passing an
invalid FILE pointer (0).
So look in the pmap output to see what shared library contains fe7fd8b4.
>Please help me to know right contact details if it is not the right place to discuss above issue
If it is some third-party module, which I think it is, you'd need to contact
the provider of it.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 55849] httpd child crashes in ct_auth_check_user
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55849
--- Comment #7 from Siva <si...@gmail.com> ---
(In reply to Jeff Trawick from comment #6)
> The module in question is /dinftksw/apache2/lib/libct_apache22_agent.so --
> RSA Access Manager Agents.
> .
> Your enterprise has presumably licensed that from RSA. They need to be
> contacted about this crash.
> .
> FWIW, you also are using the WLS plugin from Oracle
> (/dinftksw/apache2/modules/mod_wl_22.so), but it doesn't seem to be related
> to the problem.
> .
> Something else interesting:
> /dinftksw/apache2/lib/isso_wax_apache2.2.4_solaris10.so; I have no idea what
> that is.
> .
> That's all I can help with. Find somebody at your company that is the
> liason with RSA.
=========
Hi Jeff,
But we are using same setup in SIT/LT/UAT environments. There we don't have any
issue.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 55849] httpd child crashes in ct_auth_check_user
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55849
--- Comment #2 from Siva <si...@gmail.com> ---
(In reply to Jeff Trawick from comment #1)
> Is this ct_* stuff RSA ClearTrust? If so, report it to them.
>
> This third-party module passed NULL to fprintf() and crashed. We can't
> handle it here.
>
> Thread 1 is stuck in read(); it can't segfault. (I think something that
> httpd is doing with signals may trigger a common mis-identification of
> thread 1 by pflags.)
Hi Jeff,
In such case, in the same server but with different user and application having
the same problem.
Below are the findings;
STEP1:
Run pflags and look for signals that caused a crash. Here I found a SIGSEGV
signal.
bash-3.2$ pflags core
core 'core' of 24067: /dinftksw/apache2/bin/httpd -f
/dinftksw/apache2/conf/httpd.conf
data model = _ILP32 flags = MSACCT|MSFORK
/1: flags = 0
sigmask = 0xffffbefc,0x0000ffff cursig = SIGSEGV
We have only one thread and crash occured in thread number 1
STEP2:
Do pstack and look for the thread one. I got below out put for thread one
bash-3.2$ pstack core
core 'core' of 24067: /dinftksw/apache2/bin/httpd -f
/dinftksw/apache2/conf/httpd.conf
fee013f0 fprintf (0, fe955128, fe9c6148, a1010, fe7ffe04, 7ffffffe) + 24
fe7fd8b4 ct_uprint_private (4, ffbfd5f8, 1, 1, ff00, ff0000) + 210
fe7fda64 ct_debug_uprintln (fe93d150, 1e14, 35b, fe93d4d0, 0, 0) + 24
fe78f39c ct_child_process_init (19d7b0, 10cf70, 2, 0, 10cf70, 192aa8) + 70
00043680 ap_run_child_init (19d7b0, 10cf70, 0, feaa8, 0, 103228) + 40
000adeec child_main (0, acc6c, 0, 0, feea79a0, fed12a00) + 7c
000ae2c0 make_child (f7000, 0, 1, fa000, f6400, f9c00) + 128
000aeaf0 ap_mpm_run (fe5c00a8, 9, 0, 10, 1, 32) + 740
00030784 main (104a80, f4800, f7000, f7000, 102a78, 0) + 774
0002fb40 _start (0, 0, 0, 0, 0, 0) + 5c
The function fprintf is causing the crash. It is located in memory address
fee013f0.
STEP3:
Do the pmap and search for that near for that address
bash-3.2$ pmap core >pmap.txt
bash-3.2$ ls -lrt pmap.txt
-rw-r----- 1 dinftksw dinftk 5357 Dec 4 12:26 pmap.txt
bash-3.2$ grep -i "fee" pmap.txt
FEE90000 8K r-x-- /lib/libc.so.1
FEEA2000 40K rwx-- /lib/libc.so.1
FEEAC000 8K rwx-- /lib/libc.so.1
FEEB0000 576K r-x-- /lib/libnsl.so.1
Please help me to know right contact details if it is not the right place to
discuss above issue
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 55849] httpd child crashes in ct_auth_check_user
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55849
--- Comment #4 from Siva <si...@gmail.com> ---
Created attachment 31096
--> https://issues.apache.org/bugzilla/attachment.cgi?id=31096&action=edit
pmap output file
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 55849] httpd child crashes in ct_auth_check_user
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55849
--- Comment #9 from Siva <si...@gmail.com> ---
(In reply to Jeff Trawick from comment #8)
> >But we are using same setup in SIT/LT/UAT environments. There we don't have any issue.
That's not an uncommon scenario unfortunately; possibly that information will
be of use to the third-party module vendor.
If you have further questions about this issue, please bring it up on the user
support mailing list.
==========
Could you please let us know the "user support mailing list". I am new for this
site.
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org
[Bug 55849] httpd child crashes in ct_auth_check_user
Posted by bu...@apache.org.
https://issues.apache.org/bugzilla/show_bug.cgi?id=55849
Siva <si...@gmail.com> changed:
What |Removed |Added
----------------------------------------------------------------------------
Resolution|INVALID |FIXED
--- Comment #11 from Siva <si...@gmail.com> ---
Hi Jeff,
Excellent....As you suggested we went to third party and given these output to
them then they said, current using WAX and Agent files are old and need to
replace with latest files and we have done the same.
Since three months we didn't see the error and we are promoting the same to
PROD.
Thank you very very veryyyyyyy muchhhhhhhhhh........:):)
--
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org