You are viewing a plain text version of this content. The canonical link for it is here.
Posted to bugs@httpd.apache.org by bu...@apache.org on 2015/10/23 18:14:57 UTC

[Bug 58529] New: Segmentation fault after modifying a proxy BalancerMember

https://bz.apache.org/bugzilla/show_bug.cgi?id=58529

            Bug ID: 58529
           Summary: Segmentation fault after modifying a proxy
                    BalancerMember
           Product: Apache httpd-2
           Version: 2.4.16
          Hardware: Other
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P2
         Component: mod_proxy_http
          Assignee: bugs@httpd.apache.org
          Reporter: antoine.prevosto@gmail.com

With are using Apache 2.4 as an HTTP reverse proxy, and have activated the
BalancerPersist option to preserve balancer member status update when doing
graceful restart.

If we modify an existing balancer member definition (modifying the port number
or the protocol), the child process crashes with a segmentation fault when
proxifying a request

This behavior may be easily reproduced.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 58529] Segmentation fault after modifying a proxy BalancerMember

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58529

--- Comment #2 from Antoine Prevosto <an...@gmail.com> ---
Here is the main information from rhe core file :

Core was generated by `/tech/apache/runtime/httpd-2.4.16/bin/httpd -D
inst-antoine-back-tomcat7_a -d /'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007fcd5221e954 in ap_proxy_acquire_connection
(proxy_function=0x7fcd5200e1fe "HTTP", conn=0x7fcd47ffeac8, worker=0x2269438,
s=0x227c258) at proxy_util.c:2124
2124        if (worker->s->hmax && worker->cp->res) {
Missing separate debuginfos, use: debuginfo-install
AF-httpd-2.4.16-0-el6.4.x86_64
(gdb) bt
#0  0x00007fcd5221e954 in ap_proxy_acquire_connection
(proxy_function=0x7fcd5200e1fe "HTTP", conn=0x7fcd47ffeac8, worker=0x2269438,
s=0x227c258) at proxy_util.c:2124
#1  0x00007fcd5200b879 in proxy_http_handler (r=0x7fcd1c004980,
worker=0x2269438, conf=0x229a638, url=0x7fcd1c006e20
"http://http1qvit-15.xxxx/", proxyname=0x0, proxyport=0) at
mod_proxy_http.c:1957
#2  0x00007fcd52214f12 in proxy_run_scheme_handler (r=0x7fcd1c004980,
worker=0x2269438, conf=0x229a638, url=0x7fcd1c006e20
"http://http1qvit-15.xxxx/", proxyhost=0x0, proxyport=0) at mod_proxy.c:2789
#3  0x00007fcd52219fa3 in proxy_handler (r=0x7fcd1c004980) at mod_proxy.c:1161
#4  0x000000000044b210 in ap_run_handler ()
#5  0x000000000044f38e in ap_invoke_handler ()
#6  0x000000000046201a in ap_process_async_request ()
#7  0x000000000046217f in ap_process_request ()
#8  0x000000000045e4ce in ap_process_http_connection ()
#9  0x0000000000456020 in ap_run_process_connection ()
#10 0x00007fcd54e8bf2a in process_socket (thd=0x224bb90, dummy=<value optimized
out>) at worker.c:619
#11 worker_thread (thd=0x224bb90, dummy=<value optimized out>) at worker.c:978
#12 0x00007fcd55a379d1 in start_thread () from /lib64/libpthread.so.0
#13 0x00007fcd557848fd in clone () from /lib64/libc.so.6
(gdb) info threads
  27 Thread 0x7fcd4cd6c700 (LWP 22023)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  26 Thread 0x7fcd4d76d700 (LWP 22022)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  25 Thread 0x7fcd4e16e700 (LWP 22021)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  24 Thread 0x7fcd3a3e9700 (LWP 22046)  0x00007fcd55784ef3 in epoll_wait ()
from /lib64/libc.so.6
  23 Thread 0x7fcd3adea700 (LWP 22045)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  22 Thread 0x7fcd3b7eb700 (LWP 22044)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  21 Thread 0x7fcd3c1ec700 (LWP 22043)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  20 Thread 0x7fcd3cbed700 (LWP 22042)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  19 Thread 0x7fcd3d5ee700 (LWP 22041)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  18 Thread 0x7fcd3dfef700 (LWP 22040)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  17 Thread 0x7fcd3e9f0700 (LWP 22039)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  16 Thread 0x7fcd3f3f1700 (LWP 22038)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  15 Thread 0x7fcd3fdf2700 (LWP 22037)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  14 Thread 0x7fcd407f3700 (LWP 22036)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  13 Thread 0x7fcd411f4700 (LWP 22035)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  12 Thread 0x7fcd41bf5700 (LWP 22034)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  11 Thread 0x7fcd425f6700 (LWP 22033)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  10 Thread 0x7fcd42ff7700 (LWP 22032)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  9 Thread 0x7fcd439f8700 (LWP 22031)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  8 Thread 0x7fcd443f9700 (LWP 22030)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  7 Thread 0x7fcd44dfa700 (LWP 22029)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  6 Thread 0x7fcd457fb700 (LWP 22028)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  5 Thread 0x7fcd461fc700 (LWP 22027)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  4 Thread 0x7fcd46bfd700 (LWP 22026)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  3 Thread 0x7fcd475fe700 (LWP 22025)  0x00007fcd55a3b5bc in
pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
  2 Thread 0x7fcd56d47740 (LWP 22019)  0x00007fcd55a3e75d in read () from
/lib64/libpthread.so.0
* 1 Thread 0x7fcd47fff700 (LWP 22024)  0x00007fcd5221e954 in
ap_proxy_acquire_connection (proxy_function=0x7fcd5200e1fe "HTTP",
conn=0x7fcd47ffeac8, worker=0x2269438, s=0x227c258) at proxy_util.c:2124

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 58529] Segmentation fault after modifying a proxy BalancerMember

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58529

--- Comment #1 from Antoine Prevosto <an...@gmail.com> ---
Apache is 2.4.16.

Our vhost contains this directives :

<Proxy balancer://tst_antoine_um_unmanaged>
    ProxySet lbmethod=bybusyness stickysession=ASID|asid growth=1
    BalancerMember http://http1qvit-15.xxx:80 lbset=0 connectiontimeout=5
retry=60 ttl=10 timeout=60 route=a1s1
</Proxy>

Then the balancer-manager displays this balancer as follow :

************************************************
Server Version: Apache/2.4.16 (Unix) OpenSSL/1.0.1e-fips
Server Built: Oct 6 2015 15:58:34
Balancer changes will be persisted on restart.
Balancers are NOT inherited from main server.
ProxyPass settings are inherited from main server.
LoadBalancer Status for balancer://tst_antoine_um_unmanaged
[p7cf634bb_tst_antoine_um_unmanaged]

MaxMembers    StickySession    DisableFailover    Timeout    FailoverAttempts  
 Method    Path    Active
2 [1 Used]    ASID | asid    Off    0    0    bybusyness    /    Yes

Worker URL    Route    RouteRedir    Factor    Set    Status    Elected    Busy
   Load    To    From
http://http1qvit-15.xxxx a1s1        1    0    Init Ok    1    0    0    208   
1.4K
************************************************

Now, we change the port number into the configuration file, so the
BalancerMember directive is now :

BalancerMember http://http1qvit-15.xxxx:8080 lbset=0 connectiontimeout=5
retry=60 ttl=10 timeout=60 route=a1s1


Then, we reload this configuration (graceful restart). After reloading, the
balancer-manager shows that a new balancer member has been added (this seems
strange to me) :

************************************************
Server Version: Apache/2.4.16 (Unix) OpenSSL/1.0.1e-fips
Server Built: Oct 6 2015 15:58:34
Balancer changes will be persisted on restart.
Balancers are NOT inherited from main server.
ProxyPass settings are inherited from main server.
LoadBalancer Status for balancer://tst_antoine_um_unmanaged
[p7cf634bb_tst_antoine_um_unmanaged]

MaxMembers    StickySession    DisableFailover    Timeout    FailoverAttempts  
 Method    Path    Active
2 [2 Used]    ASID | asid    Off    0    1    bybusyness    /    Yes

Worker URL    Route    RouteRedir    Factor    Set    Status    Elected    Busy
   Load    To    From
http://http1qvit-15.xxxx:8080    a1s1        1    0    Init Ok    0    0    0  
 0    0
http://http1qvit-15.xxxx    a1s1        1    0    Init Ok    1    0    0    208
   1.4K
************************************************

Finally, when sending a request to this virtual host, the child process crashes
when trying to proxify the request to the new balancer member (in this case, no
server is listnening on port 8080).

Here is the error log :

************************************************
[Fri Oct 23 15:45:46.848308 2015] [mpm_worker:notice] [pid 25396:tid
140519901787968] AH00292: Apache/2.4.16 (Unix) OpenSSL/1.0.1e-fips configured
-- resuming normal operations
[Fri Oct 23 15:45:46.848329 2015] [core:notice] [pid 25396:tid 140519901787968]
AH00094: Command line: '/tech/apache/runtime/httpd-2.4.16/bin/httpd -D
inst-antoine-back-tomcat7_a -d /tech/apache/a090966 -f
/tech/apache/a090966/etc/main.conf -T -D connector -D hapi -D qos -D cache -D
ssl -D standard -D proxy -D arch-64 -D env-dev -D site-tls -D habile-r5 -D
target-any -D localIP'
[Fri Oct 23 15:45:46.848347 2015] [mpm_worker:warn] [pid 25396:tid
140519901787968] AH00291: long lost child came home! (pid 27493)
[Fri Oct 23 15:45:46.848359 2015] [mpm_worker:warn] [pid 25396:tid
140519901787968] AH00291: long lost child came home! (pid 27494)
[Fri Oct 23 15:45:46.848376 2015] [mpm_worker:warn] [pid 25396:tid
140519901787968] AH00291: long lost child came home! (pid 27495)

[Fri Oct 23 15:49:03.631823 2015] [proxy:error] [pid 22019:tid 140519652980480]
(111)Connection refused: AH00957: HTTP: attempt to connect to 10.60.88.70:8080
(http1qvit-15.xxxx) failed
[Fri Oct 23 15:49:03.631858 2015] [proxy:error] [pid 22019:tid 140519652980480]
AH00959: ap_proxy_connect_backend disabling worker for (http1qvit-15.xxxx) for
60s
[Fri Oct 23 15:49:03.631868 2015] [proxy_http:error] [pid 22019:tid
140519652980480] [client 10.70.130.50:52700] AH01114: HTTP: failed to make
connection to backend: http1qvit-15.xxxx
[Fri Oct 23 15:49:04.031590 2015] [core:notice] [pid 25396:tid 140519901787968]
AH00051: child pid 22019 exit signal Segmentation fault (11), possible coredump
in /tech/apache/a090966/var/logs/antoine-back-tomcat7_a
************************************************

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 58529] Segmentation fault after modifying a proxy BalancerMember

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=58529

Sylvain Goulmy <sy...@gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |sygoulmy@gmail.com

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org