You are viewing a plain text version of this content. The canonical link for it is here.
Posted to bugs@httpd.apache.org by bu...@apache.org on 2016/07/15 10:52:15 UTC

[Bug 59864] New: segfault when using route-redirect pairs and both servers are disabled/in error mode

https://bz.apache.org/bugzilla/show_bug.cgi?id=59864

            Bug ID: 59864
           Summary: segfault when using route-redirect pairs and both
                    servers are disabled/in error mode
           Product: Apache httpd-2
           Version: 2.5-HEAD
          Hardware: All
                OS: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: mod_proxy_balancer
          Assignee: bugs@httpd.apache.org
          Reporter: amd1212@vier-ringe.at

A little background: I am using the redirect feature for mod_proxy_balancer as
feature to pair two servers together while all servers are still used within an
cluster. 
Thats how the configuration looks like:

<Proxy balancer://fe_cluster>
    BalancerMember http://fe01:8080 min=0 smax=30 retry=30 max=100
loadfactor=50 route=fe01 redirect=fe02
    BalancerMember http://fe02:8080 min=0 smax=30 retry=30 max=100
loadfactor=50 route=fe02 redirect=fe01
    BalancerMember http://fe03:8080 min=0 smax=30 retry=30 max=100
loadfactor=50 route=fe03 redirect=fe04
    BalancerMember http://fe04:8080 min=0 smax=30 retry=30 max=100
loadfactor=50 route=fe04 redirect=fe03
    BalancerMember http://fe05:8080 min=0 smax=30 retry=30 max=100
loadfactor=50 route=fe05 redirect=fe06
    BalancerMember http://fe06:8080 min=0 smax=30 retry=30 max=100
loadfactor=50 route=fe06 redirect=fe05
</proxy>

The goal of that configuration is that the app-servers in the back do not need
to replicate their sessions to all servers but only to their redirect partner
(and sessions can become big, so this saves lot of memory).
It might by unusual to use the redirect feature in this way, but a segfault is
a bug regardless how a feature is used.

This setup works really fine if at least one server of each pair is active (not
disabled or in error mode). If both servers of a pair are not available then
the module is hanging in a deadloop doing the redirect which ends in an
segfault.
The problem seems to be that the redirection has no limit/does not check if it
already tried to redirect to a specific memeber already.

The relevant code is located in the file: modules/proxy/mod_proxy_balancer.c:
  For httpd-2.4.23 between line 214 to 277.
  For httpd-2.2.31 between line 190 to 252.
  It looks for me that the problem still exists in 2.5-trunk.




GDB output:
[Thread 0x7fffeff86700 (LWP 52185) exited]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fffcefcd700 (LWP 52245)]
0x00007ffff3288356 in find_route_worker (balancer=balancer@entry=0x320ad1c8,
route=0x7ffff0bc9ce0 "fe01", r=r@entry=0x7fff94002970)
    at /opt/httpd-2.2.31/modules/proxy/mod_proxy_balancer.c:192



What should happen:
If there is a redirect mod_proxy_balancer should keep track of the members it
already tried to redirect to or at least have a limit how often to try to
redirect -> and then return the same error code like it does when all members
of a cluster a down: 503 "Service Temporarily Unavailable".

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 59864] segfault when using route-redirect pairs and both servers are disabled/in error mode

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59864

--- Comment #4 from Ruediger Pluem <rp...@apache.org> ---
Sorry white space issue with the first post of the patch:

Index: modules/proxy/mod_proxy_balancer.c
===================================================================
--- modules/proxy/mod_proxy_balancer.c  (revision 1752793)
+++ modules/proxy/mod_proxy_balancer.c  (working copy)
@@ -212,7 +212,8 @@
 /* Find the worker that has the 'route' defined
  */
 static proxy_worker *find_route_worker(proxy_balancer *balancer,
-                                       const char *route, request_rec *r)
+                                       const char *route, request_rec *r,
+                                       int recursion)
 {
     int i;
     int checking_standby;
@@ -249,10 +250,15 @@
                          * This enables to safely remove the member from the
                          * balancer. Of course you will need some kind of
                          * session replication between those two remote.
+                         * Also check that we haven't gone thru all the
+                         * balancer members by means of redirects.
+                         * This should avoid redirect cycles.
                          */
-                        if (*worker->s->redirect) {
+                        if ((*worker->s->redirect)
+                            && (recursion < balancer->workers->nelts)) {
                             proxy_worker *rworker = NULL;
-                            rworker = find_route_worker(balancer,
worker->s->redirect, r);
+                            rworker = find_route_worker(balancer,
worker->s->redirect,
+                                                        r, recursion + 1);
                             /* Check if the redirect worker is usable */
                             if (rworker && !PROXY_WORKER_IS_USABLE(rworker)) {
                                 /*
@@ -315,7 +321,7 @@
         /* We have a route in path or in cookie
          * Find the worker that has this route defined.
          */
-        worker = find_route_worker(balancer, *route, r);
+        worker = find_route_worker(balancer, *route, r, 1);
         if (worker && strcmp(*route, worker->s->route)) {
             /*
              * Notice that the route of the worker chosen is different from

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 59864] segfault when using route-redirect pairs and both servers are disabled/in error mode

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59864

Ruediger Pluem <rp...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
           Keywords|                            |FixedInTrunk,
                   |                            |PatchAvailable

--- Comment #6 from Ruediger Pluem <rp...@apache.org> ---
Committed to trunk as r1753594.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 59864] segfault when using route-redirect pairs and both servers are disabled/in error mode

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59864

--- Comment #2 from Ruediger Pluem <rp...@apache.org> ---
Created attachment 34043
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=34043&action=edit
Patch against trunk

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 59864] segfault when using route-redirect pairs and both servers are disabled/in error mode

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59864

Ruediger Pluem <rp...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #34043|0                           |1
        is obsolete|                            |

--- Comment #3 from Ruediger Pluem <rp...@apache.org> ---
Created attachment 34044
  --> https://bz.apache.org/bugzilla/attachment.cgi?id=34044&action=edit
Patch against trunk

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 59864] segfault when using route-redirect pairs and both servers are disabled/in error mode

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59864

Ruediger Pluem <rp...@apache.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |NEEDINFO

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 59864] segfault when using route-redirect pairs and both servers are disabled/in error mode

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59864

--- Comment #1 from Ruediger Pluem <rp...@apache.org> ---
Does the below patch fix your issue?

Index: modules/proxy/mod_proxy_balancer.c                    
===================================================================
--- modules/proxy/mod_proxy_balancer.c  (revision 1752793)         
+++ modules/proxy/mod_proxy_balancer.c  (working copy)             
@@ -212,7 +212,8 @@                                                
 /* Find the worker that has the 'route' defined                   
  */                                                               
 static proxy_worker *find_route_worker(proxy_balancer *balancer,  
-                                       const char *route, request_rec *r)
+                                       const char *route, request_rec *r,
+                                       int recursion)                    
 {                                                                        
     int i;                                                               
     int checking_standby;                                                
@@ -249,10 +250,15 @@                                                     
                          * This enables to safely remove the member from the
                          * balancer. Of course you will need some kind of   
                          * session replication between those two remote.    
+                         * Also check that we haven't gone thru all the     
+                         * balancer members by means of redirects.          
+                         * This should avoid redirect cycles.                  
                          */                                                    
-                        if (*worker->s->redirect) {                            
+                        if ((*worker->s->redirect)                             
+                            && (recursion < balancer->workers->nelts)) {       
                             proxy_worker *rworker = NULL;                      
-                            rworker = find_route_worker(balancer,
worker->s->redirect, r);                                                        
+                            rworker = find_route_worker(balancer,
worker->s->redirect,                                                            
+                                                        r, recursion + 1);     
                             /* Check if the redirect worker is usable */       
                             if (rworker && !PROXY_WORKER_IS_USABLE(rworker)) { 
                                 /*                                             
@@ -315,7 +321,7 @@                                                             
         /* We have a route in path or in cookie                                
          * Find the worker that has this route defined.                        
          */                                                                    
-        worker = find_route_worker(balancer, *route, r);                       
+        worker = find_route_worker(balancer, *route, r, 1);                    
         if (worker && strcmp(*route, worker->s->route)) {                      
             /*                                                                 
              * Notice that the route of the worker chosen is different from

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 59864] segfault when using route-redirect pairs and both servers are disabled/in error mode

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59864

--- Comment #7 from Ruediger Pluem <rp...@apache.org> ---
Proposed for backport to 2.4.x as r1754075.

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 59864] segfault when using route-redirect pairs and both servers are disabled/in error mode

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59864

--- Comment #8 from Yann Ylavic <yl...@gmail.com> ---
Backported to 2.4.24 in r1756563.

-- 
You are receiving this mail because:
You are the assignee for the bug.
---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org


[Bug 59864] segfault when using route-redirect pairs and both servers are disabled/in error mode

Posted by bu...@apache.org.
https://bz.apache.org/bugzilla/show_bug.cgi?id=59864

amd1212@vier-ringe.at changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEEDINFO                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #5 from amd1212@vier-ringe.at ---
Yes, i've tested the patch successfully on version 2.2.31.
It works like a charm, I can't reproduce the segfaults anymore and if a pair is
down the request gets sent to other members of the same cluster.

I will test it in detail on monday, but it looks very promising so far.

Thanks a lot.
best regards
Stefan

-- 
You are receiving this mail because:
You are the assignee for the bug.

---------------------------------------------------------------------
To unsubscribe, e-mail: bugs-unsubscribe@httpd.apache.org
For additional commands, e-mail: bugs-help@httpd.apache.org