You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Hansjoerg Pehofer <ha...@uibk.ac.at> on 2005/09/19 15:53:11 UTC
httpd-2.1.7 Connection-pooling Problem w/ ReverseProxy, Loadbalancer
Hi,
I experienced Apache-2.1.7 (/w WorkerMPM on Solaris 9) connecting to the
wrong backends when having configured more than one balancer cluster.
(see httpd.conf below)
ErrorLog shows some odd entries (marked in the example below) during
proxy_util.c's ap_proxy_determine_connection().
I could work around this behaviour by changing line 1788 of proxy_util.c
- else if (!conn->hostname) {
+ else if (!conn->hostname || strcmp(conn->hostname, uri->hostname) != 0) {
This unlikely is a fix, however: I wonder if someone has seen this
behavior before and whether someone familiar with this modules can hint
me where to look deeper into this.
kind regards
Hansjörg
-- Example ErrorLog:
[Wed Sep 14 14:33:27 2005] [debug] mod_proxy_balancer.c(41): proxy: BALANCER: canonicalising URL //webapp-cluster/gopublic/content/bidok/library
[Wed Sep 14 14:33:27 2005] [debug] mod_proxy_balancer.c(524): proxy: BALANCER (balancer://webapp-cluster) worker (http://192.168.15.5) rewritten to http://192.168.15.5/gopublic/content/bidok/library?authors=1
[Wed Sep 14 14:33:27 2005] [debug] mod_proxy.c(726): Running scheme balancer handler (attempt 0)
[Wed Sep 14 14:33:27 2005] [debug] mod_proxy_http.c(1601): proxy: HTTP: serving URL http://192.168.15.5/gopublic/content/bidok/library?authors=1
--> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(1764): proxy: connecting http://192.168.15.5/gopublic/content/bidok/library?authors=1 to 192.168.15.5:80
--> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(1860): proxy: connected /gopublic/content/bidok/library?authors=1 to 192.168.15.3:80
--> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(1952): proxy: HTTP: fam 2 socket created to connect to 192.168.15.5
--> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(2045): proxy: HTTP: connection complete to 192.168.15.3:80 (192.168.15.3)
[Wed Sep 14 14:33:27 2005] [debug] mod_proxy_http.c(1383): proxy: start body send
[Wed Sep 14 14:33:27 2005] [debug] mod_headers.c(612): headers: ap_headers_output_filter()
[Wed Sep 14 14:33:27 2005] [debug] mod_proxy_http.c(1470): proxy: end body send
[Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(1722): proxy: HTTP: has released connection for (192.168.15.3)
[Wed Sep 14 14:33:27 2005] [debug] mod_proxy_balancer.c(552): proxy_balancer_post_request for (balancer://webapp-cluster)
-- Example httpd.conf:
ServerRoot "/usr/local/apache2-2.1.7/"
TimeOut 300
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 15
StartServers 2
MaxClients 50
MinSpareThreads 25
MaxSpareThreads 50
ThreadsPerChild 25
Listen 80
User nobody
Group nobody
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule rewrite_module modules/mod_rewrite.so
DocumentRoot "/www"
ProxyRequests Off
ProxyVia Off
EnableMMAP off
ErrorLog logs/error_log
LogLevel debug
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog logs/access_log combined env=!nolog
<Location /balancer-manager>
SetHandler balancer-manager
Order Deny,Allow
Deny from all
Allow from 192.168.15
</Location>
NameVirtualHost *:80
<VirtualHost *:80>
ServerName bidok.uibk.ac.at
ProxyPreserveHost On
<Proxy balancer://webapp-cluster>
BalancerMember http://192.168.15.5
#BalancerMember http://192.168.15.6
</Proxy>
<Proxy balancer://axkit-cluster>
BalancerMember http://192.168.15.2 loadfactor=100
BalancerMember http://192.168.15.3 loadfactor=50
</Proxy>
RewriteEngine On
RewriteRule ^/library/q(.*)$ balancer://webapp-cluster/gopublic/content/bidok/library$1 [P,L]
ProxyPassReverse /library/q balancer://webapp-cluster/gopublic/content/bidok/library
RewriteRule ^/ximsroot/?(.*)$ balancer://webapp-cluster/ximsroot/$1 [P,L]
ProxyPassReverse /ximsroot/ balancer://webapp-cluster/ximsroot/
RewriteRule ^/?(.*)$ balancer://axkit-cluster/$1 [P,L]
ProxyPassReverse / balancer://axkit-cluster/
</VirtualHost>
--
IT Services University of Innsbruck
CFB4 D6E7 33F4 34C0 18B9 6661 E355 4337 3F8B D9C2
http://purl.org/net/hansjoerg.pehofer/public_key
Re: httpd-2.1.7 Connection-pooling Problem w/ ReverseProxy,
Loadbalancer
Posted by Hansjoerg Pehofer <ha...@uibk.ac.at>.
Jim Jagielski <ji...@jaguNET.com> writes:
> Can you try HEAD on httpd-trunk for a fix until something
> more robust as far as the connections are implemented...
It gets the backend-connections right, but segfaults in the new code in
proxy_util.c.
I got the following out of a coredump:
dbx: warning: Some symbolic information might be incorrect.
t@5 (l@5) terminated by signal SEGV (Segmentation Fault)
0xfefd42cc: strcasecmp+0x0024: ldub [%o2], %o3
Current function is ap_proxy_determine_connection
1837 (strcasecmp(conn->hostname, uri->hostname) != 0) ) ) {
(dbx) where
current thread: t@5
[1] strcasecmp(0x0, 0x198238, 0xfef183b4, 0xfef183b8, 0xfef183c0, 0xfef183c4), at 0xfefd42cc
=>[2] ap_proxy_determine_connection(p = 0x18dd58, r = 0x19bdc8, conf = 0x14be88, worker = 0x14b320, conn = 0x14c1e0, uri = 0x1981f0, url = 0xfea7ba20, proxyname = (nil), proxyport = 0, server_portstr = 0xfea7b9a8 "", server_portstr_size = 32), line 1837 in "proxy_util.c"
[3] proxy_http_handler(r = 0x19bdc8, worker = 0x14b320, conf = 0x14be88, url = 0x198258 "/images/xpop-1.jpg", proxyname = (nil), proxyport = 0), line 1690 in "mod_proxy_http.c"
[4] proxy_run_scheme_handler(0x19bdc8, 0x14b320, 0x14be88, 0x19d6f0, 0x0, 0x0), at 0xfeefc8e8
[5] proxy_handler(r = 0x19bdc8), line 740 in "mod_proxy.c"
[6] ap_run_handler(0x19bdc8, 0x14, 0x19cc4d, 0x0, 0x0, 0x0), at 0x5e014
[7] ap_invoke_handler(r = 0x19bdc8), line 373 in "config.c"
[8] ap_process_async_request(r = 0x19bdc8), line 241 in "http_request.c"
[9] ap_process_request(r = 0x19bdc8), line 281 in "http_request.c"
[10] ap_process_http_connection(c = 0x18de80), line 182 in "http_core.c"
[11] ap_run_process_connection(0x18de80, 0x18dd90, 0x18dd90, 0x2, 0x18de78, 0x191d68), at 0x6a7ac
[12] ap_process_connection(c = 0x18de80, csd = 0x18dd90), line 180 in "connection.c"
[13] process_socket(p = 0x18dd58, sock = 0x18dd90, my_child_num = 0, my_thread_num = 2, bucket_alloc = 0x191d68), line 532 in "worker.c"
[14] worker_thread(thd = 0xedea0, dummy = 0xdefc0), line 878 in "worker.c"
[15] dummy_worker(opaque = 0xedea0), line 138 in "thread.c"
hope this is useful,
Hansjörg
--
IT Services University of Innsbruck
CFB4 D6E7 33F4 34C0 18B9 6661 E355 4337 3F8B D9C2
http://purl.org/net/hansjoerg.pehofer/public_key
Re: httpd-2.1.7 Connection-pooling Problem w/ ReverseProxy, Loadbalancer
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Nov 30, 2005, at 4:30 PM, Ruediger Pluem wrote:
>
> I believe the clearer approach would be to fetch the connection
> from the connection pool
> each time and do not use this module config method any longer. This
> would also resolve the
> problem of Hans-Joerg which you fixed with your recent patches as
> we would get the correct
> connection then anyway.
>> From a first quick view the whole infrastructure for doing this
>> seems to be prepared
> pretty well already.
>
Yeah, I'm still not sure what the original intent was, but
from what I can see, using the pool for each one is
certainly cleaner...
I'm looking into how much of DBD can be re-used...
Re: httpd-2.1.7 Connection-pooling Problem w/ ReverseProxy, Loadbalancer
Posted by Ruediger Pluem <rp...@apache.org>.
On 11/30/2005 08:38 PM, Jim Jagielski wrote:
>
> On Nov 29, 2005, at 4:55 PM, Ruediger Pluem wrote:
[..cut..]
>>
>> Has someone found out out why we close the connection if
>> r->proxyreq == PROXYREQ_PROXY || r->proxyreq == PROXYREQ_REVERSE?
>>
>> I fear that this is something that has survived from 2.0.x.
>> I am also wondering why mod_proxy_http is storing the connection
>> in the module config. Is this something from 2.0.x days where we had
>> no connection
>> pool or is it a security problem to share one keepalive http backend
>> connections across
>> different client connections and vice versa?
>>
>
> Doubtful, since this aspect of the code didn't exist
> in 2.0. IIRC, it had to do with some aspects of
> pooling and threads, but it's been awhile since it
> was folded in. Likely it has to do with AJPs connection
> pool, but that's a wag.
Ok, regarding the PROXYREQ_REVERSE issue you are right. I had a look at the 2.0.x
code meanwhile (sorry been too lazy for that before) and found no signs for this.
Regarding the storage of the backend connection in the module config I found
very similar code pieces in the old 2.0.x sources so I think it is an heritage from 2.0.x.
I believe the clearer approach would be to fetch the connection from the connection pool
each time and do not use this module config method any longer. This would also resolve the
problem of Hans-Joerg which you fixed with your recent patches as we would get the correct
connection then anyway.
>From a first quick view the whole infrastructure for doing this seems to be prepared
pretty well already.
>
> Anyway, I'm likely going to rework that area a bit and move
> some things around to try to optimize the flow a bit...
>
I also hope to have some time on the weekend to make some thoughts.
As soon as they are made I'll let you and everyone know :-).
Regards
Rüdiger
Re: httpd-2.1.7 Connection-pooling Problem w/ ReverseProxy, Loadbalancer
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Nov 29, 2005, at 4:55 PM, Ruediger Pluem wrote:
>
>
> On 11/29/2005 04:12 PM, Jim Jagielski wrote:
>> Can you try HEAD on httpd-trunk for a fix until something
>> more robust as far as the connections are implemented...
>>
>
> Just for convenience:
>
> http://svn.apache.org/viewcvs?rev=349723&view=rev
>
> Has someone found out out why we close the connection if
> r->proxyreq == PROXYREQ_PROXY || r->proxyreq == PROXYREQ_REVERSE?
>
> I fear that this is something that has survived from 2.0.x.
> I am also wondering why mod_proxy_http is storing the connection
> in the module config. Is this something from 2.0.x days where we
> had no connection
> pool or is it a security problem to share one keepalive http
> backend connections across
> different client connections and vice versa?
>
Doubtful, since this aspect of the code didn't exist
in 2.0. IIRC, it had to do with some aspects of
pooling and threads, but it's been awhile since it
was folded in. Likely it has to do with AJPs connection
pool, but that's a wag.
Anyway, I'm likely going to rework that area a bit and move
some things around to try to optimize the flow a bit...
Re: httpd-2.1.7 Connection-pooling Problem w/ ReverseProxy, Loadbalancer
Posted by Ruediger Pluem <rp...@apache.org>.
On 11/29/2005 04:12 PM, Jim Jagielski wrote:
> Can you try HEAD on httpd-trunk for a fix until something
> more robust as far as the connections are implemented...
>
Just for convenience:
http://svn.apache.org/viewcvs?rev=349723&view=rev
Has someone found out out why we close the connection if
r->proxyreq == PROXYREQ_PROXY || r->proxyreq == PROXYREQ_REVERSE?
I fear that this is something that has survived from 2.0.x.
I am also wondering why mod_proxy_http is storing the connection
in the module config. Is this something from 2.0.x days where we had no connection
pool or is it a security problem to share one keepalive http backend connections across
different client connections and vice versa?
Regards
Rüdiger
Re: httpd-2.1.7 Connection-pooling Problem w/ ReverseProxy, Loadbalancer
Posted by Jim Jagielski <ji...@jaguNET.com>.
Can you try HEAD on httpd-trunk for a fix until something
more robust as far as the connections are implemented...
Re: httpd-2.1.7 Connection-pooling Problem w/ ReverseProxy, Loadbalancer
Posted by Ruediger Pluem <rp...@apache.org>.
On 11/23/2005 06:11 PM, Jim Jagielski wrote:
> It seems to me that is_address_reusable should be
> assumed to be 0 if either
>
> (r->proxyreq == PROXYREQ_PROXY || r->proxyreq == PROXYREQ_REVERSE)
>
> As mentioned, we go ahead and force close the socket
Does anybody remember why we close the socket in the case of
r->proxyreq == PROXYREQ_PROXY || r->proxyreq == PROXYREQ_REVERSE
?
>From the first glance it seems to kill many of the connection pooling advantages
(especially with the ajp modul) of reusing already established connections.
Regards
Rüdiger
Re: httpd-2.1.7 Connection-pooling Problem w/ ReverseProxy, Loadbalancer
Posted by Jim Jagielski <ji...@jaguNET.com>.
It seems to me that is_address_reusable should be
assumed to be 0 if either
(r->proxyreq == PROXYREQ_PROXY || r->proxyreq == PROXYREQ_REVERSE)
As mentioned, we go ahead and force close the socket
anyway, the problem is that conn->hostname still
points to the wrong element. So:
if (r->proxyreq == PROXYREQ_PROXY || r->proxyreq ==
PROXYREQ_REVERSE ||
!worker->is_address_reusable) {
if (proxyname) {
conn->hostname = apr_pstrdup(conn->pool, proxyname);
conn->port = proxyport;
} else {
conn->hostname = apr_pstrdup(conn->pool, uri->hostname);
conn->port = uri->port;
}
}
is likely a good enough fix, until we add better logic to
the whole backend/pool/keepalive structure.
On Nov 22, 2005, at 7:25 AM, Hansjoerg Pehofer wrote:
> Rüdiger, Jim,
>
> I used the tarball from http://people.apache.org/~pquerna/dev/
> httpd-2.1.10/.
>
> Ruediger Pluem <rp...@apache.org> writes:
>
> [...]
>>
>> I think the real problem happens if you have requests to different
>> backends (webapp-cluster, axkit-cluster) on one
>> shared keepalive connection to the reverse proxy.
>>
>> So having something like
>>
>> GET /library/q
>> and
>> GET /anything
>>
>> served over the same keepalive connection between client and
>> reverse proxy should cause trouble, because of
>>
>> if (!r->main) {
>> backend = (proxy_conn_rec *) ap_get_module_config(c-
>> >conn_config,
>>
>> &proxy_http_module);
>> }
>>
>> starting at line 1668 of mod_proxy_http.c which gets the previous
>> backend
>> connection. Although the socket to the backend is closed for
>> Reverse Proxy Requests
>> in ap_proxy_determine_connection of line 1856 of proxy_util.c, the
>> new connection
>> gets created to the same address in line 2010 of proxy_util.c
>> (ap_proxy_connect_backend).
>> I have not found the time to search for a solution.
>> Just as a check to my theory can you disable keepalives and see if
>> everything works correctly
>> then?
>
> You are right, it works correctly with KeepAlive Off.
>
> Thanks,
> Hansjörg
>
> - --
> IT Services University of Innsbruck
> CFB4 D6E7 33F4 34C0 18B9 6661 E355 4337 3F8B D9C2
> http://purl.org/net/hansjoerg.pehofer/public_key
Re: httpd-2.1.7 Connection-pooling Problem w/ ReverseProxy,
Loadbalancer
Posted by Hansjoerg Pehofer <ha...@uibk.ac.at>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Rüdiger, Jim,
I used the tarball from http://people.apache.org/~pquerna/dev/httpd-2.1.10/.
Ruediger Pluem <rp...@apache.org> writes:
[...]
>
> I think the real problem happens if you have requests to different backends (webapp-cluster, axkit-cluster) on one
> shared keepalive connection to the reverse proxy.
>
> So having something like
>
> GET /library/q
> and
> GET /anything
>
> served over the same keepalive connection between client and reverse proxy should cause trouble, because of
>
> if (!r->main) {
> backend = (proxy_conn_rec *) ap_get_module_config(c->conn_config,
> &proxy_http_module);
> }
>
> starting at line 1668 of mod_proxy_http.c which gets the previous backend
> connection. Although the socket to the backend is closed for Reverse Proxy Requests
> in ap_proxy_determine_connection of line 1856 of proxy_util.c, the new connection
> gets created to the same address in line 2010 of proxy_util.c (ap_proxy_connect_backend).
> I have not found the time to search for a solution.
> Just as a check to my theory can you disable keepalives and see if everything works correctly
> then?
You are right, it works correctly with KeepAlive Off.
Thanks,
Hansjörg
- --
IT Services University of Innsbruck
CFB4 D6E7 33F4 34C0 18B9 6661 E355 4337 3F8B D9C2
http://purl.org/net/hansjoerg.pehofer/public_key
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
iD8DBQFDgw5G41VDNz+L2cIRAj7RAJ9JCAD37QsNd2I7tGuDIFaAao61qACdGUq0
ZgtLXEkd3TBZ5gM+CJmyyRU=
=eYuY
-----END PGP SIGNATURE-----
Re: httpd-2.1.7 Connection-pooling Problem w/ ReverseProxy, Loadbalancer
Posted by Ruediger Pluem <rp...@apache.org>.
On 11/21/2005 06:52 PM, Jim Jagielski wrote:
> Thanks for the report: You said that you tested against
> 2.1.10-HEAD right?
>
> On Nov 21, 2005, at 10:25 AM, Hansjoerg Pehofer wrote:
>
> Hi,
>
> I briefly tried httpd-2.1.10 today and still see this.
> (Same httpd.conf, only changed ServerRoot to
> "/usr/local/apache2-2.1.10/")
>
> kind regards
> Hansjörg
>
> Hansjoerg Pehofer <ha...@uibk.ac.at> writes:
>
>>>> Hi,
>>>>
>>>> I experienced Apache-2.1.7 (/w WorkerMPM on Solaris 9) connecting to
>>>> the
>>>> wrong backends when having configured more than one balancer cluster.
>>>> (see httpd.conf below)
>>>> ErrorLog shows some odd entries (marked in the example below) during
>>>> proxy_util.c's ap_proxy_determine_connection().
>>>>
>>>> I could work around this behaviour by changing line 1788 of
>>>> proxy_util.c
>>>>
>>>> - else if (!conn->hostname) {
>>>> + else if (!conn->hostname || strcmp(conn->hostname, uri- >hostname)
>>>> != 0) {
>>>>
>>>> This unlikely is a fix, however: I wonder if someone has seen this
>>>> behavior before and whether someone familiar with this modules can hint
>>>> me where to look deeper into this.
>>>>
>>>> kind regards
>>>> Hansjörg
I think the real problem happens if you have requests to different backends (webapp-cluster, axkit-cluster) on one
shared keepalive connection to the reverse proxy.
So having something like
GET /library/q
and
GET /anything
served over the same keepalive connection between client and reverse proxy should cause trouble, because of
if (!r->main) {
backend = (proxy_conn_rec *) ap_get_module_config(c->conn_config,
&proxy_http_module);
}
starting at line 1668 of mod_proxy_http.c which gets the previous backend
connection. Although the socket to the backend is closed for Reverse Proxy Requests
in ap_proxy_determine_connection of line 1856 of proxy_util.c, the new connection
gets created to the same address in line 2010 of proxy_util.c (ap_proxy_connect_backend).
I have not found the time to search for a solution.
Just as a check to my theory can you disable keepalives and see if everything works correctly
then?
[..cut..]
Regards
Rüdiger
Re: httpd-2.1.7 Connection-pooling Problem w/ ReverseProxy, Loadbalancer
Posted by Jim Jagielski <ji...@jaguNET.com>.
Thanks for the report: You said that you tested against
2.1.10-HEAD right?
On Nov 21, 2005, at 10:25 AM, Hansjoerg Pehofer wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hi,
>
> I briefly tried httpd-2.1.10 today and still see this.
> (Same httpd.conf, only changed ServerRoot to
> "/usr/local/apache2-2.1.10/")
>
> kind regards
> Hansjörg
>
> Hansjoerg Pehofer <ha...@uibk.ac.at> writes:
>
>> Hi,
>>
>> I experienced Apache-2.1.7 (/w WorkerMPM on Solaris 9) connecting
>> to the
>> wrong backends when having configured more than one balancer cluster.
>> (see httpd.conf below)
>> ErrorLog shows some odd entries (marked in the example below) during
>> proxy_util.c's ap_proxy_determine_connection().
>>
>> I could work around this behaviour by changing line 1788 of
>> proxy_util.c
>>
>> - else if (!conn->hostname) {
>> + else if (!conn->hostname || strcmp(conn->hostname, uri-
>> >hostname) != 0) {
>>
>> This unlikely is a fix, however: I wonder if someone has seen this
>> behavior before and whether someone familiar with this modules can
>> hint
>> me where to look deeper into this.
>>
>> kind regards
>> Hansjörg
>>
>> -- Example ErrorLog:
>>
>> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy_balancer.c(41):
>> proxy: BALANCER: canonicalising URL //webapp-cluster/gopublic/
>> content/bidok/library
>> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy_balancer.c(524):
>> proxy: BALANCER (balancer://webapp-cluster) worker (http://
>> 192.168.15.5) rewritten to http://192.168.15.5/gopublic/content/
>> bidok/library?authors=1
>> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy.c(726): Running
>> scheme balancer handler (attempt 0)
>> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy_http.c(1601):
>> proxy: HTTP: serving URL http://192.168.15.5/gopublic/content/
>> bidok/library?authors=1
>> --> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(1764): proxy:
>> connecting http://192.168.15.5/gopublic/content/bidok/library?
>> authors=1 to 192.168.15.5:80
>> --> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(1860): proxy:
>> connected /gopublic/content/bidok/library?authors=1 to
>> 192.168.15.3:80
>> --> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(1952): proxy:
>> HTTP: fam 2 socket created to connect to 192.168.15.5
>> --> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(2045): proxy:
>> HTTP: connection complete to 192.168.15.3:80 (192.168.15.3)
>> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy_http.c(1383):
>> proxy: start body send
>> [Wed Sep 14 14:33:27 2005] [debug] mod_headers.c(612):
>> headers: ap_headers_output_filter()
>> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy_http.c(1470):
>> proxy: end body send
>> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(1722): proxy:
>> HTTP: has released connection for (192.168.15.3)
>> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy_balancer.c(552):
>> proxy_balancer_post_request for (balancer://webapp-cluster)
>>
>> -- Example httpd.conf:
>>
>> ServerRoot "/usr/local/apache2-2.1.7/"
>> TimeOut 300
>> KeepAlive On
>> MaxKeepAliveRequests 100
>> KeepAliveTimeout 15
>> StartServers 2
>> MaxClients 50
>> MinSpareThreads 25
>> MaxSpareThreads 50
>> ThreadsPerChild 25
>>
>> Listen 80
>> User nobody
>> Group nobody
>>
>> LoadModule proxy_module modules/mod_proxy.so
>> LoadModule proxy_http_module modules/mod_proxy_http.so
>> LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
>> LoadModule rewrite_module modules/mod_rewrite.so
>>
>> DocumentRoot "/www"
>>
>> ProxyRequests Off
>> ProxyVia Off
>> EnableMMAP off
>>
>> ErrorLog logs/error_log
>> LogLevel debug
>> LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-
>> Agent}i\"" combined
>> CustomLog logs/access_log combined env=!nolog
>>
>> <Location /balancer-manager>
>> SetHandler balancer-manager
>>
>> Order Deny,Allow
>> Deny from all
>> Allow from 192.168.15
>> </Location>
>>
>> NameVirtualHost *:80
>> <VirtualHost *:80>
>>
>> ServerName bidok.uibk.ac.at
>> ProxyPreserveHost On
>>
>> <Proxy balancer://webapp-cluster>
>> BalancerMember http://192.168.15.5
>> #BalancerMember http://192.168.15.6
>> </Proxy>
>>
>> <Proxy balancer://axkit-cluster>
>> BalancerMember http://192.168.15.2 loadfactor=100
>> BalancerMember http://192.168.15.3 loadfactor=50
>> </Proxy>
>>
>> RewriteEngine On
>> RewriteRule ^/library/q(.*)$ balancer://webapp-cluster/
>> gopublic/content/bidok/library$1 [P,L]
>> ProxyPassReverse /library/q balancer://webapp-cluster/
>> gopublic/content/bidok/library
>> RewriteRule ^/ximsroot/?(.*)$ balancer://webapp-cluster/
>> ximsroot/$1 [P,L]
>> ProxyPassReverse /ximsroot/ balancer://webapp-cluster/
>> ximsroot/
>> RewriteRule ^/?(.*)$ balancer://axkit-cluster/
>> $1 [P,L]
>> ProxyPassReverse / balancer://axkit-cluster/
>> </VirtualHost>
>>
>> --
>> IT Services University of Innsbruck
>> CFB4 D6E7 33F4 34C0 18B9 6661 E355 4337 3F8B D9C2
>> http://purl.org/net/hansjoerg.pehofer/public_key
>
> - --
> IT Services University of Innsbruck
> CFB4 D6E7 33F4 34C0 18B9 6661 E355 4337 3F8B D9C2
> http://purl.org/net/hansjoerg.pehofer/public_key
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.1 (GNU/Linux)
>
> iD8DBQFDgecC41VDNz+L2cIRAv0CAJ9zvxnjPc3m+HAXdc4bekkybot/nQCgiFIw
> USti/+rXeppNK5QjyfqU2B8=
> =Am2K
> -----END PGP SIGNATURE-----
>
Re: httpd-2.1.7 Connection-pooling Problem w/ ReverseProxy,
Loadbalancer
Posted by Hansjoerg Pehofer <ha...@uibk.ac.at>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
I briefly tried httpd-2.1.10 today and still see this.
(Same httpd.conf, only changed ServerRoot to
"/usr/local/apache2-2.1.10/")
kind regards
Hansjörg
Hansjoerg Pehofer <ha...@uibk.ac.at> writes:
> Hi,
>
> I experienced Apache-2.1.7 (/w WorkerMPM on Solaris 9) connecting to the
> wrong backends when having configured more than one balancer cluster.
> (see httpd.conf below)
> ErrorLog shows some odd entries (marked in the example below) during
> proxy_util.c's ap_proxy_determine_connection().
>
> I could work around this behaviour by changing line 1788 of proxy_util.c
>
> - else if (!conn->hostname) {
> + else if (!conn->hostname || strcmp(conn->hostname, uri->hostname) != 0) {
>
> This unlikely is a fix, however: I wonder if someone has seen this
> behavior before and whether someone familiar with this modules can hint
> me where to look deeper into this.
>
> kind regards
> Hansjörg
>
> -- Example ErrorLog:
>
> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy_balancer.c(41): proxy: BALANCER: canonicalising URL //webapp-cluster/gopublic/content/bidok/library
> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy_balancer.c(524): proxy: BALANCER (balancer://webapp-cluster) worker (http://192.168.15.5) rewritten to http://192.168.15.5/gopublic/content/bidok/library?authors=1
> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy.c(726): Running scheme balancer handler (attempt 0)
> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy_http.c(1601): proxy: HTTP: serving URL http://192.168.15.5/gopublic/content/bidok/library?authors=1
> --> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(1764): proxy: connecting http://192.168.15.5/gopublic/content/bidok/library?authors=1 to 192.168.15.5:80
> --> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(1860): proxy: connected /gopublic/content/bidok/library?authors=1 to 192.168.15.3:80
> --> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(1952): proxy: HTTP: fam 2 socket created to connect to 192.168.15.5
> --> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(2045): proxy: HTTP: connection complete to 192.168.15.3:80 (192.168.15.3)
> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy_http.c(1383): proxy: start body send
> [Wed Sep 14 14:33:27 2005] [debug] mod_headers.c(612): headers: ap_headers_output_filter()
> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy_http.c(1470): proxy: end body send
> [Wed Sep 14 14:33:27 2005] [debug] proxy_util.c(1722): proxy: HTTP: has released connection for (192.168.15.3)
> [Wed Sep 14 14:33:27 2005] [debug] mod_proxy_balancer.c(552): proxy_balancer_post_request for (balancer://webapp-cluster)
>
> -- Example httpd.conf:
>
> ServerRoot "/usr/local/apache2-2.1.7/"
> TimeOut 300
> KeepAlive On
> MaxKeepAliveRequests 100
> KeepAliveTimeout 15
> StartServers 2
> MaxClients 50
> MinSpareThreads 25
> MaxSpareThreads 50
> ThreadsPerChild 25
>
> Listen 80
> User nobody
> Group nobody
>
> LoadModule proxy_module modules/mod_proxy.so
> LoadModule proxy_http_module modules/mod_proxy_http.so
> LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
> LoadModule rewrite_module modules/mod_rewrite.so
>
> DocumentRoot "/www"
>
> ProxyRequests Off
> ProxyVia Off
> EnableMMAP off
>
> ErrorLog logs/error_log
> LogLevel debug
> LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
> CustomLog logs/access_log combined env=!nolog
>
> <Location /balancer-manager>
> SetHandler balancer-manager
>
> Order Deny,Allow
> Deny from all
> Allow from 192.168.15
> </Location>
>
> NameVirtualHost *:80
> <VirtualHost *:80>
>
> ServerName bidok.uibk.ac.at
> ProxyPreserveHost On
>
> <Proxy balancer://webapp-cluster>
> BalancerMember http://192.168.15.5
> #BalancerMember http://192.168.15.6
> </Proxy>
>
> <Proxy balancer://axkit-cluster>
> BalancerMember http://192.168.15.2 loadfactor=100
> BalancerMember http://192.168.15.3 loadfactor=50
> </Proxy>
>
> RewriteEngine On
> RewriteRule ^/library/q(.*)$ balancer://webapp-cluster/gopublic/content/bidok/library$1 [P,L]
> ProxyPassReverse /library/q balancer://webapp-cluster/gopublic/content/bidok/library
> RewriteRule ^/ximsroot/?(.*)$ balancer://webapp-cluster/ximsroot/$1 [P,L]
> ProxyPassReverse /ximsroot/ balancer://webapp-cluster/ximsroot/
> RewriteRule ^/?(.*)$ balancer://axkit-cluster/$1 [P,L]
> ProxyPassReverse / balancer://axkit-cluster/
> </VirtualHost>
>
> --
> IT Services University of Innsbruck
> CFB4 D6E7 33F4 34C0 18B9 6661 E355 4337 3F8B D9C2
> http://purl.org/net/hansjoerg.pehofer/public_key
- --
IT Services University of Innsbruck
CFB4 D6E7 33F4 34C0 18B9 6661 E355 4337 3F8B D9C2
http://purl.org/net/hansjoerg.pehofer/public_key
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
iD8DBQFDgecC41VDNz+L2cIRAv0CAJ9zvxnjPc3m+HAXdc4bekkybot/nQCgiFIw
USti/+rXeppNK5QjyfqU2B8=
=Am2K
-----END PGP SIGNATURE-----