You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Axel-Stéphane SMORGRAV <Ax...@europe.adp.com> on 2007/08/29 19:18:04 UTC
[users@httpd] Apache 2.2.x mod_rewrite and mod_proxy and pooled connections
Hello folks,
I have been doing some testing with Apache 2.2.4 recently, and one of the things I am particularly interested in is the pooling of backend connections of mod_proxy's as this makes it possible to maintain persistent connections to backend systems across requests from different clients no matter whether the client connections are persistent or not.
>From what I have gathered through testing and peeking at logs and code, mod_proxy creates one worker pool per ProxyPass directive and associates each worker with a backend connection. Workers are added to the pool as needed, within the limits defined (or default values as documented). If the connection associated with a worker has been torn down, a new connection is established.
Now this is fine and dandy as long as you define the proxying rules using ProxyPass. If you have to use RewriteRule with the [P] flag it seems like a worker named * is used which creates a new backend connection for each new request.
In cases where you have a finite set of backend servers which addresses are known at the time the configuration is written, you can work around this problem and create the worker pools by adding bogus ProxyPass directives like:
ProxyPass / !
ProxyPass / http://10.0.0.1/
ProxyPass / !
ProxyPass / http://10.0.0.2/
[Wed Aug 29 18:45:01 2007] [debug] proxy_util.c(1625): proxy: grabbed scoreboard slot 1 in child 14450 for worker http://10.0.0.1/
[Wed Aug 29 18:45:01 2007] [debug] proxy_util.c(1708): proxy: initialized worker 1 in child 14450 for (10.0.0.1) min=0 max=100 smax=100
[Wed Aug 29 18:45:01 2007] [debug] proxy_util.c(1625): proxy: grabbed scoreboard slot 2 in child 14450 for worker http://10.0.0.2/
[Wed Aug 29 18:45:01 2007] [debug] proxy_util.c(1708): proxy: initialized worker 2 in child 14450 for (10.0.0.2) min=0 max=100 smax=100
But you may also want to use RewriteMap to determine the address of the backend server to which forward the request, the advantage of which is that you can modify the mapping and add new backend servers dynamically. In that case the complete list of backend servers is not known at the time the server is started and the above trick does not apply. You now end up with a new connection to the backend server for each client request:
[Wed Aug 29 11:58:54 2007] [debug] mod_proxy_http.c(54): proxy: HTTP: canonicalising URL //10.0.66.29/ABCDEFR/site/icons.aspx
[Wed Aug 29 11:58:54 2007] [debug] proxy_util.c(1397): [client 10.0.32.163] proxy: *: found reverse proxy worker for http://10.0.66.29/ABCDEFR/site/icons.aspx?size=normal&id=FOFDFOHAHMKMBFOMPFMBBIIPIDCBAMJK
[Wed Aug 29 11:58:54 2007] [debug] mod_proxy.c(777): Running scheme http handler (attempt 0)
[Wed Aug 29 11:58:54 2007] [debug] mod_proxy_http.c(1662): proxy: HTTP: serving URL http://10.0.66.29/ABCDEFR/site/icons.aspx?size=normal&id=FOFDFOHAHMKMBFOMPFMBBIIPIDCBAMJK
[Wed Aug 29 11:58:54 2007] [debug] proxy_util.c(1798): proxy: HTTP: has acquired connection for (*)
[Wed Aug 29 11:58:54 2007] [debug] proxy_util.c(1859): proxy: connecting http://10.0.66.29/ABCDEFR/site/icons.aspx?size=normal&id=FOFDFOHAHMKMBFOMPFMBBIIPIDCBAMJK to 10.0.66.29:80
[Wed Aug 29 11:58:54 2007] [debug] proxy_util.c(1955): proxy: connected /ABCDEFR/site/icons.aspx?size=normal&id=FOFDFOHAHMKMBFOMPFMBBIIPIDCBAMJK to 10.0.66.29:80
[Wed Aug 29 11:58:54 2007] [debug] proxy_util.c(2050): proxy: HTTP: fam 2 socket created to connect to *
[Wed Aug 29 11:58:54 2007] [debug] proxy_util.c(2146): proxy: HTTP: connection complete to 10.0.66.29:80 (10.0.66.29)
[Wed Aug 29 11:58:54 2007] [debug] mod_proxy_http.c(1448): proxy: start body send
[Wed Aug 29 11:58:54 2007] [debug] mod_proxy_http.c(1537): proxy: end body send
[Wed Aug 29 11:58:54 2007] [debug] proxy_util.c(1816): proxy: HTTP: has released connection for (*)
[Wed Aug 29 11:59:01 2007] [debug] mod_proxy_http.c(54): proxy: HTTP: canonicalising URL //10.0.66.29/ABCDEFR/site/icons.aspx
[Wed Aug 29 11:59:01 2007] [debug] proxy_util.c(1397): [client 10.0.32.163] proxy: *: found reverse proxy worker for http://10.0.66.29/ABCDEFR/site/icons.aspx?size=normal&id=FOFDFOHAHMKMBFOMPFMBBIIPIDCBAMJK
[Wed Aug 29 11:59:01 2007] [debug] mod_proxy.c(777): Running scheme http handler (attempt 0)
[Wed Aug 29 11:59:01 2007] [debug] mod_proxy_http.c(1662): proxy: HTTP: serving URL http://10.0.66.29/ABCDEFR/site/icons.aspx?size=normal&id=FOFDFOHAHMKMBFOMPFMBBIIPIDCBAMJK
[Wed Aug 29 11:59:01 2007] [debug] proxy_util.c(1798): proxy: HTTP: has acquired connection for (*)
[Wed Aug 29 11:59:01 2007] [debug] proxy_util.c(1859): proxy: connecting http://10.0.66.29/ABCDEFR/site/icons.aspx?size=normal&id=FOFDFOHAHMKMBFOMPFMBBIIPIDCBAMJK to 10.0.66.29:80
[Wed Aug 29 11:59:01 2007] [debug] proxy_util.c(1955): proxy: connected /ABCDEFR/site/icons.aspx?size=normal&id=FOFDFOHAHMKMBFOMPFMBBIIPIDCBAMJK to 10.0.66.29:80
[Wed Aug 29 11:59:01 2007] [debug] proxy_util.c(2050): proxy: HTTP: fam 2 socket created to connect to *
[Wed Aug 29 11:59:01 2007] [debug] proxy_util.c(2146): proxy: HTTP: connection complete to 10.0.66.29:80 (10.0.66.29)
[Wed Aug 29 11:59:01 2007] [debug] mod_proxy_http.c(1448): proxy: start body send
[Wed Aug 29 11:59:01 2007] [debug] mod_proxy_http.c(1537): proxy: end body send
[Wed Aug 29 11:59:01 2007] [debug] proxy_util.c(1816): proxy: HTTP: has released connection for (*)
Unless I have missed something and mod_proxy already addresses this issue, it seems to me like one might want to consider creating new worker pools dynamically so that URLs that have been rewritten by mod_rewrite with the "force proxy" flag ([P]) can benefit from persistent backend connections.
I look forward to your feedback and experience in this matter
-ascs
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] Apache 2.2.x mod_rewrite and mod_proxy and pooled connections
Posted by Axel-Stéphane SMORGRAV <Ax...@europe.adp.com>.
Let me rephrase my previous post.
One proxy worker is created for each ProxyPass directive or balancer member. A connection pool is associated with each worker. One single remote server is associated with each connection pool, and thus with each worker. This allows mod_proxy to establish persistent connections to backend servers that can be used across requests from different clients.
When using mod_rewrite for proxying (RewriteRule ... [P]), the backend connections are always created on the fly for each client request and do not benefit from the use of pooled (persistent) connections.
I am wondering whether it would be possible to enhance the implementation of mod_proxy so that workers and associated connection pools can be allocated whenever a new backend server is detected through the arrival of a request for which there is no worker.
In ap_proxy_pre_request() (modules/proxy/proxy_util.c: line 1395), if no worker is found, a default worker is returned. Instead a new worker could be created for scheme://address/
modules/proxy/proxy_util.c: line 1395 (current implementation in 2.2.4)
else if (r->proxyreq == PROXYREQ_REVERSE) {
if (conf->reverse) {
ap_log_rerror(APLOG_MARK, APLOG_DEBUG, 0, r,
"proxy: *: found reverse proxy worker for %s",
*url);
*balancer = NULL;
*worker = conf->reverse;
access_status = OK;
}
-ascs
-----Message d'origine-----
De : Axel-Stéphane SMORGRAV
Envoyé : mercredi 29 août 2007 19:18
À : users@httpd.apache.org
Objet : [users@httpd] Apache 2.2.x mod_rewrite and mod_proxy and pooled connections
Hello folks,
I have been doing some testing with Apache 2.2.4 recently, and one of the things I am particularly interested in is the pooling of backend connections of mod_proxy's as this makes it possible to maintain persistent connections to backend systems across requests from different clients no matter whether the client connections are persistent or not.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org