You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Sander Smeenk <ss...@freshdot.net> on 2004/04/23 13:49:14 UTC
[users@httpd] mod_rewrite weirdness
Hello,
This is a lot of text ;)
I'm trying to build a software-based loadbalancer with Apache 1.3.xx,
for this I used Ralf Engelschall's examples and modified it a bit.
My httpd.conf has:
| RewriteMap lb prg:/var/web/load/load.pl
| RewriteRule ^/(.*)$ ${lb:%{HTTP_HOST}:$1} [P,L]
The load.pl script works, and basically does this:
1) Accept the ${HTTP_HOST}:URI string
2) Strip the first bit of the HTTP_HOST (www.bla.com -> bla.com)
3) Find out what server has the lowest load -> www4
4) print out http://www4.bla.com/URI
The loadbalancer is on domain abcde.nl, the domain it balances is called
12345.nl.
To aid debugging I put some logging in load.pl in strategic places.
It shows what the script received from apache, and what it writes out to
apache:
| www.12345.nl:index.html
| defined uri : 1 -index.html-
| defined host: 1 -12345.nl-
| http://www4.12345.nl/index.html
This is one request. The first line is what is received from STDIN, the
last line is what is written to STDOUT (back to apache as redirect
target).
This all looks sane, and works quite okay with a low amount of hits.
But if I fire up 'ab' to torture the 'loadbalancer' with some hundred
requests, and tell it to use three (or more) concurrent connections,
weird rewrite issues begin to show up. These log-entries are subsequent
log entries of subsequent requests made by the 'ab' tool:
(The exact ab incantation was: ab -n 1500 -c 3 http://www.12345.nl/index.html)
| www.12345.nl:index.html
| defined uri : 1 -index.html-
| defined host: 1 -12345.nl-
| http://www3.12345.nl/index.html
|
| lb.abcde.nl:http:/http://www3.12345.nl/index.html
| defined uri : 1 -http-
| defined host: 1 -abcde.nl-
| http://www4.abcde.nl/http
Once again, the first line shows what is received on STDIN, the last
line what is sent to apache. The first entry shows clearly that the
correct destination url was returned to apache, but the next log entry
shows that somehow apache tried to access the loadbalancer itself with a
severely malformed URI string.
Accesslog entries for this attemt, from the access log maintained by the
loadbalancer look like this:
(.181 is the client, .170 is theloadbalancer itself)
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] "GET /index.html HTTP/1.0" 200 8438 "-" "ApacheBench/1.3d"
| xx.yy.zz.170 - - [23/Apr/2004:12:00:38 +0200] "GET /http:/http://www3.12345.nl/index.html HTTP/1.1" 404 14197 "-" "ApacheBench/1.3d"
The rewrite log is a mess. Requests are mixedup and it's hard to find
what belongs to eachother. The sid / rid numbers never seem to change
either. From timestamps & grepping I figured that his might well be
the relevant entries:
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (1) go-ahead with proxy request proxy:http://www3.12345.nl/index.html [OK]
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (2) forcing proxy-throughput with http://lb.abcde.nl/www4.12345.nl/index.html
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (1) go-ahead with proxy request proxy:http://lb.abcde.nl/www4.12345.nl/index.html [OK]
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (5) map lookup OK: map=lb key=www.12345.nl:index.html -> val=http://www3.12345.nl/index.html
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (2) rewrite /index.html -> http://www3.12345.nl/index.html
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (2) forcing proxy-throughput with http://www3.12345.nl/index.html
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (1) go-ahead with proxy request proxy:http://www3.12345.nl/index.html [OK]
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (5) map lookup OK: map=lb key=www.12345.nl:index.html -> val=http://www4.12345.nl/index.html
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (2) rewrite /index.html -> http://www4.12345.nl/index.html
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (2) forcing proxy-throughput with http://www4.12345.nl/index.html
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (1) go-ahead with proxy request proxy:http://www4.12345.nl/index.html [OK]
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (5) map lookup OK: map=lb key=www.12345.nl:index.html -> val=http:/http://www3.12345.nl/index.html
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (2) rewrite /index.html -> http:/http://www3.12345.nl/index.html
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (2) forcing proxy-throughput with http://lb.abcde.nl/http:/http://www3.12345.nl/index.html
| xx.yy.zz.170 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (2) init rewrite engine with requested uri /www4.12345.nl/index.html
| xx.yy.zz.170 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (3) applying pattern '^/(.*)$' to uri '/www4.12345.nl/index.html'
| xx.yy.zz.181 - - [23/Apr/2004:12:00:38 +0200] [lb.abcde.nl/sid#8084034][rid#80af034/initial] (1) go-ahead with proxy request proxy:http://lb.abcde.nl/http:/http://www3.12345.nl/index.html [OK]
I can't seem to track down where this corruption occurs
According to my load.pl logs it returns exactly what it should return,
but it receives 'malformed' information. This makes me tend to think
that somewhere in mod_rewrite / apache the information gets messed up.
I'm running Apache 1.3.29 on both the 'loadbalancer' and the 'balanced
servers', with load.pl running in perl 5.8.2 on FreeBSD 4.9 systems.
To rule out that my load.pl script is the bad guy here, I replaced it
with a very small shellscript, eehr... perl script: :)
| #!/usr/bin/perl -w
| $|++
| while (<>) { print "http://www3.12345.nl/index.html" }
This rewrites anything and everything to the above mentioned url.
With this script, the 'weird' requests still occured.
I hope someone can shed some light on this matter.
All I have now is /etc/shadow. :)
Kind regards,
Sander.
--
| The brain is a wonderful organ; it starts working the moment you get up
| in the morning, and does not stop until you get to work.
| 1024D/08CEC94D - 34B3 3314 B146 E13C 70C8 9BDB D463 7E41 08CE C94D
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] mod_rewrite weirdness
Posted by Sander Smeenk <ss...@freshdot.net>.
Quoting Rodrigo Campos (rodrigo@geekbunker.org):
> Are you using a RewriteLock file ?
No. I'm not. I didn't really know about RewriteLock.
Weird that I never saw that option before.
Hmm. *tappety*click*click*tappety*
Ah! *clickety*tap*
| -rw-r--r-- 1 nobody wheel 0 Apr 24 09:53 rewrite.lock
Hookay...
*tap*tap*clickety*
| Concurrency Level: 10
| Time taken for tests: 9.691 seconds
| Complete requests: 1500
| Failed requests: 0
| Broken pipe errors: 0
Looks promising. The rewrite.lock appeared when it actually rewrote
something, and it now seems to work! It even seems a tad bit faster?
Thanks :) Unfortunately I have no idea why 2.0.49 doesn't create the
rewrite.lock file. I had mine written to /tmp to ensure correct
permissions ;)
Leaving you with the question, why isn't RewriteLock enabled by default
somewhere in shared memory or something?
Kind regards,
Sander.
--
| A conscience is what hurts when all of your other parts feel so good.
| 1024D/08CEC94D - 34B3 3314 B146 E13C 70C8 9BDB D463 7E41 08CE C94D
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
RE: [users@httpd] mod_rewrite weirdness
Posted by Rodrigo Campos <ro...@geekbunker.org>.
I am having exactly the same problem with a completely different setup
tough.
Apache 2.0.49 with the worker mpm, and mod_rewrite using a external
rewrite program written in C (with the output buffer properly disabled).
Are you using a RewriteLock file ?
The symptoms are identical to what you described, I noticed that the
RewriteLock file is not being created (permissions are ok).
Any ideas anyone ?
Regards,
-----Original Message-----
From: Sander Smeenk [mailto:ssmeenk+apacheusr@freshdot.net]
Sent: sexta-feira, 23 de abril de 2004 08:49
To: users@httpd.apache.org
Subject: [users@httpd] mod_rewrite weirdness
Hello,
This is a lot of text ;)
I'm trying to build a software-based loadbalancer with Apache 1.3.xx,
for this I used Ralf Engelschall's examples and modified it a bit.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org