You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@httpd.apache.org by Josh Trutwin <jo...@trutwins.homeip.net> on 2008/06/04 18:00:44 UTC
[users@httpd] RewriteMap question
We have a RewriteMap program that is kicked off via the following:
RewriteMap smc_rewriter prg:/usr/share/php/rewriter.php
This works well most of the time, but for heavy use it really can bog
down as there is always only one of these programs running to handle
all the requests.
Is it possible to start multiple instances of this program?
Thanks,
Josh
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] RewriteMap question
Posted by Eric Covener <co...@gmail.com>.
On Wed, Jun 4, 2008 at 12:00 PM, Josh Trutwin <jo...@trutwins.homeip.net> wrote:
> We have a RewriteMap program that is kicked off via the following:
>
> RewriteMap smc_rewriter prg:/usr/share/php/rewriter.php
>
> This works well most of the time, but for heavy use it really can bog
> down as there is always only one of these programs running to handle
> all the requests.
>
I've looked at something similar recently in 2.0.x, where one of the
big recommendations was to make sure to limit the # of times per
request that you consult the map. This means making the regex on your
rewriterule as explicit as possible, and capturing the results of the
rewritemap lookup in an environment variable whenever it's going to be
used/consulted more then once.
--
Eric Covener
covener@gmail.com
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] RewriteMap question
Posted by Josh Trutwin <jo...@trutwins.homeip.net>.
On Thu, 5 Jun 2008 15:30:25 +0200
"Krist van Besien" <kr...@gmail.com> wrote:
> > 2. I see a script on the doc page to create a dbm map using perl -
> > know of any php equiv? google's not showing anything. I have
> > nothing against perl, just looking to keep everything in the same
> > software.
>
> dbm is a standard. There might indeed be a php library to produce
> these.
I notice apache 2.2 has a nice utility for converting text to dbm.
It supports GDBM, SDBM, NDBM and berkely DB. Is there any
performance differences between those?
> > 3. The text file format appears to use spaces to separate the map
> > items. If either of these contains a space assume can URL encode?
> > E.g.
>
> It doesn't have to be. It all depends on what input the script you
> use to create the dbm map expects.
Ah, I see that now in the script txt2dbm.pl on the website.
$DB{$1} = $2 if (/^\s*(\S+)\s+(\S+)/);
Thanks,
Josh
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] RewriteMap question
Posted by Krist van Besien <kr...@gmail.com>.
On Wed, Jun 4, 2008 at 7:32 PM, Josh Trutwin <jo...@trutwins.homeip.net> wrote:
>
> 1. if you update a dbm map file do you need to reload anything or
> will the apache rewrite module pull in the changes on the fly?
If you change the dbm map it is reloaded. No need to restart apache.
> 2. I see a script on the doc page to create a dbm map using perl -
> know of any php equiv? google's not showing anything. I have nothing
> against perl, just looking to keep everything in the same software.
dbm is a standard. There might indeed be a php library to produce these.
>
> 3. The text file format appears to use spaces to separate the map
> items. If either of these contains a space assume can URL encode?
> E.g.
It doesn't have to be. It all depends on what input the script you use
to create the dbm map expects.
I have a similar situation, where I have a lot of URLs that I need to
rewrite. These are mostly URLs hardcoded in to mobile phone firmware
that point to content that now lives elsewhere. My solution is a Excel
spreadsheet maintained by the people responsible for the content of
the server, which I convert to a "tab seperated" text file. This file
is converted in to a dbm file with a perl script that expects tabs as
record seperator in it's input. It can ofcourse also be something
else. My next step is to build a web frontend to the dbm file so that
our editors can change the mappings directly. I intend to do this in
perl, but I asume building something like that in php will not be a
big problem.
Krist
--
krist.vanbesien@gmail.com
krist@vanbesien.org
Bremgarten b. Bern, Switzerland
--
A: It reverses the normal flow of conversation.
Q: What's wrong with top-posting?
A: Top-posting.
Q: What's the biggest scourge on plain text email discussions?
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] RewriteMap question
Posted by Josh Trutwin <jo...@trutwins.homeip.net>.
On Wed, 4 Jun 2008 13:04:42 -0400
"Joshua Slive" <jo...@slive.ca> wrote:
> Just use a dbm rewritemap. These will be fast and scalable and don't
> suffer from all the prg map problems.
ok - I think that's the road I'll try to take - couple quick
questions:
1. if you update a dbm map file do you need to reload anything or
will the apache rewrite module pull in the changes on the fly?
2. I see a script on the doc page to create a dbm map using perl -
know of any php equiv? google's not showing anything. I have nothing
against perl, just looking to keep everything in the same software.
3. The text file format appears to use spaces to separate the map
items. If either of these contains a space assume can URL encode?
E.g.
stupid%20url stupid-url
> I haven't tried it, but if you just point directly at the php
> script, then if the script emits a Location header with the
> hostname/protocol part, you might get an internal redirect.
Interesting, apache 2 only? Think I'll steer clear of this for
now. ;)
> I don't know why .htaccess files would matter.
Since this feature for URL rewrites is enabled via the CMS for a
site, I don't want to have to play with httpd.conf so if a RewriteMap
directive could be entered into a .htaccess it'll make my life a lot
easier. Looking at the docs though it appears I'm not able to use
this in .htaccess "Context: server config, virtual host".
Thanks for you help - are you on #apache IRC?
Josh
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] RewriteMap question
Posted by Joshua Slive <jo...@slive.ca>.
On Wed, Jun 4, 2008 at 12:37 PM, Josh Trutwin <jo...@trutwins.homeip.net> wrote:
> On Wed, 4 Jun 2008 12:08:28 -0400
> "Joshua Slive" <jo...@slive.ca> wrote:
>
>> > Is it possible to start multiple instances of this program?
>>
>> No, not directly.
>>
>> A few possibilities:
>>
>> 1. Try to do whatever you are doing without an prg: rewritemap.
>> These should really be a last resort for problems that are too
>> complicated to solve in any other way. Of course, since we don't
>> have any details of the problem, we can't help there.
>
> Actually this is an issue you helped me with quite a while ago. I
> started without a rewrite map program and generated an htaccess file
> with rules to redirect our CMS's old URL's which looked like
> sitepages/pidXXX.php to a user-defined URL. The .htaccess files grew
> extremely large for some sites so switched to a rewrite map program.
> Largest site I know of would have over 30k rules. Is there a
> better solution for that kind of URL mapping size?
Just use a dbm rewritemap. These will be fast and scalable and don't
suffer from all the prg map problems.
>> 2. Call a php script directly to issue the redirect, rather than
>> going through mod_rewrite.
>
> I don't understand - can you explain?
I haven't tried it, but if you just point directly at the php script,
then if the script emits a Location header with the hostname/protocol
part, you might get an internal redirect.
>> 3. Have mod_rewrite split the request into several different paths
>> and use an independent rewritemap for each path. This could be as
>> simple as
>> RewriteCond %{REQUEST_URI} ^[a-n]
>> RewriteRule ... using map 1 [L]
>> RewriteRule ... using map 2
>
> This is a possibility assuming RewriteMap can be loaded from htaccess
> file - I currently have two rules that use the mapper for each but
> I'm not sure that will help much. The mapper itself needs a little
> optimization but there's not a whole lot I can do there.
I don't know why .htaccess files would matter.
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] RewriteMap question
Posted by Josh Trutwin <jo...@trutwins.homeip.net>.
On Wed, 4 Jun 2008 12:08:28 -0400
"Joshua Slive" <jo...@slive.ca> wrote:
> > Is it possible to start multiple instances of this program?
>
> No, not directly.
>
> A few possibilities:
>
> 1. Try to do whatever you are doing without an prg: rewritemap.
> These should really be a last resort for problems that are too
> complicated to solve in any other way. Of course, since we don't
> have any details of the problem, we can't help there.
Actually this is an issue you helped me with quite a while ago. I
started without a rewrite map program and generated an htaccess file
with rules to redirect our CMS's old URL's which looked like
sitepages/pidXXX.php to a user-defined URL. The .htaccess files grew
extremely large for some sites so switched to a rewrite map program.
Largest site I know of would have over 30k rules. Is there a
better solution for that kind of URL mapping size?
Original thread:
http://markmail.org/message/b6ye2ww4dcjand2k#query:trutwin%20rewritemap%20questions+page:1+mid:rnxk5lx5ej6gjvjt+state:results
> 2. Call a php script directly to issue the redirect, rather than
> going through mod_rewrite.
I don't understand - can you explain?
> 3. Have mod_rewrite split the request into several different paths
> and use an independent rewritemap for each path. This could be as
> simple as
> RewriteCond %{REQUEST_URI} ^[a-n]
> RewriteRule ... using map 1 [L]
> RewriteRule ... using map 2
This is a possibility assuming RewriteMap can be loaded from htaccess
file - I currently have two rules that use the mapper for each but
I'm not sure that will help much. The mapper itself needs a little
optimization but there's not a whole lot I can do there.
Josh
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org
Re: [users@httpd] RewriteMap question
Posted by Joshua Slive <jo...@slive.ca>.
On Wed, Jun 4, 2008 at 12:00 PM, Josh Trutwin <jo...@trutwins.homeip.net> wrote:
> We have a RewriteMap program that is kicked off via the following:
>
> RewriteMap smc_rewriter prg:/usr/share/php/rewriter.php
>
> This works well most of the time, but for heavy use it really can bog
> down as there is always only one of these programs running to handle
> all the requests.
>
> Is it possible to start multiple instances of this program?
No, not directly.
A few possibilities:
1. Try to do whatever you are doing without an prg: rewritemap. These
should really be a last resort for problems that are too complicated
to solve in any other way. Of course, since we don't have any details
of the problem, we can't help there.
2. Call a php script directly to issue the redirect, rather than going
through mod_rewrite.
3. Have mod_rewrite split the request into several different paths and
use an independent rewritemap for each path. This could be as simple
as
RewriteCond %{REQUEST_URI} ^[a-n]
RewriteRule ... using map 1 [L]
RewriteRule ... using map 2
Joshua.
---------------------------------------------------------------------
The official User-To-User support forum of the Apache HTTP Server Project.
See <URL:http://httpd.apache.org/userslist.html> for more info.
To unsubscribe, e-mail: users-unsubscribe@httpd.apache.org
" from the digest: users-digest-unsubscribe@httpd.apache.org
For additional commands, e-mail: users-help@httpd.apache.org