You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Nick Gearls <ni...@gmail.com> on 2009/03/09 14:34:55 UTC
mod_substitute & back-references
Hi,
Is there any plan to support back-references in mod_substitute ?
mod_sed could obviously do it, but it must be much heavier.
Thanks,
Nick
Re: mod_substitute & back-references
Posted by Nick Kew <ni...@webthing.com>.
On Wed, 18 Mar 2009 08:44:12 -0400
Jim Jagielski <ji...@jaguNET.com> wrote:
> doesn't mod_proxy_html do it's own substitutions.... :/
Only in a URL context.
Being markup-aware is what distinguishes it from the various
sed-like filter modules.
--
Nick Kew
Application Development with Apache - the Apache Modules Book
http://www.apachetutor.org/
Re: mod_substitute & back-references
Posted by Jim Jagielski <ji...@jaguNET.com>.
doesn't mod_proxy_html do it's own substitutions.... :/
On Mar 16, 2009, at 7:20 AM, Nick Gearls wrote:
> I found more info:
>
> I had mod_proxy_html loaded.
> If I remove it, it changes "abc" to "------abc++++++", thus twice
> instead of 3 times. Strange, isn't it ?
>
> I performed the test on a minimal config, and it works correctly.
> I cannot reproduce the mod_proxy_html problem on this minimal config
> either.
> I suppose there is another interaction.
>
> I'll try to make more tests.
>
> Regards,
>
> Nick
>
>
> Nick Gearls wrote:
>> Same file, mod_substitute from 2.2.11, test under Windows
>> Regards,
>> Nick
>> Plüm, Rüdiger, VF-Group wrote:
>>>
>>>> -----Ursprüngliche Nachricht-----
>>>> Von: Nick Gearls Gesendet: Montag, 16. März 2009 10:02
>>>> An: dev@httpd.apache.org
>>>> Betreff: Re: mod_substitute & back-references
>>>>
>>>> Different result, but still incorrect:
>>>> Substitute s|abc|---abc+++|n
>>>> changes "abc" to "---------abc+++++++++"
>>>
>>> I cannot reproduce this. Mind to share your configuration?
>>>
>>> I used the following file as a source for the filter:
>>>
>>> <html><body>
>>> abc
>>> </body></html>
>>>
>>> Regards
>>>
>>> Rüdiger
>>>
>>>
>
Re: mod_substitute & back-references
Posted by Nick Gearls <ni...@gmail.com>.
I found more info:
I had mod_proxy_html loaded.
If I remove it, it changes "abc" to "------abc++++++", thus twice
instead of 3 times. Strange, isn't it ?
I performed the test on a minimal config, and it works correctly.
I cannot reproduce the mod_proxy_html problem on this minimal config either.
I suppose there is another interaction.
I'll try to make more tests.
Regards,
Nick
Nick Gearls wrote:
> Same file, mod_substitute from 2.2.11, test under Windows
>
> Regards,
>
> Nick
>
>
> Plüm, Rüdiger, VF-Group wrote:
>>
>>
>>> -----Ursprüngliche Nachricht-----
>>> Von: Nick Gearls Gesendet: Montag, 16. März 2009 10:02
>>> An: dev@httpd.apache.org
>>> Betreff: Re: mod_substitute & back-references
>>>
>>> Different result, but still incorrect:
>>> Substitute s|abc|---abc+++|n
>>> changes "abc" to "---------abc+++++++++"
>>
>> I cannot reproduce this. Mind to share your configuration?
>>
>> I used the following file as a source for the filter:
>>
>> <html><body>
>> abc
>> </body></html>
>>
>> Regards
>>
>> Rüdiger
>>
>>
>
Re: mod_substitute & back-references
Posted by Nick Gearls <ni...@gmail.com>.
Same file, mod_substitute from 2.2.11, test under Windows
Regards,
Nick
Plüm, Rüdiger, VF-Group wrote:
>
>
>> -----Ursprüngliche Nachricht-----
>> Von: Nick Gearls
>> Gesendet: Montag, 16. März 2009 10:02
>> An: dev@httpd.apache.org
>> Betreff: Re: mod_substitute & back-references
>>
>> Different result, but still incorrect:
>> Substitute s|abc|---abc+++|n
>> changes "abc" to "---------abc+++++++++"
>
> I cannot reproduce this. Mind to share your configuration?
>
> I used the following file as a source for the filter:
>
> <html><body>
> abc
> </body></html>
>
> Regards
>
> Rüdiger
>
>
Re: mod_substitute & back-references
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
> -----Ursprüngliche Nachricht-----
> Von: Nick Gearls
> Gesendet: Montag, 16. März 2009 10:02
> An: dev@httpd.apache.org
> Betreff: Re: mod_substitute & back-references
>
> Different result, but still incorrect:
> Substitute s|abc|---abc+++|n
> changes "abc" to "---------abc+++++++++"
I cannot reproduce this. Mind to share your configuration?
I used the following file as a source for the filter:
<html><body>
abc
</body></html>
Regards
Rüdiger
Re: mod_substitute & back-references
Posted by Nick Gearls <ni...@gmail.com>.
Different result, but still incorrect:
Substitute s|abc|---abc+++|n
changes "abc" to "---------abc+++++++++"
Regards,
Nick
Plüm, Rüdiger, VF-Group wrote:
> Ok. Now I could reproduce it. Could you please check if the following patch
> fixes it for you?
>
> Index: modules/filters/mod_substitute.c
> ===================================================================
> --- modules/filters/mod_substitute.c (revision 753197)
> +++ modules/filters/mod_substitute.c (working copy)
> @@ -199,7 +199,6 @@
> tmp_b = apr_bucket_transient_create(s1, strlen(s1),
> f->r->connection->bucket_alloc);
> APR_BUCKET_INSERT_BEFORE(b, tmp_b);
> - tmp_b = APR_BUCKET_NEXT(b);
> apr_bucket_delete(b);
> b = tmp_b;
> }
> @@ -249,7 +248,6 @@
> tmp_b = apr_bucket_transient_create(s1, strlen(s1),
> f->r->connection->bucket_alloc);
> APR_BUCKET_INSERT_BEFORE(b, tmp_b);
> - tmp_b = APR_BUCKET_NEXT(b);
> apr_bucket_delete(b);
> b = tmp_b;
> }
>
> Regards
>
> Rüdiger
>
>> -----Ursprüngliche Nachricht-----
>> Von: Nick Gearls
>> Gesendet: Freitag, 13. März 2009 15:26
>> An: dev@httpd.apache.org
>> Betreff: Re: mod_substitute & back-references
>>
>> No, only once at a time.
>> It's just to give several examples
>>
>> Regards,
>>
>> Nick
>>
>>
>> Plüm, Rüdiger, VF-Group wrote:
>>>
>>>
>>>> -----Ursprüngliche Nachricht-----
>>>> Von: Nick Gearls
>>>> Gesendet: Freitag, 13. März 2009 13:48
>>>> An: dev@httpd.apache.org
>>>> Betreff: Re: mod_substitute & back-references
>>>>
>>>> You can reproduce the problems on this file:
>>>> <html><body>
>>>> abc
>>>> </body></html>
>>>>
>>>> 1. Substitute s|abc|abc|n -> OK: "abc"
>>>> 2. Substitute s|abc|+abc| -> NOK: "+++abc"
>>>> 3. Substitute s|abc|++abc|nq -> NOK: "++++++abc"
>>>> 4. Substitute s|abc|+++abc|q -> NOK: "+++++++++abc"
>>> Are you using all these above at the same time?
>>>
>>> Regards
>>>
>>> Rüdiger
>>>
>>>
>
Re: mod_substitute & back-references
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
Ok. Now I could reproduce it. Could you please check if the following patch
fixes it for you?
Index: modules/filters/mod_substitute.c
===================================================================
--- modules/filters/mod_substitute.c (revision 753197)
+++ modules/filters/mod_substitute.c (working copy)
@@ -199,7 +199,6 @@
tmp_b = apr_bucket_transient_create(s1, strlen(s1),
f->r->connection->bucket_alloc);
APR_BUCKET_INSERT_BEFORE(b, tmp_b);
- tmp_b = APR_BUCKET_NEXT(b);
apr_bucket_delete(b);
b = tmp_b;
}
@@ -249,7 +248,6 @@
tmp_b = apr_bucket_transient_create(s1, strlen(s1),
f->r->connection->bucket_alloc);
APR_BUCKET_INSERT_BEFORE(b, tmp_b);
- tmp_b = APR_BUCKET_NEXT(b);
apr_bucket_delete(b);
b = tmp_b;
}
Regards
Rüdiger
> -----Ursprüngliche Nachricht-----
> Von: Nick Gearls
> Gesendet: Freitag, 13. März 2009 15:26
> An: dev@httpd.apache.org
> Betreff: Re: mod_substitute & back-references
>
> No, only once at a time.
> It's just to give several examples
>
> Regards,
>
> Nick
>
>
> Plüm, Rüdiger, VF-Group wrote:
> >
> >
> >> -----Ursprüngliche Nachricht-----
> >> Von: Nick Gearls
> >> Gesendet: Freitag, 13. März 2009 13:48
> >> An: dev@httpd.apache.org
> >> Betreff: Re: mod_substitute & back-references
> >>
> >> You can reproduce the problems on this file:
> >> <html><body>
> >> abc
> >> </body></html>
> >>
> >> 1. Substitute s|abc|abc|n -> OK: "abc"
> >> 2. Substitute s|abc|+abc| -> NOK: "+++abc"
> >> 3. Substitute s|abc|++abc|nq -> NOK: "++++++abc"
> >> 4. Substitute s|abc|+++abc|q -> NOK: "+++++++++abc"
> >
> > Are you using all these above at the same time?
> >
> > Regards
> >
> > Rüdiger
> >
> >
>
Re: mod_substitute & back-references
Posted by Nick Gearls <ni...@gmail.com>.
No, only once at a time.
It's just to give several examples
Regards,
Nick
Plüm, Rüdiger, VF-Group wrote:
>
>
>> -----Ursprüngliche Nachricht-----
>> Von: Nick Gearls
>> Gesendet: Freitag, 13. März 2009 13:48
>> An: dev@httpd.apache.org
>> Betreff: Re: mod_substitute & back-references
>>
>> You can reproduce the problems on this file:
>> <html><body>
>> abc
>> </body></html>
>>
>> 1. Substitute s|abc|abc|n -> OK: "abc"
>> 2. Substitute s|abc|+abc| -> NOK: "+++abc"
>> 3. Substitute s|abc|++abc|nq -> NOK: "++++++abc"
>> 4. Substitute s|abc|+++abc|q -> NOK: "+++++++++abc"
>
> Are you using all these above at the same time?
>
> Regards
>
> Rüdiger
>
>
Re: mod_substitute & back-references
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
> -----Ursprüngliche Nachricht-----
> Von: Nick Gearls
> Gesendet: Freitag, 13. März 2009 13:48
> An: dev@httpd.apache.org
> Betreff: Re: mod_substitute & back-references
>
> You can reproduce the problems on this file:
> <html><body>
> abc
> </body></html>
>
> 1. Substitute s|abc|abc|n -> OK: "abc"
> 2. Substitute s|abc|+abc| -> NOK: "+++abc"
> 3. Substitute s|abc|++abc|nq -> NOK: "++++++abc"
> 4. Substitute s|abc|+++abc|q -> NOK: "+++++++++abc"
Are you using all these above at the same time?
Regards
Rüdiger
Re: mod_substitute & back-references
Posted by Nick Gearls <ni...@gmail.com>.
You can reproduce the problems on this file:
<html><body>
abc
</body></html>
1. Substitute s|abc|abc|n -> OK: "abc"
2. Substitute s|abc|+abc| -> NOK: "+++abc"
3. Substitute s|abc|++abc|nq -> NOK: "++++++abc"
4. Substitute s|abc|+++abc|q -> NOK: "+++++++++abc"
q & n flags have no influence.
I also managed to have a full recursive loop (Apache eating memory up to
a crash), but I cannot reproduce it anymore !?!
Regards,
Nick
Ruediger Pluem wrote:
>
> On 03/12/2009 06:13 PM, Nick Gearls wrote:
>
>>>> Anyway, a real problem:
>>>> Substitute s|(toreplace)|*replaced[$1]*|qi
>>>> translates "toreplace" into
>>>> "*replaced[*replaced[*replaced[toreplace]*]*]*"
>
> Hm. I cannot reproduce this. Mind to attach the file to which you applied this?
>
> Regards
>
> Rüdiger
>
>
Re: mod_substitute & back-references
Posted by Ruediger Pluem <rp...@apache.org>.
On 03/12/2009 06:13 PM, Nick Gearls wrote:
>>>
>>> Anyway, a real problem:
>>> Substitute s|(toreplace)|*replaced[$1]*|qi
>>> translates "toreplace" into
>>> "*replaced[*replaced[*replaced[toreplace]*]*]*"
Hm. I cannot reproduce this. Mind to attach the file to which you applied this?
Regards
Rüdiger
Re: mod_substitute & back-references
Posted by Nick Gearls <ni...@gmail.com>.
It's not replaceAll vs. replaceFirst, it's replace recursively or not.
For me, it should not be recursive (independently of the q flag, I agree
with you Jim).
Actually, if it is recursive, it should be a never-ending replacement in
this case, which would be a (not voluntary) denial of service. I do not
expect this.
Nick
Jim Jagielski wrote:
>
> On Mar 11, 2009, at 5:03 AM, Nick Gearls wrote:
>
>> Oops, stupid !
>>
>> Anyway, a real problem:
>> Substitute s|(toreplace)|*replaced[$1]*|qi
>> translates "toreplace" into
>> "*replaced[*replaced[*replaced[toreplace]*]*]*"
>>
>> Don't we expect the q flag to stop any replacement after the first one?
>>
>
> Not necessarily, no... it's not a replaceAll vs. replaceFirst
> flag.
>
>
Re: mod_substitute & back-references
Posted by Jim Jagielski <ji...@jaguNET.com>.
On Mar 11, 2009, at 5:03 AM, Nick Gearls wrote:
> Oops, stupid !
>
> Anyway, a real problem:
> Substitute s|(toreplace)|*replaced[$1]*|qi
> translates "toreplace" into
> "*replaced[*replaced[*replaced[toreplace]*]*]*"
>
> Don't we expect the q flag to stop any replacement after the first
> one?
>
Not necessarily, no... it's not a replaceAll vs. replaceFirst
flag.
Re: mod_substitute & back-references
Posted by Nick Gearls <ni...@gmail.com>.
Same thing
Plüm, Rüdiger, VF-Group wrote:
> Have you tried without the q flag?
>
> Regards
>
> Rüdiger
>
>> -----Ursprüngliche Nachricht-----
>> Von: Nick Gearls
>> Gesendet: Mittwoch, 11. März 2009 10:04
>> An: dev@httpd.apache.org
>> Betreff: Re: mod_substitute & back-references
>>
>> Oops, stupid !
>>
>> Anyway, a real problem:
>> Substitute s|(toreplace)|*replaced[$1]*|qi
>> translates "toreplace" into
>> "*replaced[*replaced[*replaced[toreplace]*]*]*"
>>
>> Don't we expect the q flag to stop any replacement after the
>> first one?
>>
>> Thanks,
>>
>> Nick
>>
>>
>> Eric Covener wrote:
>>> On Tue, Mar 10, 2009 at 8:01 AM, Nick Gearls
>> <ni...@gmail.com> wrote:
>>>> When using
>>>> Substitute s|toreplace|*$1*|ni
>>>> I get "*$1" in my page.
>>> You have to capture something.
>>>
>
Re: mod_substitute & back-references
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
Have you tried without the q flag?
Regards
Rüdiger
> -----Ursprüngliche Nachricht-----
> Von: Nick Gearls
> Gesendet: Mittwoch, 11. März 2009 10:04
> An: dev@httpd.apache.org
> Betreff: Re: mod_substitute & back-references
>
> Oops, stupid !
>
> Anyway, a real problem:
> Substitute s|(toreplace)|*replaced[$1]*|qi
> translates "toreplace" into
> "*replaced[*replaced[*replaced[toreplace]*]*]*"
>
> Don't we expect the q flag to stop any replacement after the
> first one?
>
> Thanks,
>
> Nick
>
>
> Eric Covener wrote:
> > On Tue, Mar 10, 2009 at 8:01 AM, Nick Gearls
> <ni...@gmail.com> wrote:
> >> When using
> >> Substitute s|toreplace|*$1*|ni
> >> I get "*$1" in my page.
> >
> > You have to capture something.
> >
>
Re: mod_substitute & back-references
Posted by Nick Gearls <ni...@gmail.com>.
Oops, stupid !
Anyway, a real problem:
Substitute s|(toreplace)|*replaced[$1]*|qi
translates "toreplace" into "*replaced[*replaced[*replaced[toreplace]*]*]*"
Don't we expect the q flag to stop any replacement after the first one?
Thanks,
Nick
Eric Covener wrote:
> On Tue, Mar 10, 2009 at 8:01 AM, Nick Gearls <ni...@gmail.com> wrote:
>> When using
>> Substitute s|toreplace|*$1*|ni
>> I get "*$1" in my page.
>
> You have to capture something.
>
Re: mod_substitute & \n
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
Using newlines in the expression does not make sense as
mod_substitute (similar to sed) does apply the regular expression
line by line. You cannot process multiline regexs with mod_substitute.
Regards
Rüdiger
> -----Ursprüngliche Nachricht-----
> Von: Nick Gearls
> Gesendet: Donnerstag, 19. März 2009 15:43
> An: dev@httpd.apache.org
> Betreff: mod_substitute & \n
>
> I found a problem with handling of new lines in mod_substitute.
> Take the following file as example
> <html>
> <body>
> </body>
> </html>
>
>
> 1. If I use "Substitute s/\n/1/", it works almost correctly:
> <html>
> 1<body>
> 1</body>
> 1</html>
> 1
> Note that it does not replace the new line, but adds the replacement
> after it. This is quite weird.
>
> 2. If I use "Substitute s/<body>\n/<body>2/" or
> "Substitute s/\n<body>/<body>2/", the file is unchanged
>
>
> Could somebody explain how newlines are handled ?
> Can we use them inside a pattern ?
>
> Thanks,
>
>
> Nick
>
>
mod_substitute & \n
Posted by Nick Gearls <ni...@gmail.com>.
I found a problem with handling of new lines in mod_substitute.
Take the following file as example
<html>
<body>
</body>
</html>
1. If I use "Substitute s/\n/1/", it works almost correctly:
<html>
1<body>
1</body>
1</html>
1
Note that it does not replace the new line, but adds the replacement
after it. This is quite weird.
2. If I use "Substitute s/<body>\n/<body>2/" or
"Substitute s/\n<body>/<body>2/", the file is unchanged
Could somebody explain how newlines are handled ?
Can we use them inside a pattern ?
Thanks,
Nick
Re: mod_substitute & back-references
Posted by Eric Covener <co...@gmail.com>.
On Tue, Mar 10, 2009 at 8:01 AM, Nick Gearls <ni...@gmail.com> wrote:
> When using
> Substitute s|toreplace|*$1*|ni
> I get "*$1" in my page.
You have to capture something.
--
Eric Covener
covener@gmail.com
Re: mod_substitute & back-references
Posted by Nick Gearls <ni...@gmail.com>.
When using
Substitute s|toreplace|*$1*|ni
I get "*$1" in my page.
There is no info about backtracking on the doc page, so maybe I don't
use the right syntax ?
Regards,
Nick
Plüm, Rüdiger, VF-Group wrote:
>
>
>> -----Ursprüngliche Nachricht-----
>> Von: Nick Gearls
>> Gesendet: Montag, 9. März 2009 14:35
>> An: Development Apache
>> Betreff: mod_substitute & back-references
>>
>> Hi,
>>
>> Is there any plan to support back-references in mod_substitute ?
>
> mod_substitute already supports backreferences (and always has).
>
> Regards
>
> Rüdiger
>
>
AW: mod_substitute & back-references
Posted by "Plüm, Rüdiger, VF-Group" <ru...@vodafone.com>.
> -----Ursprüngliche Nachricht-----
> Von: Nick Gearls
> Gesendet: Montag, 9. März 2009 14:35
> An: Development Apache
> Betreff: mod_substitute & back-references
>
> Hi,
>
> Is there any plan to support back-references in mod_substitute ?
mod_substitute already supports backreferences (and always has).
Regards
Rüdiger