You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Joe Aldrich <ja...@kimobility.com> on 2016/02/09 15:18:34 UTC

FW: [PossibleSpam] Re: Tomcat Rewrite Valve

Hello,

This is my first attempt at interaction with the Tomcat Users List. I haven't heard anything back on my response to the initial followup by Chris, and I was just checking if there was anything else needed from me on this.

Thanks,
Joe

-----Original Message-----
From: Joe Aldrich 
Sent: Friday, January 29, 2016 5:07 PM
To: Tomcat Users List
Subject: RE: [PossibleSpam] Re: Tomcat Rewrite Valve

Hello,

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Joe,

>On 1/29/16 9:34 AM, Joe Aldrich wrote:
>> I am using Tomcat 8.0.28 on Windows 10 and am having a problem with 
>> the Rewrite Value. I must include the escaped form of an ampersand 
>> '%26' in the output URL.
>> 
>> My rewrite.config has the following:
>> 
>> RewriteCond %{QUERY_STRING} ^(.*&)?SCID=8(&.*)?$ RewriteRule 
>> ^/(product|specs|avail-options|avail-category)\.php$
>> /Product.action?select=Model+4+\%26+4C [R=301,L,NE]
>> 
>> I am escaping the percent sign with a backslash, and I have tried 
>> using the NE flag. However, Tomcat always is treating the percent 
>> symbol as a back reference to the above RewriteCond. If I don't have 
>> a second capture group, then I get a 500 error from a 
>> NullPointerException.

>Can you please post the stack trace from that?

Here is what I get if I don’t specify a second capture group:

HTTP Status 500 - No group 2

type Exception report

message No group 2

description The server encountered an internal error that prevented it from fulfilling this request.

exception

java.lang.IndexOutOfBoundsException: No group 2
	java.util.regex.Matcher.group(Unknown Source)
	org.apache.catalina.valves.rewrite.Substitution$RewriteCondBackReferenceElement.evaluate(Substitution.java:51)
	org.apache.catalina.valves.rewrite.Substitution.evaluate(Substitution.java:238)
	org.apache.catalina.valves.rewrite.RewriteRule.evaluate(RewriteRule.java:133)
	org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:292)
	org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
	org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518)
	org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091)
	org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:673)
	org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
	org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
	java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	java.lang.Thread.run(Unknown Source)
note The full stack trace of the root cause is available in the Apache Tomcat/8.0.28 logs.

Apache Tomcat/8.0.28

>> I was working with the documentation on this page:
>> 
>> http://tomcat.apache.org/tomcat-8.0-doc/rewrite.html
>> 
>> The desired output URL would be:
>> 
>> http://www.domain.com/Product.html?select=Model+4+%26+4C

>Presumably, if you don't escape it at all, you get:
>
>http://www.domain.com/Product.html?select=Model+4+%2526+4C
>
>?

If I do not use the backslash to escape the percent sign, then (with or without the [NE] flag) I get a back-reference resulting in a 500 error if there isn't a second capture group. If there is a second capture group I get:

http://www.domain.com/Product.html?select=Model+4+null26+4C

(where again, null represents there was nothing specified after the SCID=8 in the query string).

If I omit the [NE] flag and keep the backslash to escape the percent sign, the escaping of the percent sign fails and I get similar results except for the presence of the backslash in the output URL as:

http://www.domain.com/Product.html?select=Model+4+\null26+4C


>> In the example given for the NE flag on the page reference above, the 
>> percent sign is escaped by a backslash to prevent it from being 
>> treated as a back-reference. This is not working for me. Instead I
>> get:
>> 
>> http://www.domain.com/Product.action?select=Model+4+\null6+4C
>> 
>> Where the "null" is due to an empty second back-reference.  I believe 
>> this is a bug in that it is not escaping the percent sign (making it 
>> impossible to create the %26 in the redirect URL). Or am I 
>> misunderstanding something here?
>> 
>> As a side question, shouldn't an empty back-reference be blank 
>> instead of adding 'null' to the URL?

>I agree that the "null" is incorrect. That is almost certainly a bug.
>
>[NE] should be preventing escaping of the resulting URL, but that might break if you had user-specified input being re-written, but then not escaped.
>
>I'm not entirely sure if backslash-escaping is expected to work for back-references. It's certainly a reasonable expectation, especially if that's the way that mod_rewrite >works (and I don't know if that's the case). The "escaping" section is only mentioned in the "regular expressions" section, and not in the "backreferences" section, which is >why I think there may be some room for alternative interpretations, here.
>
>I'm curious if \$25 works (as opposed to \%25), and this is merely an oversight for one type of backreference. Can you confirm whether \$25 works as you expect (i.e. >resulting in a URL containing a literal $25)?

If I use \$25 it fails as it tries to reference the second capture group of the RewriteRule. The stack trace is similar to above:

HTTP Status 500 - No group 2

type Exception report

message No group 2

description The server encountered an internal error that prevented it from fulfilling this request.

exception

java.lang.IndexOutOfBoundsException: No group 2
	java.util.regex.Matcher.group(Unknown Source)
	org.apache.catalina.valves.rewrite.Substitution$RewriteRuleBackReferenceElement.evaluate(Substitution.java:43)
	org.apache.catalina.valves.rewrite.Substitution.evaluate(Substitution.java:238)
	org.apache.catalina.valves.rewrite.RewriteRule.evaluate(RewriteRule.java:133)
	org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java:292)
	org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
	org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
	org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:518)
	org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1091)
	org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:673)
	org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1500)
	org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1456)
	java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	java.lang.Thread.run(Unknown Source)
note The full stack trace of the root cause is available in the Apache Tomcat/8.0.28 logs.

Apache Tomcat/8.0.28

The example given in the documentation references above appears to suggest the backslash would escape the percent sign. It doesn't explicitly state that, but provides this example:

RewriteRule /foo/(.*) /bar?arg=P1\%3d$1 [R,NE]

And says the resulting URL would turn '/foo/zed' into a safe request for '/bar?arg=P1=zed'.

This inclines me to believe that the backslash would be used to escape the percent symbol. I have tried without the RewriteCond and still get the java.lang.IndexOutOfBoundsException: No group 2 exception.

Let me know if you need more information. I appreciate any help on this. Thanks, Joe

>- -chris
>-----BEGIN PGP SIGNATURE-----
>Comment: GPGTools - http://gpgtools.org
>Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

>iEYEARECAAYFAlary+YACgkQ9CaO5/Lv0PAjOACgrxb5md+QtRwzENQCOWtonQft
>K70An3MWvKlh2nFgEL/mhjZK+RGIHB2y
>=0g9b
>-----END PGP SIGNATURE-----
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>For additional commands, e-mail: users-help@tomcat.apache.org

Re: FW: [PossibleSpam] Re: Tomcat Rewrite Valve

Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Joe,

On 2/9/16 9:18 AM, Joe Aldrich wrote:
> This is my first attempt at interaction with the Tomcat Users List.
>  I haven't heard anything back on my response to the initial
> followup by Chris, and I was just checking if there was anything
> else needed from me on this.

I was waiting to see if someone else with more familiarity with the
rewrite valve would speak up. I don't have time to dive into that code
right now, unfortunately.

- -chris

> -----Original Message----- From: Joe Aldrich Sent: Friday, January
> 29, 2016 5:07 PM To: Tomcat Users List Subject: RE: [PossibleSpam]
> Re: Tomcat Rewrite Valve
> 
> Hello,
> 
> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
> 
> Joe,
> 
>> On 1/29/16 9:34 AM, Joe Aldrich wrote:
>>> I am using Tomcat 8.0.28 on Windows 10 and am having a problem
>>> with the Rewrite Value. I must include the escaped form of an
>>> ampersand '%26' in the output URL.
>>> 
>>> My rewrite.config has the following:
>>> 
>>> RewriteCond %{QUERY_STRING} ^(.*&)?SCID=8(&.*)?$ RewriteRule 
>>> ^/(product|specs|avail-options|avail-category)\.php$ 
>>> /Product.action?select=Model+4+\%26+4C [R=301,L,NE]
>>> 
>>> I am escaping the percent sign with a backslash, and I have
>>> tried using the NE flag. However, Tomcat always is treating the
>>> percent symbol as a back reference to the above RewriteCond. If
>>> I don't have a second capture group, then I get a 500 error
>>> from a NullPointerException.
> 
>> Can you please post the stack trace from that?
> 
> Here is what I get if I don’t specify a second capture group:
> 
> HTTP Status 500 - No group 2
> 
> type Exception report
> 
> message No group 2
> 
> description The server encountered an internal error that prevented
> it from fulfilling this request.
> 
> exception
> 
> java.lang.IndexOutOfBoundsException: No group 2 
> java.util.regex.Matcher.group(Unknown Source) 
> org.apache.catalina.valves.rewrite.Substitution$RewriteCondBackReferen
ceElement.evaluate(Substitution.java:51)
>
> 
org.apache.catalina.valves.rewrite.Substitution.evaluate(Substitution.ja
va:238)
> org.apache.catalina.valves.rewrite.RewriteRule.evaluate(RewriteRule.ja
va:133)
>
> 
org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java
:292)
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.ja
va:79)
>
> 
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessL
ogValve.java:616)
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java
:518)
>
> 
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11P
rocessor.java:1091)
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(A
bstractProtocol.java:673)
>
> 
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint
.java:1500)
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint
.java:1456)
>
> 
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThr
ead.java:61)
>
> 
java.lang.Thread.run(Unknown Source)
> note The full stack trace of the root cause is available in the
> Apache Tomcat/8.0.28 logs.
> 
> Apache Tomcat/8.0.28
> 
>>> I was working with the documentation on this page:
>>> 
>>> http://tomcat.apache.org/tomcat-8.0-doc/rewrite.html
>>> 
>>> The desired output URL would be:
>>> 
>>> http://www.domain.com/Product.html?select=Model+4+%26+4C
> 
>> Presumably, if you don't escape it at all, you get:
>> 
>> http://www.domain.com/Product.html?select=Model+4+%2526+4C
>> 
>> ?
> 
> If I do not use the backslash to escape the percent sign, then
> (with or without the [NE] flag) I get a back-reference resulting in
> a 500 error if there isn't a second capture group. If there is a
> second capture group I get:
> 
> http://www.domain.com/Product.html?select=Model+4+null26+4C
> 
> (where again, null represents there was nothing specified after the
> SCID=8 in the query string).
> 
> If I omit the [NE] flag and keep the backslash to escape the
> percent sign, the escaping of the percent sign fails and I get
> similar results except for the presence of the backslash in the
> output URL as:
> 
> http://www.domain.com/Product.html?select=Model+4+\null26+4C
> 
> 
>>> In the example given for the NE flag on the page reference
>>> above, the percent sign is escaped by a backslash to prevent it
>>> from being treated as a back-reference. This is not working for
>>> me. Instead I get:
>>> 
>>> http://www.domain.com/Product.action?select=Model+4+\null6+4C
>>> 
>>> Where the "null" is due to an empty second back-reference.  I
>>> believe this is a bug in that it is not escaping the percent
>>> sign (making it impossible to create the %26 in the redirect
>>> URL). Or am I misunderstanding something here?
>>> 
>>> As a side question, shouldn't an empty back-reference be blank
>>>  instead of adding 'null' to the URL?
> 
>> I agree that the "null" is incorrect. That is almost certainly a
>> bug.
>> 
>> [NE] should be preventing escaping of the resulting URL, but that
>> might break if you had user-specified input being re-written, but
>> then not escaped.
>> 
>> I'm not entirely sure if backslash-escaping is expected to work
>> for back-references. It's certainly a reasonable expectation,
>> especially if that's the way that mod_rewrite >works (and I don't
>> know if that's the case). The "escaping" section is only
>> mentioned in the "regular expressions" section, and not in the
>> "backreferences" section, which is >why I think there may be some
>> room for alternative interpretations, here.
>> 
>> I'm curious if \$25 works (as opposed to \%25), and this is
>> merely an oversight for one type of backreference. Can you
>> confirm whether \$25 works as you expect (i.e. >resulting in a
>> URL containing a literal $25)?
> 
> If I use \$25 it fails as it tries to reference the second capture
> group of the RewriteRule. The stack trace is similar to above:
> 
> HTTP Status 500 - No group 2
> 
> type Exception report
> 
> message No group 2
> 
> description The server encountered an internal error that prevented
> it from fulfilling this request.
> 
> exception
> 
> java.lang.IndexOutOfBoundsException: No group 2 
> java.util.regex.Matcher.group(Unknown Source) 
> org.apache.catalina.valves.rewrite.Substitution$RewriteRuleBackReferen
ceElement.evaluate(Substitution.java:43)
>
> 
org.apache.catalina.valves.rewrite.Substitution.evaluate(Substitution.ja
va:238)
> org.apache.catalina.valves.rewrite.RewriteRule.evaluate(RewriteRule.ja
va:133)
>
> 
org.apache.catalina.valves.rewrite.RewriteValve.invoke(RewriteValve.java
:292)
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.ja
va:79)
>
> 
org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessL
ogValve.java:616)
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java
:518)
>
> 
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11P
rocessor.java:1091)
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(A
bstractProtocol.java:673)
>
> 
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint
.java:1500)
> org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint
.java:1456)
>
> 
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) 
> org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThr
ead.java:61)
>
> 
java.lang.Thread.run(Unknown Source)
> note The full stack trace of the root cause is available in the
> Apache Tomcat/8.0.28 logs.
> 
> Apache Tomcat/8.0.28
> 
> The example given in the documentation references above appears to
> suggest the backslash would escape the percent sign. It doesn't
> explicitly state that, but provides this example:
> 
> RewriteRule /foo/(.*) /bar?arg=P1\%3d$1 [R,NE]
> 
> And says the resulting URL would turn '/foo/zed' into a safe
> request for '/bar?arg=P1=zed'.
> 
> This inclines me to believe that the backslash would be used to
> escape the percent symbol. I have tried without the RewriteCond and
> still get the java.lang.IndexOutOfBoundsException: No group 2
> exception.
> 
> Let me know if you need more information. I appreciate any help on
> this. Thanks, Joe
> 
>> - -chris -----BEGIN PGP SIGNATURE----- Comment: GPGTools -
>> http://gpgtools.org Comment: Using GnuPG with Thunderbird -
>> http://www.enigmail.net/
> 
>> iEYEARECAAYFAlary+YACgkQ9CaO5/Lv0PAjOACgrxb5md+QtRwzENQCOWtonQft 
>> K70An3MWvKlh2nFgEL/mhjZK+RGIHB2y =0g9b -----END PGP
>> SIGNATURE-----
>> 
>> ---------------------------------------------------------------------
>>
>> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
>> For additional commands, e-mail: users-help@tomcat.apache.org
> 
> 
> ---------------------------------------------------------------------
>
> 
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
> 
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAla5+3sACgkQ9CaO5/Lv0PAOBACfXdPLHBiNw9LoJuBtAM6oCec7
25wAn1PbYz2yFjjy3CagmLYIZijNMqjx
=Je86
-----END PGP SIGNATURE-----

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org