You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@httpd.apache.org by Mark Stosberg <ma...@summersault.com> on 2013/02/15 16:38:40 UTC

bug? Inconsistent handling of query string

I'd like feedback on whether the following behavior is a bug, or
intentionally inconsistent.

I was looking at the environment variables generated by this case:

   Browser URL: /file%3Fa=b?c=d

   'QUERY_STRING' => 'a=b&c=d',
   'SCRIPT_URL' => '/file?a=b',
   'SCRIPT_URI' => 'http://example.com/file?a=b',
   'REQUEST_URI' => '/file%3Fa=b?c=d',

The "%3F" is an encoded question mark.

Note that SCRIPT_URI and SCRIPT_URL treat the query string as starting
after the unencoded question mark, while 'QUERY_STRING' variables treats
the query string as started at the encoded question mark.

>From my reading of RFC 3875 (CGI), my understanding is that only an
unencoded question mark should mark the beginning the query string.
Thus, it appears that the QUERY_STRING variable being returned here is
incorrect. Is it?

This data was generated with Apache/2.2.14.

Thanks!

   Mark  Stosberg



Re: bug? Inconsistent handling of query string

Posted by Mark Stosberg <ma...@summersault.com>.
On 02/15/2013 01:49 PM, Eric Covener wrote:
> On Fri, Feb 15, 2013 at 12:57 PM, Mark Stosberg <ma...@summersault.com> wrote:
>>
>> Thanks for the response, William --
>>
>> On 02/15/2013 11:49 AM, William A. Rowe Jr. wrote:
>>> On Fri, 15 Feb 2013 10:38:40 -0500
>>> Mark Stosberg <ma...@summersault.com> wrote:
>>>
>>>>
>>>> I'd like feedback on whether the following behavior is a bug, or
>>>> intentionally inconsistent.
>>>>
>>>> I was looking at the environment variables generated by this case:
>>>>
>>>>    Browser URL: /file%3Fa=b?c=d
> 
>>   # Otherwise, pass everything through to the dispatcher
>>   RewriteRule ^home/project/www/(.*)$ /cgi-bin/dispatch.cgi/$1
>> [last,qsappend]
> 
> Here you'll match the decoded version and copy it into the path as a
> literal ? -- maybe you need [B] here to be safe, or capture
> %{THE_REQUEST} in a condition which has the still-encoded request.

Eric,

You were correct. Here's the result after adding [B] to my RewriteRule:

  'QUERY_STRING' => 'c=d',
  'SCRIPT_URL' => '/file?a=b',
  'SCRIPT_URI' => 'http://www.mark.net.adoptapet.com/file?a=b',
  'REQUEST_URI' => '/file%3Fa=b?c=d',

In summary, all variables look consistent and correct. The bug was on my
end!

Thanks for the feedback,

Unfortunately, this a subtle detail of how rewriting works that is
commonly overlooked.

A quick search on Github shows shows nearly 5,000 projects that use this
pattern, and none of them in the random sampling I looked at had
included [B] in their flags:

https://github.com/search?q=+RewriteCond+%25%7BREQUEST_FILENAME%7D+%21-f++%241&type=Code&ref=searchresults



   Mark




Re: bug? Inconsistent handling of query string

Posted by Eric Covener <co...@gmail.com>.
On Fri, Feb 15, 2013 at 12:57 PM, Mark Stosberg <ma...@summersault.com> wrote:
>
> Thanks for the response, William --
>
> On 02/15/2013 11:49 AM, William A. Rowe Jr. wrote:
>> On Fri, 15 Feb 2013 10:38:40 -0500
>> Mark Stosberg <ma...@summersault.com> wrote:
>>
>>>
>>> I'd like feedback on whether the following behavior is a bug, or
>>> intentionally inconsistent.
>>>
>>> I was looking at the environment variables generated by this case:
>>>
>>>    Browser URL: /file%3Fa=b?c=d

>   # Otherwise, pass everything through to the dispatcher
>   RewriteRule ^home/project/www/(.*)$ /cgi-bin/dispatch.cgi/$1
> [last,qsappend]

Here you'll match the decoded version and copy it into the path as a
literal ? -- maybe you need [B] here to be safe, or capture
%{THE_REQUEST} in a condition which has the still-encoded request.

Re: bug? Inconsistent handling of query string

Posted by Mark Stosberg <ma...@summersault.com>.
Thanks for the response, William --

On 02/15/2013 11:49 AM, William A. Rowe Jr. wrote:
> On Fri, 15 Feb 2013 10:38:40 -0500
> Mark Stosberg <ma...@summersault.com> wrote:
> 
>>
>> I'd like feedback on whether the following behavior is a bug, or
>> intentionally inconsistent.
>>
>> I was looking at the environment variables generated by this case:
>>
>>    Browser URL: /file%3Fa=b?c=d
>>
>>    'QUERY_STRING' => 'a=b&c=d',
>>    'SCRIPT_URL' => '/file?a=b',
>>    'SCRIPT_URI' => 'http://example.com/file?a=b',
>>    'REQUEST_URI' => '/file%3Fa=b?c=d',
>>
>> The "%3F" is an encoded question mark.
>>
>> Note that SCRIPT_URI and SCRIPT_URL treat the query string as starting
>> after the unencoded question mark, while 'QUERY_STRING' variables
>> treats the query string as started at the encoded question mark.
>>
>> >From my reading of RFC 3875 (CGI), my understanding is that only an
>> unencoded question mark should mark the beginning the query string.
>> Thus, it appears that the QUERY_STRING variable being returned here is
>> incorrect. Is it?
> 
> I agree with your interpretation.  However, we pick apart QUERY_STRING
> long after %-escapes are transcoded.  That could be the issue.
> 
>> This data was generated with Apache/2.2.14.
> 
> Was mod_rewrite involved in any way with processing this request?

Yes, a standard dispatching recipe was in play:

  # If an actual file or directory is requested, serve directly
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_FILENAME} !-d

  # Otherwise, pass everything through to the dispatcher
  RewriteRule ^home/project/www/(.*)$ /cgi-bin/dispatch.cgi/$1
[last,qsappend]

Should I open a bug report? (If someone could check quickly against
HEAD, that could be helpful as well).

    Mark


Re: bug? Inconsistent handling of query string

Posted by "William A. Rowe Jr." <wr...@rowe-clan.net>.
On Fri, 15 Feb 2013 10:38:40 -0500
Mark Stosberg <ma...@summersault.com> wrote:

> 
> I'd like feedback on whether the following behavior is a bug, or
> intentionally inconsistent.
> 
> I was looking at the environment variables generated by this case:
> 
>    Browser URL: /file%3Fa=b?c=d
> 
>    'QUERY_STRING' => 'a=b&c=d',
>    'SCRIPT_URL' => '/file?a=b',
>    'SCRIPT_URI' => 'http://example.com/file?a=b',
>    'REQUEST_URI' => '/file%3Fa=b?c=d',
> 
> The "%3F" is an encoded question mark.
> 
> Note that SCRIPT_URI and SCRIPT_URL treat the query string as starting
> after the unencoded question mark, while 'QUERY_STRING' variables
> treats the query string as started at the encoded question mark.
> 
> >From my reading of RFC 3875 (CGI), my understanding is that only an
> unencoded question mark should mark the beginning the query string.
> Thus, it appears that the QUERY_STRING variable being returned here is
> incorrect. Is it?

I agree with your interpretation.  However, we pick apart QUERY_STRING
long after %-escapes are transcoded.  That could be the issue.

> This data was generated with Apache/2.2.14.

Was mod_rewrite involved in any way with processing this request?

> Thanks!
> 
>    Mark  Stosberg
> 
>