You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@tomcat.apache.org by Stephen Crawford <sr...@psu.edu> on 2017/04/27 16:23:33 UTC
URL decoding
Hello All,
We are running Tomcat 8.5.13 on Linux, mostly as a container for
Geoserver. We have a few apps (in Flash!) that have been running fine
untouched for at least six years but stopped working a few weeks ago. I
believe the issue appeared before we upgraded from Tomcat 6.0.24,
probably after a security patch. For that and other reasons we
upgraded, but the problem persists.
I believe the problem is that a "loose" URL encoding that was previously
being allowed to go through is now being stopped and returning a code
400 Bad Request. I narrowed down the culprit to this portion of the xml
filter at the end of the url string:
<PropertyIsLike wildCard="*" escape="\" singleChar="?">
which the browser encodes as:
%3CPropertyIsLike%20wildCard=%22*%22%20escape=%22\%22%20singleChar=%22?%22%3E
Note that the "*", "\" and "?" remain not encoded. If I replace
(encode) these the request is sent on through.
My question: can I configure Tomcat to return to the the previous
behavior of allowing this request? I cannot change the Flash apps
Thanks,
Steve
--
Stephen Crawford
Center for Environmental Informatics
The Pennsylvania State University
src176@psu.edu
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: URL decoding
Posted by Stephen Crawford <sr...@psu.edu>.
On 4/27/2017 2:27 PM, Mark Thomas wrote:
> On 27/04/17 17:23, Stephen Crawford wrote:
>> Hello All,
>>
>> We are running Tomcat 8.5.13 on Linux, mostly as a container for
>> Geoserver. We have a few apps (in Flash!) that have been running fine
>> untouched for at least six years but stopped working a few weeks ago. I
>> believe the issue appeared before we upgraded from Tomcat 6.0.24,
>> probably after a security patch. For that and other reasons we
>> upgraded, but the problem persists.
>>
>> I believe the problem is that a "loose" URL encoding that was previously
>> being allowed to go through is now being stopped and returning a code
>> 400 Bad Request. I narrowed down the culprit to this portion of the xml
>> filter at the end of the url string:
>>
>> <PropertyIsLike wildCard="*" escape="\" singleChar="?">
>>
>> which the browser encodes as:
>> %3CPropertyIsLike%20wildCard=%22*%22%20escape=%22\%22%20singleChar=%22?%22%3E
>>
>>
>> Note that the "*", "\" and "?" remain not encoded. If I replace
>> (encode) these the request is sent on through.
>>
>> My question: can I configure Tomcat to return to the the previous
>> behavior of allowing this request? I cannot change the Flash apps
>
> I assume that that string is part of the query string. If it was part of
> the path I'd be amazed if it ever worked. (I'd expect the '?' to be
> treated as the start of the query string.)
>
> The root cause of the tightening of the restrictions was CVE-2016-6816.
>
> Looking at the code, the '*' and the '?' should be OK. It is the '\'
> that is problematic.
>
> (Note: If the specification for the query string was enforced, '?' would
> be problematic as well.)
>
> 8.5.x and earlier allow the restrictions for some invalid characters to
> be lifted. '\' is not among them.
>
> Generally '\' is problematic because it will be treated as a path
> separator on some platforms but not on others. That has led to security
> vulnerabilities in the past when attackers put \..\ in the URL to bypass
> security restrictions. I'm a little surprised Tomcat allowed it before
> CVE-2016-6816 was fixed but digging into the code we have a check for
> that in the path (which confirms my suspicion the string in question is
> in the query string).
>
> To get to your question, it is not currently possible to configure
> Tomcat to allow this.
>
> Possible options are:
>
> 1. Find a way to fix the flash app.
>
> 2. Convince the Tomcat developers to validate the path and query string
> segments separately and be (optionally) more lenient with the query string.
>
> 3. Use a reverse proxy in front of Tomcat and encode the query string
> properly before passing it to Tomcat.
>
>
> 1. is the 'right' solution. After all the client is not specification
> compliant.
>
> 2. is unlikely. More sophisticated validation will have a performance
> impact and the trend has been for Tomcat to be more stringent with
> respect to spec compliance, not less.
>
> 3. is likely the simplest. Until the proxies implement more stringent
> rules for much the same reason as Tomcat did (CVE-2016-6816).
>
> I suspect this isn.t what you wanted to hear. Sorry.
>
> Mark
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
> For additional commands, e-mail: users-help@tomcat.apache.org
>
Thanks for such a thorough answer; not what I wanted to hear but mostly
expected. Seven years was a pretty good run. Now to see if the client
has the $$ for the HTML5 version....
--
Stephen Crawford
Center for Environmental Informatics
The Pennsylvania State University
src176@psu.edu
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: URL decoding
Posted by Mark Thomas <ma...@apache.org>.
On 27/04/17 17:23, Stephen Crawford wrote:
> Hello All,
>
> We are running Tomcat 8.5.13 on Linux, mostly as a container for
> Geoserver. We have a few apps (in Flash!) that have been running fine
> untouched for at least six years but stopped working a few weeks ago. I
> believe the issue appeared before we upgraded from Tomcat 6.0.24,
> probably after a security patch. For that and other reasons we
> upgraded, but the problem persists.
>
> I believe the problem is that a "loose" URL encoding that was previously
> being allowed to go through is now being stopped and returning a code
> 400 Bad Request. I narrowed down the culprit to this portion of the xml
> filter at the end of the url string:
>
> <PropertyIsLike wildCard="*" escape="\" singleChar="?">
>
> which the browser encodes as:
> %3CPropertyIsLike%20wildCard=%22*%22%20escape=%22\%22%20singleChar=%22?%22%3E
>
>
> Note that the "*", "\" and "?" remain not encoded. If I replace
> (encode) these the request is sent on through.
>
> My question: can I configure Tomcat to return to the the previous
> behavior of allowing this request? I cannot change the Flash apps
I assume that that string is part of the query string. If it was part of
the path I'd be amazed if it ever worked. (I'd expect the '?' to be
treated as the start of the query string.)
The root cause of the tightening of the restrictions was CVE-2016-6816.
Looking at the code, the '*' and the '?' should be OK. It is the '\'
that is problematic.
(Note: If the specification for the query string was enforced, '?' would
be problematic as well.)
8.5.x and earlier allow the restrictions for some invalid characters to
be lifted. '\' is not among them.
Generally '\' is problematic because it will be treated as a path
separator on some platforms but not on others. That has led to security
vulnerabilities in the past when attackers put \..\ in the URL to bypass
security restrictions. I'm a little surprised Tomcat allowed it before
CVE-2016-6816 was fixed but digging into the code we have a check for
that in the path (which confirms my suspicion the string in question is
in the query string).
To get to your question, it is not currently possible to configure
Tomcat to allow this.
Possible options are:
1. Find a way to fix the flash app.
2. Convince the Tomcat developers to validate the path and query string
segments separately and be (optionally) more lenient with the query string.
3. Use a reverse proxy in front of Tomcat and encode the query string
properly before passing it to Tomcat.
1. is the 'right' solution. After all the client is not specification
compliant.
2. is unlikely. More sophisticated validation will have a performance
impact and the trend has been for Tomcat to be more stringent with
respect to spec compliance, not less.
3. is likely the simplest. Until the proxies implement more stringent
rules for much the same reason as Tomcat did (CVE-2016-6816).
I suspect this isn.t what you wanted to hear. Sorry.
Mark
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: URL decoding
Posted by Stephen Crawford <sr...@psu.edu>.
On 4/27/2017 4:52 PM, Christopher Schultz wrote:
> When I take your browser-encoded query string from above and send it
> to Tomcat, I don't get any errors or anything.
>
> Are you sure this isn't a problem with the application?
Hi Chris,
That happens to me too, in the sense that usually Tomcat throws an error
or Geoserver throws an error, but in this case nothing happens except
for the 400 BAD REQUEST. Nothing in logs of Tomcat or Geoserver. I
have not yet tried changing the logging level.
Thanks,
Steve
--
Stephen Crawford
Center for Environmental Informatics
The Pennsylvania State University
src176@psu.edu
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org
Re: URL decoding
Posted by Christopher Schultz <ch...@christopherschultz.net>.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Steve,
(Bringing this back into the list)
On 4/27/17 1:49 PM, Stephen Crawford wrote:
> Hi Chris,
>
> On 4/27/2017 12:47 PM, Christopher Schultz wrote:
>> Steve,
>>
>> On 4/27/17 12:23 PM, Stephen Crawford wrote:
>>> Hello All,
>>>
>>> We are running Tomcat 8.5.13 on Linux, mostly as a container
>>> for Geoserver. We have a few apps (in Flash!) that have been
>>> running fine untouched for at least six years but stopped
>>> working a few weeks ago. I believe the issue appeared before we
>>> upgraded from Tomcat 6.0.24, probably after a security patch.
>>> For that and other reasons we upgraded, but the problem
>>> persists.
>>>
>>> I believe the problem is that a "loose" URL encoding that was
>>> previously being allowed to go through is now being stopped and
>>> returning a code 400 Bad Request. I narrowed down the culprit
>>> to this portion of the xml filter at the end of the url
>>> string:
>>>
>>> <PropertyIsLike wildCard="*" escape="\" singleChar="?">
>>>
>>> which the browser encodes as:
>>> %3CPropertyIsLike%20wildCard=%22*%22%20escape=%22\%22%20singleChar=%
22?%22%3E
>>>
>>>
>>>
>>>
>>>
Note that the "*", "\" and "?" remain not encoded. If I replace
>>> (encode) these the request is sent on through.
>>>
>>> My question: can I configure Tomcat to return to the the
>>> previous behavior of allowing this request? I cannot change
>>> the Flash apps
>>
>> Can you give an example of the whole URL you are trying to
>> process?
>>
>> This URL seems to work just fine for me:
>>
>> https://host/application/resource?%3CPropertyIsLike%20wildCard=%22*%2
2%20escape=%22\%22%20singleChar=%22?%22%3E
>>
>>
>>
>>
Where I'm running my application on Tomcat 8.5.14.
>>
>> -chris
>>
>
> It is long one, in plain english first:
>
> http://geo.cei.psu.edu:8080/geoserver/wfs?version=1.0.0&request=GetFea
ture&typeName=cei:taxon_acres08_48&PropertyName=the_geom,areasymbol,comp
name,total,gid,areaname,class,sdmlegac,semac,cordate&Filter=<Filter><Or>
<PropertyIsLike
>
>
wildCard="*" escape="\"
> singleChar="?"><PropertyName>compname</PropertyName><Literal>lucy</Lit
eral></PropertyIsLike><PropertyIsLike
>
>
wildCard="*" escape="\"
> singleChar="?"><PropertyName>compname</PropertyName><Literal>LUCY</Lit
eral></PropertyIsLike></Or></Filter>
>
>
>
> And now as the browser encodes it and sends to Tomcat:
>
> http://geo.cei.psu.edu:8080/geoserver/wfs?version=1.0.0&request=GetFea
ture&typeName=cei:taxon_acres08_48&PropertyName=the_geom,areasymbol,comp
name,total,gid,areaname,class,sdmlegac,semac,cordate&Filter=%3CFilter%3E
%3COr%3E%3CPropertyIsLike%20wildCard=%22*%22%20escape=%22\%22%20singleCh
ar=%22?%22%3E%3CPropertyName%3Ecompname%3C/PropertyName%3E%3CLiteral%3El
ucy%3C/Literal%3E%3C/PropertyIsLike%3E%3CPropertyIsLike%20wildCard=%22*%
22%20escape=%22\%22%20singleChar=%22?%22%3E%3CPropertyName%3Ecompname%3C
/PropertyName%3E%3CLiteral%3ELUCY%3C/Literal%3E%3C/PropertyIsLike%3E%3C/
Or%3E%3C/Filter%3E
>
>
>
> And if I take the 2nd example and replace the \ = %5C and * = %2A,
> it works:
>
> http://geo.cei.psu.edu:8080/geoserver/wfs?version=1.0.0&request=GetFea
ture&typeName=cei:taxon_acres08_48&PropertyName=the_geom,areasymbol,comp
name,total,gid,areaname,class,sdmlegac,semac,cordate&Filter=%3CFilter%3E
%3COr%3E%3CPropertyIsLike%20wildCard=%22%2A%22%20escape=%22%5C%22%20sing
leChar=%22?%22%3E%3CPropertyName%3Ecompname%3C/PropertyName%3E%3CLiteral
%3Elucy%3C/Literal%3E%3C/PropertyIsLike%3E%3CPropertyIsLike%20wildCard=%
22%2A%22%20escape=%22%5C%22%20singleChar=%22?%22%3E%3CPropertyName%3Ecom
pname%3C/PropertyName%3E%3CLiteral%3ELUCY%3C/Literal%3E%3C/PropertyIsLik
e%3E%3C/Or%3E%3C/Filter%3E
>
>
>
> Since the snippet in question is part of the xml filter, I imagine
> it could be the decoding of xml specifically.
When I take your browser-encoded query string from above and send it
to Tomcat, I don't get any errors or anything.
Are you sure this isn't a problem with the application?
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
iQIcBAEBCAAGBQJZAlofAAoJEBzwKT+lPKRYnNMP/1F1VkM8522Sc2dDmn/1sY26
Q9c2t4ldwars9h9a0gxP7Sx2WGOuYf4yZkAdopU2VvpeXWImMh394k2QPEMIeg/0
rGWDpNYI0DZobxFXyHLe69nMKXGC7xSGt0IkEO5y0DlU50m8Ee/BcglwPsapYGng
2F8LWEJEGwZlmKrxN1kVLuy+q6Vo+s3YH8bt3i9T2bStascH6H8lDxJed6U/Td5s
3po7KJu18Lj0NySgBhj0mbb2EqQkZDlGK1ZgOsYyrP8xe5Pa0sg/xizPXx7Ykinq
Nk2F8W0xnYIAXKHr3ftPZKP+U+DkXXRNrhN+AkkgJ7Y2gEii/ILyHLgcISn3iCNF
T4cc2UpekmcTcAmGdISRbscNxneFeJyPs31FmosxLA/i2VVL6dxI/mcWNjPBEjD5
1q4X+uVWci9rGPI0vUqWJL1h7kH5DZzhsgzqi3w04Iuk7byUbDoeswhBWJUkTtO6
fq6vMWSoWoiG/KHO1ZTyEt6RFmEj97qj/K83KLZSN2DX6KaGeAZV+UfhzJSuAHrL
PwwF0N17pBXw+Iyq3q4RdVnm/nyJkXbrHrpuYw6XMKP2xKv3Ph3bE7SgURKmttMx
wdPKoOGloncgd1sNRvnIVdGSKtjHIX73bS4HlayZY36GOXDm9NGsREYfnFEaJC5X
477TjSBIMuX1fc77LCse
=zmbn
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@tomcat.apache.org
For additional commands, e-mail: users-help@tomcat.apache.org