You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Eli Collins (JIRA)" <ji...@apache.org> on 2011/08/08 07:39:27 UTC
[jira] [Created] (HADOOP-7527) Make URL encoding consistent
Make URL encoding consistent
----------------------------
Key: HADOOP-7527
URL: https://issues.apache.org/jira/browse/HADOOP-7527
Project: Hadoop Common
Issue Type: Improvement
Affects Versions: 0.23.0
Reporter: Eli Collins
URL encoding is currently handled in at least 4 different ways. We should make these consistent:
# Parameters are encoded when a URI object is created
# HttpServlet uses RequestQuoter to html escape parameter names and values
# StringEscapeUtils is used to escape parameters in ReconfigurationServlet and DatanodeJspHelper
# URLEncoder and URLDecoder are used in multiple places
We should also be consistent about how we pass file names in URLs, some times they're passed in the path segment, sometimes they're passed in the query fragment as parameters.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7527) Make URL encoding consistent
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081055#comment-13081055 ]
Todd Lipcon commented on HADOOP-7527:
-------------------------------------
The QuotingInputFilter is also totally misguided - yes, it prevents you from accidentally forgetting to encode something, but it's done the wrong layer. Encoding is output-type dependent -- just because we usually output to HTML doesn't mean that the parameters should be HTML-encoded "on the way in".
The correct way to do this is to make sure all interpolation of user input escapes at interpolation time -- the encoding should then be the proper context-dependent escaping scheme (eg URL escaping, javascript escaping, or HTML escaping, which are all different and sometimes need to be layered)
> Make URL encoding consistent
> ----------------------------
>
> Key: HADOOP-7527
> URL: https://issues.apache.org/jira/browse/HADOOP-7527
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 0.23.0
> Reporter: Eli Collins
>
> URL encoding is currently handled in at least 4 different ways. We should make these consistent:
> # Parameters are encoded when a URI object is created
> # HttpServlet uses RequestQuoter to html escape parameter names and values
> # StringEscapeUtils is used to escape parameters in ReconfigurationServlet and DatanodeJspHelper
> # URLEncoder and URLDecoder are used in multiple places
> We should also be consistent about how we pass file names in URLs, some times they're passed in the path segment, sometimes they're passed in the query fragment as parameters.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7527) Make URL encoding consistent
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081691#comment-13081691 ]
Owen O'Malley commented on HADOOP-7527:
---------------------------------------
{quote}
Anyone who has done substantial web development would disagree with you that this is the correct way.
{quote}
*Laugh* It was the Yahoo paranoids, who have done *thousands* of properties who strongly suggested it as by far the most reliable way of avoiding problems. XSS problems are endemic and very hard to catch without tools. Mechanisms that cause the dev's code to fail in a safe way are far preferable to ones that fail with a XSS that lays unfixed for years.
> Make URL encoding consistent
> ----------------------------
>
> Key: HADOOP-7527
> URL: https://issues.apache.org/jira/browse/HADOOP-7527
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 0.23.0
> Reporter: Eli Collins
>
> URL encoding is currently handled in at least 5 different ways. We should make these consistent:
> # Parameters are encoded when a URI object is created
> # HttpServlet uses RequestQuoter to html escape parameter names and values
> # StringEscapeUtils is used to escape parameters in ReconfigurationServlet and DatanodeJspHelper
> # URLEncoder and URLDecoder are used in multiple places
> # encodePath from Jetty's URIUtil
> We should also be consistent about how we pass file names in URLs, some times they're passed in the path segment, sometimes they're passed in the query fragment as parameters.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7527) Make URL encoding consistent
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081109#comment-13081109 ]
Owen O'Malley commented on HADOOP-7527:
---------------------------------------
You're missing the point.
The reality is the developers miss cases of untrusted input. Let's take the 404 page that is built into jetty. It echos the parameters blindly. The jetty developers, who should know better, missed it. Hadoop developers have missed it many many more times. The only reliable way to fix the problem is on input. As long as it is done consistently, it protects against the majority of attacks. It isn't fool-proof, but it is far safer than assuming all uses in output will be caught.
> Make URL encoding consistent
> ----------------------------
>
> Key: HADOOP-7527
> URL: https://issues.apache.org/jira/browse/HADOOP-7527
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 0.23.0
> Reporter: Eli Collins
>
> URL encoding is currently handled in at least 4 different ways. We should make these consistent:
> # Parameters are encoded when a URI object is created
> # HttpServlet uses RequestQuoter to html escape parameter names and values
> # StringEscapeUtils is used to escape parameters in ReconfigurationServlet and DatanodeJspHelper
> # URLEncoder and URLDecoder are used in multiple places
> We should also be consistent about how we pass file names in URLs, some times they're passed in the path segment, sometimes they're passed in the query fragment as parameters.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7527) Make URL encoding consistent
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081174#comment-13081174 ]
Todd Lipcon commented on HADOOP-7527:
-------------------------------------
The 404 page built into jetty was a bug - it's fixed.
Anyone who has done substantial web development would disagree with you that this is the correct way. It most be the most _expedient_ way when dealing with systems people writing JSPs and not understanding escaping, but it's certainly not correct, and it causes other bugs like we see with HFTP.
> Make URL encoding consistent
> ----------------------------
>
> Key: HADOOP-7527
> URL: https://issues.apache.org/jira/browse/HADOOP-7527
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 0.23.0
> Reporter: Eli Collins
>
> URL encoding is currently handled in at least 4 different ways. We should make these consistent:
> # Parameters are encoded when a URI object is created
> # HttpServlet uses RequestQuoter to html escape parameter names and values
> # StringEscapeUtils is used to escape parameters in ReconfigurationServlet and DatanodeJspHelper
> # URLEncoder and URLDecoder are used in multiple places
> We should also be consistent about how we pass file names in URLs, some times they're passed in the path segment, sometimes they're passed in the query fragment as parameters.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7527) Make URL encoding consistent
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13082680#comment-13082680 ]
Luke Lu commented on HADOOP-7527:
---------------------------------
bq. Mechanisms that cause the dev's code to fail in a safe way are far preferable to ones that fail with a XSS that lays unfixed for years.
I agree that this is a reasonable stop gap solution until we moved to more secure web UI framework (cf. HADOOP-7532) ;)
> Make URL encoding consistent
> ----------------------------
>
> Key: HADOOP-7527
> URL: https://issues.apache.org/jira/browse/HADOOP-7527
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 0.23.0
> Reporter: Eli Collins
>
> URL encoding is currently handled in at least 5 different ways. We should make these consistent:
> # Parameters are encoded when a URI object is created
> # HttpServlet uses RequestQuoter to html escape parameter names and values
> # StringEscapeUtils is used to escape parameters in ReconfigurationServlet and DatanodeJspHelper
> # URLEncoder and URLDecoder are used in multiple places
> # encodePath from Jetty's URIUtil
> We should also be consistent about how we pass file names in URLs, some times they're passed in the path segment, sometimes they're passed in the query fragment as parameters.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7527) Make URL encoding consistent
Posted by "Eli Collins (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Eli Collins updated HADOOP-7527:
--------------------------------
Description:
URL encoding is currently handled in at least 5 different ways. We should make these consistent:
# Parameters are encoded when a URI object is created
# HttpServlet uses RequestQuoter to html escape parameter names and values
# StringEscapeUtils is used to escape parameters in ReconfigurationServlet and DatanodeJspHelper
# URLEncoder and URLDecoder are used in multiple places
# encodePath from Jetty's URIUtil
We should also be consistent about how we pass file names in URLs, some times they're passed in the path segment, sometimes they're passed in the query fragment as parameters.
was:
URL encoding is currently handled in at least 4 different ways. We should make these consistent:
# Parameters are encoded when a URI object is created
# HttpServlet uses RequestQuoter to html escape parameter names and values
# StringEscapeUtils is used to escape parameters in ReconfigurationServlet and DatanodeJspHelper
# URLEncoder and URLDecoder are used in multiple places
We should also be consistent about how we pass file names in URLs, some times they're passed in the path segment, sometimes they're passed in the query fragment as parameters.
> Make URL encoding consistent
> ----------------------------
>
> Key: HADOOP-7527
> URL: https://issues.apache.org/jira/browse/HADOOP-7527
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 0.23.0
> Reporter: Eli Collins
>
> URL encoding is currently handled in at least 5 different ways. We should make these consistent:
> # Parameters are encoded when a URI object is created
> # HttpServlet uses RequestQuoter to html escape parameter names and values
> # StringEscapeUtils is used to escape parameters in ReconfigurationServlet and DatanodeJspHelper
> # URLEncoder and URLDecoder are used in multiple places
> # encodePath from Jetty's URIUtil
> We should also be consistent about how we pass file names in URLs, some times they're passed in the path segment, sometimes they're passed in the query fragment as parameters.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7527) Make URL encoding consistent
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081045#comment-13081045 ]
Owen O'Malley commented on HADOOP-7527:
---------------------------------------
The QuotingInputFilter has a very different purpose than the others. It is ensuring that parameters that are echoed back to the user don't create XSS vulnerabilities. In particular, they are using HTML quoting and not URL quoting.
> Make URL encoding consistent
> ----------------------------
>
> Key: HADOOP-7527
> URL: https://issues.apache.org/jira/browse/HADOOP-7527
> Project: Hadoop Common
> Issue Type: Improvement
> Affects Versions: 0.23.0
> Reporter: Eli Collins
>
> URL encoding is currently handled in at least 4 different ways. We should make these consistent:
> # Parameters are encoded when a URI object is created
> # HttpServlet uses RequestQuoter to html escape parameter names and values
> # StringEscapeUtils is used to escape parameters in ReconfigurationServlet and DatanodeJspHelper
> # URLEncoder and URLDecoder are used in multiple places
> We should also be consistent about how we pass file names in URLs, some times they're passed in the path segment, sometimes they're passed in the query fragment as parameters.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira