You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Dongjoon Hyun (JIRA)" <ji...@apache.org> on 2016/04/07 01:36:25 UTC
[jira] [Commented] (SPARK-14443) parse_url() does not escape query
parameters
[ https://issues.apache.org/jira/browse/SPARK-14443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15229345#comment-15229345 ]
Dongjoon Hyun commented on SPARK-14443:
---------------------------------------
I seems that the root cause is the third `key` parameter: `url[]`. Actually, `hive` also makes errors on the key `url[]`.
{code}
hive> select parse_url('http://1168.xg4ken.com/media/redir.php?prof=457&camp=67116&affcode=kw54&k_inner_url_encoded=1&cid=adwords&kdv=Desktop&url[]=http%3A%2F%2Fwww.landroverusa.com%2Fvehicles%2Frange-rover-sport-off-road-suv%2Findex.html%3Futm_content%3Dcontent%26utm_source%fb%26utm_medium%3Dcpc%26utm_term%3DAdwords_Brand_Range_Rover_Sport%26utm_campaign%3DFB_Land_Rover_Brand', 'QUERY', 'url[]');
FAILED: SemanticException [Error 10014]: Line 1:7 Wrong arguments ''url[]'': org.apache.hadoop.hive.ql.metadata.HiveException: Unable to execute method public java.lang.String org.apache.hadoop.hive.ql.udf.UDFParseUrl.evaluate(java.lang.String,java.lang.String,java.lang.String) on object org.apache.hadoop.hive.ql.udf.UDFParseUrl@2935fd2c of class org.apache.hadoop.hive.ql.udf.UDFParseUrl with arguments {http://1168.xg4ken.com/media/redir.php?prof=457&camp=67116&affcode=kw54&k_inner_url_encoded=1&cid=adwords&kdv=Desktop&url[]=http%3A%2F%2Fwww.landroverusa.com%2Fvehicles%2Frange-rover-sport-off-road-suv%2Findex.html%3Futm_content%3Dcontent%26utm_source%fb%26utm_medium%3Dcpc%26utm_term%3DAdwords_Brand_Range_Rover_Sport%26utm_campaign%3DFB_Land_Rover_Brand:java.lang.String, QUERY:java.lang.String, url[]:java.lang.String} of size 3
{code}
> parse_url() does not escape query parameters
> --------------------------------------------
>
> Key: SPARK-14443
> URL: https://issues.apache.org/jira/browse/SPARK-14443
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.0
> Environment: Databricks
> Reporter: Simeon Simeonov
> Labels: functions, sql
>
> To reproduce, run the following SparkSQL statement:
> {code}
> select parse_url('http://1168.xg4ken.com/media/redir.php?prof=457&camp=67116&affcode=kw54&k_inner_url_encoded=1&cid=adwords&kdv=Desktop&url[]=http%3A%2F%2Fwww.landroverusa.com%2Fvehicles%2Frange-rover-sport-off-road-suv%2Findex.html%3Futm_content%3Dcontent%26utm_source%fb%26utm_medium%3Dcpc%26utm_term%3DAdwords_Brand_Range_Rover_Sport%26utm_campaign%3DFB_Land_Rover_Brand', 'QUERY', 'url[]')
> {code}
> The exception is ultimately caused by
> {code}
> java.util.regex.PatternSyntaxException: Unclosed character class near index 17
> (&|^)url[]=([^&]*)
> ^
> {code}
> Looks like the code is building a regex internally without escaping the passed in query parameter name.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org