You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "Shixiong Zhu (JIRA)" <ji...@apache.org> on 2018/09/10 17:51:00 UTC
[jira] [Commented] (SPARK-23425) load data for hdfs file path with
wild card usage is not working properly
[ https://issues.apache.org/jira/browse/SPARK-23425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16609595#comment-16609595 ]
Shixiong Zhu commented on SPARK-23425:
--------------------------------------
Added "release-note" label.
Previously, when INPATH contains special characters (such as " "), the user has to manually escape them, e.g., use "/a/b/foo%20bar" rather than "/a/b/foo bar" because the former will throw "URISyntaxException: Illegal character in path at index XX: /a/b/foo bar".
After this patch, the above workaround will throw "AnalysisException: LOAD DATA input path does not exist: /a/b/foo%20bar;".
The root cause is we changed from "new URI(user_specified_path)" to "new Path(user_specified_path)". I believe this patch is indeed a bug fix but it's worth to highlight in the release note.
> load data for hdfs file path with wild card usage is not working properly
> -------------------------------------------------------------------------
>
> Key: SPARK-23425
> URL: https://issues.apache.org/jira/browse/SPARK-23425
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.2.1, 2.3.0
> Reporter: Sujith
> Assignee: Sujith
> Priority: Major
> Labels: release-notes
> Fix For: 2.4.0
>
> Attachments: wildcard_issue.PNG
>
>
> load data command for loading data from non local file paths by using wild card strings lke * are not working
> eg:
> "load data inpath 'hdfs://hacluster/user/ext* into table t1"
> Getting Analysis excepton while executing this query
> !image-2018-02-14-23-41-39-923.png!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org