You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "ASF GitHub Bot (JIRA)" <ji...@apache.org> on 2018/12/10 17:25:01 UTC
[jira] [Commented] (SPARK-24930) Exception information is not
accurate when using `LOAD DATA LOCAL INPATH`
[ https://issues.apache.org/jira/browse/SPARK-24930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16715171#comment-16715171 ]
ASF GitHub Bot commented on SPARK-24930:
----------------------------------------
AmplabJenkins commented on issue #21881: [SPARK-24930][SQL] Improve exception information when using LOAD DATA LOCAL INPATH
URL: https://github.com/apache/spark/pull/21881#issuecomment-445898969
Can one of the admins verify this patch?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> Exception information is not accurate when using `LOAD DATA LOCAL INPATH`
> --------------------------------------------------------------------------
>
> Key: SPARK-24930
> URL: https://issues.apache.org/jira/browse/SPARK-24930
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.2.2, 2.3.1
> Reporter: Xiaochen Ouyang
> Priority: Minor
>
> # root user create a test.txt file contains a record '123' in /root/ directory
> # switch mr user to execute spark-shell --master local
> {code:java}
> scala> spark.version
> res2: String = 2.2.1
> scala> spark.sql("create table t1(id int) partitioned by(area string)");
> 2018-07-26 17:20:37,523 WARN org.apache.hadoop.hive.metastore.HiveMetaStore: Location: hdfs://nameservice/spark/t1 specified for non-external table:t1
> res4: org.apache.spark.sql.DataFrame = []
> scala> spark.sql("load data local inpath '/root/test.txt' into table t1 partition(area ='025')")
> org.apache.spark.sql.AnalysisException: LOAD DATA input path does not exist: /root/test.txt;
> at org.apache.spark.sql.execution.command.LoadDataCommand.run(tables.scala:339)
> at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)
> at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)
> at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67)
> at org.apache.spark.sql.Dataset.<init>(Dataset.scala:183)
> at org.apache.spark.sql.Dataset$.ofRows(Dataset.scala:68)
> at org.apache.spark.sql.SparkSession.sql(SparkSession.scala:639)
> ... 48 elided
> scala>
> {code}
> In fact, the input path exists, but the mr user does not have permission to access the directory `/root/` ,so the message throwed by `AnalysisException` can confuse user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org