You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@spark.apache.org by "bailin (Jira)" <ji...@apache.org> on 2019/09/11 06:46:00 UTC
[jira] [Created] (SPARK-29047) use spark-submit Not a file:
hdfs://
bailin created SPARK-29047:
------------------------------
Summary: use spark-submit Not a file: hdfs://
Key: SPARK-29047
URL: https://issues.apache.org/jira/browse/SPARK-29047
Project: Spark
Issue Type: Question
Components: Deploy, Spark Submit
Affects Versions: 2.1.3
Environment: spark1.3 ；hadoop2.6
Reporter: bailin
when i submit a spark application :
{code:java}
/spark-submit --class com.yto.log.SparkStatCleanJobYARN --name TopNStatJobYARN --master yarn --executor-memory 1G --num-executors 1 /home/hadoop/lib/sql-1.0-jar-with-dependencies.jar hdfs://hadoop001:8020/clean 20161110{code}
the hadfs file is already existed，but not recognition
{code:java}
19/09/11 14:18:50 INFO mapred.FileInputFormat: Total input paths to process : 219/09/11 14:18:50 INFO mapred.FileInputFormat: Total input paths to process : 219/09/11 14:18:50 ERROR datasources.FileFormatWriter: Aborting job null.java.io.IOException: Not a file: hdfs://hadoop001:8020/clean/day= at org.apache.hadoop.mapred.FileInputFormat.getSplits
{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org