spark-submit: file not found exception occurs

[Shupeng Geng <sh...@envisioncn.com>]

Hi, everyone,
         An annoying problem occurs to me.
         When submitting a spark job, the jar file not found exception is thrown as follows:

does not existread "main" java.io.FileNotFoundException: File file:/home/algo/shupeng/eeop_bridger/EeopBridger-1.0-SNAPSHOT.jar
         at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:534)
         at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:747)
         at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:524)
         at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:409)
         at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:337)
         at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
         at org.apache.spark.deploy.yarn.Client.copyFileToRemote(Client.scala:340)
         at org.apache.spark.deploy.yarn.Client.org$apache$spark$deploy$yarn$Client$$distribute$1(Client.scala:433)
         at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$10.apply(Client.scala:530)
         at org.apache.spark.deploy.yarn.Client$$anonfun$prepareLocalResources$10.apply(Client.scala:529)

I'm pretty sure the file is located just where it should be.

Here is the submit shell cmd:

/usr/local/hadoop/spark-release/bin/spark-submit --master yarn --deploy-mode cluster --queue root.algo --name eeop_bridiger --class Bridger EeopBridger-1.0-SNAPSHOT.jar

Somebody give me some help, please. Many Thanks.




???(????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????,?????????????????????
This email message (including any attachments) is confidential and may be legally privileged. If you have received it by mistake, please notify the sender by return email and delete this message from your system. Any unauthorized use or dissemination of this message in whole or in part is strictly prohibited. Envision Energy Limited and all its subsidiaries shall not be liable for the improper or incomplete transmission of the information contained in this email nor for any delay in its receipt or damage to your system. Envision Energy Limited does not guarantee the integrity of this email message, nor that this email message is free of viruses, interceptions, or interference.