[jira] [Resolved] (SPARK-19992) spark-submit on deployment-mode cluster

["Sean Owen (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/SPARK-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Sean Owen resolved SPARK-19992.
-------------------------------
    Resolution: Not A Problem

[~jerryshao] is "/*" even going to work?
This must be an env problem because YARN cluster mode certainly works, and it's not clear the env is correct here. Here the Spark JAR is also being submitted with --jars, and it's not a matching version even (though I doubt that's the problem).

> spark-submit on deployment-mode cluster
> ---------------------------------------
>
>                 Key: SPARK-19992
>                 URL: https://issues.apache.org/jira/browse/SPARK-19992
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Submit
>    Affects Versions: 2.0.2
>         Environment: spark version 2.0.2
> hadoop version 2.6.0
>            Reporter: narendra maru
>
> spark version 2.0.2
> hadoop version 2.6.0
> spark -submit command
> "spark-submit --class spark.mongohadoop.testing3 --master yarn --deploy-mode cluster --jars /home/ec2-user/jars/hgmongonew.jar, /home/ec2-user/jars/mongo-hadoop-spark-2.0.1.jar"
> after adding following in
> 1 Spark-default.conf
> spark.executor.extraJavaOptions -Dconfig.fuction.conf 
> spark.yarn.jars=local:/usr/local/spark-2.0.2-bin-hadoop2.6/yarn/*
> spark.eventLog.dir=hdfs://localhost:9000/user/spark/applicationHistory
> spark.eventLog.enabled=true
> 2yarn-site.xml
> <property>
> <name>yarn.application.classpath</name>
> <value>
> /usr/local/hadoop-2.6.0/etc/hadoop,
> /usr/local/hadoop-2.6.0/,
> /usr/local/hadoop-2.6.0/lib/,
> /usr/local/hadoop-2.6.0/share/hadoop/common/,
> /usr/local/hadoop-2.6.0/share/hadoop/common/lib/
> /usr/local/hadoop-2.6.0/share/hadoop/hdfs/,
> /usr/local/hadoop-2.6.0/share/hadoop/hdfs/lib/,
> /usr/local/hadoop-2.6.0/share/hadoop/mapreduce/,
> /usr/local/hadoop-2.6.0/share/hadoop/mapreduce/lib/,
> /usr/local/hadoop-2.6.0/share/hadoop/tools/lib/,
> /usr/local/hadoop-2.6.0/share/hadoop/yarn/,
> /usr/local/hadoop-2.6.0/share/hadoop/yarn/lib/*,
> /usr/local/spark-2.0.2-bin-hadoop2.6/jars/spark-yarn_2.11-2.0.2.jar
> </value>
> </property>
> Error on log:-
> Error: Could not find or load main class org.apache.spark.deploy.yarn.ApplicationMaster
> Error on terminal:-
> diagnostics: Application application_1489673977198_0002 failed 2 times due to AM Container for appattempt_1489673977198_0002_000002 exited with exitCode: 1 For more detailed output, check application tracking page:http://bdg-hdp-sparkmaster:8088/proxy/application_1489673977198_0002/Then, click on links to logs of each attempt.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org