[jira] [Resolved] (OOZIE-2730) Spark submission fails, when spark-mode is specified through spark-opts instead of mode tag in action

["Rinat Sharipov (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OOZIE-2730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Rinat Sharipov resolved OOZIE-2730.
-----------------------------------
    Resolution: Won't Fix

> Spark submission fails, when spark-mode is specified through spark-opts instead of mode tag in action
> -----------------------------------------------------------------------------------------------------
>
>                 Key: OOZIE-2730
>                 URL: https://issues.apache.org/jira/browse/OOZIE-2730
>             Project: Oozie
>          Issue Type: Bug
>          Components: action
>    Affects Versions: 4.1.0
>         Environment: cloudera 4.8.0, oozie 4.1.0, java 8, java version "1.8.0_101"/ Java(TM) SE Runtime Environment (build 1.8.0_101-b13)/ Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode), Linux host 4.2.0-42-generic #49~14.04.1-Ubuntu SMP Wed Jun 29 20:22:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
>            Reporter: Rinat Sharipov
>            Priority: Major
>         Attachments: failed_job.png, job_configuration.xml, job_definition.xml
>
>
> Hi !
> I'm trying to submit spark job through oozie spark action and specify spark-mode in spark-options instead of mode tag (xsd schema specifies that it's optional). But submission fails with further message "Error: Could not find or load main class org.apache.spark.deploy.yarn.ApplicationMaster"
> I've attached further files:
>  - job_configuration.xml - properties, that is used to run a spark job
>  - job_definition.xml - spark action config
>  - failed_job.png - is an example of failed oozie job from oozie UI
> I think, that this behaviour is unexpected, because xsd schema specifies, that <mode> tag is optional
> Also, it's not very clear, what will happen, if I specify different values in spark-action mode tag and spark-opts (--deploy-mode), which option will have the highest priority ?



--
This message was sent by Atlassian Jira
(v8.3.4#803005)