[jira] [Updated] (OOZIE-2389) Spark action failed with error starting MRAppMaster

["Andras Salamon (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/OOZIE-2389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Andras Salamon updated OOZIE-2389:
----------------------------------
    Priority: Major  (was: Blocker)

> Spark action failed with error starting MRAppMaster
> ---------------------------------------------------
>
>                 Key: OOZIE-2389
>                 URL: https://issues.apache.org/jira/browse/OOZIE-2389
>             Project: Oozie
>          Issue Type: Bug
>    Affects Versions: 4.2.0
>            Reporter: Hunt Tang
>            Priority: Major
>         Attachments: application_1445333963855_0050.log
>
>
> I used spark-examples-1.5.1-hadoop2.6.0.jar to generate a test spark action in Oozie, it succeeded sometimes, but in most of the times, it failed.
> I checked the Hadoop job history, and it said
> {quote}
> ERROR [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
> java.lang.NoSuchMethodError: org.apache.hadoop.mapred.TaskLog.createLogSyncer()Ljava/util/concurrent/ScheduledExecutorService;
>         at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.<init>(MRAppMaster.java:244)
>         at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.<init>(MRAppMaster.java:227)
>         at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1412)
> 2015-10-22 17:01:56,203 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1
> {quote}
> I'm using Hadoop 2.6.0, Spark 1.5.1, Oozie 4.2.0, and I put spark-assembly-1.5.1-hadoop2.6.0.jar together with the example jar in the lib, and my workflow.xml is as followings.
> {code:xml}
>     <action name="sparkTest">
>         <spark xmlns="uri:oozie:spark-action:0.1">
>             <job-tracker>${jobTracker}</job-tracker>
>             <name-node>${nameNode}</name-node>
>             <master>yarn-cluster</master>
>             <name>SparkPi</name>
>             <class>org.apache.spark.examples.JavaSparkPi</class>
>             <jar>${nameNode}/sparkOozieTest/workflow/lib/spark-examples-1.5.1-hadoop2.6.0.jar</jar>
>             <arg>10</arg>
>         </spark>
>         <ok to="end"/>
>         <error to="fail"/>
>     </action>
> {code}
> As it indeed succeeded sometimes, I don't think there is anything wrong with my configs. So is it an Oozie bug or is there anything I missed?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)