You are viewing a plain text version of this content. The canonical link for it is here.

Posted to hdfs-user@hadoop.apache.org by Divya Gehlot <di...@gmail.com> on 2016/03/07 10:14:51 UTC

Steps to Run Spark Scala job from Oozie on EC2 Hadoop clsuter

Hi,

Could somebody help me by providing the steps /redirect me  to
blog/documentation on how to run Spark job written in scala through Oozie.

Would really appreciate the help.



Thanks,
Divya

Re: Steps to Run Spark Scala job from Oozie on EC2 Hadoop clsuter

Posted by Ashutosh Kumar <as...@gmail.com>.

Is using oozie binding for you? If you have flexibility to change workflow
engine then take a look at Luigi (https://github.com/spotify/luigi) .

Thanks
Ashutosh

On Mon, Mar 7, 2016 at 9:22 PM, Benjamin Kim <bb...@gmail.com> wrote:

> To comment…
>
> At my company, we have not gotten it to work in any other mode than local.
> If we try any of the yarn modes, it fails with a “file does not exist”
> error when trying to locate the executable jar. I mentioned this to the Hue
> users group, which we used for this, and they replied that the Spark Action
> is very basic implementation and that they will be writing their own for
> production use.
>
> That’s all I know...
>
> On Mar 7, 2016, at 1:18 AM, Deepak Sharma <de...@gmail.com> wrote:
>
> There is Spark action defined for oozie workflows.
> Though I am not sure if it supports only Java SPARK jobs or Scala jobs as
> well.
> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html
> Thanks
> Deepak
>
> On Mon, Mar 7, 2016 at 2:44 PM, Divya Gehlot <di...@gmail.com>
> wrote:
>
>> Hi,
>>
>> Could somebody help me by providing the steps /redirect me  to
>> blog/documentation on how to run Spark job written in scala through Oozie.
>>
>> Would really appreciate the help.
>>
>>
>>
>> Thanks,
>> Divya
>>
>
>
>
> --
> Thanks
> Deepak
> www.bigdatabig.com
> www.keosha.net
>
>
>

Re: Steps to Run Spark Scala job from Oozie on EC2 Hadoop clsuter

Posted by dileep kumar <di...@gmail.com>.

Hi Divya,

Please find below code to invoke spark from oozie.

Oozie file:
+++++++++++++

<workflow-app xmlns="uri:oozie:workflow:0.2" name="DE_2_workflow_US">
        <start to="spark-scala"/>
        <action name="mysparkscala">
                <shell xmlns="uri:oozie:shell-action:0.1">
                        <job-tracker>maprfs:///</job-tracker>
                        <name-node>maprfs:///</name-node>
                        <configuration>
                                <property>
                                        <name>mapred.job.queue.name</name>

                                        <value>dileep</value>
                                </property>
                        </configuration>
                        <exec>sparkshell.sh</exec>

<file>/ggg/gms/gmsrffr/dev/dileep/sparkshell.sh</file>
                  <file>/axp/gms/gmsrffr/dev/dileep/sparkshell.scala</file>
                </shell>
                <ok to="end" />
                <error to="fail" />
        </action>
        <kill name="fail">
                 <message>Java failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
                      </kill>
        <end name="end" />
</workflow-app>


#############
SparkShell.sh
##############

/opt/mapr/spark/spark-1.2.1/bin/spark-shell --name perf108pret1
--num-executors 1 --executor-cores 1 --executor-memory 1G --driver-memory
2G -i sparkshell.scala
exit 0

On Mon, Mar 7, 2016 at 9:32 PM, Chandeep Singh <cs...@chandeep.com> wrote:

> As a work around you could put your spark-submit statement in a shell
> script and then use Oozie’s SSH action to execute that script.
>
> On Mar 7, 2016, at 3:58 PM, Neelesh Salian <ns...@cloudera.com> wrote:
>
> Hi Divya,
>
> This link should have the details that you need to begin using the Spark
> Action on Oozie:
> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html
>
> Thanks.
>
> On Mon, Mar 7, 2016 at 7:52 AM, Benjamin Kim <bb...@gmail.com> wrote:
>
>> To comment…
>>
>> At my company, we have not gotten it to work in any other mode than
>> local. If we try any of the yarn modes, it fails with a “file does not
>> exist” error when trying to locate the executable jar. I mentioned this to
>> the Hue users group, which we used for this, and they replied that the
>> Spark Action is very basic implementation and that they will be writing
>> their own for production use.
>>
>> That’s all I know...
>>
>> On Mar 7, 2016, at 1:18 AM, Deepak Sharma <de...@gmail.com> wrote:
>>
>> There is Spark action defined for oozie workflows.
>> Though I am not sure if it supports only Java SPARK jobs or Scala jobs as
>> well.
>> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html
>> Thanks
>> Deepak
>>
>> On Mon, Mar 7, 2016 at 2:44 PM, Divya Gehlot <di...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Could somebody help me by providing the steps /redirect me  to
>>> blog/documentation on how to run Spark job written in scala through Oozie.
>>>
>>> Would really appreciate the help.
>>>
>>>
>>>
>>> Thanks,
>>> Divya
>>>
>>
>>
>>
>> --
>> Thanks
>> Deepak
>> www.bigdatabig.com
>> www.keosha.net
>>
>>
>>
>
>
> --
> Neelesh Srinivas Salian
> Customer Operations Engineer
>
>
>
>
>


-- 
Regards
Dileep Kumar
+91 9742443302

Re: Steps to Run Spark Scala job from Oozie on EC2 Hadoop clsuter

Posted by dileep kumar <di...@gmail.com>.

Hi Divya,

Please find below code to invoke spark from oozie.

Oozie file:
+++++++++++++

<workflow-app xmlns="uri:oozie:workflow:0.2" name="DE_2_workflow_US">
        <start to="spark-scala"/>
        <action name="mysparkscala">
                <shell xmlns="uri:oozie:shell-action:0.1">
                        <job-tracker>maprfs:///</job-tracker>
                        <name-node>maprfs:///</name-node>
                        <configuration>
                                <property>
                                        <name>mapred.job.queue.name</name>

                                        <value>dileep</value>
                                </property>
                        </configuration>
                        <exec>sparkshell.sh</exec>

<file>/ggg/gms/gmsrffr/dev/dileep/sparkshell.sh</file>
                  <file>/axp/gms/gmsrffr/dev/dileep/sparkshell.scala</file>
                </shell>
                <ok to="end" />
                <error to="fail" />
        </action>
        <kill name="fail">
                 <message>Java failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
                      </kill>
        <end name="end" />
</workflow-app>


#############
SparkShell.sh
##############

/opt/mapr/spark/spark-1.2.1/bin/spark-shell --name perf108pret1
--num-executors 1 --executor-cores 1 --executor-memory 1G --driver-memory
2G -i sparkshell.scala
exit 0

On Mon, Mar 7, 2016 at 9:32 PM, Chandeep Singh <cs...@chandeep.com> wrote:

> As a work around you could put your spark-submit statement in a shell
> script and then use Oozie’s SSH action to execute that script.
>
> On Mar 7, 2016, at 3:58 PM, Neelesh Salian <ns...@cloudera.com> wrote:
>
> Hi Divya,
>
> This link should have the details that you need to begin using the Spark
> Action on Oozie:
> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html
>
> Thanks.
>
> On Mon, Mar 7, 2016 at 7:52 AM, Benjamin Kim <bb...@gmail.com> wrote:
>
>> To comment…
>>
>> At my company, we have not gotten it to work in any other mode than
>> local. If we try any of the yarn modes, it fails with a “file does not
>> exist” error when trying to locate the executable jar. I mentioned this to
>> the Hue users group, which we used for this, and they replied that the
>> Spark Action is very basic implementation and that they will be writing
>> their own for production use.
>>
>> That’s all I know...
>>
>> On Mar 7, 2016, at 1:18 AM, Deepak Sharma <de...@gmail.com> wrote:
>>
>> There is Spark action defined for oozie workflows.
>> Though I am not sure if it supports only Java SPARK jobs or Scala jobs as
>> well.
>> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html
>> Thanks
>> Deepak
>>
>> On Mon, Mar 7, 2016 at 2:44 PM, Divya Gehlot <di...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Could somebody help me by providing the steps /redirect me  to
>>> blog/documentation on how to run Spark job written in scala through Oozie.
>>>
>>> Would really appreciate the help.
>>>
>>>
>>>
>>> Thanks,
>>> Divya
>>>
>>
>>
>>
>> --
>> Thanks
>> Deepak
>> www.bigdatabig.com
>> www.keosha.net
>>
>>
>>
>
>
> --
> Neelesh Srinivas Salian
> Customer Operations Engineer
>
>
>
>
>


-- 
Regards
Dileep Kumar
+91 9742443302

Re: Steps to Run Spark Scala job from Oozie on EC2 Hadoop clsuter

Posted by dileep kumar <di...@gmail.com>.

Hi Divya,

Please find below code to invoke spark from oozie.

Oozie file:
+++++++++++++

<workflow-app xmlns="uri:oozie:workflow:0.2" name="DE_2_workflow_US">
        <start to="spark-scala"/>
        <action name="mysparkscala">
                <shell xmlns="uri:oozie:shell-action:0.1">
                        <job-tracker>maprfs:///</job-tracker>
                        <name-node>maprfs:///</name-node>
                        <configuration>
                                <property>
                                        <name>mapred.job.queue.name</name>

                                        <value>dileep</value>
                                </property>
                        </configuration>
                        <exec>sparkshell.sh</exec>

<file>/ggg/gms/gmsrffr/dev/dileep/sparkshell.sh</file>
                  <file>/axp/gms/gmsrffr/dev/dileep/sparkshell.scala</file>
                </shell>
                <ok to="end" />
                <error to="fail" />
        </action>
        <kill name="fail">
                 <message>Java failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
                      </kill>
        <end name="end" />
</workflow-app>


#############
SparkShell.sh
##############

/opt/mapr/spark/spark-1.2.1/bin/spark-shell --name perf108pret1
--num-executors 1 --executor-cores 1 --executor-memory 1G --driver-memory
2G -i sparkshell.scala
exit 0

On Mon, Mar 7, 2016 at 9:32 PM, Chandeep Singh <cs...@chandeep.com> wrote:

> As a work around you could put your spark-submit statement in a shell
> script and then use Oozie’s SSH action to execute that script.
>
> On Mar 7, 2016, at 3:58 PM, Neelesh Salian <ns...@cloudera.com> wrote:
>
> Hi Divya,
>
> This link should have the details that you need to begin using the Spark
> Action on Oozie:
> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html
>
> Thanks.
>
> On Mon, Mar 7, 2016 at 7:52 AM, Benjamin Kim <bb...@gmail.com> wrote:
>
>> To comment…
>>
>> At my company, we have not gotten it to work in any other mode than
>> local. If we try any of the yarn modes, it fails with a “file does not
>> exist” error when trying to locate the executable jar. I mentioned this to
>> the Hue users group, which we used for this, and they replied that the
>> Spark Action is very basic implementation and that they will be writing
>> their own for production use.
>>
>> That’s all I know...
>>
>> On Mar 7, 2016, at 1:18 AM, Deepak Sharma <de...@gmail.com> wrote:
>>
>> There is Spark action defined for oozie workflows.
>> Though I am not sure if it supports only Java SPARK jobs or Scala jobs as
>> well.
>> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html
>> Thanks
>> Deepak
>>
>> On Mon, Mar 7, 2016 at 2:44 PM, Divya Gehlot <di...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Could somebody help me by providing the steps /redirect me  to
>>> blog/documentation on how to run Spark job written in scala through Oozie.
>>>
>>> Would really appreciate the help.
>>>
>>>
>>>
>>> Thanks,
>>> Divya
>>>
>>
>>
>>
>> --
>> Thanks
>> Deepak
>> www.bigdatabig.com
>> www.keosha.net
>>
>>
>>
>
>
> --
> Neelesh Srinivas Salian
> Customer Operations Engineer
>
>
>
>
>


-- 
Regards
Dileep Kumar
+91 9742443302

Re: Steps to Run Spark Scala job from Oozie on EC2 Hadoop clsuter

Posted by dileep kumar <di...@gmail.com>.

Hi Divya,

Please find below code to invoke spark from oozie.

Oozie file:
+++++++++++++

<workflow-app xmlns="uri:oozie:workflow:0.2" name="DE_2_workflow_US">
        <start to="spark-scala"/>
        <action name="mysparkscala">
                <shell xmlns="uri:oozie:shell-action:0.1">
                        <job-tracker>maprfs:///</job-tracker>
                        <name-node>maprfs:///</name-node>
                        <configuration>
                                <property>
                                        <name>mapred.job.queue.name</name>

                                        <value>dileep</value>
                                </property>
                        </configuration>
                        <exec>sparkshell.sh</exec>

<file>/ggg/gms/gmsrffr/dev/dileep/sparkshell.sh</file>
                  <file>/axp/gms/gmsrffr/dev/dileep/sparkshell.scala</file>
                </shell>
                <ok to="end" />
                <error to="fail" />
        </action>
        <kill name="fail">
                 <message>Java failed, error
message[${wf:errorMessage(wf:lastErrorNode())}]</message>
                      </kill>
        <end name="end" />
</workflow-app>


#############
SparkShell.sh
##############

/opt/mapr/spark/spark-1.2.1/bin/spark-shell --name perf108pret1
--num-executors 1 --executor-cores 1 --executor-memory 1G --driver-memory
2G -i sparkshell.scala
exit 0

On Mon, Mar 7, 2016 at 9:32 PM, Chandeep Singh <cs...@chandeep.com> wrote:

> As a work around you could put your spark-submit statement in a shell
> script and then use Oozie’s SSH action to execute that script.
>
> On Mar 7, 2016, at 3:58 PM, Neelesh Salian <ns...@cloudera.com> wrote:
>
> Hi Divya,
>
> This link should have the details that you need to begin using the Spark
> Action on Oozie:
> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html
>
> Thanks.
>
> On Mon, Mar 7, 2016 at 7:52 AM, Benjamin Kim <bb...@gmail.com> wrote:
>
>> To comment…
>>
>> At my company, we have not gotten it to work in any other mode than
>> local. If we try any of the yarn modes, it fails with a “file does not
>> exist” error when trying to locate the executable jar. I mentioned this to
>> the Hue users group, which we used for this, and they replied that the
>> Spark Action is very basic implementation and that they will be writing
>> their own for production use.
>>
>> That’s all I know...
>>
>> On Mar 7, 2016, at 1:18 AM, Deepak Sharma <de...@gmail.com> wrote:
>>
>> There is Spark action defined for oozie workflows.
>> Though I am not sure if it supports only Java SPARK jobs or Scala jobs as
>> well.
>> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html
>> Thanks
>> Deepak
>>
>> On Mon, Mar 7, 2016 at 2:44 PM, Divya Gehlot <di...@gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> Could somebody help me by providing the steps /redirect me  to
>>> blog/documentation on how to run Spark job written in scala through Oozie.
>>>
>>> Would really appreciate the help.
>>>
>>>
>>>
>>> Thanks,
>>> Divya
>>>
>>
>>
>>
>> --
>> Thanks
>> Deepak
>> www.bigdatabig.com
>> www.keosha.net
>>
>>
>>
>
>
> --
> Neelesh Srinivas Salian
> Customer Operations Engineer
>
>
>
>
>


-- 
Regards
Dileep Kumar
+91 9742443302

Re: Steps to Run Spark Scala job from Oozie on EC2 Hadoop clsuter

Posted by Chandeep Singh <cs...@chandeep.com>.

As a work around you could put your spark-submit statement in a shell script and then use Oozie’s SSH action to execute that script.

> On Mar 7, 2016, at 3:58 PM, Neelesh Salian <ns...@cloudera.com> wrote:
> 
> Hi Divya,
> 
> This link should have the details that you need to begin using the Spark Action on Oozie:
> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html <https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html>
> 
> Thanks.
> 
> On Mon, Mar 7, 2016 at 7:52 AM, Benjamin Kim <bbuild11@gmail.com <ma...@gmail.com>> wrote:
> To comment…
> 
> At my company, we have not gotten it to work in any other mode than local. If we try any of the yarn modes, it fails with a “file does not exist” error when trying to locate the executable jar. I mentioned this to the Hue users group, which we used for this, and they replied that the Spark Action is very basic implementation and that they will be writing their own for production use.
> 
> That’s all I know...
> 
>> On Mar 7, 2016, at 1:18 AM, Deepak Sharma <deepakmca05@gmail.com <ma...@gmail.com>> wrote:
>> 
>> There is Spark action defined for oozie workflows.
>> Though I am not sure if it supports only Java SPARK jobs or Scala jobs as well.
>> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html <https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html>
>> Thanks
>> Deepak
>> 
>> On Mon, Mar 7, 2016 at 2:44 PM, Divya Gehlot <divya.htconex@gmail.com <ma...@gmail.com>> wrote:
>> Hi,
>> 
>> Could somebody help me by providing the steps /redirect me  to blog/documentation on how to run Spark job written in scala through Oozie.
>> 
>> Would really appreciate the help.
>> 
>> 
>> 
>> Thanks,
>> Divya 
>> 
>> 
>> 
>> -- 
>> Thanks
>> Deepak
>> www.bigdatabig.com <http://www.bigdatabig.com/>
>> www.keosha.net <http://www.keosha.net/>
> 
> 
> 
> -- 
> Neelesh Srinivas Salian
> Customer Operations Engineer
> 
> 
>

Re: Steps to Run Spark Scala job from Oozie on EC2 Hadoop clsuter

Posted by Chandeep Singh <cs...@chandeep.com>.

As a work around you could put your spark-submit statement in a shell script and then use Oozie’s SSH action to execute that script.

> On Mar 7, 2016, at 3:58 PM, Neelesh Salian <ns...@cloudera.com> wrote:
> 
> Hi Divya,
> 
> This link should have the details that you need to begin using the Spark Action on Oozie:
> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html <https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html>
> 
> Thanks.
> 
> On Mon, Mar 7, 2016 at 7:52 AM, Benjamin Kim <bbuild11@gmail.com <ma...@gmail.com>> wrote:
> To comment…
> 
> At my company, we have not gotten it to work in any other mode than local. If we try any of the yarn modes, it fails with a “file does not exist” error when trying to locate the executable jar. I mentioned this to the Hue users group, which we used for this, and they replied that the Spark Action is very basic implementation and that they will be writing their own for production use.
> 
> That’s all I know...
> 
>> On Mar 7, 2016, at 1:18 AM, Deepak Sharma <deepakmca05@gmail.com <ma...@gmail.com>> wrote:
>> 
>> There is Spark action defined for oozie workflows.
>> Though I am not sure if it supports only Java SPARK jobs or Scala jobs as well.
>> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html <https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html>
>> Thanks
>> Deepak
>> 
>> On Mon, Mar 7, 2016 at 2:44 PM, Divya Gehlot <divya.htconex@gmail.com <ma...@gmail.com>> wrote:
>> Hi,
>> 
>> Could somebody help me by providing the steps /redirect me  to blog/documentation on how to run Spark job written in scala through Oozie.
>> 
>> Would really appreciate the help.
>> 
>> 
>> 
>> Thanks,
>> Divya 
>> 
>> 
>> 
>> -- 
>> Thanks
>> Deepak
>> www.bigdatabig.com <http://www.bigdatabig.com/>
>> www.keosha.net <http://www.keosha.net/>
> 
> 
> 
> -- 
> Neelesh Srinivas Salian
> Customer Operations Engineer
> 
> 
>

Re: Steps to Run Spark Scala job from Oozie on EC2 Hadoop clsuter

Posted by Chandeep Singh <cs...@chandeep.com>.

As a work around you could put your spark-submit statement in a shell script and then use Oozie’s SSH action to execute that script.

> On Mar 7, 2016, at 3:58 PM, Neelesh Salian <ns...@cloudera.com> wrote:
> 
> Hi Divya,
> 
> This link should have the details that you need to begin using the Spark Action on Oozie:
> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html <https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html>
> 
> Thanks.
> 
> On Mon, Mar 7, 2016 at 7:52 AM, Benjamin Kim <bbuild11@gmail.com <ma...@gmail.com>> wrote:
> To comment…
> 
> At my company, we have not gotten it to work in any other mode than local. If we try any of the yarn modes, it fails with a “file does not exist” error when trying to locate the executable jar. I mentioned this to the Hue users group, which we used for this, and they replied that the Spark Action is very basic implementation and that they will be writing their own for production use.
> 
> That’s all I know...
> 
>> On Mar 7, 2016, at 1:18 AM, Deepak Sharma <deepakmca05@gmail.com <ma...@gmail.com>> wrote:
>> 
>> There is Spark action defined for oozie workflows.
>> Though I am not sure if it supports only Java SPARK jobs or Scala jobs as well.
>> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html <https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html>
>> Thanks
>> Deepak
>> 
>> On Mon, Mar 7, 2016 at 2:44 PM, Divya Gehlot <divya.htconex@gmail.com <ma...@gmail.com>> wrote:
>> Hi,
>> 
>> Could somebody help me by providing the steps /redirect me  to blog/documentation on how to run Spark job written in scala through Oozie.
>> 
>> Would really appreciate the help.
>> 
>> 
>> 
>> Thanks,
>> Divya 
>> 
>> 
>> 
>> -- 
>> Thanks
>> Deepak
>> www.bigdatabig.com <http://www.bigdatabig.com/>
>> www.keosha.net <http://www.keosha.net/>
> 
> 
> 
> -- 
> Neelesh Srinivas Salian
> Customer Operations Engineer
> 
> 
>

Re: Steps to Run Spark Scala job from Oozie on EC2 Hadoop clsuter

Posted by Chandeep Singh <cs...@chandeep.com>.

As a work around you could put your spark-submit statement in a shell script and then use Oozie’s SSH action to execute that script.

> On Mar 7, 2016, at 3:58 PM, Neelesh Salian <ns...@cloudera.com> wrote:
> 
> Hi Divya,
> 
> This link should have the details that you need to begin using the Spark Action on Oozie:
> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html <https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html>
> 
> Thanks.
> 
> On Mon, Mar 7, 2016 at 7:52 AM, Benjamin Kim <bbuild11@gmail.com <ma...@gmail.com>> wrote:
> To comment…
> 
> At my company, we have not gotten it to work in any other mode than local. If we try any of the yarn modes, it fails with a “file does not exist” error when trying to locate the executable jar. I mentioned this to the Hue users group, which we used for this, and they replied that the Spark Action is very basic implementation and that they will be writing their own for production use.
> 
> That’s all I know...
> 
>> On Mar 7, 2016, at 1:18 AM, Deepak Sharma <deepakmca05@gmail.com <ma...@gmail.com>> wrote:
>> 
>> There is Spark action defined for oozie workflows.
>> Though I am not sure if it supports only Java SPARK jobs or Scala jobs as well.
>> https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html <https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html>
>> Thanks
>> Deepak
>> 
>> On Mon, Mar 7, 2016 at 2:44 PM, Divya Gehlot <divya.htconex@gmail.com <ma...@gmail.com>> wrote:
>> Hi,
>> 
>> Could somebody help me by providing the steps /redirect me  to blog/documentation on how to run Spark job written in scala through Oozie.
>> 
>> Would really appreciate the help.
>> 
>> 
>> 
>> Thanks,
>> Divya 
>> 
>> 
>> 
>> -- 
>> Thanks
>> Deepak
>> www.bigdatabig.com <http://www.bigdatabig.com/>
>> www.keosha.net <http://www.keosha.net/>
> 
> 
> 
> -- 
> Neelesh Srinivas Salian
> Customer Operations Engineer
> 
> 
>