Fwd: Spark Node with Mesos and Spark Standalone

[Ahmed Kamal <ah...@badrit.com>]

---------- Forwarded message ----------
From: Ahmed Kamal <ah...@badrit.com>
Date: Mon, Feb 29, 2016 at 4:18 PM
Subject: Spark Node with Mesos and Spark Standalone
To: dev@oozie.apache.org


Hi guys,

I'm investigating the Oozie Spark Node and I came across this sentence "The
master element indicates the url of the Spark Master. Ex: spark://host:port,
<https://oozie.apache.org/docs/4.2.0/spark://host:port,.html>
mesos://host:port,
<https://oozie.apache.org/docs/4.2.0/mesos://host:port,.html> yarn-cluster,
yarn-master, or local.
" in the documentation of Oozie

https://oozie.apache.org/docs/4.2.0/DG_SparkActionExtension.html

from what I understood oozie uses Spark Submit script to either run spark
jobs in local mode or to run on yarn and it depend on the Yarn API to get
the job status but it isn't possible to use it with spark standalone or
spark mesos as the current implementation doesn't have a valid way to check
the status. Am I right ? If so , I'm interested in extending the current
spark node to support them and contributing this to the project. I'm
preparing a design document for the change. Should I open an issue on JIRA
?
-- 
Regards ,

Ahmed Kamal
Junior Big Data Engineer , BadrIT



-- 
JAK ,

Ahmed Kamal
Junior Big Data Engineer , BadrIT