You are viewing a plain text version of this content. The canonical link for it is here.

Posted to user@spark.apache.org by nsalian <ne...@gmail.com> on 2015/06/15 23:32:28 UTC

Re: Spark application in production without HDFS

Hi,

Spark on YARN should help in the memory management for Spark jobs.
Here is a good starting point:
https://spark.apache.org/docs/latest/running-on-yarn.html
YARN integrates well with HDFS and should be a good solution for a large
cluster.
What specific features are you looking for that HDFS does not satisfy?

Thank you.




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-application-in-production-without-HDFS-tp23260p23320.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org

Re: Spark application in production without HDFS

Posted by rahulkumar-aws <ra...@gmail.com>.

Hi If your data is not so huge you can use both cloudera and HDP's free
stack. Cloudera Express is 100% opensource free. 



-----
Software Developer
SigmoidAnalytics, Bangalore

--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-application-in-production-without-HDFS-tp23260p23322.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org