Apache Spark Video Processing from NFS Shared storage: Advise needed

[mobcdi <b0...@student.itb.ie>]

Hi all,

Is it advisable to use nfs as shared storage for a small Spark cluster to
process video and images? I have a total of 20 vms (2vCPU, 6GB Ram, 20GB
Local Disk) connected to 500GB nfs shared storage (mounted the same in each
of the vms) at my disposal and I'm wondering if I can avoid the need for
hdfs and instead use the larger capacity nfs to work with my videos and
images in Spark?

I have spun up a master node (using maven not sbt) and connected 1 slave to
it but I haven't made any configuration changes to Spark. On
masternode:4040/ I don't see anything under storage. Is that to be expected
and if I do need to spin up hadoop can I double job the 20vms by running
both hadoop and spark on all 20 machines or would the recommendation be I
split them into separate hadoop and spark clusters 

Michael



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Apache-Spark-Video-Processing-from-NFS-Shared-storage-Advise-needed-tp27030.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
For additional commands, e-mail: user-help@spark.apache.org