Doubt: Regarding running Hadoop on a cluster with shared disk.

[Udaya Lakshmi <ud...@gmail.com>]

Hi,
   I have an account on a cluster which is having a file system similar to
NFS. If I create a file on one machine it is being shown on all the machines
in the cluster. But hadoop will work on a cluster of machines, where in ,
each machine has a disk of its own. Can someone please help me use hadoop on
my cluster.
Thanks,
Udaya.

RE: Doubt: Regarding running Hadoop on a cluster with shared disk.

[Michael Segel <mi...@hotmail.com>]

Udaya,

You can use non-local disk on your hadoop cloud, however it will have sub-optimal performance, and you will have to tune accordingly.

If its a shared drive on all of your nodes, you need to create different directories for each machine.

Suppose your shared drive is /foo  then you would need to set up a /foo/<name of node>/data for each machine in your cluster.

The drawback is not only I/O traffic and constraints but you'll have to tune ZK and watch out for timing issues as your disk i/o is your constraint.

Definitely not recommended.


> Date: Wed, 5 May 2010 15:52:11 +0530
> Subject: Doubt: Regarding running Hadoop on a cluster with shared disk.
> From: udaya603@gmail.com
> To: common-user@hadoop.apache.org
> 
> Hi,
>    I have an account on a cluster which is having a file system similar to
> NFS. If I create a file on one machine it is being shown on all the machines
> in the cluster. But hadoop will work on a cluster of machines, where in ,
> each machine has a disk of its own. Can someone please help me use hadoop on
> my cluster.
> Thanks,
> Udaya.
 		 	   		  
_________________________________________________________________
Hotmail is redefining busy with tools for the New Busy. Get more from your inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_2