how much time to run a Hadoop cluster ?

[Nick Rathke <ni...@sci.utah.edu>]

Hello Hadoop Users,

Now that we have our cluster up and running ( mostly ) the question has 
come up about much time will be required to run the system.

We have Hadoop running on 64 nodes, 40TB of storage with only 6 or 7 
Hadoop users right now. The number of users is likely to grow as more of 
our researchers begin to use the system but will likely not be more the 
20 users.

I am trying to get a sense of how much of an FTE ( Full Time Employee ) 
I should plan on for managing the system. We also run MPI and CUDA on 
the cluster and from a system administrator point of view these are 
fairly low overhead, but I have no concept yet of what the time 
commitment to Hadoop will be.

Thanks,

Nick Rathke
Scientific Computing and Imaging Institute
Sr. Systems Administrator
nick@sci.utah.edu
www.sci.utah.edu
801-587-9933
801-557-3832

"I came I saw I made it possible" Royal Bliss - Here They Come

Re: how much time to run a Hadoop cluster ?

[Amandeep Khurana <am...@gmail.com>]

Hadoop does need someone to administer it. You can be doing other
stuff and doing a half time to take care of the cluster. That's how I
do it at my job. However your cluster is bigger and will be doing more
stuff than the one I work on. So, that might change the equation a
bit.

On 10/9/09, Nick Rathke <ni...@sci.utah.edu> wrote:
> Hello Hadoop Users,
>
> Now that we have our cluster up and running ( mostly ) the question has
> come up about much time will be required to run the system.
>
> We have Hadoop running on 64 nodes, 40TB of storage with only 6 or 7
> Hadoop users right now. The number of users is likely to grow as more of
> our researchers begin to use the system but will likely not be more the
> 20 users.
>
> I am trying to get a sense of how much of an FTE ( Full Time Employee )
> I should plan on for managing the system. We also run MPI and CUDA on
> the cluster and from a system administrator point of view these are
> fairly low overhead, but I have no concept yet of what the time
> commitment to Hadoop will be.
>
> Thanks,
>
> Nick Rathke
> Scientific Computing and Imaging Institute
> Sr. Systems Administrator
> nick@sci.utah.edu
> www.sci.utah.edu
> 801-587-9933
> 801-557-3832
>
> "I came I saw I made it possible" Royal Bliss - Here They Come
>


-- 


Amandeep Khurana
Computer Science Graduate Student
University of California, Santa Cruz