MR Jobs managing on Hadoop cluster

[Ophir Cohen <op...@gmail.com>]

Hi,
Recently we deployed 20-nodes cluster in our organization. Shortly it would
doubled (at least) and will start to handle billions of rows.

My question concerns the managing option.

I would like to let users (i.e. internal developers) to submit, schedule and
monitor their jobs.
Of course, I can give them command line access and make them submit jobs
directly.
The problem is that its not a whole management system.

In my vision I can see one web page that holds possibility to submit job
(i.e. add jar and set scheduling), monitor the jobs and even add alert
options (we work with HPOV).

In the current state there is Ganglia, command line and, say, Oozie - but
not in one handy place

Is there any app that do so?
Any other ideas?

Thanks,
Ophir

Re: MR Jobs managing on Hadoop cluster

[Ted Dunning <td...@maprtech.com>]

There is also Azkaban (http://sna-projects.com/azkaban/) which provides the
scheduling and some historical statistics.  Azkaban is much simpler than
oozie, but lacks some capabilities.


On Sun, Jul 3, 2011 at 11:19 PM, Ophir Cohen <op...@gmail.com> wrote:

> Hi,
> Recently we deployed 20-nodes cluster in our organization. Shortly it would
> doubled (at least) and will start to handle billions of rows.
>
> My question concerns the managing option.
>
> I would like to let users (i.e. internal developers) to submit, schedule
> and
> monitor their jobs.
> Of course, I can give them command line access and make them submit jobs
> directly.
> The problem is that its not a whole management system.
>
> In my vision I can see one web page that holds possibility to submit job
> (i.e. add jar and set scheduling), monitor the jobs and even add alert
> options (we work with HPOV).
>
> In the current state there is Ganglia, command line and, say, Oozie - but
> not in one handy place
>
> Is there any app that do so?
> Any other ideas?
>
> Thanks,
> Ophir
>