Running as multiple users on a cluster

[Steve Lewis <lo...@gmail.com>]

We have recently set up a hadoop 0.21 cluster. It looks as if jobs are
designed to run as a user www

We would like to run jobs as individual users - say joe, bill, sue

This raises several issues -
   we can disable permissions - otherwise we see permission issues when jobs
are staged

 What about staging - are there conflicts if multiple users try to submit
jobs at the same time - as long as there is no conflict on
input or output directories

are threre other issues or reasons why different users cannot run hadoop
jobs on the same cluster

-- 
Steven M. Lewis PhD
4221 105th Ave Ne
Kirkland, WA 98033
206-384-1340 (cell)
Institute for Systems Biology
Seattle WA

Re: Running as multiple users on a cluster

[abhishek sharma <ab...@usc.edu>]

Hi Steven,

I have run jobs from multiple users on the same cluster. The dfs
permission and output directory naming were the only issues I had to
worry about.

Things ran correctly even when two different users submitted jobs that
read the same input.

Thanks,
Abhishek

On Wed, Nov 10, 2010 at 4:19 PM, Steve Lewis <lo...@gmail.com> wrote:
> We have recently set up a hadoop 0.21 cluster. It looks as if jobs are
> designed to run as a user www
>
> We would like to run jobs as individual users - say joe, bill, sue
>
> This raises several issues -
>   we can disable permissions - otherwise we see permission issues when jobs
> are staged
>
>  What about staging - are there conflicts if multiple users try to submit
> jobs at the same time - as long as there is no conflict on
> input or output directories
>
> are threre other issues or reasons why different users cannot run hadoop
> jobs on the same cluster
>
> --
> Steven M. Lewis PhD
> 4221 105th Ave Ne
> Kirkland, WA 98033
> 206-384-1340 (cell)
> Institute for Systems Biology
> Seattle WA
>